Public Member Functions | Static Public Member Functions | Protected Attributes

CStreamingSparseFeatures< T > Class Template Reference


Detailed Description

template<class T>
class shogun::CStreamingSparseFeatures< T >

This class implements streaming features with sparse feature vectors. The vector is represented as an SGSparseVector<T>. Each entry is of type SGSparseVectorEntry<T> with members `feat_index' and `entry'.

This class expects the input from the StreamingFile object to be zero-based, i.e., a feature entered as 1:6.5 would have feat_index=0 and entry=6.5.

The current example is stored as a combination of current_vector and current_label. current_num_features stores the highest dimensionality of examples encountered upto the point of the function call. For example, if the first example is '1:6.5 7:10.0', then current_num_features would be 7 after the first function call.

Since the dimensionality of the feature space is not immediately known initially, current_num_features may increase as more examples are processed and larger dimensions are seen. For this purpose, `expand_if_required()' is provided which when called with a dynamically allocated float or double array and the length, reallocates that array to the new dimensionality (if necessary), setting the newer dimensions to zero, and updates the length parameter to equal the new length of the array.

Definition at line 43 of file StreamingSparseFeatures.h.

Inheritance diagram for CStreamingSparseFeatures< T >:
Inheritance graph
[legend]

List of all members.

Public Member Functions

 CStreamingSparseFeatures ()
 CStreamingSparseFeatures (CStreamingFile *file, bool is_labelled, int32_t size)
 ~CStreamingSparseFeatures ()
virtual void set_vector_reader ()
virtual void set_vector_and_label_reader ()
virtual void start_parser ()
virtual void end_parser ()
virtual bool get_next_example ()
get_feature (int32_t index)
SGSparseVector< T > get_vector ()
virtual float64_t get_label ()
virtual void release_example ()
virtual void reset_stream ()
int32_t set_num_features (int32_t num)
virtual int32_t get_dim_feature_space () const
virtual void expand_if_required (float32_t *&vec, int32_t &len)
virtual void expand_if_required (float64_t *&vec, int32_t &len)
virtual float32_t dot (CStreamingDotFeatures *df)
dense_dot (T alpha, T *vec, int32_t dim, T b)
virtual float64_t dense_dot (const float64_t *vec2, int32_t vec2_len)
virtual float32_t dense_dot (const float32_t *vec2, int32_t vec2_len)
virtual void add_to_dense_vec (float64_t alpha, float64_t *vec2, int32_t vec2_len, bool abs_val=false)
virtual void add_to_dense_vec (float32_t alpha, float32_t *vec2, int32_t vec2_len, bool abs_val=false)
int64_t get_num_nonzero_entries ()
float32_t compute_squared ()
void sort_features ()
virtual int32_t get_num_features ()
virtual int32_t get_nnz_features_for_vector ()
virtual EFeatureType get_feature_type ()
virtual EFeatureClass get_feature_class ()
virtual CFeaturesduplicate () const
virtual const char * get_name () const
virtual int32_t get_num_vectors () const
virtual int32_t get_size ()

Static Public Member Functions

static T sparse_dot (T alpha, SGSparseVectorEntry< T > *avec, int32_t alen, SGSparseVectorEntry< T > *bvec, int32_t blen)

Protected Attributes

CInputParser
< SGSparseVectorEntry< T > > 
parser
 The parser object, which reads from input and returns parsed example objects.
CStreamingFileworking_file
 The StreamingFile object to read from.
SGSparseVector< T > current_sgvector
 The current example's feature vector as an SGVector<T>
SGSparseVectorEntry< T > * current_vector
 The current example's feature vector as an SGSparseVectorEntry<T>*.
index_t current_vec_index
 The current vector index.
float64_t current_label
 The current example's label.
int32_t current_length
 Number of set indices in current example.
int32_t current_num_features
 Number of features in current vector (as seen so far upto the current vector).

Constructor & Destructor Documentation

Default constructor.

Sets the reading functions to be CStreamingFile::get_*_vector and get_*_vector_and_label depending on the type T.

Definition at line 54 of file StreamingSparseFeatures.h.

CStreamingSparseFeatures ( CStreamingFile file,
bool  is_labelled,
int32_t  size 
)

Constructor taking args. Initializes the parser with the given args.

Parameters:
file StreamingFile object, input file.
is_labelled Whether examples are labelled or not.
size Number of example objects to be stored in the parser at a time.

Definition at line 69 of file StreamingSparseFeatures.h.

Destructor.

Ends the parsing thread. (Waits for pthread_join to complete)

Definition at line 83 of file StreamingSparseFeatures.h.


Member Function Documentation

virtual void add_to_dense_vec ( float64_t  alpha,
float64_t vec2,
int32_t  vec2_len,
bool  abs_val = false 
) [virtual]

Add alpha*current_vector to another float64_t type dense vector. Takes the absolute value of current_vector if specified.

Parameters:
alpha alpha
vec2 vector to add to, float64_t*
vec2_len length of vector
abs_val true if abs of current_vector should be taken

Definition at line 415 of file StreamingSparseFeatures.h.

virtual void add_to_dense_vec ( float32_t  alpha,
float32_t vec2,
int32_t  vec2_len,
bool  abs_val = false 
) [virtual]

Add alpha*current_vector to another dense vector. Takes the absolute value of current_vector if specified.

Parameters:
alpha alpha
vec2 vector to add to
vec2_len length of vector
abs_val true if abs of current_vector should be taken

Implements CStreamingDotFeatures.

Definition at line 451 of file StreamingSparseFeatures.h.

float32_t compute_squared (  ) 

Compute sum of squares of features on current vector.

Returns:
sum of squares for current vector

Definition at line 493 of file StreamingSparseFeatures.h.

virtual float32_t dense_dot ( const float32_t vec2,
int32_t  vec2_len 
) [virtual]

Dot product with another dense vector.

Parameters:
vec2 The dense vector with which to take the dot product.
vec2_len length of vector
Returns:
Dot product as a float32_t.

Implements CStreamingDotFeatures.

Definition at line 387 of file StreamingSparseFeatures.h.

virtual float64_t dense_dot ( const float64_t vec2,
int32_t  vec2_len 
) [virtual]

Dot product with another float64_t type dense vector.

Parameters:
vec2 The dense vector with which to take the dot product.
vec2_len length of vector
Returns:
Dot product as a float64_t.

Definition at line 360 of file StreamingSparseFeatures.h.

T dense_dot ( alpha,
T *  vec,
int32_t  dim,
b 
)

compute the dot product between dense weights and a sparse feature vector alpha * sparse^T * w + b

Parameters:
alpha scalar to multiply with
vec dense vector to compute dot product with
dim length of the dense vector
b bias
Returns:
dot product between dense weights and a sparse feature vector

Definition at line 334 of file StreamingSparseFeatures.h.

float32_t dot ( CStreamingDotFeatures df  )  [virtual]

Dot product taken with another StreamingDotFeatures object.

Currently only works if it is a CStreamingSparseFeatures object. It takes the dot product of the current_vectors of both objects.

Parameters:
df CStreamingDotFeatures object.
Returns:
Dot product.

Implements CStreamingDotFeatures.

Definition at line 772 of file StreamingSparseFeatures.h.

virtual CFeatures* duplicate (  )  const [virtual]

Duplicate the object.

Returns:
a duplicate object as CFeatures*

Implements CFeatures.

Definition at line 579 of file StreamingSparseFeatures.h.

void end_parser (  )  [virtual]

Ends the parsing thread.

Waits for the thread to join.

Implements CStreamingFeatures.

Definition at line 714 of file StreamingSparseFeatures.h.

virtual void expand_if_required ( float32_t *&  vec,
int32_t &  len 
) [virtual]

Expand the vector passed so that it its length is equal to the dimensionality of the features. The previous values are kept intact through realloc, and the new ones are set to zero.

Parameters:
vec float32_t* vector
len length of the vector

Reimplemented from CStreamingDotFeatures.

Definition at line 225 of file StreamingSparseFeatures.h.

virtual void expand_if_required ( float64_t *&  vec,
int32_t &  len 
) [virtual]

Expand the vector passed so that it its length is equal to the dimensionality of the features. The previous values are kept intact through realloc, and the new ones are set to zero.

Parameters:
vec float64_t* vector
len length of the vector

Reimplemented from CStreamingDotFeatures.

Definition at line 244 of file StreamingSparseFeatures.h.

int32_t get_dim_feature_space (  )  const [virtual]

obtain the dimensionality of the feature space

(not mix this up with the dimensionality of the input space, usually obtained via get_num_features())

Returns:
dimensionality

Implements CStreamingDotFeatures.

Definition at line 766 of file StreamingSparseFeatures.h.

T get_feature ( int32_t  index  ) 

get a single feature

Parameters:
index index of feature in this vector
Returns:
sum of features that match dimension index and 0 if none is found

Definition at line 140 of file StreamingSparseFeatures.h.

EFeatureClass get_feature_class (  )  [virtual]

Return the feature class

Returns:
C_STREAMING_SPARSE

Implements CFeatures.

Definition at line 791 of file StreamingSparseFeatures.h.

virtual EFeatureType get_feature_type (  )  [virtual]

Return the feature type, depending on T.

Returns:
Feature type as EFeatureType

Implements CFeatures.

float64_t get_label (  )  [virtual]

Return the label of the current example as a float.

Examples must be labelled, otherwise an error occurs.

Returns:
The label as a float64_t.

Implements CStreamingFeatures.

Definition at line 752 of file StreamingSparseFeatures.h.

virtual const char* get_name ( void   )  const [virtual]

Return the name.

Returns:
StreamingSparseFeatures

Implements CSGObject.

Definition at line 589 of file StreamingSparseFeatures.h.

bool get_next_example (  )  [virtual]

Instructs the parser to return the next example.

This example is stored as the current_example in this object.

Returns:
True on success, false if there are no more examples, or an error occurred.

Implements CStreamingFeatures.

Definition at line 720 of file StreamingSparseFeatures.h.

int32_t get_nnz_features_for_vector (  )  [virtual]

Return the number of non-zero features in vector

Returns:
number of sparse features in vector

Reimplemented from CStreamingDotFeatures.

Definition at line 785 of file StreamingSparseFeatures.h.

int32_t get_num_features (  )  [virtual]

Return the number of features in the current example.

Returns:
number of features as int

Implements CStreamingFeatures.

Definition at line 779 of file StreamingSparseFeatures.h.

int64_t get_num_nonzero_entries (  ) 

Get number of non-zero entries in current sparse vector

Returns:
number of features explicity set in the sparse vector

Definition at line 483 of file StreamingSparseFeatures.h.

virtual int32_t get_num_vectors (  )  const [virtual]

Return the number of vectors stored in this object.

Returns:
1 if current_vector exists, else 0.

Implements CFeatures.

Definition at line 596 of file StreamingSparseFeatures.h.

virtual int32_t get_size (  )  [virtual]

Return the size of one T object.

Returns:
Size of T.

Implements CFeatures.

Definition at line 608 of file StreamingSparseFeatures.h.

SGSparseVector< T > get_vector (  ) 

Return the current feature vector as an SGSparseVector<T>.

Returns:
The vector as SGSparseVector<T>

Definition at line 742 of file StreamingSparseFeatures.h.

void release_example (  )  [virtual]

Release the current example, indicating to the parser that it has been processed by the learning algorithm.

The parser is then free to throw away that example.

Implements CStreamingFeatures.

Definition at line 760 of file StreamingSparseFeatures.h.

virtual void reset_stream (  )  [virtual]

Reset the file back to the first example if possible.

Reimplemented from CStreamingFeatures.

Definition at line 184 of file StreamingSparseFeatures.h.

int32_t set_num_features ( int32_t  num  ) 

set number of features

Sometimes when loading sparse features not all possible dimensions are used. This may pose a problem to classifiers when being applied to higher dimensional test-data. This function allows to artificially explode the feature space

Parameters:
num the number of features, must be larger than the current number of features
Returns:
previous number of features

Definition at line 200 of file StreamingSparseFeatures.h.

void set_vector_and_label_reader (  )  [virtual]

Sets the read function (in case the examples are labelled) to get_*_vector_and_label from CStreamingFile.

The exact function depends on type T.

The parser uses the function set by this while reading labelled examples.

Implements CStreamingFeatures.

Definition at line 657 of file StreamingSparseFeatures.h.

void set_vector_reader (  )  [virtual]

Sets the read function (in case the examples are unlabelled) to get_*_vector() from CStreamingFile.

The exact function depends on type T.

The parser uses the function set by this while reading unlabelled examples.

Implements CStreamingFeatures.

Definition at line 652 of file StreamingSparseFeatures.h.

void sort_features (  ) 

Ensure features of the current vector are in ascending order. It modifies the current_vector in-place, though a temporary vector is created and later freed.

Definition at line 510 of file StreamingSparseFeatures.h.

static T sparse_dot ( alpha,
SGSparseVectorEntry< T > *  avec,
int32_t  alen,
SGSparseVectorEntry< T > *  bvec,
int32_t  blen 
) [static]

compute the dot product between two sparse feature vectors alpha * vec^T * vec

Parameters:
alpha scalar to multiply with
avec first sparse feature vector
alen avec's length
bvec second sparse feature vector
blen bvec's length
Returns:
dot product between the two sparse feature vectors

Definition at line 277 of file StreamingSparseFeatures.h.

void start_parser (  )  [virtual]

Starts the parsing thread.

To be called before trying to use any feature vectors from this object.

Implements CStreamingFeatures.

Definition at line 707 of file StreamingSparseFeatures.h.


Member Data Documentation

float64_t current_label [protected]

The current example's label.

Definition at line 643 of file StreamingSparseFeatures.h.

int32_t current_length [protected]

Number of set indices in current example.

Definition at line 646 of file StreamingSparseFeatures.h.

int32_t current_num_features [protected]

Number of features in current vector (as seen so far upto the current vector).

Definition at line 649 of file StreamingSparseFeatures.h.

The current example's feature vector as an SGVector<T>

Definition at line 634 of file StreamingSparseFeatures.h.

The current vector index.

Definition at line 640 of file StreamingSparseFeatures.h.

The current example's feature vector as an SGSparseVectorEntry<T>*.

Definition at line 637 of file StreamingSparseFeatures.h.

The parser object, which reads from input and returns parsed example objects.

Definition at line 628 of file StreamingSparseFeatures.h.

The StreamingFile object to read from.

Reimplemented from CStreamingFeatures.

Definition at line 631 of file StreamingSparseFeatures.h.


The documentation for this class was generated from the following file:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines

SHOGUN Machine Learning Toolbox - Documentation