SHOGUN  v1.1.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
List of all members | Public Member Functions | Protected Attributes
CStreamingVwFeatures Class Reference

Detailed Description

This class implements streaming features for use with VW.

Each example is stored in a VwExample object, which also contains label and other information. Features are hashed and are supposed to be used with a weight vector of preallocated dimensions.

Definition at line 39 of file StreamingVwFeatures.h.

Inheritance diagram for CStreamingVwFeatures:
Inheritance graph
[legend]

Public Member Functions

 CStreamingVwFeatures ()
 CStreamingVwFeatures (CStreamingVwFile *file, bool is_labelled, int32_t size)
 CStreamingVwFeatures (CStreamingVwCacheFile *file, bool is_labelled, int32_t size)
 ~CStreamingVwFeatures ()
CFeaturesduplicate () const
virtual void set_vector_reader ()
virtual void set_vector_and_label_reader ()
virtual void start_parser ()
virtual void end_parser ()
virtual void reset_stream ()
virtual CVwEnvironmentget_env ()
virtual void set_env (CVwEnvironment *vw_env)
virtual bool get_next_example ()
virtual VwExampleget_example ()
virtual float64_t get_label ()
virtual void release_example ()
virtual void expand_if_required (float32_t *&vec, int32_t &len)
virtual void expand_if_required (float64_t *&vec, int32_t &len)
virtual int32_t get_dim_feature_space () const
virtual float32_t real_weight (float32_t w, float32_t gravity)
virtual float32_t dot (CStreamingDotFeatures *df)
virtual float32_t dense_dot (VwExample *&ex, const float32_t *vec2)
virtual float32_t dense_dot (const float32_t *vec2, int32_t vec2_len)
virtual float32_t dense_dot (SGSparseVector< float32_t > *vec1, const float32_t *vec2)
virtual float32_t dense_dot_truncated (const float32_t *vec2, VwExample *&ex, float32_t gravity)
virtual void add_to_dense_vec (float32_t alpha, VwExample *&ex, float32_t *vec2, int32_t vec2_len, bool abs_val=false)
virtual void add_to_dense_vec (float32_t alpha, float32_t *vec2, int32_t vec2_len, bool abs_val=false)
virtual int32_t get_nnz_features_for_vector ()
virtual int32_t get_num_features ()
virtual EFeatureType get_feature_type ()
virtual EFeatureClass get_feature_class ()
virtual const char * get_name () const
virtual int32_t get_num_vectors () const
virtual int32_t get_size ()
- Public Member Functions inherited from CStreamingDotFeatures
 CStreamingDotFeatures ()
 CStreamingDotFeatures (CStreamingFile *file, bool is_labelled, int32_t size)
 CStreamingDotFeatures (CDotFeatures *dot_features, float64_t *lab=NULL)
virtual ~CStreamingDotFeatures ()
virtual void dense_dot_range (float32_t *output, float32_t *alphas, float32_t *vec, int32_t dim, float32_t b, int32_t num_vec=0)
virtual void * get_feature_iterator ()
virtual bool get_next_feature (int32_t &index, float32_t &value, void *iterator)
virtual void free_feature_iterator (void *iterator)
- Public Member Functions inherited from CStreamingFeatures
 CStreamingFeatures ()
 CStreamingFeatures (CStreamingFile *file, bool is_labelled, int32_t size)
virtual ~CStreamingFeatures ()
void set_read_functions ()
virtual bool get_has_labels ()
virtual bool is_seekable ()
- Public Member Functions inherited from CFeatures
 CFeatures (int32_t size=0)
 CFeatures (const CFeatures &orig)
 CFeatures (CFile *loader)
virtual ~CFeatures ()
virtual int32_t add_preprocessor (CPreprocessor *p)
 set preprocessor
virtual CPreprocessordel_preprocessor (int32_t num)
 del current preprocessor
CPreprocessorget_preprocessor (int32_t num)
 get current preprocessor
void set_preprocessed (int32_t num)
bool is_preprocessed (int32_t num)
int32_t get_num_preprocessed ()
 get whether specified preprocessor (or all if num=1) was/were already applied
int32_t get_num_preprocessors () const
void clean_preprocessors ()
int32_t get_cache_size ()
virtual bool reshape (int32_t num_features, int32_t num_vectors)
void list_feature_obj ()
virtual void load (CFile *loader)
virtual void save (CFile *writer)
bool check_feature_compatibility (CFeatures *f)
bool has_property (EFeatureProperty p)
void set_property (EFeatureProperty p)
void unset_property (EFeatureProperty p)
virtual void set_subset (CSubset *subset)
virtual void remove_subset ()
virtual void subset_changed_post ()
index_t subset_idx_conversion (index_t idx) const
bool has_subset () const
virtual CFeaturescopy_subset (SGVector< index_t > indices)
- Public Member Functions inherited from CSGObject
 CSGObject ()
 CSGObject (const CSGObject &orig)
virtual ~CSGObject ()
virtual bool is_generic (EPrimitiveType *generic) const
template<class T >
void set_generic ()
void unset_generic ()
virtual void print_serializable (const char *prefix="")
virtual bool save_serializable (CSerializableFile *file, const char *prefix="")
virtual bool load_serializable (CSerializableFile *file, const char *prefix="")
void set_global_io (SGIO *io)
SGIOget_global_io ()
void set_global_parallel (Parallel *parallel)
Parallelget_global_parallel ()
void set_global_version (Version *version)
Versionget_global_version ()
SGVector< char * > get_modelsel_names ()
char * get_modsel_param_descr (const char *param_name)
index_t get_modsel_param_index (const char *param_name)

Protected Attributes

CInputParser< VwExampleparser
 The parser object, which reads from input and returns parsed example objects.
vw_size_t example_count
 Number of examples processed at a point of time.
float64_t current_label
 The current example's label.
int32_t current_length
 Number of features in current example.
CVwEnvironmentenv
 Environment for VW.
VwExamplecurrent_example
 Example currently being processed.
- Protected Attributes inherited from CStreamingDotFeatures
float32_t combined_weight
 feature weighting in combined dot features
- Protected Attributes inherited from CStreamingFeatures
bool has_labels
 Whether examples are labelled or not.
CStreamingFileworking_file
 The StreamingFile object to read from.
bool seekable
 Whether the stream is seekable.
- Protected Attributes inherited from CFeatures
CSubsetm_subset

Additional Inherited Members

- Public Attributes inherited from CSGObject
SGIOio
Parallelparallel
Versionversion
Parameterm_parameters
Parameterm_model_selection_parameters
- Protected Member Functions inherited from CSGObject
virtual void load_serializable_pre () throw (ShogunException)
virtual void load_serializable_post () throw (ShogunException)
virtual void save_serializable_pre () throw (ShogunException)
virtual void save_serializable_post () throw (ShogunException)

Constructor & Destructor Documentation

Default constructor.

Sets the reading functions to be CStreamingFile::get_*_vector and get_*_vector_and_label depending on the type T.

Definition at line 20 of file StreamingVwFeatures.cpp.

CStreamingVwFeatures ( CStreamingVwFile file,
bool  is_labelled,
int32_t  size 
)

Constructor taking args. Initializes the parser with the given args.

Parameters
fileStreamingFile object, input file.
is_labelledWhether examples are labelled or not.
sizeNumber of example objects to be stored in the parser at a time.

Definition at line 26 of file StreamingVwFeatures.cpp.

CStreamingVwFeatures ( CStreamingVwCacheFile file,
bool  is_labelled,
int32_t  size 
)

Constructor used when initialized with a cache file.

Parameters
fileStreamingVwCacheFile object
is_labelledWhether examples are labelled or not
sizeNumber of example objects to be stored in the parser at a time

Definition at line 34 of file StreamingVwFeatures.cpp.

Destructor.

Ends the parsing thread. (Waits for pthread_join to complete)

Definition at line 42 of file StreamingVwFeatures.cpp.

Member Function Documentation

void add_to_dense_vec ( float32_t  alpha,
VwExample *&  ex,
float32_t vec2,
int32_t  vec2_len,
bool  abs_val = false 
)
virtual

Add alpha*an example's feature vector to another dense vector. Takes the absolute value of current_vector if specified

Parameters
alphaalpha
exexample whose vector should be used
vec2vector to add to
vec2_lenlength of vector
abs_valtrue if abs of example's vector should be taken

Definition at line 350 of file StreamingVwFeatures.cpp.

void add_to_dense_vec ( float32_t  alpha,
float32_t vec2,
int32_t  vec2_len,
bool  abs_val = false 
)
virtual

Add alpha*current_vector to another dense vector. Takes the absolute value of current_vector if specified

Parameters
alphaalpha
vec2vector to add to
vec2_lenlength of vector
abs_valtrue if abs of current_vector should be taken

Implements CStreamingDotFeatures.

Definition at line 370 of file StreamingVwFeatures.cpp.

float32_t dense_dot ( VwExample *&  ex,
const float32_t vec2 
)
virtual

Dot product of an example with a vector

Parameters
exexample, as VwExample
vec2vector to take dot product with
Returns
dot product

Definition at line 309 of file StreamingVwFeatures.cpp.

float32_t dense_dot ( const float32_t vec2,
int32_t  vec2_len 
)
virtual

Dot product of current feature vector with a dense vector which stores weights in hashed indices

Parameters
vec2dense weight vector
vec2_lenlength of weight vector (not used)
Returns
dot product

Implements CStreamingDotFeatures.

Definition at line 320 of file StreamingVwFeatures.cpp.

float32_t dense_dot ( SGSparseVector< float32_t > *  vec1,
const float32_t vec2 
)
virtual

Dot product between a dense weight vector and a sparse feature vector. Assumes the features to belong to the constant namespace.

Parameters
vec1sparse feature vector
vec2weight vector
Returns
dot product between dense weights and a sparse feature vector

Definition at line 325 of file StreamingVwFeatures.cpp.

float32_t dense_dot_truncated ( const float32_t vec2,
VwExample *&  ex,
float32_t  gravity 
)
virtual

Calculate dot product of features with another vector, truncating the elements of that vector by magnitude 'gravity' to a minimum final magnitude of zero.

Parameters
vec2vector to take dot product with
exexample whose features have to be taken
gravityvalue to use for truncating
Returns
dot product

Definition at line 334 of file StreamingVwFeatures.cpp.

float32_t dot ( CStreamingDotFeatures df)
virtual

Dot product taken with another StreamingDotFeatures object.

Currently only works if it is a CStreamingVwFeatures object. It takes the dot product of the current_vectors of both objects.

Parameters
dfCStreamingDotFeatures object.
Returns
Dot product.

Implements CStreamingDotFeatures.

Definition at line 303 of file StreamingVwFeatures.cpp.

CFeatures * duplicate ( ) const
virtual

Duplicate this object

Returns
a copy of this object

Implements CFeatures.

Definition at line 48 of file StreamingVwFeatures.cpp.

void end_parser ( )
virtual

Ends the parsing thread.

Waits for the thread to join.

Implements CStreamingFeatures.

Definition at line 244 of file StreamingVwFeatures.cpp.

void expand_if_required ( float32_t *&  vec,
int32_t &  len 
)
virtual

Expand the vector passed so that it its length is equal to the dimensionality of the features. The previous values are kept intact through realloc, and the new ones are set to zero.

Parameters
vecfloat32_t* vector
lenlength of the vector

Reimplemented from CStreamingDotFeatures.

Definition at line 89 of file StreamingVwFeatures.cpp.

void expand_if_required ( float64_t *&  vec,
int32_t &  len 
)
virtual

Expand the vector passed so that it its length is equal to the dimensionality of the features. The previous values are kept intact through realloc, and the new ones are set to zero.

Parameters
vecfloat64_t* vector
lenlength of the vector

Reimplemented from CStreamingDotFeatures.

Definition at line 100 of file StreamingVwFeatures.cpp.

int32_t get_dim_feature_space ( ) const
virtual

obtain the dimensionality of the feature space

(not mix this up with the dimensionality of the input space, usually obtained via get_num_features())

Returns
dimensionality

Implements CStreamingDotFeatures.

Definition at line 298 of file StreamingVwFeatures.cpp.

CVwEnvironment * get_env ( )
virtual

Get the environment

Returns
environment

Definition at line 77 of file StreamingVwFeatures.cpp.

VwExample * get_example ( )
virtual

Returns the current example.

Returns
current example as VwExample*

Definition at line 269 of file StreamingVwFeatures.cpp.

EFeatureClass get_feature_class ( )
virtual

Return the feature class

Returns
C_STREAMING_VW

Implements CFeatures.

Definition at line 380 of file StreamingVwFeatures.cpp.

EFeatureType get_feature_type ( )
virtual

Return the feature type, depending on T.

Returns
Feature type as EFeatureType

Implements CFeatures.

Definition at line 137 of file StreamingVwFeatures.cpp.

float64_t get_label ( )
virtual

Return the label of the current example as a float.

Examples must be labelled, otherwise an error occurs.

Returns
The label as a float64_t.

Implements CStreamingFeatures.

Definition at line 274 of file StreamingVwFeatures.cpp.

virtual const char* get_name ( ) const
virtual

Return the name.

Returns
StreamingVwFeatures

Implements CSGObject.

Definition at line 329 of file StreamingVwFeatures.h.

bool get_next_example ( )
virtual

Instructs the parser to return the next example.

This example is stored as the current_example in this object.

Returns
True on success, false if there are no more examples, or an error occurred.

Implements CStreamingFeatures.

Definition at line 249 of file StreamingVwFeatures.cpp.

int32_t get_nnz_features_for_vector ( )
virtual

get number of non-zero features in vector

Returns
number of non-zero features in vector

Reimplemented from CStreamingDotFeatures.

Definition at line 119 of file StreamingVwFeatures.cpp.

int32_t get_num_features ( )
virtual

Return the number of features in the current example.

Returns
number of features as int

Implements CStreamingFeatures.

Definition at line 375 of file StreamingVwFeatures.cpp.

int32_t get_num_vectors ( ) const
virtual

Return the number of vectors stored in this object.

Returns
1 if current_example exists, else 0.

Implements CFeatures.

Definition at line 124 of file StreamingVwFeatures.cpp.

int32_t get_size ( )
virtual

Return the size of one T object.

Returns
Size of T.

Implements CFeatures.

Definition at line 132 of file StreamingVwFeatures.cpp.

float32_t real_weight ( float32_t  w,
float32_t  gravity 
)
virtual

Reduce element 'w' to max(w-gravity, 0)

Parameters
wvalue to truncate
gravityvalue to truncate using
Returns
truncated value

Definition at line 111 of file StreamingVwFeatures.cpp.

void release_example ( )
virtual

Release the current example, indicating to the parser that it has been processed by the learning algorithm.

The parser is then free to throw away that example.

Implements CStreamingFeatures.

Definition at line 281 of file StreamingVwFeatures.cpp.

void reset_stream ( )
virtual

Reset the file back to the first example. Only works for cache files.

Reimplemented from CStreamingFeatures.

Definition at line 63 of file StreamingVwFeatures.cpp.

void set_env ( CVwEnvironment vw_env)
virtual

Set the environment

Parameters
vw_envenvironment

Definition at line 83 of file StreamingVwFeatures.cpp.

void set_vector_and_label_reader ( )
virtual

Sets the read function (in case the examples are labelled) to get_*_vector_and_label from CStreamingFile.

The exact function depends on type T.

The parser uses the function set by this while reading labelled examples.

Implements CStreamingFeatures.

Definition at line 58 of file StreamingVwFeatures.cpp.

void set_vector_reader ( )
virtual

Sets the read function (in case the examples are unlabelled) to get_*_vector() from CStreamingFile.

The exact function depends on type T.

The parser uses the function set by this while reading unlabelled examples.

Implements CStreamingFeatures.

Definition at line 53 of file StreamingVwFeatures.cpp.

void start_parser ( )
virtual

Starts the parsing thread.

To be called before trying to use any feature vectors from this object.

Implements CStreamingFeatures.

Definition at line 238 of file StreamingVwFeatures.cpp.

Member Data Documentation

VwExample* current_example
protected

Example currently being processed.

Definition at line 396 of file StreamingVwFeatures.h.

float64_t current_label
protected

The current example's label.

Definition at line 387 of file StreamingVwFeatures.h.

int32_t current_length
protected

Number of features in current example.

Definition at line 390 of file StreamingVwFeatures.h.

CVwEnvironment* env
protected

Environment for VW.

Definition at line 393 of file StreamingVwFeatures.h.

vw_size_t example_count
protected

Number of examples processed at a point of time.

Definition at line 384 of file StreamingVwFeatures.h.

CInputParser<VwExample> parser
protected

The parser object, which reads from input and returns parsed example objects.

Definition at line 381 of file StreamingVwFeatures.h.


The documentation for this class was generated from the following files:

SHOGUN Machine Learning Toolbox - Documentation