SHOGUN  v1.1.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
List of all members | Public Member Functions | Protected Attributes
CStreamingFile Class Reference

Detailed Description

A Streaming File access class.

Definition at line 38 of file StreamingFile.h.

Inheritance diagram for CStreamingFile:
Inheritance graph
[legend]

Public Member Functions

 CStreamingFile ()
 CStreamingFile (char *fname, char rw='r')
virtual ~CStreamingFile ()
void close ()
virtual bool is_seekable ()
virtual void reset_stream ()
virtual void get_vector (VwExample *&ex, int32_t &len)
virtual void get_vector_and_label (VwExample *&ex, int32_t &len, float64_t &label)
virtual const char * get_name () const
Dense Vector Access Functions

Functions to access dense vectors of one of several base data types. These functions are used when reading one dense vector at a time from an input source and return the vector and length of the vector by reference.

virtual void get_vector (bool *&vector, int32_t &len)
virtual void get_vector (uint8_t *&vector, int32_t &len)
virtual void get_vector (char *&vector, int32_t &len)
virtual void get_vector (int32_t *&vector, int32_t &len)
virtual void get_vector (float32_t *&vector, int32_t &len)
virtual void get_vector (float64_t *&vector, int32_t &len)
virtual void get_vector (int16_t *&vector, int32_t &len)
virtual void get_vector (uint16_t *&vector, int32_t &len)
virtual void get_vector (int8_t *&vector, int32_t &len)
virtual void get_vector (uint32_t *&vector, int32_t &len)
virtual void get_vector (int64_t *&vector, int32_t &len)
virtual void get_vector (uint64_t *&vector, int32_t &len)
virtual void get_vector (floatmax_t *&vector, int32_t &len)
Dense Vector And Label Access Functions

Functions to access dense vectors of one of several base data types. These functions are used when reading one dense vector at a time from an input source and return the vector, length and label of the vector by reference.

virtual void get_vector_and_label (bool *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint8_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (char *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int32_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (float32_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (float64_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int16_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint16_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int8_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint32_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int64_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint64_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (floatmax_t *&vector, int32_t &len, float64_t &label)
String Access Functions

Functions to access string of one of several base data types. These functions are used when reading one string vector at a time from an input source and return the vector and length of the vector by reference.

virtual void get_string (bool *&vector, int32_t &len)
virtual void get_string (uint8_t *&vector, int32_t &len)
virtual void get_string (char *&vector, int32_t &len)
virtual void get_string (int32_t *&vector, int32_t &len)
virtual void get_string (float32_t *&vector, int32_t &len)
virtual void get_string (float64_t *&vector, int32_t &len)
virtual void get_string (int16_t *&vector, int32_t &len)
virtual void get_string (uint16_t *&vector, int32_t &len)
virtual void get_string (int8_t *&vector, int32_t &len)
virtual void get_string (uint32_t *&vector, int32_t &len)
virtual void get_string (int64_t *&vector, int32_t &len)
virtual void get_string (uint64_t *&vector, int32_t &len)
virtual void get_string (floatmax_t *&vector, int32_t &len)
String And Label Access Functions

Functions to access strings of one of several base data types. These functions are used when reading one string vector at a time from an input source and return the vector, length and label of the vector by reference.

virtual void get_string_and_label (bool *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint8_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (char *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int32_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (float32_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (float64_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int16_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint16_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int8_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint32_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int64_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint64_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (floatmax_t *&vector, int32_t &len, float64_t &label)
Sparse Vector Access Functions

Functions to access sparse vectors of one of several base data types. These functions are used when reading one sparse vector at a time from an input source and return the vector and length of the vector by reference.

virtual void get_sparse_vector (SGSparseVectorEntry< bool > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint8_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< char > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int32_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< float32_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< float64_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int16_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint16_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int8_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint32_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int64_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint64_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< floatmax_t > *&vector, int32_t &len)
Sparse Vector And Label Access Functions

Functions to access sparse vectors of one of several base data types. These functions are used when reading one sparse vector at a time from an input source and return the vector, length and label of the vector by reference.

virtual void get_sparse_vector_and_label (SGSparseVectorEntry< bool > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint8_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< char > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int32_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< float32_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< float64_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int16_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint16_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int8_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint32_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int64_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint64_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< floatmax_t > *&vector, int32_t &len, float64_t &label)
- Public Member Functions inherited from CSGObject
 CSGObject ()
 CSGObject (const CSGObject &orig)
virtual ~CSGObject ()
virtual bool is_generic (EPrimitiveType *generic) const
template<class T >
void set_generic ()
void unset_generic ()
virtual void print_serializable (const char *prefix="")
virtual bool save_serializable (CSerializableFile *file, const char *prefix="")
virtual bool load_serializable (CSerializableFile *file, const char *prefix="")
void set_global_io (SGIO *io)
SGIOget_global_io ()
void set_global_parallel (Parallel *parallel)
Parallelget_global_parallel ()
void set_global_version (Version *version)
Versionget_global_version ()
SGVector< char * > get_modelsel_names ()
char * get_modsel_param_descr (const char *param_name)
index_t get_modsel_param_index (const char *param_name)

Protected Attributes

CIOBufferbuf
 Buffer to hold stuff in memory.
char task
 Task.
char * filename
 Name of the handled file.

Additional Inherited Members

- Public Attributes inherited from CSGObject
SGIOio
Parallelparallel
Versionversion
Parameterm_parameters
Parameterm_model_selection_parameters
- Protected Member Functions inherited from CSGObject
virtual void load_serializable_pre () throw (ShogunException)
virtual void load_serializable_post () throw (ShogunException)
virtual void save_serializable_pre () throw (ShogunException)
virtual void save_serializable_post () throw (ShogunException)

Constructor & Destructor Documentation

default constructor

Definition at line 195 of file StreamingFile.cpp.

CStreamingFile ( char *  fname,
char  rw = 'r' 
)

constructor

Parameters
fnamefilename to open
rwmode, 'r' or 'w'

Definition at line 201 of file StreamingFile.cpp.

~CStreamingFile ( )
virtual

default destructor

Definition at line 232 of file StreamingFile.cpp.

Member Function Documentation

void close ( )

Closes the file

Definition at line 57 of file StreamingFile.h.

virtual const char* get_name ( ) const
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< bool > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< uint8_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< char > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< int32_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< float32_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< float64_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< int16_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< uint16_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< int8_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< uint32_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< int64_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< uint64_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector ( SGSparseVectorEntry< floatmax_t > *&  vector,
int32_t &  len 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< bool > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< uint8_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< char > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< int32_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< float32_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< float64_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< int16_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< uint16_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< int8_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< uint32_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< int64_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< uint64_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_sparse_vector_and_label ( SGSparseVectorEntry< floatmax_t > *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string ( bool *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( uint8_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( char *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( int32_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( float32_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( float64_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( int16_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( uint16_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( int8_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( uint32_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( int64_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( uint64_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string ( floatmax_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_string_and_label ( bool *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( uint8_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( char *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( int32_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( float32_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( float64_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( int16_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( uint16_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( int8_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( uint32_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( int64_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( uint64_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_string_and_label ( floatmax_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector ( bool *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( uint8_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( char *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( int32_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( float32_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( float64_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( int16_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( uint16_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( int8_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( uint32_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( int64_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( uint64_t *&  vector,
int32_t &  len 
)
virtual
virtual void get_vector ( floatmax_t *&  vector,
int32_t &  len 
)
virtual
void get_vector ( VwExample *&  ex,
int32_t &  len 
)
virtual

Function to read VW examples without labels

Parameters
exexample
lenlength of feature vector

Reimplemented in CStreamingVwCacheFile, and CStreamingVwFile.

Definition at line 182 of file StreamingFile.cpp.

virtual void get_vector_and_label ( bool *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( uint8_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( char *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( int32_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( float32_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( float64_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( int16_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( uint16_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( int8_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( uint32_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( int64_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( uint64_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
virtual void get_vector_and_label ( floatmax_t *&  vector,
int32_t &  len,
float64_t label 
)
virtual
void get_vector_and_label ( VwExample *&  ex,
int32_t &  len,
float64_t label 
)
virtual

Function to read VW examples with labels

Parameters
exexample
lenlength of feature vector
labellabel

Reimplemented in CStreamingVwCacheFile, and CStreamingVwFile.

Definition at line 187 of file StreamingFile.cpp.

virtual bool is_seekable ( )
virtual

Whether the stream is seekable/resettable

Returns
false by default, unless overloaded

Reimplemented in CStreamingVwFile, and CStreamingVwCacheFile.

Definition at line 67 of file StreamingFile.h.

virtual void reset_stream ( )
virtual

Reset the stream, should be overloaded if possible

Reimplemented in CStreamingVwCacheFile, CStreamingFileFromSimpleFeatures< T >, CStreamingFileFromSparseFeatures< T >, and CStreamingFileFromStringFeatures< T >.

Definition at line 72 of file StreamingFile.h.

Member Data Documentation

CIOBuffer* buf
protected

Buffer to hold stuff in memory.

Definition at line 294 of file StreamingFile.h.

char* filename
protected

Name of the handled file.

Definition at line 298 of file StreamingFile.h.

char task
protected

Task.

Definition at line 296 of file StreamingFile.h.


The documentation for this class was generated from the following files:

SHOGUN Machine Learning Toolbox - Documentation