Detailed Description

template<class ST>
class shogun::CFeatureSelection< ST >

Template class CFeatureSelection, base class for all feature selection preprocessors which select a subset of features (dimensions in the feature matrix) to achieve a specified number of dimensions, m_target_dim from a given set of features. This class showcases all feature selection algorithms via a generic interface. Supported algorithms are specified by the enum EFeatureSelectionAlgorithm which can be set via set_algorithm() call. Supported wrapper algorithms are.

shogun::BACKWARD_ELIMINATION: apply_backward_elimination() method implements this algorithm. This runs inside a loop till it reaches m_target_dim. In each iteration, inside another loop that runs for all current dimensions we compute measures and store the scores for each current dimension. Based on those measures a number of features are removed at once.
shogun::FORWARD_SELECTION: apply_forward_selection() method implements this algorithm. Inside a loop it adds selected features to an empty feature set till it reaches m_target_dim. In each iteration, inside another loop that runs for all current dimensions we compute measures and store the scores for each current dimension. Based on those measures a number of features are removed from the original feature set and added to the new feature set.

Since all these algorithm cannot be applied for all the feature selection approaches, the method set_algorithm() is kept abstract which is defined in the subclasses as appropriate.

The apply() method acts as a wrapper which decides which above methods to call based on the algorithm specified by m_algorithm. This method makes a deep copy of the original feature object using CSGObject::clone and then performs feature selection on it. The actual memory requirement depends on how copying a dimension subset is handled in CFeatures::copy_dimension_subset implementation.

For computing the measures that are used to rank the features for feature selection task, it relies on an abstract method compute_measures() which is defined in the subclasses.

Due to the difference in the measure, the removal policy for features can be different which is specified by the EFeatureRemovalPolicy enum and can be set by set_policy() call. The supported policies are

shogun::N_SMALLEST: Features corresponding to N smallest measures are removed
shogun::PERCENTILE_SMALLEST: Features corresponding to smallest N% measures are removed
shogun::N_LARGEST: Features corresponding to N largeest measures are removed
shogun::PERCENTILE_LARGEST: Features corresponding to largest N% measures are removed

Note that not all policies can be adapted for a specific feature seleciton approaches. In general, in classes where feature selection is performed by removing the features which correspond to lowest measure, the policies shogun::N_SMALLEST and shogun::PERCENTILE_SMALLEST are appropriate. When features corresponding to highest measures are removed (e.g. training error in a cross-validation scenario), shogun::N_LARGEST and shogun::PERCENTILE_LARGEST are applicable. Therefore, set_policy() is kept abstract and subclasses define this to allow specific policies to be set.

Removal of features in each iteration is handled by an abstract method remove_feats() that removes a set of features at once based on the ranks of the features on the measure. Internally this method also updates a subset stack to keep track of the selected features. After calling apply(), the indices of the original features that are selected can be obtained by calling get_selected_feats() method. Please note that the selected features are always kept in the original order.

Some of the methods are for internal purpose and are not exposed to the public API. For example,

method precompute() is provided which is intended to be overridden in the subclasses to perform specific tasks that can be completed beforehand in the feature selection algorithms. For example, see CKernelDependenceMaximization.
method adapt_params() is also overridden in the subclasses which tunes the parameters based on current features that are then used to compute the measure.

Definition at line 139 of file FeatureSelection.h.

Inheritance diagram for CFeatureSelection< ST >:

[legend]

Public Member Functions
	CFeatureSelection ()

virtual	~CFeatureSelection ()

virtual CFeatures *	apply (CFeatures *features)

virtual float64_t	compute_measures (CFeatures *features, index_t idx)=0

virtual CFeatures *	remove_feats (CFeatures *features, SGVector< index_t > argsorted)=0

SGVector< index_t >	get_selected_feats ()

virtual EFeatureClass	get_feature_class ()

virtual EFeatureType	get_feature_type ()

virtual EPreprocessorType	get_type () const

void	set_target_dim (index_t target_dim)

index_t	get_target_dim () const

virtual void	set_algorithm (EFeatureSelectionAlgorithm algorithm)=0

EFeatureSelectionAlgorithm	get_algorithm () const

virtual void	set_policy (EFeatureRemovalPolicy policy)=0

EFeatureRemovalPolicy	get_policy () const

void	set_num_remove (index_t num_remove)

index_t	get_num_remove () const

virtual void	set_labels (CLabels *labels)

CLabels *	get_labels () const

virtual void	cleanup ()

virtual const char *	get_name () const

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

template<>
EFeatureType	get_feature_type ()

virtual bool	init (CFeatures *features)=0

virtual CSGObject *	shallow_copy () const

virtual CSGObject *	deep_copy () const

virtual bool	is_generic (EPrimitiveType *generic) const

template<class T >
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

template<>
void	set_generic ()

void	unset_generic ()

virtual void	print_serializable (const char *prefix="")

virtual bool	save_serializable (CSerializableFile file, const char prefix="")

virtual bool	load_serializable (CSerializableFile file, const char prefix="")

void	set_global_io (SGIO *io)

SGIO *	get_global_io ()

void	set_global_parallel (Parallel *parallel)

Parallel *	get_global_parallel ()

void	set_global_version (Version *version)

Version *	get_global_version ()

SGStringList< char >	get_modelsel_names ()

void	print_modsel_params ()

char *	get_modsel_param_descr (const char *param_name)

index_t	get_modsel_param_index (const char *param_name)

void	build_gradient_parameter_dictionary (CMap< TParameter , CSGObject > *dict)

bool	has (const std::string &name) const

template<typename T >
bool	has (const Tag< T > &tag) const

template<typename T , typename U = void>
bool	has (const std::string &name) const

template<typename T >
void	set (const Tag< T > &_tag, const T &value)

template<typename T , typename U = void>
void	set (const std::string &name, const T &value)

template<typename T >
T	get (const Tag< T > &_tag) const

template<typename T , typename U = void>
T	get (const std::string &name) const

virtual void	update_parameter_hash ()

virtual bool	parameter_hash_changed ()

virtual bool	equals (CSGObject *other, float64_t accuracy=0.0, bool tolerant=false)

virtual CSGObject *	clone ()

Public Attributes
SGIO *	io

Parallel *	parallel

Version *	version

Parameter *	m_parameters

Parameter *	m_model_selection_parameters

Parameter *	m_gradient_parameters

uint32_t	m_hash

Protected Member Functions
virtual CFeatures *	apply_backward_elimination (CFeatures *features)

virtual void	precompute ()

virtual void	adapt_params (CFeatures *features)

index_t	get_num_features (CFeatures *features) const

virtual void	load_serializable_pre () throw (ShogunException)

virtual void	load_serializable_post () throw (ShogunException)

virtual void	save_serializable_pre () throw (ShogunException)

virtual void	save_serializable_post () throw (ShogunException)

template<typename T >
void	register_param (Tag< T > &_tag, const T &value)

template<typename T >
void	register_param (const std::string &name, const T &value)

Protected Attributes
index_t	m_target_dim

EFeatureSelectionAlgorithm	m_algorithm

EFeatureRemovalPolicy	m_policy

index_t	m_num_remove

CLabels *	m_labels

CSubsetStack *	m_subset

Constructor & Destructor Documentation

CFeatureSelection ( )

Default constructor

Definition at line 43 of file FeatureSelection.cpp.

~CFeatureSelection ( )

virtual

Destructor

Definition at line 73 of file FeatureSelection.cpp.

Member Function Documentation

void adapt_params ( CFeatures * features )

protectedvirtual

Tunes the parameters which are required to compute the measure based on current features. Overridden in the subclasses. Here it does nothing.

Parameters

features the features based on which parameters are needed to be tuned for computing measures

Definition at line 219 of file FeatureSelection.cpp.

CFeatures * apply ( CFeatures * features )

virtual

Generic interface for applying the feature selection preprocessor. Acts as a wrapper which decides which actual method to call based on the algorithm specified.

Parameters

features the input features

Returns: the result feature object after applying the preprocessor

Implements CPreprocessor.

Definition at line 173 of file FeatureSelection.cpp.

CFeatures * apply_backward_elimination ( CFeatures * features )

protectedvirtual

Applies backward elimination algorithm for performing feature selection. After performing necessary precomputing (defined by subclasses), it iteratively eliminates a number of features based on a measure until target dimension is reached.

Parameters

features the input features

Returns: the result feature object after applying the preprocessor

Definition at line 87 of file FeatureSelection.cpp.

void build_gradient_parameter_dictionary ( CMap< TParameter *, CSGObject * > * dict )

inherited

Builds a dictionary of all parameters in SGObject as well of those of SGObjects that are parameters of this object. Dictionary maps parameters to the objects that own them.

Parameters

dict	dictionary of parameters to be built.

Definition at line 630 of file SGObject.cpp.

void cleanup ( )

virtual

performs cleanup

Implements CPreprocessor.

Definition at line 81 of file FeatureSelection.cpp.

CSGObject * clone ( )

virtualinherited

Creates a clone of the current object. This is done via recursively traversing all parameters, which corresponds to a deep copy. Calling equals on the cloned object always returns true although none of the memory of both objects overlaps.

Returns: an identical copy of the given object, which is disjoint in memory. NULL if the clone fails. Note that the returned object is SG_REF'ed

Definition at line 747 of file SGObject.cpp.

virtual float64_t compute_measures	(	CFeatures *	features,
		index_t	idx
	)

pure virtual

Abstract method that is defined in the subclasses to compute the measures for the provided features based on which feature selection is performed.

Parameters

features	the features on which the measure has to be computed
idx	the index that decides which features should we compute the measure on

Returns: the measure based on which features are selected

Implemented in CDependenceMaximization.

CSGObject * deep_copy ( ) const

virtualinherited

A deep copy. All the instance variables will also be copied.

Definition at line 231 of file SGObject.cpp.

bool equals	(	CSGObject *	other,
		float64_t	accuracy = `0.0`,
		bool	tolerant = `false`
	)

virtualinherited

Recursively compares the current SGObject to another one. Compares all registered numerical parameters, recursion upon complex (SGObject) parameters. Does not compare pointers!

May be overwritten but please do with care! Should not be necessary in most cases.

Parameters

other	object to compare with
accuracy	accuracy to use for comparison (optional)
tolerant	allows linient check on float equality (within accuracy)

Returns: true if all parameters were equal, false if not

Definition at line 651 of file SGObject.cpp.

T get ( const Tag< T > & _tag ) const

inherited

Getter for a class parameter, identified by a Tag. Throws an exception if the class does not have such a parameter.

Parameters

_tag	name and type information of parameter

Returns: value of the parameter identified by the input tag

Definition at line 367 of file SGObject.h.

T get ( const std::string & name ) const

inherited

Getter for a class parameter, identified by a name. Throws an exception if the class does not have such a parameter.

Parameters

name	name of the parameter

Returns: value of the parameter corresponding to the input name and type

Definition at line 388 of file SGObject.h.

EFeatureSelectionAlgorithm get_algorithm ( ) const

Returns: the feature removal algorithm being used

Definition at line 283 of file FeatureSelection.cpp.

EFeatureClass get_feature_class ( )

virtual

Returns: the feature class, shogun::C_ANY

Implements CPreprocessor.

Definition at line 322 of file FeatureSelection.cpp.

virtual EFeatureType get_feature_type ( )

virtual

Returns: feature type

Implements CPreprocessor.