SHOGUN
v3.0.0
|
This page lists ready to run shogun examples for the Octave Modular interface.
To run the examples issue
octave name_of_example.m
or start up octave and then type
name_of_example
% In this example we demonstrate how to use SVMs in a domain adaptation % scenario. Here, we assume that we have two problem domains, one with % an abundance of training data (source domain) and one with only a few % training examples (target domain). These domains are assumed to be % different but related enough to transfer information between them. % Thus, we first train an SVM on the source domain and then subsequently % pass this previously trained SVM object to the DASVM, that we train % on the target domain. The DASVM internally computes a custom linear term % (for the underlying quadratic program of the dual formulation of the SVM) % based on the support vectors of the source SVM and the training examples % of the target SVM. Finally, it can be used for prediction just as any other % SVM object. % addpath('tools'); modshogun; tmp=load_matrix('../data/label_train_dna.dat'); label_train_dna=tmp(1:50); tmp=load_matrix('../data/fm_train_dna.dat'); fm_train_dna=tmp(:,1:50); tmp=load_matrix('../data/label_train_dna.dat'); label_train_dna2=tmp(50:92); tmp=load_matrix('../data/fm_train_dna.dat'); fm_train_dna2=tmp(:, 50:92); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); fm_test_dna2=tmp(:,50:92); %if exist('SVMLight') disp('Domain Adaptation SVM') C = 1.0; degree=3; feats_train=StringCharFeatures(DNA); feats_test=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test.set_features(fm_test_dna); kernel=WeightedDegreeStringKernel(feats_train, feats_train, degree); labels=BinaryLabels(label_train_dna); svm=SVMLight(C, kernel, labels); svm.train(); %##################################### disp('obtaining DA SVM from previously trained SVM') feats_train2=StringCharFeatures(DNA); feats_test2=StringCharFeatures(DNA); feats_train2.set_features(fm_train_dna2); feats_test2.set_features(fm_test_dna2); kernel2=WeightedDegreeStringKernel(feats_train, feats_train, degree); labels2=BinaryLabels(label_train_dna); % we regularize versus the previously obtained solution dasvm = DomainAdaptationSVM(C, kernel2, labels2, svm, 1.0); dasvm.train(); out = dasvm.apply(feats_test2).get_labels(); %else %disp('No support for SVMLight available.') %end
% In this example a multi-class support vector machine is trained on a toy data % set and the trained classifier is then used to predict labels of test % examples. The training algorithm is based on BSVM formulation (L2-soft margin % and the bias added to the objective function) which is solved by the Improved % Mitchell-Demyanov-Malozemov algorithm. The training algorithm uses the Gaussian % kernel of width 2.1 and the regularization constant C=1. The solver stops if the % relative duality gap falls below 1e-5. % % For more details on the used SVM solver see % V.Franc: Optimization Algorithms for Kernel Methods. Research report. % CTU-CMP-2005-22. CTU FEL Prague. 2005. % ftp://cmp.felk.cvut.cz/pub/cmp/articles/franc/Franc-PhD.pdf . % modshogun addpath('tools'); label_train_multiclass=load_matrix('../data/label_train_multiclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % gmnpsvm disp('GMNPSVM') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; num_threads=1; labels=MulticlassLabels(label_train_multiclass); svm=GMNPSVM(C, kernel, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.train(); kernel.init(feats_train, feats_test); svm.apply().get_labels();
% In this example a two-class support vector machine classifier is trained on a % toy data set and the trained classifier is then used to predict labels of test % examples. As training algorithm Gradient Projection Decomposition Technique % (GPDT) is used with SVM regularization parameter C=1 and a Gaussian % kernel of width 2.1. The solver returns an epsilon-precise (epsilon=1e-5) solution. % % For more details on GPDT solver see http://dm.unife.it/gpdt . % modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % gpbtsvm disp('GPBTSVM') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; num_threads=2; labels=BinaryLabels(label_train_twoclass); svm=GPBTSVM(C, kernel, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.train(); kernel.init(feats_train, feats_test); svm.apply().get_labels();
% This example shows usage of a k-nearest neighbor (KNN) classification rule on % a toy data set. The number of the nearest neighbors is set to k=3 and the distances % are measured by the Euclidean metric. Finally, the KNN rule is applied to predict % labels of test examples. modshogun addpath('tools'); label_train_multiclass=load_matrix('../data/label_train_multiclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % knn disp('KNN') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=EuclideanDistance(feats_train, feats_train); k=3; num_threads=1; labels=MulticlassLabels(label_train_multiclass); knn=KNN(k, distance, labels); knn.parallel.set_num_threads(num_threads); knn.train(); output=knn.apply(feats_test).get_labels();
% In this example a two-class linear classifier based on the Linear Discriminant % Analysis (LDA) is trained on a toy data set and then the trained classifier is % used to predict test examples. The regularization parameter, which corresponds % to a weight of a unitary matrix added to the covariance matrix, is set to % gamma=3. % % For more details on the LDA see e.g. % http://en.wikipedia.org/wiki/Linear_discriminant_analysis modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % lda disp('LDA') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); gamma=3; num_threads=1; labels=BinaryLabels(label_train_twoclass); lda=LDA(gamma, feats_train, labels); lda.parallel.set_num_threads(num_threads); lda.train(); lda.get_bias(); lda.get_w(); lda.set_features(feats_test); lda.apply().get_labels();
% In this example a two-class linear support vector machine classifier is trained % on a toy data set and the trained classifier is then used to predict labels of % test examples. As training algorithm the LIBLINEAR solver is used with the SVM % regularization parameter C=0.9 and the bias in the classification rule switched % on and the precision parameters epsilon=1e-5. % % For more details on LIBLINEAR see % http://www.csie.ntu.edu.tw/~cjlin/liblinear/ modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % liblinear disp('LibLinear') realfeat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(realfeat); realfeat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(realfeat); C=1.2; epsilon=1e-5; num_threads=1; labels=BinaryLabels(label_train_twoclass); svm=LibLinear(C, feats_train, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.set_bias_enabled(true); svm.train(); svm.set_features(feats_test); svm.apply().get_labels();
% In this example a two-class support vector machine classifier is trained on a % 2-dimensional randomly generated data set and the trained classifier is used to % predict labels of test examples. As training algorithm the LIBSVM solver is used % with SVM regularization parameter C=1 and a Gaussian kernel of width 2.1. % % For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/ modshogun num=1000; dist=1; width=2.1; C=1; traindata_real=[randn(2,num)-dist, randn(2,num)+dist]; testdata_real=[randn(2,num)-dist, randn(2,num)+dist]; trainlab=[-ones(1,num), ones(1,num)]; testlab=[-ones(1,num), ones(1,num)]; feats_train=RealFeatures(traindata_real); feats_test=RealFeatures(testdata_real); kernel=GaussianKernel(feats_train, feats_train, width); labels=BinaryLabels(trainlab); svm=LibSVM(C, kernel, labels); svm.parallel.set_num_threads(8); svm.train(); kernel.init(feats_train, feats_test); out=svm.apply().get_labels(); testerr=mean(sign(out)~=testlab)
% In this example a two-class support vector machine classifier is trained on a % toy data set and the trained classifier is used to predict labels of test % examples. As training algorithm the LIBSVM solver is used with SVM % regularization parameter C=1 and a Gaussian kernel of width 2.1 and the % precision parameter epsilon=1e-5. The example also shows how to retrieve the % support vectors from the train SVM model. % % For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/ modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % libsvm disp('LibSVM') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; num_threads=2; labels=BinaryLabels(label_train_twoclass); svm=LibSVM(C, kernel, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.train(); kernel.init(feats_train, feats_test); svm.apply().get_labels();
% In this example a one-class support vector machine classifier is trained on a % toy data set. The training algorithm finds a hyperplane in the RKHS which % separates the training data from the origin. The one-class classifier is % typically used to estimate the support of a high-dimesnional distribution. % For more details see e.g. % B. Schoelkopf et al. Estimating the support of a high-dimensional % distribution. Neural Computation, 13, 2001, 1443-1471. % % In the example, the one-class SVM is trained by the LIBSVM solver with the % regularization parameter C=1 and the Gaussian kernel of width 2.1 and the % precision parameter epsilon=1e-5. % % For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/ modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % libsvm twoclass disp('LibSVMOneClass') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; num_threads=4; svm=LibSVMOneClass(C, kernel); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.train(); kernel.init(feats_train, feats_test); svm.apply().get_labels();
% In this example a two-class support vector machine classifier is trained on a % toy data set and the trained classifier is used to predict labels of test % examples. As training algorithm the Minimal Primal Dual SVM is used with SVM % regularization parameter C=1 and a Gaussian kernel of width 1.2 and the % precision parameter 1e-5. % % For more details on the MPD solver see % Kienzle, W. and B. Schölkopf: Training Support Vector Machines with Multiple % Equality Constraints. Machine Learning: ECML 2005, 182-193. (Eds.) Carbonell, % J. G., J. Siekmann, Springer, Berlin, Germany (11 2005) modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % mpdsvm disp('MPDSVM') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; num_threads=1; labels=BinaryLabels(label_train_twoclass); svm=MPDSVM(C, kernel, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.train(); kernel.init(feats_train, feats_test); svm.apply().get_labels();
modshogun addpath('tools'); label_train_multiclass=load_matrix('../data/label_train_multiclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); disp('MulticlassLibSVM') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; num_threads=8; labels=MulticlassLabels(label_train_multiclass); svm=MulticlassLibSVM(C, kernel, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.train(); kernel.init(feats_train, feats_test); svm.apply().get_labels();
modshogun addpath('tools'); label=load_matrix('../data/label_train_dna.dat'); data=load_matrix('../data/fm_train_dna.dat'); fm_test_real=load_matrix('../data/fm_test_dna.dat'); % Newton SVM disp('NewtonSVM') data=double(data); %fm_test_real=double(fm_test_real); realfeat=RealFeatures(data); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(realfeat); %realfeat=RealFeatures(fm_test_real); %feats_test=SparseRealFeatures(); %feats_test.obtain_from_simple(realfeat); C=1.0; epsilon=1e-5; num_threads=1; label=double(label); labels=BinaryLabels(label); svm=NewtonSVM(C, feats_train, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.set_bias_enabled(true); svm.train(); %svm.set_features(feats_test); svm.get_bias(); svm.get_w(); %svm.apply().get_labels();
% This example shows usage of the Perceptron algorithm for training a two-class % linear classifier, i.e. y = sign( <x,w>+b). The Perceptron algorithm works by % iteratively passing though the training examples and applying the update rule on % those examples which are misclassified by the current classifier. The Perceptron % update rule reads % % w(t+1) = w(t) + alpha * y_t * x_t % b(t+1) = b(t) + alpha * y_t % % where (x_t,y_t) is feature vector and label (must be +1/-1) of the misclassified example % (w(t),b(t)) are the current parameters of the linear classifier % (w(t+1),b(t+1)) are the new parameters of the linear classifier % alpha is the learning rate; in this examples alpha=1 % % The Perceptron algorithm iterates until all training examples are correctly % classified or the prescribed maximal number of iterations, in this example % max_iter=1000, is reached. modshogun num=50; label_train_twoclass=[-ones(1,num/2) ones(1,num/2)]; fm_train_real=[randn(5,num/2)-1, randn(5,num/2)+1]; fm_test_real=[randn(5,num)-1, randn(5,num)+1]; % perceptron disp('Perceptron') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); learn_rate=1.; max_iter=1000; num_threads=1; labels=BinaryLabels(label_train_twoclass); perceptron=Perceptron(feats_train, labels); perceptron.set_learn_rate(learn_rate); perceptron.set_max_iter(max_iter); perceptron.parallel.set_num_threads(num_threads); perceptron.train(); perceptron.set_features(feats_test); perceptron.apply().get_labels();
% initialize modular shogun interface modshogun % add path to load matrix script addpath('tools'); % some defines C = 1.2; width = 2.1; epsilon = 1e-5; num_threads = 2; % get train features and labels fm_train_real = load_matrix('../data/fm_train_real.dat'); fm_train_labels = load_matrix('../data/label_train_twoclass.dat'); % get test features and labels %fixme need example w/ test data/labels - using training data instead fm_test_real = load_matrix('../data/fm_train_real.dat'); fm_test_labels = load_matrix('../data/label_train_twoclass.dat'); % create feature and label objects feats_train = RealFeatures(fm_train_real); feats_test = RealFeatures(fm_test_real); labels = BinaryLabels(fm_train_labels); % create kernel kernel = GaussianKernel(feats_train, feats_train, width); % create support vector machine svm = LibSVM(C, kernel, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); % train svm.train(); % save to file file = SerializableAsciiFile('test_svm.dat', 'w'); svm.save_serializable(file); file.close(); % load classifier and verify with test features file_new = SerializableAsciiFile('test_svm.dat', 'r'); svm_new = LibSVM(); svm_new.load_serializable(file_new); file_new.close(); result = svm_new.apply(feats_test).get_labels(); result = sum(sign(result) == fm_test_labels) / columns(fm_test_labels);
% In this example a two-class support vector machine classifier is trained on a % DNA splice-site detection data set and the trained classifier is used to predict % labels on test set. As training algorithm SVM^light is used with SVM % regularization parameter C=1.2 and the Weighted Degree kernel of degree 20 and % the precision parameter epsilon=1e-5. % % For more details on the SVM^light see % T. Joachims. Making large-scale SVM learning practical. In Advances in Kernel % Methods -- Support Vector Learning, pages 169-184. MIT Press, Cambridge, MA USA, 1999. % % For more details on the Weighted Degree kernel see % G. Raetsch, S.Sonnenburg, and B. Schoelkopf. RASE: recognition of alternatively % spliced exons in C. elegans. Bioinformatics, 21:369-377, June 2005. modshogun % Explicit examples on how to use the different classifiers addpath('tools'); label_train_dna=load_matrix('../data/label_train_dna.dat'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % svm light try disp('SVMLight') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); degree=20; kernel=WeightedDegreeStringKernel(feats_train, feats_train, degree); C=1.2; epsilon=1e-5; num_threads=3; labels=BinaryLabels(label_train_dna); svm=SVMLight(C, kernel, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.train(); kernel.init(feats_train, feats_test); svm.apply().get_labels(); catch disp('No support for SVMLight available.') end
% In this example a two-class linear support vector machine classifier (SVM) is % trained on a toy data set and the trained classifier is used to predict labels % of test examples. As training algorithm the SVMLIN solver is used with the SVM % regularization parameter C=0.9 and the bias in the classification rule switched % on and the precision parameter epsilon=1e-5. The example also shows how to % retrieve parameters (vector w and bias b)) of the trained linear classifier. % % For more details on the SVMLIN solver see % V. Sindhwani, S.S. Keerthi. Newton Methods for Fast Solution of Semi-supervised % Linear SVMs. Large Scale Kernel Machines MIT Press (Book Chapter), 2007 modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % svm lin disp('SVMLin') realfeat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(realfeat); realfeat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(realfeat); C=0.9; epsilon=1e-5; num_threads=1; labels=BinaryLabels(label_train_twoclass); svm=SVMLin(C, feats_train, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.set_bias_enabled(true); svm.train(); svm.set_features(feats_test); svm.get_bias(); svm.get_w(); svm.apply().get_labels();
% In this example a two-class linear support vector machine classifier is trained % on a toy data set and the trained classifier is used to predict labels of test % examples. As training algorithm the OCAS solver is used with the SVM % regularization parameter C=0.9 and the bias term in the classification rule % switched off and the precision parameter epsilon=1e-5 (duality gap). % % For more details on the OCAS solver see % V. Franc, S. Sonnenburg. Optimized Cutting Plane Algorithm for Large-Scale Risk % Minimization.The Journal of Machine Learning Research, vol. 10, % pp. 2157--2192. October 2009. % modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % svm ocas disp('SVMOcas') realfeat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(realfeat); realfeat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(realfeat); C=0.9; epsilon=1e-5; num_threads=1; labels=BinaryLabels(label_train_twoclass); svm=SVMOcas(C, feats_train, labels); svm.set_epsilon(epsilon); svm.parallel.set_num_threads(num_threads); svm.set_bias_enabled(false); svm.train(); svm.set_features(feats_test); svm.apply().get_labels();
% In this example a two-class linear support vector machine classifier is trained % on a toy data set and the trained classifier is used to predict labels of test % examples. As training algorithm the Stochastic Gradient Descent (SGD) solver is % used with the SVM regularization parameter C=0.9. The number of iterations, i.e. % passes though all training examples, is set to num_iter=5 . % % For more details on the SGD solver see % L. Bottou, O. Bousquet. The tradeoff of large scale learning. In NIPS 20. MIT % Press. 2008. modshogun addpath('tools'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % sgd disp('SVMSGD') realfeat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(realfeat); realfeat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(realfeat); C=0.9; num_iter=5 num_threads=1; labels=BinaryLabels(label_train_twoclass); svm=SVMSGD(C, feats_train, labels); svm.set_epochs(num_iter) %svm.io.set_loglevel(0); svm.train(); svm.set_features(feats_test); svm.apply().get_labels();
% In this example an agglomerative hierarchical single linkage clustering method % is used to cluster a given toy data set. Starting with each object being % assigned to its own cluster clusters are iteratively merged. Here the clusters % are merged that have the closest (minimum distance, here set via the Euclidean % distance object) two elements. modshogun % Explicit examples on how to use clustering addpath('tools'); fm_train=load_matrix('../data/fm_train_real.dat'); % Hierarchical disp('Hierarchical') merges=4; feats_train=RealFeatures(fm_train); feats_test=RealFeatures(fm_train); distance=EuclideanDistance(feats_train, feats_train); hierarchical=Hierarchical(merges, distance); hierarchical.train(); distance.init(feats_train, feats_test); mdist=hierarchical.get_merge_distances(); pairs=hierarchical.get_cluster_pairs();
% In this example the k-means clustering method is used to cluster a given toy % data set. In k-means clustering one tries to partition n observations into k % clusters in which each observation belongs to the cluster with the nearest mean. % The algorithm class constructor takes the number of clusters and a distance to % be used as input. The distance used in this example is Euclidean distance. % After training one can fetch the result of clustering by obtaining the cluster % centers and their radiuses. modshogun % Explicit examples on how to use clustering addpath('tools'); fm_train=load_matrix('../data/fm_train_real.dat'); % KMeans disp('KMeans') k=4; feats_train=RealFeatures(fm_train); distance=EuclideanDistance(feats_train, feats_train); kmeans=KMeans(k, distance); kmeans.train(); c=kmeans.get_cluster_centers(); r=kmeans.get_radiuses();
% Blind Source Separation using the Jade Algorithm with Shogun % % Based on the example from scikit-learn % http://scikit-learn.org/ % % Kevin Hughes 2013 modshogun % Generate sample data n_samples = 2000; time = linspace(0,10,n_samples); % Source Signals S = zeros(2, length(time)); S(1,:) = sin(2*time); S(2,:) = sign(sin(3*time)); S += 0.2*rand(size(S)); % Standardize data S = S ./ (std(S,0,2) * ones(1,n_samples)); % Mixing Matrix A = [1 0.5; 0.5 1] % Mix Signals X = A*S; mixed_signals = RealFeatures(X); % Separating jade = Jade(); signals = jade.apply(mixed_signals); signals = RealFeatures.obtain_from_generic(signals); S_ = signals.get_feature_matrix(); A_ = jade.get_mixing_matrix() % Plot figure(); subplot(311); plot(time, S(1,:), 'b'); hold on; plot(time, S(2,:), 'g'); set(gca, 'xtick', []) title("True Sources"); subplot(312); plot(time, X(1,:), 'b'); hold on; plot(time, X(2,:), 'g'); set(gca, 'xtick', []) title("Mixed Sources"); subplot(313); plot(time, S_(1,:), 'b'); hold on; plot(time, S_(2,:), 'g'); title("Estimated Sources");
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance matrix is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % matrix between these two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CBrayCurtisDistance.html. % % Obviously, using the Bray Curtis distance is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % bray curtis distance disp('BrayCurtisDistance') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=BrayCurtisDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance (dissimilarity ratio) matrix is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % matrix between these two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CCanberraMetric.html. % % Obviously, using the Canberra distance is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % canberra metric disp('CanberaMetric') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=CanberraMetric(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% This example shows how to compute the Canberra Word Distance. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % canberra word distance disp('CanberraWordDistance') order=3; gap=0; reverse=false; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); distance=CanberraWordDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance (maximum of absolute feature dimension differences) matrix is % computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % (maximum of absolute feature dimension differences) matrix between these % two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CChebyshewMetric.html. % % Obviously, using the Chebyshew distance is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % chebyshew metric disp('ChebyshewMetric') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=ChebyshewMetric(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance matrix is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % matrix between these two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CChiSquareDistance.html. % % Obviously, using the ChiSquare distance is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % chi square distance disp('ChiSquareDistance') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=ChiSquareDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance matrix is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % matrix between these two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CCosineDistance.html. % % Obviously, using the Cosine distance is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % cosine distance disp('Cosine Distance') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=CosineDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance matrix is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % matrix between these two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CEuclidianDistance.html. % % Obviously, using the Euclidian distance is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % euclidean distance disp('EuclideanDistance') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=EuclideanDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a % pairwise distance (shortest path on a sphere) matrix is computed % by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % (shortest path on a sphere) matrix between these two data sets is % computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CGeodesicMetric.html. % % Obviously, using the Geodesic distance is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % geodesic metric disp('GeodesicMetric') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=GeodesicMetric(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% This example shows how to compute the Hamming Word Distance for string features. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % hamming word distance disp('HammingWordDistance') order=3; gap=0; reverse=false; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); use_sign=false; distance=HammingWordDistance(feats_train, feats_train, use_sign); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance (divergence measure based on the Kullback-Leibler divergence) matrix % is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % (divergence measure based on the Kullback-Leibler divergence) matrix between % these two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CJensenMetric.html. % % Obviously, using the Jensen-Shannon distance/divergence is not limited to % this showcase example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % jensen metric disp('JensenMetric') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=JensenMetric(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% This example shows how to compute the Manhatten Distance. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % manhattan metric disp('ManhattanMetric') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=ManhattanMetric(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% This example shows how to compute the Manahattan Distance for string features. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % manhattan word distance disp('ManhattanWordDistance') order=3; gap=0; reverse=false; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); distance=ManhattanWordDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) and norm 'k' controls the processing of the given data points, % where a pairwise distance matrix is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % matrix between these two data sets is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CMinkowskiMetric.html. % % Obviously, using the Minkowski metric is not limited to this showcase % example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % minkowski metric disp('MinkowskiMetric') k=3; feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=MinkowskiMetric(feats_train, feats_train, k); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% This example shows how to compute the Eucledian Distance using a sparse encoding. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % sparse euclidean distance disp('SparseEuclideanDistance') realfeat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(realfeat); realfeat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(realfeat); distance=SparseEuclideanDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% An approach as applied below, which shows the processing of input data % from a file becomes a crucial factor for writing your own sample applications. % This approach is just one example of what can be done using the distance % functions provided by shogun. % % First, you need to determine what type your data will be, because this % will determine the distance function you can use. % % This example loads two stored matrices of real values from different % files and initializes the matrices to 'RealFeatures'. % Each column of the matrices corresponds to one data point. % % The distance initialized by two data sets (the same data set as shown in the % first call) controls the processing of the given data points, where a pairwise % distance (extended Jaccard coefficient) matrix is computed by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % The method call 'init'* binds the given data sets, where a pairwise distance % (extended Jaccard coefficient) matrix between these two data sets is computed % by 'get_distance_matrix'. % % The resulting distance matrix can be reaccessed by 'get_distance_matrix'. % % *Note that the previous computed distance matrix can no longer be % reaccessed by 'get_distance_matrix'. % % For more details see doc/classshogun_1_1CTanimotoDistance.html. % % Obviously, using the Tanimoto distance/coefficient is not limited to % this showcase example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % tanimoto distance disp('TanimotoDistance') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); distance=TanimotoDistance(feats_train, feats_train); dm_train=distance.get_distance_matrix(); distance.init(feats_train, feats_test); dm_test=distance.get_distance_matrix();
% In this example the Histogram algorithm object computes a histogram over all % 16bit unsigned integers in the features. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); % Histogram disp('Histogram') order=3; gap=0; reverse=false; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats=StringWordFeatures(charfeat.get_alphabet()); feats.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats); feats.add_preprocessor(preproc); feats.apply_preprocessor(); histo=Histogram(feats); histo.train(); histo.get_histogram(); num_examples=feats.get_num_vectors(); num_param=histo.get_num_model_parameters(); % for i=0:(num_examples-1), % for j=0:(num_param-1), % histo.get_log_derivative(j, i); % end % end histo.get_log_likelihood(); histo.get_log_likelihood_sample();
% In this example a hidden markov model with 3 states and 6 transitions is trained % on a string data set. After calling the constructor of the HMM class specifying % the number of states and transitions the model is trained. Via the Baum-Welch % algorithm the optimal transition and emission probabilities are estimated. The % best path, i.e. the path with highest probability given the model can then be % calculated using get_best_path_state. modshogun addpath('tools'); leng=50; rep=5; weight=0.3; % generate a sequence with characters 1-6 drawn from 3 loaded cubes for i = 1:3, a{i}= [ ones(1,ceil(leng*rand)) 2*ones(1,ceil(leng*rand)) 3*ones(1,ceil(leng*rand)) 4*ones(1,ceil(leng*rand)) 5*ones(1,ceil(leng*rand)) 6*ones(1,ceil(leng*rand)) ]; a{i}= a{i}(randperm(length(a{i}))); end s=[]; for i = 1:size(a,2), s= [ s i*ones(1,ceil(rep*rand)) ]; end s=s(randperm(length(s))); cubesequence={''}; for i = 1:length(s), f(i)=ceil(((1-weight)*rand+weight)*length(a{s(i)})); t=randperm(length(a{s(i)})); r=a{s(i)}(t(1:f(i))); cubesequence{1}=[cubesequence{1} char(r+'0')]; end % HMM disp('HMM') N=3; M=6; pseudo=1e-1; order=1; gap=0; reverse=false; num_examples=2; charfeat=StringCharFeatures(CUBE); charfeat.set_features(cubesequence); feats=StringWordFeatures(charfeat.get_alphabet()); feats.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats); feats.add_preprocessor(preproc); feats.apply_preprocessor(); hmm=HMM(feats, N, M, pseudo); hmm.train(); % cheating, but enum BaumWelchViterbiType does not seem available BW_NORMAL=0; hmm.baum_welch_viterbi_train(BW_NORMAL); num_examples=feats.get_num_vectors(); num_param=hmm.get_num_model_parameters(); for i=0:(num_examples-1), for j=0:(num_param-1), hmm.get_log_derivative(j, i); end end best_path=0; best_path_state=0; for i=0:(num_examples-1), best_path = best_path + hmm.best_path(i); for j=0:(N-1), best_path_state = best_path_state + hmm.get_best_path_state(i, j); end end hmm.get_log_likelihood(); hmm.get_log_likelihood_sample();
% Trains an inhomogeneous Markov chain of order 3 on a DNA string data set. Due to % the structure of the Markov chain it is very similar to a HMM with just one % chain of connected hidden states - that is why we termed this linear HMM. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); leng=50; rep=5; weight=0.3; % generate a sequence with characters 1-6 drawn from 3 loaded cubes for i = 1:3, a{i}= [ ones(1,ceil(leng*rand)) 2*ones(1,ceil(leng*rand)) 3*ones(1,ceil(leng*rand)) 4*ones(1,ceil(leng*rand)) 5*ones(1,ceil(leng*rand)) 6*ones(1,ceil(leng*rand)) ]; a{i}= a{i}(randperm(length(a{i}))); end s=[]; for i = 1:size(a,2), s= [ s i*ones(1,ceil(rep*rand)) ]; end s=s(randperm(length(s))); cubesequence={''}; for i = 1:length(s), f(i)=ceil(((1-weight)*rand+weight)*length(a{s(i)})); t=randperm(length(a{s(i)})); r=a{s(i)}(t(1:f(i))); cubesequence{1}=[cubesequence{1} char(r+'0')]; end % Linear HMM disp('LinearHMM') order=3; gap=0; reverse=false; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats=StringWordFeatures(charfeat.get_alphabet()); feats.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats); feats.add_preprocessor(preproc); feats.apply_preprocessor(); hmm=LinearHMM(feats); hmm.train(); hmm.get_transition_probs(); num_examples=feats.get_num_vectors(); num_param=hmm.get_num_model_parameters(); for i=0:(num_examples-1), for j=0:(num_param-1), hmm.get_log_derivative(j, i); end end hmm.get_log_likelihood(); hmm.get_log_likelihood_sample();
% This example demsonstrates how to encode sparse (most entries zero), % real-valued features in shogun using SparseRealFeatures. modshogun rand('state',0); disp('Sparse Features') A=rand(3,5); A(A<0.7)=0; full(A) % sparse representation X of dense matrix A X = sparse(A) % create sparse shogun features from dense matrix A a=SparseRealFeatures(A) a_out = a.get_full_feature_matrix() % create sparse shogun features from sparse matrix X a.set_sparse_feature_matrix(X) a_out=a.get_full_feature_matrix() % create sparse shogun features from sparse matrix X a=SparseRealFeatures(X) a_out=a.get_full_feature_matrix() z=a.get_sparse_feature_matrix()
% This example demonstrates the use of the AUC Kernel, which % can be used to maximize AUC instead of margin in SVMs. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); % auc disp('AUC') feats_train=RealFeatures(fm_train_real); width=1.7; subkernel=GaussianKernel(feats_train, feats_train, width); kernel=AUCKernel(0, subkernel); kernel.setup_auc_maximization( BinaryLabels(label_train_twoclass) ); km_train=kernel.get_kernel_matrix();
% This is an example for the initialization of the chi2-kernel on real data, where % each column of the matrices corresponds to one training/test example. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % chi2 disp('Chi2') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=1.4; size_cache=10; kernel=Chi2Kernel(feats_train, feats_train, width, size_cache); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of a combined kernel, which is a weighted sum of % in this case three kernels on real valued data. The sub-kernel weights are all set to 1. % modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % combined disp('Combined') kernel=CombinedKernel(); feats_train=CombinedFeatures(); feats_test=CombinedFeatures(); subkfeats_train=RealFeatures(fm_train_real); subkfeats_test=RealFeatures(fm_test_real); subkernel=GaussianKernel(10, 1.2); feats_train.append_feature_obj(subkfeats_train); feats_test.append_feature_obj(subkfeats_test); kernel.append_kernel(subkernel); subkfeats_train=StringCharFeatures(DNA); subkfeats_train.set_features(fm_train_dna); subkfeats_test=StringCharFeatures(DNA); subkfeats_test.set_features(fm_test_dna); degree=3; subkernel=FixedDegreeStringKernel(10, degree); feats_train.append_feature_obj(subkfeats_train); feats_test.append_feature_obj(subkfeats_test); kernel.append_kernel(subkernel); subkfeats_train=StringCharFeatures(DNA); subkfeats_train.set_features(fm_train_dna); subkfeats_test=StringCharFeatures(DNA); subkfeats_test.set_features(fm_test_dna); subkernel=LocalAlignmentStringKernel(10); feats_train.append_feature_obj(subkfeats_train); feats_test.append_feature_obj(subkfeats_test); kernel.append_kernel(subkernel); kernel.init(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of the CommUlongString-kernel. This kernel % sums over k-mere matches (k='order'). For efficient computing a preprocessor is used % that extracts and sorts all k-mers. If 'use_sign' is set to one each k-mere is counted % only once. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % comm_ulong_string disp('CommUlongString') order=3; gap=0; reverse=false; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringUlongFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortUlongString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringUlongFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); use_sign=false; kernel=CommUlongStringKernel(feats_train, feats_train, use_sign); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of the CommWordString-kernel (aka % Spectrum or n-gram kernel; its name is derived from the unix command comm). This kernel % sums over k-mere matches (k='order'). For efficient computing a preprocessor is used % that extracts and sorts all k-mers. If 'use_sign' is set to one each k-mere is counted % only once. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); order=4; gap=0; reverse=false; use_sign=true; % comm_word_string disp('CommWordString') charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); use_sign=false; kernel=CommWordStringKernel(feats_train, feats_train, use_sign); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The constant kernel gives a trivial kernel matrix with all entries set to the same value % defined by the argument 'c'. % modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % const disp('Const') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); c=23.; kernel=ConstKernel(feats_train, feats_train, c); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% A user defined custom kernel is assigned in this example, for which only the lower triangle % may be given (set_triangle_kernel_matrix_from_triangle) or % a full matrix (set_full_kernel_matrix_from_full), or a full matrix which is then internally stored as a % triangle (set_triangle_kernel_matrix_from_full). Labels for the examples are given, a svm is trained and % the svm is used to classify the examples. % modshogun; C=1; dim=7; lab=sign(2*rand(1,dim) - 1); data=rand(dim, dim); symdata=data+data'; % custom disp('Custom') dim=7 data=rand(dim, dim); symdata=data+data'; %lowertriangle=array([symdata[(x,y)] for x in xrange(symdata.shape[1]); % for y in xrange(symdata.shape[0]) if y<=x]); % kernel=CustomKernel(); %kernel.set_triangle_kernel_matrix_from_triangle(lowertriangle); %km_triangletriangle=kernel.get_kernel_matrix(); % kernel.set_triangle_kernel_matrix_from_full(symdata); km_fulltriangle=kernel.get_kernel_matrix(); % kernel.set_full_kernel_matrix_from_full(data); km_fullfull=kernel.get_kernel_matrix();
% This is an example for the initialization of the diag-kernel. % The diag kernel has all kernel matrix entries but those on % the main diagonal set to zero. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % diag disp('Diag') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); diag=23.; kernel=DiagKernel(feats_train, feats_train, diag); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% With the distance kernel one can use any of the following distance metrics: % BrayCurtisDistance() % CanberraMetric() % CanberraWordDistance() % ChebyshewMetric() % ChiSquareDistance() % CosineDistance() % Distance() % EuclidianDistance() % GeodesicMetric() % HammingWordDistance() % JensenMetric() % ManhattanMetric() % ManhattanWordDistance() % MinkowskiMetric() % RealDistance() % SimpleDistance() % SparseDistance() % SparseEuclidianDistance() % StringDistance() % TanimotoDistance() % modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); label_train_twoclass=load_matrix('../data/label_train_twoclass.dat'); % distance disp('Distance') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=1.7; distance=EuclideanDistance(); kernel=DistanceKernel(feats_train, feats_test, width, distance); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The FixedDegree String kernel takes as input two strings of same size and counts the number of matches of length d. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % fixed_degree_string disp('FixedDegreeString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); degree=3; kernel=FixedDegreeStringKernel(feats_train, feats_train, degree); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The well known Gaussian kernel (swiss army knife for SVMs) on dense real valued features. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % gaussian disp('Gaussian') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=1.9; kernel=GaussianKernel(feats_train, feats_train, width); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% An experimental kernel inspired by the WeightedDegreePositionStringKernel and the Gaussian kernel. % The idea is to shift the dimensions of the input vectors against eachother. 'shift_step' is the step % size of the shifts and max_shift is the maximal shift. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % gaussian_shift disp('GaussianShift') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); width=1.8; max_shift=2; shift_step=1; kernel=GaussianShiftKernel( feats_train, feats_train, width, max_shift, shift_step); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The HistogramWordString computes the TOP kernel on inhomogeneous Markov Chains. modshogun addpath('tools'); label_train_dna=load_matrix('../data/label_train_dna.dat'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % plugin_estimate disp('PluginEstimate w/ HistogramWord') order=3; gap=0; reverse=false; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); pie=PluginEstimate(); labels=BinaryLabels(label_train_dna); pie.set_labels(labels); pie.set_features(feats_train); pie.train(); kernel=HistogramWordStringKernel(feats_train, feats_train, pie); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); pie.set_features(feats_test); pie.apply().get_labels(); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of a linear kernel on raw byte % data. modshogun addpath('tools'); fm_train_byte=uint8(load_matrix('../data/fm_train_byte.dat')); fm_test_byte=uint8(load_matrix('../data/fm_test_byte.dat')); % linear byte disp('LinearByte') feats_train=ByteFeatures(RAWBYTE); feats_train.set_feature_matrix(fm_train_byte); feats_test=ByteFeatures(RAWBYTE); feats_test.set_feature_matrix(fm_test_byte); kernel=LinearKernel(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of a linear kernel on real valued % data using scaling factor 1.2. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % linear disp('Linear') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); scale=1.2; kernel=LinearKernel(); kernel.set_normalizer(AvgDiagKernelNormalizer(scale)); kernel.init(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of a linear kernel on string data. The % strings are all of the same length and consist of the characters 'ACGT' corresponding % to the DNA-alphabet. Each column of the matrices of type char corresponds to % one training/test example. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % linear_string disp('LinearString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); kernel=LinearStringKernel(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of a linear kernel on word (2byte) % data. modshogun addpath('tools'); fm_train_word=uint16(load_matrix('../data/fm_train_word.dat')); fm_test_word=uint16(load_matrix('../data/fm_test_word.dat')); % linear_word disp('LinearWord') feats_train=WordFeatures(fm_train_word); feats_test=WordFeatures(fm_test_word); do_rescale=true; scale=1.4; kernel=LinearKernel(); kernel.set_normalizer(AvgDiagKernelNormalizer(scale)); kernel.init(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of the local alignment kernel on % DNA sequences, where each column of the matrices of type char corresponds to % one training/test example. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % local_alignment_strin disp('LocalAlignmentString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); kernel=LocalAlignmentStringKernel(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This example initializes the locality improved string kernel. The locality improved string % kernel is defined on sequences of the same length and inspects letters matching at % corresponding positions in both sequences. The kernel sums over all matches in windows of % length l and takes this sum to the power of 'inner_degree'. The sum over all these % terms along the sequence is taken to the power of 'outer_degree'. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % locality_improved_string disp('LocalityImprovedString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); l=5; inner_degree=5; outer_degree=7; kernel=LocalityImprovedStringKernel( feats_train, feats_train, l, inner_degree, outer_degree); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The class MatchWordStringKernel computes a variant of the polynomial % kernel on strings of same length converted to a word alphabet. % % It is computed as % % k(x,x')= \sum_{i=0}^L I(x_i=x'_i)+c)^d % % where I is the indicator function which evaluates to 1 if its argument is % true and to 0 otherwise. % % Note that additional normalisation is applied, i.e. % k'(x, x')=\frac{k(x, x')}{\sqrt{k(x, x)k( x', x')}} % modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % match_word_string disp('MatchWordString') order=4; gap=0; reverse=false; degree=3; scale=1.4; size_cache=10; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); kernel=MatchWordStringKernel(size_cache, degree); kernel.set_normalizer(AvgDiagKernelNormalizer(scale)); kernel.init(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example initializing the oligo string kernel which takes distances % between matching oligos (k-mers) into account via a gaussian. Variable 'k' defines the length % of the oligo and variable 'w' the width of the gaussian. The oligo string kernel is % implemented for the DNA-alphabet 'ACGT'. % modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % oligo_string disp('OligoString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); k=3; width=1.2; size_cache=10; kernel=OligoStringKernel(size_cache, k, width); kernel.init(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of the PolyMatchString kernel on string data. % The PolyMatchString kernel sums over the matches of two stings of the same length and % takes the sum to the power of 'degree'. The strings consist of the characters 'ACGT' corresponding % to the DNA-alphabet. Each column of the matrices of type char corresponds to % one training/test example. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); order=3; gap=0; reverse=false; % poly_match_word_string disp('PolyMatchWordString') degree=2; inhomogene=true; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); kernel=PolyMatchWordStringKernel(feats_train, feats_train, degree, inhomogene); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This example initializes the polynomial kernel with real data. % If variable 'inhomogene' is 'True' +1 is added to the scalar product % before taking it to the power of 'degree'. If 'use_normalization' is % set to 'true' then kernel matrix will be normalized by the square roots % of the diagonal entries. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % poly disp('Poly') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); degree=4; inhomogene=false; use_normalization=true; kernel=PolyKernel( feats_train, feats_train, degree, inhomogene, use_normalization); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% This is an example for the initialization of the PolyMatchString kernel on string data. % The PolyMatchString kernel sums over the matches of two stings of the same length and % takes the sum to the power of 'degree'. The strings consist of the characters 'ACGT' corresponding % to the DNA-alphabet. Each column of the matrices of type char corresponds to % one training/test example. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % poly_match_string disp('PolyMatchString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); degree=3; inhomogene=false; kernel=PolyMatchStringKernel(feats_train, feats_train, degree, inhomogene); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The standard Sigmoid kernel computed on dense real valued features. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % sigmoid disp('Sigmoid') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); size_cache=10; gamma=1.2; coef0=1.3; kernel=SigmoidKernel(feats_train, feats_train, size_cache, gamma, coef0); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% SimpleLocalityImprovedString kernel, is a `simplified' and better performing version of the Locality improved kernel. modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % simple_locality_improved_string disp('SimpleLocalityImprovedString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); l=5; inner_degree=5; outer_degree=7; kernel=SimpleLocalityImprovedStringKernel( feats_train, feats_train, l, inner_degree, outer_degree); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The well known Gaussian kernel (swiss army knife for SVMs) on sparse real valued features. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % sparse_gaussian - b0rked? disp('SparseGaussian') feat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(feat); feat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(feat); width=1.1; kernel=GaussianKernel(feats_train, feats_train, width); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% Computes the standard linear kernel on sparse real valued features. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % sparse_linear disp('SparseLinear') feat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(feat); feat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(feat); scale=1.1; kernel=LinearKernel(); kernel.set_normalizer(AvgDiagKernelNormalizer(scale)); kernel.init(feats_train, feats_train); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% Computes the standard polynomial kernel on sparse real valued features. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); % sparse_poly disp('SparsePoly') feat=RealFeatures(fm_train_real); feats_train=SparseRealFeatures(); feats_train.obtain_from_simple(feat); feat=RealFeatures(fm_test_real); feats_test=SparseRealFeatures(); feats_test.obtain_from_simple(feat); size_cache=10; degree=3; inhomogene=true; kernel=PolyKernel(feats_train, feats_train, degree, inhomogene, size_cache); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The class TOPFeatures implements TOP kernel features obtained from % two Hidden Markov models. % % It was used in % % K. Tsuda, M. Kawanabe, G. Raetsch, S. Sonnenburg, and K.R. Mueller. A new % discriminative kernel from probabilistic models. Neural Computation, % 14:2397-2414, 2002. % % which also has the details. % % Note that TOP-features are computed on the fly, so to be effective feature % caching should be enabled. % % It inherits its functionality from CSimpleFeatures, which should be % consulted for further reference. % modshogun addpath('tools'); leng=28; rep=5; weight=0.3; % generate a sequence with characters 1-6 drawn from 3 loaded cubes for i = 1:3, a{i}= [ ones(1,ceil(leng*rand)) 2*ones(1,ceil(leng*rand)) 3*ones(1,ceil(leng*rand)) 4*ones(1,ceil(leng*rand)) 5*ones(1,ceil(leng*rand)) 6*ones(1,ceil(leng*rand)) ]; a{i}= a{i}(randperm(length(a{i}))); end s=[]; for i = 1:size(a,2), s= [ s i*ones(1,ceil(rep*rand)) ]; end s=s(randperm(length(s))); cubesequence={''}; for i = 1:length(s), f(i)=ceil(((1-weight)*rand+weight)*length(a{s(i)})); t=randperm(length(a{s(i)})); r=a{s(i)}(t(1:f(i))); cubesequence{1}=[cubesequence{1} char(r+'0')]; end % top_fisher disp('TOP/Fisher on PolyKernel') N=3; M=6; pseudo=1e-1; order=1; gap=0; reverse=false; charfeat=StringCharFeatures(CUBE); charfeat.set_features(cubesequence); wordfeats_train=StringWordFeatures(charfeat.get_alphabet()); wordfeats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(wordfeats_train); wordfeats_train.add_preprocessor(preproc); wordfeats_train.apply_preprocessor(); charfeat=StringCharFeatures(CUBE); charfeat.set_features(cubesequence); wordfeats_test=StringWordFeatures(charfeat.get_alphabet()); wordfeats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); wordfeats_test.add_preprocessor(preproc); wordfeats_test.apply_preprocessor(); % cheating, BW_NORMAL is somehow not available BW_NORMAL=0; pos=HMM(wordfeats_train, N, M, pseudo); pos.train(); pos.baum_welch_viterbi_train(BW_NORMAL); neg=HMM(wordfeats_train, N, M, pseudo); neg.train(); neg.baum_welch_viterbi_train(BW_NORMAL); pos_clone=HMM(pos); neg_clone=HMM(neg); pos_clone.set_observations(wordfeats_test); neg_clone.set_observations(wordfeats_test); feats_train=TOPFeatures(10, pos, neg, false, false); feats_test=TOPFeatures(10, pos_clone, neg_clone, false, false); kernel=PolyKernel(feats_train, feats_train, 1, false, true); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix(); feats_train=FKFeatures(10, pos, neg); feats_train.set_opt_a(-1); %estimate prior feats_test=FKFeatures(10, pos_clone, neg_clone); feats_test.set_a(feats_train.get_a()); %use prior from training data kernel=PolyKernel(feats_train, feats_train, 1, false, true); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The WeightedCommWordString kernel may be used to compute the weighted % spectrum kernel (i.e. a spectrum kernel for 1 to K-mers, where each k-mer % length is weighted by some coefficient \f$\beta_k\f$) from strings that have % been mapped into unsigned 16bit integers. % % These 16bit integers correspond to k-mers. To applicable in this kernel they % need to be sorted (e.g. via the SortWordString pre-processor). % % It basically uses the algorithm in the unix "comm" command (hence the name) % to compute: % % k({\bf x},({\bf x'})= \sum_{k=1}^K\beta_k\Phi_k({\bf x})\cdot \Phi_k({\bf x'}) % % where \f$\Phi_k\f$ maps a sequence \f${\bf x}\f$ that consists of letters in % \f$\Sigma\f$ to a feature vector of size \f$|\Sigma|^k\f$. In this feature % vector each entry denotes how often the k-mer appears in that \f${\bf x}\f$. % % Note that this representation is especially tuned to small alphabets % (like the 2-bit alphabet DNA), for which it enables spectrum kernels % of order 8. % % For this kernel the linadd speedups are quite efficiently implemented using % direct maps. % modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % weighted_comm_word_string disp('WeightedCommWordString') order=3; gap=0; reverse=true; charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_train_dna); feats_train=StringWordFeatures(charfeat.get_alphabet()); feats_train.obtain_from_char(charfeat, order-1, order, gap, reverse); preproc=SortWordString(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); charfeat=StringCharFeatures(DNA); charfeat.set_features(fm_test_dna); feats_test=StringWordFeatures(charfeat.get_alphabet()); feats_test.obtain_from_char(charfeat, order-1, order, gap, reverse); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); use_sign=false; kernel=WeightedCommWordStringKernel(feats_train, feats_train, use_sign); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The Weighted Degree Position String kernel (Weighted Degree kernel with shifts). % % The WD-shift kernel of order d compares two sequences X and % Y of length L by summing all contributions of k-mer matches of % lengths k in 1...d, weighted by coefficients beta_k % allowing for a positional tolerance of up to shift s. % modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % weighted_degree_position_string disp('WeightedDegreePositionString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); degree=20; kernel=WeightedDegreePositionStringKernel(feats_train, feats_train, degree); %kernel.set_shifts(zeros(len(fm_train_dna[0]), dtype=int)); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% The Weighted Degree String kernel. % % The WD kernel of order d compares two sequences X and % Y of length L by summing all contributions of k-mer matches of % lengths k in 1...d , weighted by coefficients beta_k. It % is defined as % % k(X, Y)=\sum_{k=1}^d\beta_k\sum_{l=1}^{L-k+1}I(u_{k,l}(X)=u_{k,l}(Y)). % % Here, $u_{k,l}(X)$ is the string of length k starting at position % l of the sequence X and I(.) is the indicator function % which evaluates to 1 when its argument is true and to 0 % otherwise. % modshogun addpath('tools'); fm_train_dna=load_matrix('../data/fm_train_dna.dat'); fm_test_dna=load_matrix('../data/fm_test_dna.dat'); % weighted_degree_string disp('WeightedDegreeString') feats_train=StringCharFeatures(DNA); feats_train.set_features(fm_train_dna); feats_test=StringCharFeatures(DNA); feats_test.set_features(fm_test_dna); degree=20; kernel=WeightedDegreeStringKernel(feats_train, feats_train, degree); %weights=arange(1,degree+1,dtype=double)[::-1]/ \ % sum(arange(1,degree+1,dtype=double)); %kernel.set_wd_weights(weights); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
modshogun; label_train_fname='../data/label_train_multiclass.dat'; train_fname='../data/fm_train_real.dat'; test_fname='../data/fm_test_real.dat'; % wrap data into Shogun objects feats_train=RealFeatures(CSVFile(train_fname)); feats_test=RealFeatures(CSVFile(test_fname)); labels_train=MulticlassLabels(CSVFile(label_train_fname)); % distance learning with LMNN disp('LMNN') % number of target neighbours per example k=3; lmnn=LMNN(feats_train,labels_train,k); init_transform=eye(feats_train.get_num_features()); lmnn.train(init_transform); % retrieve distance learnt by LMNN lmnn_distance=lmnn.get_distance(); % perform multiclass classification using KNN with the distance learnt knn=KNN(k,lmnn_distance,labels_train); knn.train(); output=knn.apply(feats_test).get_labels();
% In this example we show how to perform Multiple Kernel Learning (MKL) % with the modular interface for multi-class classification. % First, we create a number of base kernels and features. % These kernels can capture different views of the same features, or actually % consider entirely different features associated with the same example % (e.g. DNA sequences = strings AND gene expression data = real values of the same tissue sample). % The base kernels are then subsequently added to a CombinedKernel, which % contains a weight for each kernel and encapsulates the base kernels % from the training procedure. When the CombinedKernel between two examples is % evaluated it computes the corresponding linear combination of kernels according to their weights. % We then show how to create an MKLMultiClass classifier that trains an SVM and learns the optimal % weighting of kernels (w.r.t. a given norm q) at the same time. The main difference to the binary % classification version of MKL is that we can use more than two values as labels, when training % the classifier. % Finally, the example shows how to classify with a trained MKLMultiClass classifier. % modshogun addpath('tools'); label_train_multiclass=load_matrix('../data/label_train_multiclass.dat'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); disp('Octave_modular') % combined disp('Combined') kernel=CombinedKernel(); feats_train=CombinedFeatures(); feats_test=CombinedFeatures(); subkfeats_train=RealFeatures(fm_train_real); subkfeats_test=RealFeatures(fm_test_real); subkernel=GaussianKernel(10, 1.2); feats_train.append_feature_obj(subkfeats_train); feats_test.append_feature_obj(subkfeats_test); kernel.append_kernel(subkernel); subkfeats_train=RealFeatures(fm_train_real); subkfeats_test=RealFeatures(fm_test_real); subkernel=LinearKernel(); feats_train.append_feature_obj(subkfeats_train); feats_test.append_feature_obj(subkfeats_test); kernel.append_kernel(subkernel); subkfeats_train=RealFeatures(fm_train_real); subkfeats_test=RealFeatures(fm_test_real); subkernel=PolyKernel(10,2); feats_train.append_feature_obj(subkfeats_train); feats_test.append_feature_obj(subkfeats_test); kernel.append_kernel(subkernel); kernel.init(feats_train, feats_train); C=1.2; epsilon=1e-5; num_threads=1; labels=MulticlassLabels(label_train_multiclass); % MKL_MULTICLASS disp('MKL_MULTICLASS') mkl=MKLMulticlass(C, kernel, labels); mkl.set_epsilon(epsilon); mkl.parallel.set_num_threads(num_threads); mkl.set_mkl_epsilon(0.001); mkl.set_mkl_norm(1.5); mkl.train(); kernel.init(feats_train, feats_test); result=mkl.apply().get_labels(); result
% In this example a kernel matrix is computed for a given real-valued data set. % The kernel used is the Chi2 kernel which operates on real-valued vectors. It % computes the chi-squared distance between sets of histograms. It is a very % useful distance in image recognition (used to detect objects). The preprocessor % LogPlusOne adds one to a dense real-valued vector and takes the logarithm of % each component of it. It is most useful in situations where the inputs are % counts: When one compares differences of small counts any difference may matter % a lot, while small differences in large counts don't. This is what this log % transformation controls for. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); %LogPlusOne disp('LogPlusOne') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); preproc=LogPlusOne(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); width=1.4; size_cache=10; kernel=Chi2Kernel(feats_train, feats_train, width, size_cache); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% In this example a kernel matrix is computed for a given real-valued data set. % The kernel used is the Chi2 kernel which operates on real-valued vectors. It % computes the chi-squared distance between sets of histograms. It is a very % useful distance in image recognition (used to detect objects). The preprocessor % NormOne, normalizes vectors to have norm 1. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); %NormOne disp('NormOne') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); preproc=NormOne(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); width=1.4; size_cache=10; kernel=Chi2Kernel(feats_train, feats_train, width, size_cache); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
% In this example a kernel matrix is computed for a given real-valued data set. % The kernel used is the Chi2 kernel which operates on real-valued vectors. It % computes the chi-squared distance between sets of histograms. It is a very % useful distance in image recognition (used to detect objects). The preprocessor % PruneVarSubMean substracts the mean from each feature and removes features that % have zero variance. modshogun addpath('tools'); fm_train_real=load_matrix('../data/fm_train_real.dat'); fm_test_real=load_matrix('../data/fm_test_real.dat'); %PruneVarSubMean disp('PruneVarSubMean') feats_train=RealFeatures(fm_train_real); feats_test=RealFeatures(fm_test_real); preproc=PruneVarSubMean(); preproc.init(feats_train); feats_train.add_preprocessor(preproc); feats_train.apply_preprocessor(); feats_test.add_preprocessor(preproc); feats_test.apply_preprocessor(); width=1.4; size_cache=10; kernel=Chi2Kernel(feats_train, feats_train, width, size_cache); km_train=kernel.get_kernel_matrix(); kernel.init(feats_train, feats_test); km_test=kernel.get_kernel_matrix();
modshogun addpath('tools'); label_train=load_matrix('../data/label_train_twoclass.dat'); fm_train=load_matrix('../data/fm_train_real.dat'); fm_test=load_matrix('../data/fm_test_real.dat'); % kernel ridge regression disp('KRR') feats_train=RealFeatures(fm_train); feats_test=RealFeatures(fm_test); width=0.8; kernel=GaussianKernel(feats_train, feats_train, width); C=0.9; tau=1e-6; num_threads=1; labels=RegressionLabels(label_train); krr=KernelRidgeRegression(tau, kernel, labels); krr.parallel.set_num_threads(num_threads); krr.train(); kernel.init(feats_train, feats_test); out=krr.apply().get_labels();
% In this example a support vector regression algorithm is trained on a % real-valued toy data set. The underlying library used for the SVR training is % LIBSVM. Both the epsilon-SVR and the nu-SVR formulation are used. % Data is a noisy sine wave on which a Gaussian kernel is used. % % For more details on LIBSVM solver see http://www.csie.ntu.edu.tw/~cjlin/libsvm/ . modshogun addpath('tools'); label_train=load_matrix('../data/label_train_twoclass.dat'); fm_train=load_matrix('../data/fm_train_real.dat'); fm_test=load_matrix('../data/fm_test_real.dat'); %% libsvm based support vector regression disp('LibSVR') feats_train=RealFeatures(fm_train); feats_test=RealFeatures(fm_test); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; tube_epsilon=1e-2; num_threads=3; labels=RegressionLabels(label_train); svr=LibSVR(C, epsilon, kernel, labels); svr.set_tube_epsilon(tube_epsilon); svr.parallel.set_num_threads(num_threads); svr.train(); kernel.init(feats_train, feats_test); out=svr.apply().get_labels();
% In this example a support vector regression algorithm is trained on a % real-valued toy data set. The underlying library used for the SVR training is % SVM^light. The SVR is trained with regularization parameter C=1 and a gaussian % kernel with width=2.1. The the label of both the train and the test data are % fetched via svr.classify().get_labels(). % % For more details on the SVM^light see % T. Joachims. Making large-scale SVM learning practical. In Advances in Kernel % Methods -- Support Vector Learning, pages 169-184. MIT Press, Cambridge, MA USA, 1999. modshogun addpath('tools'); label_train=load_matrix('../data/label_train_twoclass.dat'); fm_train=load_matrix('../data/fm_train_real.dat'); fm_test=load_matrix('../data/fm_test_real.dat'); % svmlight based support vector regression try disp('SVRLight') feats_train=RealFeatures(fm_train); feats_test=RealFeatures(fm_test); width=2.1; kernel=GaussianKernel(feats_train, feats_train, width); C=1.2; epsilon=1e-5; tube_epsilon=1e-2; num_threads=3; labels=RegressioLabels(label_train); svr=SVRLight(C, epsilon, kernel, labels); svr.set_tube_epsilon(tube_epsilon); svr.parallel.set_num_threads(num_threads); svr.train(); kernel.init(feats_train, feats_test); svr.apply().get_labels(); catch disp('No support for SVRLight available.') end