Averaged Perceptron¶

The averaged Perceptron is an online binary classifier. It is an extension of the standard Perceptron algorithm; it uses the averaged weight and bias. Given a vector $$\mathbf{x}$$, the predicted class is given by:

$\theta\left(\mathbf{w} \cdot \mathbf{x}+b\right)$

Here, $$\mathbf{w}$$ is the average weight vector, $$b$$ is the average bias and $$\theta$$ is a step function:

$\begin{split}\theta(x) = \begin{cases} 1 & x > 0 \\ 0 & x = 0 \\ -1 & x < 0 \end{cases}\end{split}$

See chapter 17 in [Bar12] for a brief explanation of the Perceptron.

Example¶

Given a linearly separable dataset, we create some CDenseFeatures (RealFeatures, here 64 bit float values) and some CBinaryLabels to set up the training and validation sets.

features_train = RealFeatures(f_feats_train)
features_test = RealFeatures(f_feats_test)
labels_train = BinaryLabels(f_labels_train)
labels_test = BinaryLabels(f_labels_test)

features_train = RealFeatures(f_feats_train);
features_test = RealFeatures(f_feats_test);
labels_train = BinaryLabels(f_labels_train);
labels_test = BinaryLabels(f_labels_test);

RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
BinaryLabels labels_train = new BinaryLabels(f_labels_train);
BinaryLabels labels_test = new BinaryLabels(f_labels_test);

features_train = Shogun::RealFeatures.new f_feats_train
features_test = Shogun::RealFeatures.new f_feats_test
labels_train = Shogun::BinaryLabels.new f_labels_train
labels_test = Shogun::BinaryLabels.new f_labels_test

features_train <- RealFeatures(f_feats_train)
features_test <- RealFeatures(f_feats_test)
labels_train <- BinaryLabels(f_labels_train)
labels_test <- BinaryLabels(f_labels_test)

features_train = shogun.RealFeatures(f_feats_train)
features_test = shogun.RealFeatures(f_feats_test)
labels_train = shogun.BinaryLabels(f_labels_train)
labels_test = shogun.BinaryLabels(f_labels_test)

RealFeatures features_train = new RealFeatures(f_feats_train);
RealFeatures features_test = new RealFeatures(f_feats_test);
BinaryLabels labels_train = new BinaryLabels(f_labels_train);
BinaryLabels labels_test = new BinaryLabels(f_labels_test);

auto features_train = some<CDenseFeatures<float64_t>>(f_feats_train);
auto features_test = some<CDenseFeatures<float64_t>>(f_feats_test);
auto labels_train = some<CBinaryLabels>(f_labels_train);
auto labels_test = some<CBinaryLabels>(f_labels_test);


We create the CAveragedPerceptron instance by passing it the traning features and labels. We also set its learn rate and its maximum iterations.

learn_rate = 1.0
max_iter = 1000
perceptron = AveragedPerceptron(features_train, labels_train)
perceptron.set_learn_rate(learn_rate)
perceptron.set_max_iter(max_iter)

learn_rate = 1.0;
max_iter = 1000;
perceptron = AveragedPerceptron(features_train, labels_train);
perceptron.set_learn_rate(learn_rate);
perceptron.set_max_iter(max_iter);

double learn_rate = 1.0;
int max_iter = 1000;
AveragedPerceptron perceptron = new AveragedPerceptron(features_train, labels_train);
perceptron.set_learn_rate(learn_rate);
perceptron.set_max_iter(max_iter);

learn_rate = 1.0
max_iter = 1000
perceptron = Shogun::AveragedPerceptron.new features_train, labels_train
perceptron.set_learn_rate learn_rate
perceptron.set_max_iter max_iter

learn_rate <- 1.0
max_iter <- 1000
perceptron <- AveragedPerceptron(features_train, labels_train)
perceptron$set_learn_rate(learn_rate) perceptron$set_max_iter(max_iter)

learn_rate = 1.0
max_iter = 1000
perceptron = shogun.AveragedPerceptron(features_train, labels_train)
perceptron:set_learn_rate(learn_rate)
perceptron:set_max_iter(max_iter)

double learn_rate = 1.0;
int max_iter = 1000;
AveragedPerceptron perceptron = new AveragedPerceptron(features_train, labels_train);
perceptron.set_learn_rate(learn_rate);
perceptron.set_max_iter(max_iter);

auto learn_rate = 1.0;
auto max_iter = 1000;
auto perceptron = some<CAveragedPerceptron>(features_train, labels_train);
perceptron->set_learn_rate(learn_rate);
perceptron->set_max_iter(max_iter);


Then we train the CAveragedPerceptron and we apply it to the test data, which gives the predicted CBinaryLabels.

perceptron.train()
perceptron.set_features(features_test)
labels_predict = perceptron.apply()

perceptron.train();
perceptron.set_features(features_test);
labels_predict = perceptron.apply();

perceptron.train();
perceptron.set_features(features_test);
Labels labels_predict = perceptron.apply();

perceptron.train
perceptron.set_features features_test
labels_predict = perceptron.apply

perceptron$train() perceptron$set_features(features_test)
labels_predict <- perceptron$apply()  perceptron:train() perceptron:set_features(features_test) labels_predict = perceptron:apply()  perceptron.train(); perceptron.set_features(features_test); Labels labels_predict = perceptron.apply();  perceptron->train(); perceptron->set_features(features_test); auto labels_predict = perceptron->apply();  We can also extract the average weights $$\mathbf{w}$$ and the bias $$b$$. weights = perceptron.get_w() bias = perceptron.get_bias()  weights = perceptron.get_w(); bias = perceptron.get_bias();  DoubleMatrix weights = perceptron.get_w(); double bias = perceptron.get_bias();  weights = perceptron.get_w bias = perceptron.get_bias  weights <- perceptron$get_w()
bias <- perceptron$get_bias()  weights = perceptron:get_w() bias = perceptron:get_bias()  double[] weights = perceptron.get_w(); double bias = perceptron.get_bias();  auto weights = perceptron->get_w(); auto bias = perceptron->get_bias();  Finally, we can evaluate the performance, e.g. using CAccuracyMeasure. eval = AccuracyMeasure() accuracy = eval.evaluate(labels_predict, labels_test)  eval = AccuracyMeasure(); accuracy = eval.evaluate(labels_predict, labels_test);  AccuracyMeasure eval = new AccuracyMeasure(); double accuracy = eval.evaluate(labels_predict, labels_test);  eval = Shogun::AccuracyMeasure.new accuracy = eval.evaluate labels_predict, labels_test  eval <- AccuracyMeasure() accuracy <- eval$evaluate(labels_predict, labels_test)

eval = shogun.AccuracyMeasure()
accuracy = eval:evaluate(labels_predict, labels_test)

AccuracyMeasure eval = new AccuracyMeasure();
double accuracy = eval.evaluate(labels_predict, labels_test);

auto eval = some<CAccuracyMeasure>();
auto accuracy = eval->evaluate(labels_predict, labels_test);


References¶

Wikipedia: Perceptron

 [Bar12] D. Barber. Bayesian reasoning and machine learning. Cambridge University Press, 2012.