A Gaussian mixture model is a probabilistic model that assumes that data are generated from a finite mixture of Gaussians with unknown parameters. The model likelihood can be written as:
where \(p(x|\theta)\) is probability distribution given \(\theta:=\{\pi_i, \mu_i, \Sigma_i\}_{i=1}^K\), \(K\) denotes number of mixture components, \(\pi_i\) denotes weight for \(i\)-th component, \(\mathcal{N}\) denotes a multivariate normal distribution with mean vector \(\mu_i\) and covariance matrix \(\Sigma_i\).
The expectation maximization (EM) algorithm is used to learn parameters of the model, via finding a local maximum of a lower bound on the likelihood.
See Chapter 20 in [Bar12] for a detailed introduction.
We start by creating CDenseFeatures (here 64 bit floats aka RealFeatures) as
features_train = RealFeatures(f_feats_train)
features_train = RealFeatures(f_feats_train);
RealFeatures features_train = new RealFeatures(f_feats_train);
features_train = Modshogun::RealFeatures.new f_feats_train
features_train <- RealFeatures(f_feats_train)
RealFeatures features_train = new RealFeatures(f_feats_train);
auto features_train = some<CDenseFeatures<float64_t>>(f_feats_train);
We initialize GMM, passing the desired number of mixture components.
num_components = 3
gmm = GMM(num_components)
num_components = 3;
gmm = GMM(num_components);
int num_components = 3;
GMM gmm = new GMM(num_components);
num_components = 3
gmm = Modshogun::GMM.new num_components
num_components <- 3
gmm <- GMM(num_components)
int num_components = 3;
GMM gmm = new GMM(num_components);
auto num_components = 3;
auto gmm = some<CGMM>(num_components);
We provide training features to the GMM object, train it by using EM algorithm and sample data-points from the trained model.
gmm.set_features(features_train)
gmm.train_em()
output = gmm.sample()
gmm.set_features(features_train);
gmm.train_em();
output = gmm.sample();
gmm.set_features(features_train);
gmm.train_em();
DoubleMatrix output = gmm.sample();
gmm.set_features features_train
gmm.train_em
output = gmm.sample
gmm$set_features(features_train)
gmm$train_em()
output <- gmm$sample()
gmm.set_features(features_train);
gmm.train_em();
double[] output = gmm.sample();
gmm->set_features(features_train);
gmm->train_em();
auto output = gmm->sample();
We extract parameters like \(\pi\), \(\mu_i\) and \(\Sigma_i\) for any componenet from the trained model.
component_num = 1
nth_mean = gmm.get_nth_mean(component_num)
nth_cov = gmm.get_nth_cov(component_num)
coef = gmm.get_coef()
component_num = 1;
nth_mean = gmm.get_nth_mean(component_num);
nth_cov = gmm.get_nth_cov(component_num);
coef = gmm.get_coef();
int component_num = 1;
DoubleMatrix nth_mean = gmm.get_nth_mean(component_num);
DoubleMatrix nth_cov = gmm.get_nth_cov(component_num);
DoubleMatrix coef = gmm.get_coef();
component_num = 1
nth_mean = gmm.get_nth_mean component_num
nth_cov = gmm.get_nth_cov component_num
coef = gmm.get_coef
component_num <- 1
nth_mean <- gmm$get_nth_mean(component_num)
nth_cov <- gmm$get_nth_cov(component_num)
coef <- gmm$get_coef()
int component_num = 1;
double[] nth_mean = gmm.get_nth_mean(component_num);
double[,] nth_cov = gmm.get_nth_cov(component_num);
double[] coef = gmm.get_coef();
auto component_num = 1;
auto nth_mean = gmm->get_nth_mean(component_num);
auto nth_cov = gmm->get_nth_cov(component_num);
auto coef = gmm->get_coef();
We obtain log likelihood of belonging to clusters and being generated by this model.
log_likelihoods = gmm.cluster(nth_mean)
log_likelihoods = gmm.cluster(nth_mean);
DoubleMatrix log_likelihoods = gmm.cluster(nth_mean);
log_likelihoods = gmm.cluster nth_mean
log_likelihoods <- gmm$cluster(nth_mean)
double[] log_likelihoods = gmm.cluster(nth_mean);
auto log_likelihoods = gmm->cluster(nth_mean);
We can also use Split-Merge Expectation-Maximization algorithm [UNGH00] for training.
gmm.train_smem()
gmm.train_smem();
gmm.train_smem();
gmm.train_smem
gmm$train_smem()
gmm.train_smem();
gmm->train_smem();
Wikipedia: Expectation–maximization_algorithm
| [Bar12] | David Barber. Bayesian reasoning and machine learning. Cambridge University Press, 2012. |
| [UNGH00] | N. Ueda, R. Nakano, Z. Ghahramani, and G.E. Hinton. Smem algorithm for mixture models. Neural Computation, 12(9):2109–2128, 2000. |