Kernel mean matching in metric transfer learning framework

Abstract

Metric-based algorithms are a subset of machine learning techniques which are used to extract patterns from training set and classification problems. Metric learning using the distance metric tries to classify the similar samples into one group by measuring the similarity or dissimilarity of sample pairs for classification. Conventional metric learning approaches will receive appropriate performance only by assuming that the source and target samples have the same distribution, however this assumption may not work in real world applications and the source and target samples may not have the same distribution. As a result, the model created on the source domain will not perform well on the target domain. In this situation, knowledge transfer or transfer learning may be beneficial for accurate classification. Since, the metric learning methods in transfer learning needs to measure the similarity or dissimilarity of samples using an appropriate distance metric, some of the proposed approaches use Euclidean distance metric. Since the Euclidean metric performs poorly in measuring sample pair similarity, both of the proposed approaches in this study utilize the Mahalanobis distance metric, which take into account the correlation between samples in their calculations. The first approach, known as kernel mean matching in metric transfer learning framework , reduces the existing distribution gap between source and target domain by re-weighting source domain samples as well as using Mahalanobis distance metric, simultaneously. KMTLF minimizes intra-class distance and maximizes inter-class distance on target domain while maintains the geometric information of samples. KMTLF tries to perform well in target domain by learning sample weights and utilizing the Mahalanobis distance metric in a pipelined framework. Second approach, metric transfer learning via geometric knowledge embedding , uses two projection matrices, one for each domain to map source and target samples onto a shared common subspace. In the new space, Mahalanobis distance metric is learned for target domain to minimize the inter-class distance and maximize the intra- class distance, while a novel instance reweighting scheme on the graph optimization is proposed, simultaneously, to exploit the instance weights for distribution matching. In order to evaluate the proposed methods, they have been examined on different benchmark datasets including Office-Caltech and USPS-MNIST datasets. The results of our experiments shows a significant improvement against state-of-the-arts methods.

Key Words : Metric Learning, Mahalanobis Distance Metric , Transfer Learning, Sample Re-weighting