Image Reconstruction Error Minimization via Distribution Adaptation and Low-rank representation

Abstract

Training a classifier with training data and intending it to accurately predict labels of test data with different distributions makes machine learning algorithms inefficient. Transfer learning and domain adaptation is a promising solution for tackling the cross-domain problems where the distribution gap exists between the training data (source domain) and test data (target domain). Visual domain adaptation aims to learn robust models for the target domain by knowledge transferring from a source domain. Therefore, DA boosts the performance of the model trained using source domain, for labelling the target samples. Since in real-world applications, target samples have no labels, we propose four unsupervised domain adaptation methods. In the first method, we propose a two-phases unsupervised approach referred as image reconstruction Error minimization via Distribution Adaptation and low-rank constraint (EDA), which benefits from both the distribution adaptation and the low-rank constraints to tackle distribution mismatch across domains. In the first phase, our proposed approach projects the training and test data onto a common subspace in which the marginal and conditional distribution differences of domains are minimized. Moreover, EDA benefits from domain invariant clustering to discriminate between various classes of data. In the second phase, for preserving data structure in the shared subspace, EDA minimizes the data reconstruction error using low-rank and sparse constraints. Overall, EDA solves the domain mismatch problem in cubic time complexity. The proposed approach is evaluated on variety of visual benchmark datasets and its performance is compared with the other state-of-the- art domain adaptation methods. The average accuracy of EDA on 32 experiments is determined 68.33% where outperforms other state-of-the-art domain adaptation methods with 4.28% improvement. In the second method, we propose a domain adaptation method namely Image classification via geometrical and statistical knowledge transfer (GSKT), to preserve geometrical and statistical information of the source and target domains. GSKT seeks to find a common subspace where the distribution disparity across the source and target domains are minimized. The performance of our proposed method is evaluated using the variety of standard visual datasets and 34 experiments, which shows a significant improvement against other state-of-the-art domain adaptation methods. In the third method, we propose an unsupervised domain adaptation method namely Discriminative and Invariant Subspace-Alignment (DISA). DISA transfers the source and target domains into the respective subspaces and adapts both global and local distributions in a unified framework. DISA minimizes the marginal and conditional probability disparities between the mapped source and target domains via MMD, during global adaptation. Different classes in the source domain are discriminated by maximizing and minimizing between-class and within-class distances. During local adaptation, DISA preserves the source and target domains information via labels of samples. The proposed method is verified using various visual benchmarks and compared with the state-of-the-art domain adaptation methods. The results prove that DISA outperforms other state-of-the-art methods. The fourth proposed method namely transductive transfer learning approach for multi- modal image classification (TTLM), seeks to find the specific and shared features across the source and target domains to map both domains into the respective subspaces with the least marginal and conditional distribution divergences. Moreover, discriminative learning in both domains leads to boost the model. TTLM intends to minimize the distances between all instance-pairs belonging to the same class, and apply the proposed procedure to all of the source and target classes for creating the condensed clusters. We verified the proposed method using standard visual benchmarks. Thus, we prove its superiority in comparison with the state-of-the-art domain adaptation methods through 36 cross-domain tasks.

Key Words: Machine learning, Transfer learning, Visual domain adaptation, Domain shift problem, Distribution adaptation, , Knowledge transfer, Discriminative learning.