调研不平衡场景下的领域自适应问题

No abstract available

No abstract available

No abstract available

Unsupervised domain adaptation (UDA) has become an appealing approach for knowledge transfer from a labeled source domain to an unlabeled target domain. However, when the classes in source and target domains are imbalanced, most existing UDA methods experience significant performance drop, as the decision boundary usually favors the majority classes. Some recent class-imbalanced domain adaptation (CDA) methods aim to tackle the challenge of biased label distribution by exploiting pseudo-labeled target samples during the training process. However, these methods suffer from the issues with unreliable pseudo labels and error accumulation during training. In this paper, we propose a pairwise adversarial training approach for class-imbalanced domain adaptation. Unlike conventional adversarial training in which the adversarial samples are obtained from the $\ell _{p}$ ball of the original samples, we generate adversarial samples from the interpolated line of the aligned pairwise samples from source and target domains. The pairwise adversarial training (PAT) is a novel data-augmentation method which can be integrated into existing unsupervised domain adaptation (UDA) models to tackle the CDA problem. Inspired by the noise injection, we also extend the pairwise adversarial training to noisy pairwise adversarial training (nPAT), in which the random noise is injected into the generation of the adversarial samples. In our study, we evaluate our proposed methods as well as the baselines on three major benchmark datasets, namely Office-Home, DomainNet and Office-31. For Office-Home and Office-31, we sample the data according to the Reversely-unbalanced Source and Unbalanced Target (RS-UT) protocol so that the class distribution can be imbalanced. The extensive experimental results show that UDA models integrated with our proposed nPAT can achieve prominent improvements on most tasks compared to the baseline methods as well as the state-of-the-art CDA methods. The average accuracy of our nPAT can achieve 66.56% and 80.22% on Office-Home and DomainNet, respectively, which are higher than that of the second-best methods. Besides, Experiments also show that our method is robust to the unreliability of the pseudo labels.

Many unsupervised domain adaptation (UDA) algorithms have been shown to fail when the target data is class-imbalanced – a scenario known as imbalanced domain adaptation (IDA). However, the lack of labels means this data cannot be balanced in the usual way. The most common workaround – to balance the data using pseudo labels – makes the strong assumption that the predicted labels are accurate. In this paper, we propose instead to balance the target data implicitly, by performing a diversity-based resampling of that data. This reduces the strength of the assumptions required to obtain an accurate balancing. We describe three diversity-based sampling algorithms, two based on k-means clustering and the other on the determinantal point process (DPP), which can be used to implicitly balance unlabelled data. These algorithms readily scale to large datasets, and are compatible with both shallow and deep UDA methods. Extensive analysis on a cross-dataset bioacoustic event detection task shows that this approach significantly outperforms prior work on IDA, and can achieve up to oracle-level performance for a wide range of imbalances.

In non-cooperative communication, complex channel conditions can cause data distribution shift (DDS), which can significantly degrade the performance of existing automatic modulation classification (AMC) models. Initial attempts have explored the use of unsupervised domain adaptation (UDA) to mitigate this issue. However, the considerations for these works are still limited as they assume that the label distribution is domain invariant. In real-world scenarios, significant variations in the usage frequencies of different modulation types often result in imbalanced data volumes and inconsistent label distributions. This challenge, known as label distribution shift (LDS), poses a substantial challenge for cross-domain alignment. To address this limitation, this letter considers AMC in imbalanced domain adaptation (IDA) scenarios, which handle the DDS and LDS simultaneously. We propose a novel method called Pseudo label-based Imbalanced Alignment (PIA). Specifically, our PIA leverages pseudo labels in the target domain to achieve implicit and explicit conditional feature alignment, through re-weighting self-training and centroid feature alignment, respectively. We construct various IDA scenarios for experiments using custom and public datasets, and the results prove the effectiveness and superiority of PIA.

Unsupervised domain adaptation (UDA) has become an appealing approach for knowledge transfer from a labeled source domain to an unlabeled target domain. However, when the classes in source and target domains are imbalanced, most existing UDA methods experience significant performance drop, as the decision boundary usually favors the majority classes. Some recent class-imbalanced domain adaptation (CDA) methods aim to tackle the challenge of biased label distribution by exploiting pseudo-labeled target samples during training process. However, these methods suffer from the issues with unreliable pseudo labels and error accumulation during training. In this paper, we propose a pairwise adversarial training approach for class-imbalanced domain adaptation. Unlike conventional adversarial training in which the adversarial samples are obtained from the lp ball of the original samples, we generate adversarial samples from the interpolated line of the aligned pairwise samples from source and target domains. The pairwise adversarial training (PAT) is a novel data-augmentation method which can be integrated into existing UDA models to tackle with the CDA problem. Experimental results and ablation studies show that the UDA models integrated with our method achieve considerable improvements on benchmarks compared with the original models as well as the state-of-the-art CDA methods. Our source code is available at: https://github.com/DamoSWL/Pairwise-Adversarial-Training

Many existing unsupervised domain adaptation (UDA) methods primarily focus on covariate shift, limiting their effectiveness in imbalanced domain adaptation (IDA) where both covariate shift and label shift coexist. Recent IDA methods have achieved promising results based on self-training using target pseudo labels. However, under the IDA scenarios, the classifier learned in the source domain will exhibit different decision bias from the target domain. It will potentially make target pseudo labels unreliable, and will further lead to error accumulation with incorrect class alignment. Thus, we propose contrastive conditional alignment based on label shift calibration (CCA-LSC) for IDA, to address both covariate shift and label shift. Initially, our contrastive conditional alignment resolve covariate shift to learn representations with domain invariance and class discriminability, which include domain adversarial learning, sample-weighted moving average centroid alignment and discriminative feature alignment. Subsequently, we estimate the probability distribution of the target domain, and calibrate target sample classification predictions based on label shift metrics to encourage labeling pseudo-labels more consistently with the distribution of real target data. Extensive experiments are conducted and demonstrate that our method outperforms existing UDA and IDA methods on benchmarks with both label shift and covariate shift. Our code is available at https://github.com/ysxcj-hub/CCA-LSC.

Domain adaptation aims to transfer knowledge between different domains to develop an effective hypothesis in the target domain with scarce labeled data, which is an effective method for remedying the problem of labeled data requirement in deep learning. In reality, it is unavoidable that the dataset has a large gap in the number of positive and negative instances across different categories in source and target domains, which is the imbalanced domain adaptation problem. However, since the imbalanced degree always varies greatly in different source- and target-domain datasets, most of the existing imbalanced domain adaptation models fix the imbalanced parameters, which cannot adapt to the change of the proportion between positive and negative instances in different domains. To address this problem, in this article, we propose a self-adaptive imbalanced domain adaptation method via a deep sparse autoencoder, which can adjust the model automatically according to the imbalanced extent for bridging the chasm of domains. More specifically, the self-adaptive imbalanced cross-entropy loss is designed for emphasizing more on minority categories and compensating the bias of training loss automatically. In addition, to alleviate the deficient problem of labeled data, we further propose the unlabeled information incorporating method by minimizing the distribution discrepancy of high-level representation space between the source and target domains. Experiments on several real-world datasets demonstrate the effectiveness of our method compared to other state-of-the-art methods.

Unsupervised domain adaptation methods have been proposed to tackle the problem of covariate shift by minimizing the distribution discrepancy between the feature embeddings of source domain and target domain. However, the standard evaluation protocols assume that the conditional label distributions of the two domains are invariant, which is usually not consistent with the real-world scenarios such as long-tailed distribution of visual categories. In this article, the imbalanced domain adaptation (IDA) is formulated for a more realistic scenario where both label shift and covariate shift occur between the two domains. Theoretically, when label shift exists, aligning the marginal distributions may result in negative transfer. Therefore, a novel cluster-level discrepancy minimization (CDM) is developed. CDM proposes cross-domain similarity learning to learn tight and discriminative clusters, which are utilized for both feature-level and distribution-level discrepancy minimization, palliating the negative effect of label shift during domain transfer. Theoretical justifications further demonstrate that CDM minimizes the target risk in a progressive manner. To corroborate the effectiveness of CDM, we propose two evaluation protocols according to the real-world situation and benchmark existing domain adaptation approaches. Extensive experiments demonstrate that negative transfer does occur due to label shift, while our approach achieves significant improvement on imbalanced datasets, including Office-31, Image-CLEF, and Office-Home.

In this paper, we study the problem of legal domain adaptation problem from an imbalanced source domain to a partial target domain. The task aims to improve legal judgment predictions for non-professional fact descriptions. We formulate this task as a partial-and-imbalanced domain adaptation problem. Though deep domain adaptation has achieved cutting-edge performance in many unsupervised domain adaptation tasks. However, due to the negative transfer of samples in non-shared classes, it is hard for current domain adaptation model to solve the partial-and-imbalanced transfer problem. In this work, we explore large-scale non-shared but related classes data in the source domain with a hierarchy weighting adaptation to tackle this limitation. We propose to embed a novel pArtial Imbalanced Domain Adaptation technique (AIDA) in the deep learning model, which can jointly borrow sibling knowledge from non-shared classes to shared classes in the source domain and further transfer the shared classes knowledge from the source domain to the target domain. Experimental results show that our model outperforms the state-of-the-art algorithms.

Unsupervised Domain Adaptation (UDA) approaches address the covariate shift problem by minimizing the distribution discrepancy between the source and target domains, assuming that the label distribution is invariant across domains. However, in the imbalanced domain adaptation (IDA) scenario, covariate and long-tailed label shifts both exist across domains. To tackle the IDA problem, some current research focus on minimizing the distribution discrepancies of each corresponding class between source and target domains. Such methods rely much on the reliable pseudo labels' selection and the feature distributions estimation for target domain, and the minority classes with limited numbers makes the estimations more uncertainty, which influences the model's performance. In this paper, we propose a cross-domain class discrepancy minimization method based on accumulative class-centroids for IDA (centroIDA). Firstly, class-based re-sampling strategy is used to obtain an unbiased classifier on source domain. Secondly, the accumulative class-centroids alignment loss is proposed for iterative class-centroids alignment across domains. Finally, class-wise feature alignment loss is used to optimize the feature representation for a robust classification boundary. A series of experiments have proved that our method outperforms other SOTA methods on IDA problem, especially with the increasing degree of label shift.

. Electroencephalography (EEG) based brain-computer interfaces (BCIs) face great challenges in generalizing across different domains (i.e., sessions and subjects) without costly supervised calibration. To avoid supervised calibration, transfer learning, particularly unsupervised domain adaptation, has been a popular approach. In this work, we focus on a geometric deep learning framework previously proposed for EEG-based mental imagery BCIs. The framework aligns marginal feature distributions in latent space, assuming identical label distributions across domains. Here, we propose a novel approach integrating data augmentation and clustering techniques to align the latent distributions under label shifts.

Domain adaptation (DA) becomes an up-and-coming technique to address the insufficient or no annotation issue by exploiting external source knowledge. Existing DA algorithms mainly focus on practical knowledge transfer through domain alignment. Unfortunately, they ignore the fairness issue when the auxiliary source is extremely imbalanced across different categories, which results in severe under-presented knowledge adaptation of minority source set. To this end, we propose a Towards Fair Knowledge Transfer (TFKT) framework to handle the fairness challenge in imbalanced cross-domain learning. Specifically, a novel cross-domain knowledge propagation technique is proposed with the guidance of within-source and cross-domain structure graphs to smooth the manifold of the minority source set. Besides, a cross-domain fulfillment augmentation strategy is exploited achieve domain adaptation. Moreover, hybrid distinct classifiers and cross-domain prototype alignment are adopted to seek a more robust classifier boundary and mitigate the domain shift. Such three strategies are formulated into a unified framework to address the fairness issue and domain shift challenge. Extensive experiments over two popular benchmarks have verified the effectiveness of our proposed model by comparing to existing state-of-the-art DA models, and especially our model significantly improves over 20% on two benchmarks in terms of the overall accuracy.

No abstract available

Graph domain adaptation is a key subfield of graph transfer learning that aims to bridge domain gaps by transferring knowledge from a label-rich source graph to an unlabeled target graph. However, most existing methods assume balanced labels in the source graph, which often fails in practice and leads to biased knowledge transfer. To address this, in this paper, we propose a prototype-anchored learning and alignment framework for class-imbalanced graph domain adaptation. Specifically, we incorporate pointwise node mutual information into the graph encoder to capture high-order topological proximity and learn generalized node representations. Leveraging this, we then introduce categorical prototypes with adversarial proto-instances for prototype-anchored learning and recalibration to represent the source graph under an imbalanced class distribution. Finally, we introduce a weighted prototype contrastive adaptation strategy that aligns target pseudo-labels with source prototypes to handle class imbalance during adaptation. Extensive experiments show that our PALA outperforms the state-of-the-art methods. Our code is available at https://github.com/maxin88scu/PALA.

Utilizing unsupervised domain adaptation for intelligent fault diagnosis (IFD) has demonstrated significant potential for ensuring the security of machinery systems. Nonetheless, the inherent imbalance attribute of collected data affects the performance of diagnostic model. Especially, for machines working under varied conditions, the acquired unlabeled data frequently exhibits diverse degrees of distributional deviations, thus further undermining the transferable model’s generalization capability. To address this challenge, we introduce a method termed Dynamic Unsupervised Imbalanced Domain Adaptation (DUIDA) for IFD. Employment of class rebalancing and label-dependent margin regularization strategies optimizes the selection of decision boundaries which counteract the distributional deviations introduced by the imbalance. In addition, by integrating a dynamic weighting mechanism, encompassing both adversarial-based and MMD-based domain adaptation, our model becomes versatile across varied UIDA tasks, assigning higher weights to fundamental faulty features. Finally, our empirical analyses on two faulty bearing datasets substantiate the efficacy and superior performance of the proposed framework across diverse operational scenarios.

Software vulnerabilities (SVs) have become a common, serious, and crucial concern due to the ubiquity of computer software. Many AI-based approaches have been proposed to solve the software vulnerability detection (SVD) problem to ensure the security and integrity of software applications (in both the development and testing phases). However, there are still two open and significant issues for SVD in terms of (i) learning automatic representations to improve the predictive performance of SVD, and (ii) tackling the scarcity of labeled vulnerability datasets that conventionally need laborious labeling effort by experts. In this paper, we propose a novel approach to tackle these two crucial issues. We first exploit the automatic representation learning with deep domain adaptation for SVD. We then propose a novel cross-domain kernel classifier leveraging the max-margin principle to significantly improve the transfer learning process of SVs from imbalanced labeled into imbalanced unlabeled projects. Our approach is the first work that leverages solid body theories of the max-margin principle, kernel methods, and bridging the gap between source and target domains for imbalanced domain adaptation (DA) applied in cross-project SVD. The experimental results on real-world software datasets show the superiority of our proposed method over state-of-the-art baselines. In short, our method obtains a higher performance on F1-measure, one of the most important measures in SVD, from 1.83% to 6.25% compared to the second highest method in the used datasets.

No abstract available

Modern industrial fault diagnosis tasks often face the combined challenge of distribution discrepancy and bi-imbalance. Existing domain adaptation approaches pay little attention to the prevailing bi-imbalance, leading to poor domain adaptation performance or even negative transfer. In this work, we propose a self-degraded contrastive domain adaptation (Sd-CDA) diagnosis framework to handle the domain discrepancy under the bi-imbalanced data. It first pre-trains the feature extractor via imbalance-aware contrastive learning based on model pruning to learn the feature representation efficiently in a self-supervised manner. Then it forces the samples away from the domain boundary based on supervised contrastive domain adversarial learning (SupCon-DA) and ensures the features generated by the feature extractor are discriminative enough. Furthermore, we propose the pruned contrastive domain adversarial learning (PSupCon-DA) to pay automatically re-weighted attention to the minorities to enhance the performance towards bi-imbalanced data. We show the superiority of the proposed method via two experiments.

The existing self-supervised Multi-Source Domain Adaptation (MSDA) methods often suffer an imbalanced characteristic among the distribution of pseudo-labels. Such imbalanced characteristic results in many labels with too many or too few pseudo-labeled samples on the target domain, referred to as easy-to-learn label and hard-to-learn label, respectively. Both of these labels hurt the generalization performance on the target domain. To alleviate this problem, in this paper we propose a novel multi-source domain adaptation method, namely Self-Supervised multi-Source Domain Adaptation with Label-specific Confidence (S3DA-LC). Specifically, we estimate the label-specific confidences, i.e., the learning difficulties of labels, and adopt them to generate the pseudo-labels for target samples, enabling to simultaneously constrain and enrich the pseudo supervised signals for easy-to-learn and hard-to-learn labels. We evaluate S3DA-LC on several benchmark datasets, indicating its superior performance compared with the existing MSDA baselines.

No abstract available

Fault data from in-service rotating machines are extremely scarce. This is usually true even when healthy data are abundant, leading to the problem of class imbalance. Numerous solutions have been proposed to cope with the problem of class imbalance; each solution has its own advantages and disadvantages in implementation. This paper proposes a much simpler and efficient method for fault diagnosis of rotating machines. By employing pseudo-labeling, weighted random sampling, and time-shifting, the proposed repetitive learning method generates pseudo-augmented source and target fault data. Deep convolutional domain adaptation networks are followed to extract features by minimizing different losses. The evaluation results demonstrate the effectiveness of the proposed method, achieving accuracy rates of 90.79% (CWRU), 76.26% (XJTU), and 86.45% (GIST) under extreme imbalance conditions ( $\rho =0.01$ ), outperforming existing methods by 10-30% while maintaining computational efficiency. The evaluation results show that repetitive learning produces accurate prediction performance even in situations with extremely imbalanced data, which corroborates the effectiveness offered by the proposed method, despite its simplicity.

No abstract available

Multimedia applications are often associated with cross-domain knowledge transfer, where Unsupervised Domain Adaptation (UDA) can be used to reduce the domain shifts. Open Set Domain Adaptation (OSDA) aims to transfer knowledge from a well-labeled source domain to an unlabeled target domain under the assumption that the target domain contains unknown classes. Existing OSDA methods consistently lay stress on the covariate shift, ignoring the potential label shift problem. The performance of OSDA methods degrades drastically under intra-domain class imbalance and inter-domain label shift. However, little attention has been paid to this issue in the community. In this paper, the Imbalanced Open Set Domain Adaptation (IOSDA) is explored where the covariate shift, label shift and category mismatch exist simultaneously. To alleviate the negative effects raised by label shift in OSDA, we propose Open-set Moving-threshold Estimation and Gradual Alignment (OMEGA) - a novel architecture that improves existing OSDA methods on class-imbalanced data. Specifically, a novel unknown-aware target clustering scheme is proposed to form tight clusters in the target domain to reduce the negative effects of label shift and intra-domain class imbalance. Furthermore, moving-threshold estimation is designed to generate specific thresholds for each target sample rather than using one for all. Extensive experiments on IOSDA, OSDA and OPDA benchmarks demonstrate that our method could significantly outperform existing state-of-the-arts.

Unsupervised domain adaptation (UDA) aims to transfer knowledge via domain alignment, and typically assumes balanced data distribution. When deployed in real tasks, however, (i) each domain usually suffers from class imbalance, and (ii) different domains may have different class imbalance ratios. In such bi-imbalanced cases with both within-domain and across-domain imbalance, source knowledge transfer may degenerate the target performance. Some recent efforts have adopted source re-weighting to this issue, in order to align label distributions across domains. However, since target label distribution is unknown, the alignment might be incorrect or even risky. In this paper, we propose an alternative solution named TIToK for bi-imbalanced UDA, by directly Transferring Imbalance-Tolerant Knowledge across domains. In TIToK, a class contrastive loss is presented for classification, in order to alleviate the sensitivity to imbalance in knowledge transfer. Meanwhile, knowledge of class correlation is transferred as a supplementary, which is commonly invariant to imbalance. Finally, discriminative feature alignment is developed for a more robust classifier boundary. Experiments over benchmark datasets show that TIToK achieves competitive performance with the state-of-the-arts, and its performance is less sensitive to imbalance.

Deep learning (DL) techniques are highly effective for defect detection from images. Training DL classification models, however, requires vast amounts of labeled data which is often expensive to collect. In many cases, not only the available training data is limited but may also imbalanced. In this paper, we propose a novel domain adaptation (DA) approach to address the problem of labeled training data scarcity for a target learning task by transferring knowledge gained from an existing source dataset used for a similar learning task. Our approach works for scenarios where the source dataset and the dataset available for the target learning task have same or different feature spaces. We combine our DA approach with an autoencoder-based data augmentation approach to address the problem of imbalanced target datasets. We evaluate our combined approach using image data for wafer defect prediction. The experiments show its superior performance against other algorithms when the number of labeled samples in the target dataset is significantly small and the target dataset is imbalanced.

Semi-supervised domain adaptation is a technique to build a classifier for a target domain by modifying a classifier in another (source) domain using many unlabeled samples and a small number of labeled samples from the target domain. In this paper, we develop a semi-supervised domain adaptation method, which has robustness to class-imbalanced situations, which are common in medical image classification tasks. For robustness, we propose a weakly-supervised clustering pipeline to obtain high-purity clusters and utilize the clusters in representation learning for domain adaptation. The proposed method showed state-of-the-art performance in the experiment using severely class-imbalanced pathological image patches.

In dealing with the lack of sufficient annotated data and in contrast to supervised learning, unsupervised, self-supervised, and semi-supervised domain adaptation methods are promising approaches, enabling us to transfer knowledge from rich labeled source domains to different (but related) unlabeled target domains, reducing distribution discrepancy between the source and target domains. However, most existing domain adaptation methods do not consider the imbalanced nature of the real-world data, affecting their performance in practice. We propose to overcome this limitation by proposing a novel domain adaptation approach that includes two modifications to the existing models. Firstly, we leverage the focal loss function in response to class-imbalanced labeled data in the source domain. Secondly, we introduce a novel co-training approach to involve pseudo-labeled target data points in the training process. Experiments show that the proposed model can be effective in transferring knowledge from source to target domain. As an example, we use the classification of prostate cancer images into low-cancerous and high-cancerous regions.

Recent deep learning based methods have demonstrated promising results in scene text recognition. One of the major difficulty is the lack of manually annotated data. Synthetic data are then used to eliminate the requirement for human annotation. However, the domain gap between synthetic and real-world data remains a challenging issue. To bridge the gap, unsupervised domain adaptation (UDA) was introduced to transfer knowledge from a labeled source domain to a target domain. In this work, we introduce an unsupervised domain adaptation method based on a sequence-to-sequence attention model. We take into account imbalanced distribution of characters to optimize the adaptation process. We propose to use focal loss as the classification loss for the labeled source domain and focal entropy as the entropy loss for the unlabeled target domain. Our proposed method, named ICD-DA, outperforms other UDA methods on official benchmarks.

With the technology development, information networks continuously generate a large amount of integrated labeled Big Data. Some types of labeled data in real scenes are scarce and difficult to obtain, such as some aerospace data. It is important to address the problem of nonshared and imbalanced unsupervised domain adaptation (NI-UDA) from the labeled Big Data with nonshared and long-tail distribution to unlabeled specified small and imbalanced space applications, where nonshared classes mean the label space out of the target domain. Previous methods proposed to integrate the semantic knowledge of Big Data to help the unsupervised domain adaptation for sparse data. However, they have the challenges of limited effect of knowledge sharing for long-tail Big Data and the imbalanced domain adaptation. To solve them, our goal is to leverage priori hierarchy knowledge to enhance domain contrastive aligned feature representation with graph reasoning. Our method consists of hierarchy graph reasoning (HGR) layer and K-positive contrastive domain adaptation (K-CDA). Our HGR contributes to learn direct semantic patterns for sparse classes by hierarchy attention in self-attention, nonlinear mapping, and graph normalization. For alleviating imbalanced domain adaptation, we proposed K-CDA, which explores k-positive instances for each class to every mini-batch with contrastive learning to align imbalanced feature representations. Compared with the previous contrastive UDA, our K-CDA alleviates the problems of large memory consumption and high computational cost. Experiments on three benchmark datasets shows our methods consistently improve the state-of-the-art contrastive UDA algorithms.

The SCADA-driven fault diagnosis algorithm has been extensively researched and applied. However, the highly imbalanced distribution between normal and fault data poses significant challenges in establishing high-performance fault diagnosis models. To achieve accurate fault classification on imbalanced data, this paper proposes a fault diagnosis algorithm framework that combines the SMOTE oversampling method with domain adaptive transfer learning. Firstly, the sliding window sampling technique is used to transform data into two-dimensional spatiotemporal window data. Subsequently, SMOTE oversampling is applied, preserving and enriching the complete temporal fault features. To address the issue of noise introduced by the oversampling algorithm, domain adaptive transfer learning is introduced to extract invariant features between the original and oversampled data, effectively filtering out noise introduced by the oversampling. Experimental results from a real SCADA data demonstrate that the proposed method achieves model training on highly imbalanced data, accurately identifies various fault types, and precisely discerns the corresponding time windows of fault occurrences. The diagnostic performance significantly outperforms models obtained using commonly used oversampling methods applied to handle data imbalance.

Conventional Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a well-labeled source domain to an unlabeled target domain only when data from both domains is simultaneously accessible, which is challenged by the recent Source-free Domain Adaptation (SFDA). However, we notice that the performance of existing SFDA methods would be dramatically degraded by intra-domain class imbalance and inter-domain label shift. Unfortunately, class-imbalance is a common phenomenon in real-world domain adaptation applications. To address this issue, we present Imbalanced Source-free Domain Adaptation (ISFDA) in this paper. Specifically, we first train a uniformed model from the source domain, and then propose secondary label correction, curriculum sampling, plus intra-class tightening and inter-class separation to overcome the joint presence of covariate shift and label shift. Extensive experiments on three imbalanced benchmarks verify that ISFDA could perform favorably against existing UDA and SFDA methods under various conditions of class-imbalance, and outperform existing SFDA methods by over 15% in terms of per-class average accuracy on a large-scale long-tailed imbalanced dataset.

No abstract available

No abstract available

No abstract available

Traditional machine learning methods rely on the training data and target data having the same feature space and data distribution. The performance may be unacceptable if there is a difference in data distribution between the training and target data, which is called cross-domain learning problem. In recent years, many domain adaptation methods have been proposed to solve this kind of problems and make much progress. However, existing domain adaptation approaches have a common assumption that the number of the data in source domain (labeled data) and target domain (unlabeled data) is matched. In this paper, the scenarios in real manufacturing site are considered, that the target domain data is much less than source domain data at the beginning, but the number of target domain data will increase as time goes by. A novel method is proposed for fault diagnosis of rolling bearing with online imbalanced cross-domain data. Finally, the proposed method which is tested on bearing dataset (CWRU) has achieved prediction accuracy of 95.89% with only 40 target samples. The results have been compared with other traditional methods. The comparisons show that the proposed online domain adaptation fault diagnosis method has achieved significant improvements. In addition, the deep transfer learning model by adaptive- network-based fuzzy inference system (ANFIS) is introduced to interpretation the results.

Existing source-free unsupervised domain adaptation uses the source model and unlabeled target data to train the target model under the assumption that the dataset is relatively balanced. However, in many practical applications, the class distribution is often skewed. Existing works tackle this problem mainly by oversampling the minority classes or undersampling the majority classes, which are not applicable when data are unlabeled. To address these challenges, we estimate class distribution by summing over model prediction probabilities to more flexibly tackle the uncertainty in an imbalanced dataset and employ information entropy of the classification results to select reliable balancing samples. Furthermore, since information entropy of the classification results tends to decrease as the model iterates in training, we propose Dynamic Entropy-based Balance (DEB) to more precisely determine the threshold for reliable balancing sample selection. Experimental results on several real datasets validate the effectiveness of our method.

Sonar image classification is challenging due to the limited availability and long-tail distribution of labeled sonar samples. In this work, a Feature Contrastive Transfer Learning (FCTL) framework is proposed for few-shot long-tailed sonar image classification. The proposed framework combines transfer learning and contrastive learning to improve model performance under limited labeled data. First, a deep convolutional neural network (CNN) is pre-trained on a large-scale image dataset to learn general feature representations. Then, contrastive learning is employed to maximize the similarity between positive sample pairs and minimize the similarity between positive and negative sample pairs. Specifically, positive samples are generated through a Gaussian feature enhancement method, while the remaining samples in a batch are negative. In addition, a balanced sampling strategy is employed to optimize the unbalanced feature distribution of long-tailed samples. Experiments on two different sonar image datasets demonstrate that the FCTL framework outperforms existing methods in few-shot long-tailed sonar image classification tasks.

Since the actual fault samples of electro-hydrostatic actuators are difficult to obtain and the number of fault samples is small, an effective and sufficient simulation fault data set is obtained by constructing a simulation model of an electro-hydrostatic actuator, the method of migration component analysis is used to reduce the difference of data distribution. The fault feature knowledge is learned from the simulation data set through the deep migration network and migrated to the actual fault data to solve the problem of fault classification under the condition of small samples and scarce fault data. The experimental results show that the algorithm can construct the network model based on many sample data, through the transfer learning method, the learning task of the model is significantly reduced, and the diagnosis accuracy and robustness of the model are improved. It can effectively solve the fault diagnosis problem in the case of sparse fault data and unbalanced distribution.

The transient stability assessment based on machine learning faces challenges such as sample data imbalance and poor generalization. To address these problems, this paper proposes an intelligent enhancement method for real-time adaptive assessment of transient stability. In the offline phase, a convolutional neural network (CNN) is used as the base classifier. A model training method based on contrastive learning is introduced, aiming to increase the spatial distance between positive and negative samples in the mapping space. This approach effectively improves the accuracy of the model in recognizing unbalanced samples. In the online phase, when real data with different distribution characteristics from the offline data are encountered, an active transfer strategy is employed to update the model. New system samples are obtained through instance transfer from the original system, and an active sampling strategy considering uncertainty is designed to continuously select high-value samples from the new system for labeling. The model parameters are then updated by fine-tuning. This approach drastically reduces the cost of updating while improving the model’s adaptability. Experiments on the IEEE39-node system verify the effectiveness of the proposed method.

Bearing fault diagnosis plays a pivotal role in the safe and reliable operation of modern mechanical systems. However, the existing fault diagnosis methods rarely deal with the problem of category imbalance and various speeds concurrently, which cannot work effectively in practical scenarios. Considering the underlying similarities of data in frequency domain, data mining under various speeds can help to reduce the deviation of domain distribution. Therefore, a novel fault diagnosis method based on spectrum alignment (SA) and deep transfer convolution neural network (DTCNN) is proposed, where the SA and data augmentation module are designed to extract SA features from the unbalanced bearing data. The DTCNN model based on joint distribution adaptation is built to facilitate learning reliable domain-invariant features. Different from the existing studies, a more general transfer task with time-varying speed is considered, even with complex faults. For 14 transfer tasks in two unbalanced fault diagnosis cases under variable speed, the average accuracy, F1-score, and area under curve of the proposed method can reach more than 97.76%, 97.57%, and 98.75%, respectively. The results show that this method has superior diagnostic effect and better generalization ability than various state-of-the-art methods.

Modern data analytics take advantage of ensemble learning and transfer learning approaches to tackle some of the most relevant issues in data analysis, such as lack of labeled data to use to train the analysis models, sparsity of the information, and unbalanced distributions of the records. Nonetheless, when applied to multimodal datasets (i.e., datasets acquired by means of multiple sensing techniques or strategies), the state-of-theart methods for ensemble learning and transfer learning might show some limitations. In fact, in multimodal data analysis, not all observations would show the same level of reliability or information quality, nor an homogeneous distribution of errors and uncertainties. This condition might undermine the classic assumptions ensemble learning and transfer learning methods rely on. In this work, we propose an adaptive approach for dimensionality reduction to overcome this issue. By means of a graph theory-based approach, the most relevant features across variable size subsets of the considered datasets are identified. This information is then used to set-up ensemble learning and transfer learning architectures. We test our approach on multimodal datasets acquired in diverse research fields (remote sensing, brain-computer interfaces, photovoltaic energy). Experimental results show the validity and the robustness of our approach, able to outperform state-of-the-art techniques.

No abstract available

AI algorithms have become valuable in aiding professionals in healthcare. The increasing confidence obtained by these models is helpful in critical decision demands. In clinical dermatology, classification models can detect malignant lesions on patients' skin using only RGB images as input. However, most learning-based methods employ data acquired from dermoscopic datasets on training, which are large and validated by a gold standard. Clinical models aim to deal with classification on users' smartphone cameras that do not contain the corresponding resolution provided by dermoscopy. Also, clinical applications bring new challenges. It can contain captures from uncontrolled environments, skin tone variations, viewpoint changes, noises in data and labels, and unbalanced classes. A possible alternative would be to use transfer learning to deal with the clinical images. However, as the number of samples is low, it can cause degradations on the model's performance; the source distribution used in training differs from the test set. This work aims to evaluate the gap between dermoscopic and clinical samples and understand how the dataset variations impact training. It assesses the main differences between distributions that disturb the model's prediction. Finally, from experiments on different architectures, we argue how to combine the data from divergent distributions, decreasing the impact on the model's final accuracy.

Rolling bearings are an essential core component in the power transmission systems of rotating machinery. Many previous dictionary learning (DL) methods have proven to be powerful for rolling bearing fault diagnosis. Existing diagnostic methods based on DL usually involve analyzing training and test data drawn from the same distribution. However, an unbalanced data distribution is a common phenomenon in practice. In addition, the signals in the source and target domains are often mixed with noise and irrelevant information in complex environments, which greatly reduces the performance of DL. Therefore, a new DL method based on a mixed noise model for transfer DL (TDL-MN) is presented. Specifically, the TDL-MN model is constructed to overcome noise and irrelevant signal interference in the DL process under variable conditions. Moreover, rolling bearing health status is diagnosed through sparse representation (SR) classification by calculating the target domain samples corresponding to the redundancy error in the SR vector. The results for two cases verify the ability of the TDL-MN method to accurately identify bearing fault types under complex and variable working conditions. Comparisons with other methods verify the applicability and superiority of the proposed method.

Due to the strict observation conditions and special target attributes, inverse synthetic aperture radar (ISAR) may suffer with insufficient number of images for certain space targets, which leads to a considerable decline in the recognition performance. In this article, we propose a robust space target recognition method for sequence ISAR images based on feature distribution transfer learning. To obtain deformation robust sequential features, a sequence homography network is first proposed and trained by semi-supervised learning. Then the extracted embedding features are aligned and transferred to the class label domain by optimal transport mapping. Target recognition experiments on a few-shot satellite data set illustrate that the proposed method has higher average accuracy and better robustness for scaled, rotated, and combined image deformation.

In the actual working environment, mechanical equipment is in normal operation for a long time, which leads to unbalanced sample data collected and will be accompanied by problems such as changes in working conditions. All these issues contribute to problems such as the low accuracy of intelligent fault diagnosis models. In this regard, this article proposes a new method for fault diagnosis based on hybrid data-assisted multisource domain transfer learning under unbalanced conditions. First, fault signals simulated and real vibration signals are converted into 2-D images using the Gram angle product field (GAPF), and the conditional Wasserstein generative adversarial network with gradient penalty (CWGAN-GP) is used to augment the real data of the source domain to get high-quality, usable samples. Second, the mixed data consisting of these three are input into a residual network with receptive field attention–graph convolutional networks (ResRFA-GCN) common feature extractor as source domain samples and target domain samples to train the model and the extraction of common features. Next, the features extracted from each pair of sources and target domains are mapped into a specific space, the local maximum mean discrepancy (LMMD) is employed to minimize the difference in the distribution of the domains, and the module allows it to learn the domain-invariant properties between each pair of source and target domain subclasses. Finally, after realizing the classification in a specific classifier. The model is tested on two datasets, and satisfactory results are achieved, thus reducing the dependence of the intelligent fault diagnosis model on measured data of fault samples.

In gearbox fault diagnosis, the performance of diagnostic accuracy, generalization capability in data-driven diagnostic models is greatly impacted due to imbalanced small datasets. Besides, significant time investment for hyperparameters tuning of diagnostic model makes it difficult for classifier to obtain an accurate identification. To tackle the above issues, in this article, a novel method named cross-domain transfer learning with fine-tuning mechanism (CTL-FTM) is proposed for fault diagnosis of gearbox using unbalanced samples, and the fault features can be extracted with pretraining model and fault types can be recognized with shallow network. Two fault datasets of gearbox are employed to verify the effectiveness of the proposed algorithm. Experimental results demonstrate that the proposed CTL-FTM algorithm is superior than state-of-the-art benchmark transfer learning methods such as pure VGG16 and VGG16 with attention mechanism models, in terms of convergence speed, recognition accuracy, loss rates, time consumption, and generalization ability.

Transfer learning generally addresses to reduce the distribution distance between source-domain and target-domain. However, it is unreasonable to use a distribution to represent the life-cycle signals as they are always time-varying, and the improper assumption affects the efficacy of transfer remaining useful life (RUL) prediction. To fill this gap, this research proposes a micro transfer learning mechanism for multiple differentiated distributions, and a transfer RUL prediction model is constructed. First, a multi-cellular long short-term memory (MCLSTM) neural network is applied to obtain multiple differentiated distributions of the monitoring data at some point. Then the domain adversarial mechanism is used to achieve the knowledge transfer of multiple differentiated distributions at the cell level. Furthermore, an active screen mechanism is designed for weighting the domain discrimination losses of multiple differentiated distributions. Through the transfer RUL prediction experiments on aero-engines and actual wind turbine gearboxes, the superiority of this model over the advanced transfer prediction models is verified. Note to Practitioners—The work is motivated by the accuracy reduction problem caused by the time-varying characteristics of life-cycle data in the cross domain equipment RUL prediction scenario, where a fixed single distribution is difficult to cover the full life-cycle data. This article proposes a micro transfer learning mechanism containing multiple differentiated distributions, and a novel transfer RUL prediction model based on the mechanism is constructed for solving the problem caused by the time-varying characteristics of life-cycle data. There are four steps for implementing this method in practice: 1) collecting the full-life cycle signals of historical equipment; 2) modeling the degradation curves of equipment by MCLSTM; 3) solving the cross domain RUL prediction by narrowing the distributions of degradation curves by the micro transfer learning mechanism; and 4) making prognostics for new equipment. The novelty is that the proposed mechanism can self-adaptively align multiple differentiated subspaces of the source domain and the target domain, that is, it can adaptively extract the domain invariant features over time. As a result, the proposed method has two main advantages: 1) capable of characterizing the degradation processes of different equipment; and 2) superior prognostic results on cross domain RUL prediction.

No abstract available

The problem of different characters of heterogeneous synthetic aperture radar (SAR) images leads to poor performances for transfer learning of SAR image classification. To address this issue, a semisupervised model named as deep joint distribution adaptation networks (DJDANs) is proposed for transfer learning from a source SAR image to a different but similar target SAR image, which aims to match the joint probability distributions between the source domain and target domain. In the proposed DJDAN, a marginal distribution adaptation network is developed to map features across the domains into an augmented common feature subspace, which aims to match the marginal probability distributions and unify the dimensions. Then, a conditional distribution adaptation network is proposed to transfer knowledge across the domains, which aims to reduce the discrepancies of the conditional probability distributions and enhance the effectiveness of feature representation. Moreover, one-versus-rest classification is utilized in the proposed framework, which aims to improve the discrimination between the inside and outside class. Experimental results demonstrate the effectiveness of the proposed deep networks.

The deep learning-based fault diagnosis approaches have shown great advantages in ensuring rotating machinery (RM) work normally and safely. However, in real industrial applications, due to the influence of speed fluctuation, the differences in distribution of training samples and testing samples are inevitable, which will greatly affect the diagnosis result of the model. In the article, a novel end-to-end method for the fault transfer diagnosis of RM under time-varying speeds, named semisupervised subdomain adaptation graph convolutional network (SSAGCN) is proposed by integrating SSA and GCN. To begin with, a feature extractor based on GCN is designed to obtain the transferable information of the source domain (SD) and target domain (TD) data. In addition, the closely watched fault diagnosis (FD) approach based on global domain shift and unsupervised domain adaptation is improved by employing an adaptive layer based on SSA to reduce the distribution difference of the same fault type in SD and TD. The proposed SSAGCN approach takes full advantage of the powerful capability of GCN in capturing the relationship between signals, and the excellent performance of SSA in the use of unlabeled samples and idle labeled samples, thus overcoming the distribution differences caused by speed fluctuation. Two experimental cases are carried out to prove its effectiveness under time-varying speeds, and their results indicate our presented SSAGCN approach can realize more excellent performance on diagnosis accuracy and model complexity compared with existing methods.

No abstract available

With the increasing number of hyperspectral remote sensing images, it is of great research significance to utilize labeled source domain images to achieve accurate classification of unlabeled target domain images quickly. However, due to variations in atmospheric conditions, lighting conditions, temporal factors, and spatial morphologies of materials within the same object category, the spectral shift frequently occurs, leading to significant data distribution differences between the source and target domains. This phenomenon hinders the adaptability of the classification model trained on source domain data to the target domain data, thereby seriously affecting the effectiveness of domain adaptation methods. To address the issue of significant distribution differences and lack of labels between domains, this paper proposes a hyperspectral image classification method based on attention subdomain adaptation(ASDA). Firstly, a spatial-spectral feature extractor is used to extract features, aiming to make full use of the information in HSI. Then, a attention-based conditional distribution alignment method is proposed to align the features of interest, which can alleviate the negative impact of redundant pixels and redundant bands on subdomain alignment. Specifically, spatial attention is used to remove the influence of spatial interference pixels, while channel attention is employed to eliminate redundant bands in the spectrum. Finally, the subdomains are aligned using the Local Maximum Mean Discrepancy (LMMD) criterion, which focuses on aligning more effective features of the attention network. This alignment enables accurate classification of the target domain data. The effectiveness of this method is validated on two datasets.

The subdomain adaptation (SA) based intelligent cross-domain fault diagnosis methods aim to reduce the conditional distribution shift caused by variable working conditions. However, existing SA methods may be limited by the quality of pseudolabels, since misclassified pseudolabels will lead to alignment between irrelevant subdomains, resulting in erroneous category-invariant knowledge being accumulated. To tackle this, we present a dynamic subdomain pseudolabel correction and adaptation (DSPC-A) framework. Specifically, we propose an end-to-end pseudolabel correction algorithm, which integrates an auxiliary network to learn clean and general target label distribution from noisy pseudolabels. So that, the auxiliary network can guide the SA model to perform precise subdomain alignment using learned label distribution. Moreover, to allow the synergy training of the additional auxiliary network and SA model, we introduce an iterative learning strategy to dynamically perform pseudolabel correction and subdomain alignment. The iterative training makes two models complement each other, thus achieving better SA ability and diagnosis performance. The DSPC-A framework has been thoroughly verified under three fault diagnostic scenarios: cross load, cross fault severity, and cross mechanical equipment. Case study results demonstrate the superiority of the DSPC-A, which improves the SA performance by solely implementing simple pseudolabel correction methods without other complex techniques.

In the electroencephalography (EEG) based cross-subject motor imagery (MI) classification task, the device and subject problems can cause the time-related data distribution shift problem. In a single-source to single-target (STS) MI classification task, such a shift problem will certainly provoke an increase in the overall data distribution difference between the source and target domains, giving rise to poor classification accuracy. In this paper, a novel multi-subdomain adaptation method (MSDAN) is proposed to solve the shift problem and improve the classification accuracy of the traditional approaches. In the proposed MSDAN, the adaptation losses in both class-related and time-related subdomains (that are divided by different data labels and session labels) are obtained by measuring the distribution differences between the source and target subdomains. Then, the adaptation and classification losses in the loss function of MSDAN are minimized concurrently. To illustrate the application value of the proposed method, our method is applied to solve the STS MI classification task about data analysis with respect to the brain-computer interface (BCI) competition III-IVa dataset. The resultant experiment results demonstrate that compared with other well-known domain adaptation and deep learning methods, the proposed method is capable of solving the time-related data distribution problem at higher classification accuracy.

No abstract available

Affected by the sensor, shooting environment, and other aspects, hyperspectral images (HSIs) in the source and target domains exhibit phenomenon of difficult feature extraction and domain shift. The above phenomena pose challenges to the cross-scene HSI classification task. Therefore, a focal transfer graph network (FTGN) for cross-scene HSI classification is proposed. First, FTGN leverages graph sample and aggregate (GraphSAGE) to capture spatial–spectral features by aggregating partial adjacency nodes, ensuring the acquisition of contextual information. The neighbor weighting strategy based on spatial–spectral information is proposed to solve the information interference caused by excessive node aggregation. Second, a pseudolabel trimming strategy based on class metrics is proposed to reduce the adverse effects of pseudolabel noise in the transfer process. Then, a specification subdomain adaptation (SSA) module is proposed, which helps to achieve effective distribution alignment by reducing the feature distance of intraclass samples and widening the feature distance of interclass samples during the subdomain adaptation process. Finally, the focal loss is utilized to help FTGN focus on hard-to-classify samples. The experimental results on eight data pairs show that the proposed method outperforms several state-of-the-art methods.

EEG emotion recognition has been hampered by the clear individual differences in the electroencephalogram (EEG). Nowadays, domain adaptation is a good way to deal with this issue because it aligns the distribution of data across subjects. However, the performance for EEG emotion recognition is limited by the existing research, which mainly focuses on the global alignment between the source domain and the target domain and ignores much fine-grained information. In this study, we propose a method called Graph-based Unsupervised Subdomain Adaptation (Gusa), which simultaneously aligns the distribution between the source and target domains in a fine-grained way from both the channel and emotion subdomains. Gusa employs three modules, such as the Node-wise Domain Constraints Module to align each EEG channel and obtain a domain-variant representation, the Class-level Distribution Constraints Module, and the Emotion-wise Domain Constraints Module, to collect more fine-grained information, create more discriminative representations for each emotion, and lessen the impact of noisy emotion labels. The studies on the SEED, SEED-IV, and MPED datasets demonstrate that Gusa significantly improves the ability of EEG to recognize emotions and can extract more granular and discriminative representations for EEG.

No abstract available

Surface electromyography (sEMG)-based hand gesture recognition has garnered widespread attention in rehabilitation robotics due to its noninvasive measurement and intuitive motion decoding. However, affected by various factors such as individual differences, achieving cross-user adaptability and long-term reliability for sEMG classification poses a significant challenge. Existing domain adaptation (DA) methods primarily focus on global distribution alignment to mitigate statistical distribution discrepancies across domains, yielding significant achievements. Nevertheless, these methods often overlook fine-grained category-level subdomain distribution alignment, leading to discriminative structure confusion and subdomain misalignment, which hinder cross-subject and cross-session gesture recognition. To tackle these issues, this article proposes a plug-and-play subdomain adaptation method called PPSDA to enhance the classification performance and generalization ability for gesture recognition across domains. Specifically, handcrafted features are extracted and utilized for source domain supervised training to preserve discriminative structures. Subsequently, source and target domains co-training is performed, wherein the local maximum mean discrepancy (LMMD) is minimized to capture fine-grained information on relevant subdomains for subdomain distribution alignment. To validate the performance of the proposed PPSDA, we recruited 12 healthy subjects and developed an sEMG-driven exoskeleton rehabilitation glove for cross-subject and cross-session evaluations. Extensive experimental results demonstrate the effectiveness and superiority of the proposed PPSDA over existing DA approaches.

No abstract available

Unsupervised domain adaptation (UDA) has demonstrated significant success in intelligent mechanical fault diagnosis. However, these methods primarily focus on transferring knowledge from laboratory test rigs or physical entities, which can be costly or unavailable in certain occasions. Moreover, most methods emphasize global distribution alignment while neglecting fine-grained distribution alignment in knowledge transfer, which can lead to misclassification of certain fault categories and subsequently decrease the diagnostic accuracy of the deep model. In response to these issues, a novel simulation-to-real transfer-based deep unsupervised subdomain adaptation method (Sim2Real-DUSDA) is proposed. Taking bearing-fault diagnosis as an example, a high-fidelity 4-DOF dynamic model of rolling bearings is constructed to generate simulation data across various health states and multiple working conditions. The local maximum mean discrepancy (LMMD) is incorporated to achieve a fine-grained distribution alignment between the source and target domains. A deep unsupervised subdomain adaptation transfer learning network is designed to learn knowledge from the sufficiently labeled simulation data and realize fault mode recognition on the unlabeled measured data. Experimental and empirical analyses on three cross-domain diagnostic tasks demonstrate the effectiveness and superiority of the proposed method.

The reliable operation of rolling bearings has a direct impact on the performance and reliability of the mechanical systems in which they are installed. Thus, accurate fault detection of these bearings is a critical requirement for ensuring stable system operations. This study addresses the limitations of single-scale models in fully extracting comprehensive fault features from bearing signals and the subdomain boundary discrepancies created by traditional global domain adaptation techniques. Specifically, this paper proposes a deep transfer learning model based on a multi-scale residual network (ResNet) with Laplace wavelet convolutional layers combined with a local maximum mean discrepancy (MMD) method. The multi-scale ResNet module leverages Laplace wavelet convolutional layers to comprehensively extract deep, multi-scale fault features from both the source and target domains. Concurrently, we employed the local MMD strategy to quantify the feature distribution differences between subdomains, allowing more precise feature alignment and effective adaptation at the subdomain level. Experiments conducted on two publicly available datasets validate that the proposed model demonstrates robust competence and stability in fault recognition tasks.

Bearing fault diagnosis under varying working conditions faces challenges, including a lack of labeled data, distribution discrepancies, and resource constraints. To address these issues, we propose a progressive knowledge distillation framework that transfers knowledge from a complex teacher model, utilizing a Graph Convolutional Network (GCN) with Autoregressive moving average (ARMA) filters, to a compact and efficient student model. To mitigate distribution discrepancies and labeling uncertainty, we introduce Enhanced Local Maximum Mean Squared Discrepancy (ELMMSD), which leverages mean and variance statistics in the Reproducing Kernel Hilbert Space (RKHS) and incorporates a priori probability distributions between labels. This approach increases the distance between clustering centers, bridges subdomain gaps, and enhances subdomain alignment reliability. Experimental results on benchmark datasets (CWRU and JNU) demonstrate that the proposed method achieves superior diagnostic accuracy while significantly reducing computational costs. Comprehensive ablation studies validate the effectiveness of each component, highlighting the robustness and adaptability of the approach across diverse working conditions.

We consider the problem of active domain adaptation (ADA) to unlabeled target data, of which subset is actively selected and labeled given a budget constraint. Inspired by recent analysis on a critical issue from label distribution mismatch between source and target in domain adaptation, we devise a method that addresses the issue for the first time in ADA. At its heart lies a novel sampling strategy, which seeks target data that best approximate the entire target distribution as well as being representative, diverse, and uncertain. The sampled target data are then used not only for supervised learning but also for matching label distributions of source and target domains, leading to remarkable performance improvement. On four public benchmarks, our method substantially outperforms existing methods in every adaptation scenario.

Objective. Extracting discriminative spatial information from multiple electrodes is a crucial and challenging problem for electroencephalogram (EEG)-based emotion recognition. Additionally, the domain shift caused by the individual differences degrades the performance of cross-subject EEG classification. Approach. To deal with the above problems, we propose the cerebral asymmetry representation learning-based deep subdomain adaptation network (CARL-DSAN) to enhance cross-subject EEG-based emotion recognition. Specifically, the CARL module is inspired by the neuroscience findings that asymmetrical activations of the left and right brain hemispheres occur during cognitive and affective processes. In the CARL module, we introduce a novel two-step strategy for extracting discriminative features through intra-hemisphere spatial learning and asymmetry representation learning. Moreover, the transformer encoders within the CARL module can emphasize the contributive electrodes and electrode pairs. Subsequently, the DSAN module, known for its superior performance over global domain adaptation, is adopted to mitigate domain shift and further improve the cross-subject performance by aligning relevant subdomains that share the same class samples. Main Results. To validate the effectiveness of the CARL-DSAN, we conduct subject-independent experiments on the DEAP database, achieving accuracies of 68.67% and 67.11% for arousal and valence classification, respectively, and corresponding accuracies of 67.70% and 67.18% on the MAHNOB-HCI database. Significance. The results demonstrate that CARL-DSAN can achieve an outstanding cross-subject performance in both arousal and valence classification.

In real-world scenarios, tabular data often suffer from distribution shifts that threaten the performance of machine learning models. Despite its prevalence and importance, handling distribution shifts in the tabular domain remains underexplored due to the inherent challenges within the tabular data itself. In this sense, test-time adaptation (TTA) offers a promising solution by adapting models to target data without accessing source data, crucial for privacy-sensitive tabular domains. However, existing TTA methods either 1) overlook the nature of tabular distribution shifts, often involving label distribution shifts, or 2) impose architectural constraints on the model, leading to a lack of applicability. To this end, we propose AdapTable, a novel TTA framework for tabular data. AdapTable operates in two stages: 1) calibrating model predictions using a shift-aware uncertainty calibrator, and 2) adjusting these predictions to match the target label distribution with a label distribution handler. We validate the effectiveness of AdapTable through theoretical analysis and extensive experiments on various distribution shift scenarios. Our results demonstrate AdapTable's ability to handle various real-world distribution shifts, achieving up to a 16% improvement on the HELOC dataset.

How can we address distribution shifts in stock price data to improve stock price prediction accuracy? Stock price prediction has attracted attention from both academia and industry, driven by its potential to uncover complex market patterns and enhance decisionmaking. However, existing methods often fail to handle distribution shifts effectively, focusing on scaling or representation adaptation without fully addressing distributional discrepancies and shape misalignments between training and test data. We propose ReVol (Return-Volatility Normalization for Mitigating Distribution Shift in Stock Price Data), a robust method for stock price prediction that explicitly addresses the distribution shift problem. ReVol leverages three key strategies to mitigate these shifts: (1) normalizing price features to remove sample-specific characteristics, including return, volatility, and price scale, (2) employing an attention-based module to estimate these characteristics accurately, thereby reducing the influence of market anomalies, and (3) reintegrating the sample characteristics into the predictive process, restoring the traits lost during normalization. Additionally, ReVol combines geometric Brownian motion for long-term trend modeling with neural networks for short-term pattern recognition, unifying their complementary strengths. Extensive experiments on real-world datasets demonstrate that ReVol enhances the performance of the state-of-the-art backbone models in most cases, achieving an average improvement of more than 0.03 in IC and over 0.7 in SR across various settings.

Anomaly detection plays a crucial role in quality control for industrial applications. However, ensuring robustness under unseen domain shifts such as lighting variations or sensor drift remains a significant challenge. Existing methods attempt to address domain shifts by training generalizable models but often rely on prior knowledge of target distributions and can hardly generalise to backbones designed for other data modalities. To overcome these limitations, we build upon memory-bank-based anomaly detection methods, optimizing a robust Sinkhorn distance on limited target training data to enhance generalization to unseen target domains. We evaluate the effectiveness on both 2D and 3D anomaly detection benchmarks with simulated distribution shifts. Our proposed method demonstrates superior results compared with state-of-the-art anomaly detection and domain adaptation methods.

The brake pad plays a crucial role in the operation of high-speed train brake systems. However, it is prone to abnormal wear over prolonged periods of service, posing a safety risk to train operations. High-speed trains operate in environments with varying rotational speeds, which leads to features change of brake interface signals. Consequently, monitoring the state of brake systems across different rotation speeds becomes a challenging cross-domain recognition problem. To solve the substantial differences in vibration signals of brake pads at different rotation speeds, we propose a guided subdomain adaptation network for cross-speed state monitoring of brakes. By minimizing the distribution gaps between relevant subdomains, our network effectively aligns the feature distributions in different rotation speeds of brake pad. Besides, a training guidance mechanism is designed to significantly reduce the misclassification of pseudolabels for target signals, facilitating more accurate identification of the appropriate common subspace for consistent-labeled subdomains. We validate the practicality of our approach under balanced datasets, small sample conditions, and noisy environments. The experimental results demonstrate that the proposed method can achieve cross-rotation speed state monitoring of high-speed train brake systems even with minimal labeled samples and exhibit superior stability and accuracy compared to other transfer learning algorithms in cross-domain state monitoring.

Subdomain adaptation plays a significant role in the field of bearing fault diagnosis. It effectively aligns the pertinent distributions across subdomains and addresses the frequent issue of lacking local category information in domain adaptation. Nonetheless, this approach overlooks the quantitative discrepancies in distribution between samples from the source and target domains, leading to the vanishing gradient issue during the training of models. To tackle this challenge, this paper proposes a bearing fault diagnosis method based on Wasserstein metric residual adversarial subdomain adaptation. The Wasserstein metric is introduced as the optimized objective function of the domain discriminator in RASAN-W. The distribution discrepancy between the source domain and target domain samples is quantitatively measured, achieving the alignment of the relevant subdomain distributions between the source domain and the target domain. Ultimately, extensive experiments conducted on two real-world datasets reveal that the diagnostic accuracy of this method is significantly enhanced when compared to various leading bearing fault diagnosis techniques.

Marine engines have a high risk of failure during navigation and face issues such as varying operating conditions and complex faults. Existing research mainly focuses on fault diagnosis of marine engines under single operating conditions, neglecting the applicability of these methods under other conditions. To address this issue, this paper employs domain adaptation techniques to study cross-condition fault diagnosis of marine engines. Firstly, a self-calibrating convolutional neural network is used to extract health state features from engine operation data. Meanwhile, by integrating adversarial and subdomain adaptation techniques, the distribution differences in global domains and local subdomains under different operating conditions are reduced, thereby improving the fault diagnosis accuracy under variable conditions. Finally, the effectiveness of the diagnostic model is validated based on simulated marine engine fault data. For the designed one-to-one and one-to-many condition transfer tasks, the average fault diagnosis accuracy reaches approximately 90%, achieving higher recognition accuracy compared to other methods. The research results can provide theoretical reference for fault diagnosis of marine engines under variable operating conditions.

In the condition monitoring of rotating systems, overfitting is a common challenge due to limited data history, which reduces the effectiveness of fault detection frameworks; this limitation often leads to unreliable diagnostics, resulting in unexpected machine failures and increased operational costs in industrial applications. Advances in deep learning suggest using simulated data to address this issue, but operational variabilities still cause significant data distribution shifts, affecting model accuracy. This article presents a new vibration-based monitoring framework that improves fault detection in rotary machines by effectively managing these shifts. It features a novel fine-tuning approach within sequential domain adaptation, requiring only a limited number of observations from the target domain for accurate model adjustment. The domain adaptation process is elucidated through a novel visualization of internal activation patterns within the sequential network. This method is further enhanced by a hybrid algorithm that combines wavelet transformation, a multi-layer perceptron, and a transformer encoder, followed by domain-specific fine-tuning. The framework’s effectiveness is demonstrated through experimental data from two different rotor systems, validated by sensitivity and comparative analyses, highlighting its robustness, generalizability, and practical applicability as a baseline in industrial fault detection scenarios.

Transfer learning-based methods for the remaining useful life (RUL) prediction of bearings require accessing target domains at model training stages, which limit the practical value as many real-world cases are with totally unseen domains. In addition, the life-stage domain shift within time-series data is rarely considered in tackling the unseen domain problems. To this end, this study is motivated to develop a novel network model possessing the awareness of the life-stage domain shift information to overcome challenges induced by unseen working conditions and intradomain distribution shifts. The development of the resulting model, the life-stage domain aware network, is composed of two parts. In the first part, an unsupervised learning scheme is proposed for handling the life-stage division via sensing intradomain distribution shifts. In the second part, a domain aware network tailored for life-stages is developed to build a shared domain-invariant latent space through the subdomain alignment at each stage. A preliminary theoretical analysis is conducted to show that invariant features can be learned under the proposed learning framework. The superiority of the RUL prediction model developed by the proposed method is validated through comparisons with the state-of-the-art methods on different bearing datasets.

For a target task where the labeled data are unavailable, domain adaptation can transfer a learner from a different source domain. Previous deep domain adaptation methods mainly learn a global domain shift, i.e., align the global source and target distributions without considering the relationships between two subdomains within the same category of different domains, leading to unsatisfying transfer learning performance without capturing the fine-grained information. Recently, more and more researchers pay attention to subdomain adaptation that focuses on accurately aligning the distributions of the relevant subdomains. However, most of them are adversarial methods that contain several loss functions and converge slowly. Based on this, we present a deep subdomain adaptation network (DSAN) that learns a transfer network by aligning the relevant subdomain distributions of domain-specific layer activations across different domains based on a local maximum mean discrepancy (LMMD). Our DSAN is very simple but effective, which does not need adversarial training and converges fast. The adaptation can be achieved easily with most feedforward network models by extending them with LMMD loss, which can be trained efficiently via backpropagation. Experiments demonstrate that DSAN can achieve remarkable results on both object recognition tasks and digit classification tasks. Our code will be available at https://github.com/easezyc/deep-transfer-learning.

Multi-source domain adaptation (MSDA) in remote sensing (RS) scene classification has recently gained significant attention in the visual recognition community. It leverages multiple well-labeled source domains to train a model capable of achieving strong generalization on the target domain with little to no labeled data from the target domain. However, the distribution shifts among multiple source domains make it more challenging to align the distributions between the target domain and all source domains concurrently. Moreover, relying solely on global alignment risks losing fine-grained information for each class, especially in the task of RS scene classification. To alleviate these issues, we present a Multi-Source Subdomain Distribution Alignment Network (MSSDANet), which introduces novel network structures and loss functions for subdomain-oriented MSDA. By adopting a two-level feature extraction strategy, this model attains better global alignment between the target domain and multiple source domains, as well as alignment at the subdomain level. First, it includes a pre-trained convolutional neural network (CNN) as a common feature extractor to fully exploit the shared invariant features across one target and multiple source domains. Secondly, a dual-domain feature extractor is used after the common feature extractor, which maps the data from each pair of target and source domains to a specific dual-domain feature space and performs subdomain alignment. Finally, a dual-domain feature classifier is employed to make predictions by averaging the outputs from multiple classifiers. Accompanied by the above network, two novel loss functions are proposed to boost the classification performance. Discriminant Semantic Transfer (DST) loss is exploited to force the model to effectively extract semantic information among target and source domain samples, while Class Correlation (CC) loss is introduced to reduce the feature confusion from different classes within the target domain. It is noteworthy that our MSSDANet is developed in an unsupervised manner for domain adaptation, indicating that no label information from the target domain is required during training. Extensive experiments on four common RS image datasets demonstrate that the proposed method achieves state-of-the-art performance for cross-domain RS scene classification. Specifically, in the dual-source and three-source settings, MSSDANet outperforms the second-best algorithm in terms of overall accuracy (OA) by 2.2% and 1.6%, respectively.

Many domain adaptation models have been explored for fault transfer diagnosis. However, most of them only consider the global domain adaptation of two domains while neglecting the fine-grained class-wise distribution alignment between the source and target domains. Thus, these models cannot satisfy the diagnostic requirement in some cases. In this article, a new ensemble weighting subdomain adaptation network (EWSAN) diagnostic model is established to improve the degree of domain confusion. In EWSAN, an enhanced joint distribution alignment (EJDA) mechanism is proposed. A multiscale top classifier with multiple diverse branches is designed based on ensemble learning to better achieve EJDA. Ensemble voting with the multiscale top classifier can obtain more reliable pseudolabels in the EJDA mechanism. An ensemble weighting maximum mean discrepancy with the class weight is constructed to enhance the fine-grained domain confusion. Moreover, the closed and partial transfer diagnostic tasks are made available. Furthermore, the information entropy is introduced to increase the confidence coefficient of the pseudo label. The proposed EWSAN diagnostic model is evaluated via multiple closed and partial fault transfer diagnosis experiments cross machines. The experimental results validate its effectiveness and superiority.

No abstract available

An innovative class subdomain adaptation network (CSAN) is proposed in response to the problem that the distribution of bearing data is inconsistent under variable working conditions, and the network trained on data from the source working condition cannot be applied to the target working condition. The proposed CSAN model is divided into two components. The first component is a lightweight channel convolution neural network (CCNN) designed to fully utilize channel information and perform feature extraction and classification. The second component is an innovative domain adaptation algorithm that can be used as a loss function embedded in a deep learning network. The embedded domain adaptation loss function consists of two terms. The first is a class-domain loss term, which achieves domain adaptation by measuring the correlation alignment (CORAL) distance between the source domain and target domain in each subdomain divided by category. The second term is a class-margin loss term. Through the idea of maximizing the probability difference, it can not only lessen the unreliability of pseudolabels generated by the network, but also maximize the feature differences among classes to achieve better classification performance. A multitask experiment under changeable working conditions on three public bearing datasets proves the superiority of the proposed CSAN model. Compared with other methods, CSAN can achieve the optimal best performance and has desirable generalization performance, whose value remains near the best-performance value. Hence, CSAN is relatively stable and reliable, proving that it has universal and practical application in bearing fault diagnosis under variable working conditions.

Software aging, which is caused by Aging-Related Bugs (ARBs), refers to the phenomenon of performance degradation and eventual crash in long running systems. In order to discover and remove ARBs, ARB prediction is proposed. However, due to the low presence and reproducing difficulty of ARBs, it is usually difficult to collect sufficient ARB data within a project. Therefore, cross-project ARB prediction is proposed as a solution to build the target project’s ARB predictor by using the labeled data from the source project. A key point for cross-project ARB prediction is to reduce distribution difference between source and target project. However, existing approaches mainly focus on the marginal distribution difference while somehow overlook the conditional distribution difference, and they mainly use random oversampling to alleviate the class imbalance which may lead to overfitting. To address these problems, we propose a new crossproject ARB prediction approach based on Joint Distribution Adaptation (JDA) and Improved Subclass Discriminant Analysis (ISDA), called JDA-ISDA. The key idea of JDA-ISDA is first to use JDA to reduce the marginal distribution and conditional distribution difference jointly and then apply ISDA to alleviate the severe class imbalance problem. A set of experiments are carried out on two large open-source projects with six different machine learning (ML) classifiers. The experimental results demonstrate that compared with the state-of-the-art Transfer Learning based Aging-related bug Prediction (TLAP) and Supervised Representation Learning Approach (SRLA), JDA-ISDA is much more robust to different ML classifiers than TLAP, and the average improvement in terms of the balance value can be achieved up to 31.8%, and JDA-ISDA also outperforms TLAP and SRLA on average when logistic regression is chosen as the classifier for best performance prediction.

Rotating parts are the heart of the machinery industry, so various industries put forward higher requirements for the stability and reliability of mechanical work. However, relying on manual fault diagnosis is not only time-consuming and laborious, but also the accuracy rate is not ideal, so machine learning was introduced to the machinery fault diagnosis industry, but the training of fault diagnosis models depends on a large amount of data, and the working conditions of mechanical faults are variable, and it is difficult to apply the model trained under one working condition to another working condition. Therefore, we introduce the domain adaptive method of migration learning in fault diagnosis. However, most of the samples in real production are normal samples, and fault samples only account for a few. The sample imbalance problem needs to be solved urgently, so we adopt the method based on joint distribution adaptation, which firstly uses the model itself to make a priori judgment, and learns on the basis of the judgment; moreover, in order to solve the problem of ignoring the small samples and weight bias when the model samples are imbalanced, we propose the inverse sparsity loss function and entropy increasing loss function. Under the operating conditions of the bearing sample imbalance variable, fault tests are designed to verify the accuracy and effectiveness of the proposed method to solve the sample imbalance problem.

Aligning and balancing the marginal and conditional feature distributions are two critical procedures for unsupervised domain adaptation (UDA) problems. However, existing methods usually consider the former while ignoring the latter. To improve the problems of instability and imbalance, we propose the Adaptative Joint Distribution Adaptation Network (AJDAN) by analyzing the multi-modal interactions between the two types of distributions and adding a self-learning network to simultaneously balance them. Furthermore, we give higher weights to samples that are far away from the domain boundary (easy-to-classify samples) using Strong Binary Cross-Entropy (SBCE). The strong alignment strategy is adjustable and allows the network to better train easy-to-classify samples than traditional Binary Cross-Entropy (BCE) in various scenarios. The experiment shows that AJDAN with SBCE (AJDAN+S) provides an average of 68.3% accuracy on the Office-Home dataset, and 89.1% accuracy on the Office31 dataset, showing its superiority by 2~3 percent above the existing state-of-the-art methods.

Domain adaptation has received a lot of attention in recent years, and many algorithms have been proposed with impressive progress. However, it is still not fully explored concerning the joint probability distribution (P(X, Y)) distance for this problem, since its empirical estimation derived from the maximum mean discrepancy (joint maximum mean discrepancy, JMMD) will involve complex tensor-product operator that is hard to manipulate. To solve this issue, this paper theoretically derives a unified form of JMMD that is easy to optimize, and proves that the marginal, class conditional and weighted class conditional probability distribution distances are our special cases with different label kernels, among which the weighted class conditional one not only can realize feature alignment across domains in the category level, but also deal with imbalance dataset using the class prior probabilities. From the revealed unified JMMD, we illustrate that JMMD degrades the feature-label dependence (discriminability) that benefits to classification, and it is sensitive to the label distribution shift when the label kernel is the weighted class conditional one. Therefore, we leverage Hilbert Schmidt independence criterion and propose a novel MMD matrix to promote the dependence, and devise a novel label kernel that is robust to label distribution shift. Finally, we conduct extensive experiments on several cross-domain datasets to demonstrate the validity and effectiveness of the revealed theoretical results.

In long-running systems, the phenomenon of performance degradation and failure rate increase caused by Aging-Related Bugs (ARBs) is known as software aging. Because of the low presence and reproducing difficulty of ARBs, collecting enough training data to predict ARBs in a project is not easy. Thus, cross-project ARB prediction has been proposed. There are two main challenges in cross-project ARB prediction, namely distribution differences and severe class imbalance. As for the first challenge, existing cross-project ARB prediction approaches only focus on the transferability between domains while ignoring the discriminability between classes. As for the second challenge, existing approaches only consider the imbalance between the classes while ignoring the within-class imbalance problem. To solve these problems, a cross-project ARB prediction approach based on Joint Probability Domain Adaptation (JPDA) and k- means SMOTE (KS), called JPKS, is proposed. JPDA is used to consider the transferability and discriminability simultaneously, and KS solves the within-class and between-class imbalance problems. Experiments are conducted on two natural software systems to verify the performance of JPKS. The results show that JPKS can improve the performance of ARB prediction.

Domain adaptation aims to achieve label transfer from a labeled source domain to an unlabeled target domain, where the two domains exhibit different distributions. Existing methods primarily concentrate on designing a feature extractor to learn better domain-invariant features, along with developing an effective classifier for reliable predictions. In this paper, we introduce optimal graph learning to generate a cross-domain graph that effectively connects the two domains, and two domain-specific graphs to capture domain-specific structures. On the one hand, we incorporate the three graphs into the label propagation (LP) classifier to enhance its robustness to distribution difference. On the other hand, we leverage the three graphs to introduce graph embedding losses, promoting the learning of locally discriminative and domain-invariant features. Furthermore, we maximize the nuclear norm of predictions in LP to enhance class diversity, thereby improving its robustness to class imbalance problem. Correspondingly, we develop an efficient algorithm to solve the associated optimization problem. Finally, we integrate the proposed LP and graph embedding losses into a deep neural network, resulting in our proposed deep cross-domain robust LP. Extensive experiments conducted on three cross-domain benchmark datasets demonstrate that our proposed approach could outperform existing state-of-the-art domain adaptation methods.

A variety of methods based on data analysis have demonstrated considerable effectiveness in diagnosing bearing faults across various domains and transfer tasks. However, without considering how condition distributions among source and target domains affect diagnostic results, most studies focus only on the marginal distribution between them. Furthermore, discussion and investigation of prior knowledge are often absent in existing feature extraction methods based on deep learning. Therefore, this paper introduces a feature fusion and joint distribution adaptation (JDA) method to diagnose bearing faults under variable working conditions. In addition, to enhance the diagnostic accuracy of bearing faults, this paper proposes a method combining feature fusion and domain adaptation (DA). In the feature extraction stage, data-based features extracted from the convolutional autoencoder are integrated with the time-frequency domain features based on prior knowledge to improve the overall generalization performance of the framework. A JDA network that considers both marginal distribution adaptation and conditional distribution adaptation (CDA) has been developed. An intradomain distribution statistical metric is designed to minimize the deviation between different domains, and an improved CDA mechanism facilitates the matching of conditional probability distributions between two domains. Furthermore, a weight allocation algorithm is developed to dynamically adjust the source and target domain weights, facilitating adaptive detection and restriction of irregular samples and improving the adjustment of data distributions between them. To verify the effectiveness of the developed method, we validate it on two bearing fault datasets. The test results show that the proposed network achieves excellent results in terms of both generalization and diagnostic performance.

In speaker-independent speech emotion recognition, the training and testing samples are collected from diverse speakers, leading to a multi-domain shift challenge across the feature distributions of data from different speakers. Consequently, when the trained model is confronted with data from new speakers, its performance tends to degrade. To address the issue, we propose a Dynamic Joint Distribution Adaptation (DJDA) method under the framework of multi-source domain adaptation. DJDA firstly utilizes joint distribution adaptation (JDA), involving marginal distribution adaptation (MDA) and conditional distribution adaptation (CDA), to more precisely measure the multi-domain distribution shifts caused by different speakers. This helps eliminate speaker bias in emotion features, allowing for learning discriminative and speaker-invariant speech emotion features from coarse-level to fine-level. Furthermore, we quantify the adaptation contributions of MDA and CDA within JDA by using a dynamic balance factor based on $\mathcal{A}$-Distance, promoting to effectively handle the unknown distributions encountered in data from new speakers. Experimental results demonstrate the superior performance of our DJDA as compared to other state-of-the-art (SOTA) methods.

To address the degradation of diagnostic performance due to data distribution differences and the scarcity of labeled fault data, this study has focused on transfer learning-based cross-domain fault diagnosis, which attracts considerable attention. However, deep transfer learning-based methods often present a challenge due to their time-consuming and costly nature, particularly in tuning hyperparameters. For this issue, on the basis of classical features-based transfer learning method, this study introduces a new framework for bearing fault diagnosis based on supervised joint distribution adaptation and feature refinement. It first utilizes ensemble empirical mode decomposition to process raw signals, and statistical features extraction is implemented. Then, a new feature refinement module is designed to refine domain adaptation features from high-dimensional feature set by evaluating the fault distinguishability and working-condition invariance of feature data. Next, it proposes a supervised joint distribution adaptation method to conduct improved joint distribution alignment that preserves neighborhood relationships within a manifold subspace. Finally, an adaptive classifier is trained to predict fault labels of feature data across varying working conditions. To prove the cross-domain fault diagnosis performance and superiority of the proposed methods, two bearing datasets are applied for experiments, and the experimental results verify that the model built by the proposed framework can achieve desirable diagnosis performance under different working conditions and that it apparently outperforms comparative models.

Effectively reducing the distribution discrepancy between domains is essential for enhancing the accuracy of multichannel fault diagnosis under cross-domain conditions. When extracting domain-invariant features from multichannel data, existing transfer learning methods struggle to minimize the conditional distribution discrepancy between different class-level subdomains while preserving intrinsic structural information. To address the above issue, this article proposes a novel tensor transfer learning approach termed multilinear joint distribution adaptation (MJDA). Specifically, inspired by JDA, we extend the JDA model into tensor space and formulate a corresponding optimization problem, which is efficiently solved by an alternating iterative algorithm. By estimating a set of transformation matrices without vectorization, MJDA directly minimizes the joint distribution discrepancy to extract domain-invariant features. Furthermore, twin support higher-order tensor machines are embedded as a tensor classifier, which not only provides pseudo labels for the target domain but also performs fault pattern recognition on the testing data. Extensive transfer experiments on planetary gearbox datasets demonstrate that the proposed MJDA consistently outperforms other typical tensor transfer learning models.

No abstract available

Most emotion recognition systems still present limited applicability to new users due to the intersubject variability of electroencephalogram (EEG) signals. Although domain adaptation methods have been adopted to tackle this problem, most methodologies deal with unlabeled data from a target subject. However, a few labeled samples from a target subject could also be included to boost cross-subject emotion recognition. In this article, we present a semisupervised domain adaptation (SSDA) framework to align the joint distributions of subjects, assuming that fine-grained structures must be aligned to perform a greater knowledge transfer. To achieve this, the proposed framework performs a multisource alignment of features at the subject level, while predictions are aligned over the global feature space. To support joint distribution alignment, interclass separation and consistent predictions are ensured on the target subject. We perform experiments using two public benchmark datasets, SEED and SEED-IV, with two different sampling strategies to incorporate a few labeled samples from a target subject. Our proposal achieves an average accuracy of 93.55% and 87.96% on SEED and SEED-IV, using three labeled target samples of each class. Moreover, we obtained an average accuracy of 91.79% and 85.45% on SEED and SEED-IV by incorporating ten labeled samples from the first EEG trial of each class.

Electricity theft has been a major concern all over the world. There are great differences in electricity consumption among residents from different regions. However, the existing supervised methods of machine learning are not in detecting electricity theft from different regions, while the development of transfer learning provides a new view for solving the problem. Hence, an electricity-theft detection method based on convolutional and joint distribution adaptation (CJDA) is proposed. In particular, the model consists of three components: convolutional component (Conv), marginal distribution adaptation (MDA), and conditional distribution adaptation (CDA). The convolutional component can efficiently extract the customer’s electricity characteristics. The MDA can match marginal probability distributions and solve the discrepancies of residents from different regions, while CDA can reduce the difference of the conditional probability distributions and enhance the discrimination of features between energy thieves and normal residents. As a result, the model can find a matrix to adapt the electricity residents in different regions to achieve electricity-theft detection. The experiments are conducted on electricity consumption data from the Irish Smart Energy Trial (ISET) and State Grid Corporation of China (SGCC), and metrics, including ACC, recall, false positive rate (FPR), area under curve (AUC), and $F1$ score, are used for evaluation. Compared with other methods, including some machine learning methods, such as decision tree (DT), random forest (RF), and extreme gradient boosting (XGBoost), some deep learning methods, such as recurrent neural network (RNN), convolutional neural network (CNN), and wide and deep CNN (WCNN), and some up-to-date methods, such as balanced distribution adaptation (BDA), weighted and BDA (WBDA), random convolutional kernel transform (ROCKET), and minimally ROCKET (MiniROCKET), our proposed method has a better effect on identifying electricity theft from different regions.

The present research on intelligent bearing fault diagnosis assumes that the same feature distribution is used to obtain training and testing data. However, the domain shift (distribution discrepancy) issue generally occurs in both datasets because of different operational conditions. The domain adaptation techniques are preferably applied for fault diagnosis to handle the domain shift issue. Moreover, collecting sufficient testing data or labelled data in real industries is a challenging task. Therefore, the multi-kernel joint distribution adaptation (MKJDA) with dynamic distribution alignment is proposed for bearing fault diagnosis. This method dynamically joins both the marginal and conditional distributions and uses the multi-kernel to solve the non-linear problems to extract the most effective and robust representation for cross-domain issues. Moreover, it runs with the unlabelled task domain to perform the diagnosis by iteratively updating the pseudo code. The experimental results (two public datasets and one experimental dataset) demonstrated that the proposed method (MKJDA) exhibited stable and robust accuracy while conducting bearing fault diagnosis. It can effectively address the most crucial issue: intelligent diagnosis methods must re-train the model when the distribution differs between the source domain (the model is learned) and the target domain (the learned model is applied).

Most existing intelligent fault diagnosis schemes rely on the assumption that the training and test samples are independent and identically distributed, ignoring the domain distribution shift caused by diverse operating conditions, which may limit their flexible applications in practical diagnostic tasks. To address this problem, we propose a novel unsupervised transfer learning method, namely, sparse filtering with joint distribution adaptation (SFJDA) for mechanical fault diagnosis. Specifically, two sparse filters (SFs) are used to jointly extract features from each domain, and the final feature space is formed by stacking the subspaces obtained from double SFs. Then, the maximum mean difference (MMD) is introduced to measure the distribution discrepancy between different domains. By extending the marginal distribution adaptation (MDA) to joint distribution adaptation (JDA), the constructed framework can capture domain-invariant and class-separable features. Finally, the effectiveness of the proposed scheme is verified by a motor bearing dataset.

Deep network fault diagnosis requires a lot of labeled data and assumes identical data distributions for training and testing. In industry, varying equipment conditions lead to different data distributions, making it challenging to maintain consistent fault diagnosis performance across conditions. To this end, this paper designs a transfer learning model named the multi-adversarial joint distribution adaptation network (MAJDAN) to achieve effective fault diagnosis across operating conditions. MAJDAN uses a one-dimensional lightweight convolutional neural network (1DLCNN) to directly extract features from the original bearing vibration signal. Combining the distance-based domain-adaptive method, maximum mean difference (MMD), with the multi-adversarial network will simultaneously reduce the conditional and marginal distribution differences between the domains. As a result, MAJDAN can efficiently acquire domain-invariant feature information, addressing the challenge of cross-domain bearing fault diagnosis. The effectiveness of the model was verified based on two sets of different bearing vibration signals, and one-to-one and one-to-many working condition migration task experiments were carried out. Simultaneously, various levels of noise were introduced to the signal to enable analysis and comparison. The findings demonstrate that the suggested approach achieves exceptional diagnostic accuracy and exhibits robustness.

Abstract Numerous intelligent methods have been developed to approach the challenges of fault diagnosis. However, due to the different distributions of training samples and test samples, and the lack of information on test samples, most of these methods cannot directly handle the unsupervised cross-domain fault diagnosis issues. In this paper, a joint distribution adaptation network with adversarial learning is developed to effectively tackle the mentioned fault diagnosis issues. Firstly, deep convolutional neural network (CNN) is constructed to extract the features of training samples and test samples. Secondly, since the joint maximum mean discrepancy (JMMD) cannot precisely measure the joint distribution discrepancy between different domains, an improved joint maximum mean discrepancy (IJMMD) is proposed to accurately match the feature distributions. Finally, adversarial domain adaptation is also developed to help the constructed CNN to extract the domain-invariant features. Therefore, the proposed method can achieve precisely distribution matching, and extract the category-discriminative and domain-invariant features between the source and target domains. Substantial transfer fault diagnosis cases based on three rolling bearing datasets fully demonstrate the effectiveness and generalization ability of the proposed method.

Transfer learning (TL) is one of the most important sub-realms in the field of machine learning. The goal is to achieve knowledge transfer from source domain to target domain by utilizing the similarities among data, tasks, or models. Domain adaptation (DA) is actually a type of transductive TL with assumption that both the marginal and the conditional distribution adaptation are different. In this paper, the process of joint distribution adaptation implemented on the universal quantum computer is presented. Compared to the classical joint distribution adaptation, the performance and application scope of the algorithm are promoted by the advantages of the quantum operations. Specifically, the quantum joint distribution adaptation algorithm proposed in our work can be performed exponentially efficient with quantum computing techniques in contrast to the corresponding classical procedure.

Multimodal physiological data from electroencephalogram (EEG) and eye movement (EM) signals have been shown to be useful in effectively recognizing human emotional states. Unfortunately, individual differences reduce the applicability of existing multimodal classifiers to new users, as low performance is usually observed. Indeed, existing works mainly focus on multimodal domain adaptation from a labeled source domain and unlabeled target domain to address the mentioned problem, transferring knowledge from known subjects to new one. However, a limited set of labeled target data has not been effectively exploited to enhance the knowledge transfer between subjects. In this article, we propose a multimodal semi-supervised domain adaptation (SSDA) method, called cross-modal learning and joint distribution alignment (CMJDA), to address the limitations of existing works, following three strategies: 1) discriminative features are exploited per modality through independent neural networks; 2) correlated features and consistent predictions are produced between modalities; and 3) marginal and conditional distributions are encouraged to be similar between the labeled source data, limited labeled target data, and abundant unlabeled target data. We conducted comparison experiments on two public benchmarks for emotion recognition, SEED-IV and SEED-V, using leave-one-out cross-validation (LOOCV). Our proposal achieves an average accuracy of 92.50%–96.13% across the three available sessions on SEED-IV and SEED-V, only including three labeled target samples per class from the first recorded trial.

As a key component indispensable to the normal operation of rotating machinery, the fault diagnosis of rolling bearings is beneficial to ensure the normal operation of industrial production. Many existing fault diagnosis methods based on domain adaptation (DA) rely on the assumption that the training and testing data share the same label type, whereas in practical industrial scenarios, due to the impossibility of predicting failure modes in the testing phase, it is highly likely that unknown fault types will appear during the test phase that did not occur during the training phase. To deal with this open-set fault diagnosis (OSFD) problem, this study proposes a DA method based on joint distribution alignment adversarial learning. This method is based on a novel triple-indicator metric that utilizes three estimation parameters, mean, variance, and covariance, to better measure distribution discrepancy, and combines the joint distribution alignment DA technique to align the distribution of training and testing data. Furthermore, we propose an enhanced feature extraction and classification adversarial neural network (EFECANN) that can efficiently process vibration signals, which consist of three parts: feature extractor, domain discriminator, and label classifier. These three parts are combined to promote the effective extraction of fault features for classification and diagnosis. Bearing datasets from Paderborn University (PU) and Case Western Reverse University (CWRU) are used to carry out comparative and ablation experiments to validate the proposed method for the OSFD problem, and the experimental results show that the proposed method outperforms other typical methods in fault diagnosis.

Online damage quantification suffers from insufficient labeled data that weakens its accuracy. In this context, adopting the domain adaptation on historical labeled data from similar structures/damages or simulated digital twin data to assist the current diagnosis task would be beneficial. However, most domain adaptation methods are designed for classification and cannot efficiently address damage quantification, a regression problem with continuous real-valued labels. This study first proposes a novel domain adaptation method, the Online Fuzzy-set-based Joint Distribution Adaptation for Regression, to address this challenge. By converting the continuous real-valued labels to fuzzy class labels via fuzzy sets, the marginal and conditional distribution discrepancy are simultaneously measured to achieve the domain adaptation for the damage quantification task. Thanks to the superiority of the proposed method, a state-of-the-art online damage quantification framework based on domain adaptation is presented. Finally, the framework has been comprehensively demonstrated with a damaged helicopter panel, in which three types of damage domain adaptations (across different damage locations, across different damage types, and from simulation to experiment) are all conducted, proving the accuracy of damage quantification can be significantly improved in a realistic environment. It is expected that the proposed approach to be applied to the fleet-level digital twin considering the individual differences.

No abstract available

Distribution discrepancy between training data and testing data caused by varying working conditions limits the wide applications of deep learning-based methods for process fault diagnosis. Generally, this issue is addressed by transfer learning (TL) effectively. However, previous works on TL mainly focus on aligning the marginal distribution only or ignoring the different impacts of the marginal and conditional distributions of the data. Thus, it remains challenging to reduce domain shifts by considering marginal and conditional distributions adaptatively and simultaneously. In this article, a novel deep transfer network (DTN) with adaptive joint distribution adaptation (AJDA) is proposed to solve the problem of process fault diagnosis under varying working conditions. First, an adaptive joint distribution module is proposed to implement domain adaptation both in feature space and label space. AJDA not only aligns the marginal and conditional distribution simultaneously but also quantifies the importance of the two distributions. Moreover, a novel feature generator, self-calibrated-based 1-D convolutional neural network (SC-1DCNN), is developed to effectively learn shared feature representations from the process data. The adversarial training with gradient penalty is adopted to guide SC-1DCNN to provide domain-invariant features between the two domains. The testing results on four experimental cases under varying working conditions, including two simulation cases and two real cases, have demonstrated the effectiveness of AJDA in process fault diagnosis.

Deep learning techniques have been widely applied for intelligent fault diagnosis. However, these techniques require large amounts of labeled data from a particular machine, which is demanding for real-world applications. Alternatively, models can be developed based on artificial damages and be applied for industrial data with real damages. In that case, a major challenge arises since the distributions of those artificial and real damages are greatly different, which results in severe performance degradation of conventional deep models. In this work, a model named deep coupled joint distribution adaptation network (DCJDAN) is proposed to address the large domain discrepancy between artificial and real damages. By utilizing two untied deep convolutional networks, the proposed method allows the source- and target-stream networks to focus on learning domain-representative features, providing flexibility for explicitly modeling the domain discrepancy. To ensure a more effective knowledge transferring, a regulation term is adopted to force the untied coupled networks to stay similar since the source domain and the target domain are related. The joint distribution adaptation module is further adapted to minimize the domain discrepancy, which considers both the marginal and conditional distribution differences and provides more accurate distribution matching. The effectiveness of the proposed method is evaluated based on three bearing data sets with artificial and real damages. As reported, the proposed method achieves an average accuracy of 98.17% for all tasks, which outperforms several state-of-the-art deep domain adaptation models and improves the diagnosis performance compared with the conventional deep learning models.

Universal Domain Adaptation (UniDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain, even when their classes are not fully shared. Few dedicated UniDA methods exist for Time Series (TS), which remains a challenging case. In general, UniDA approaches align common class samples and detect unknown target samples from emerging classes. Such detection often results from thresholding a discriminability metric. The threshold value is typically either a fine-tuned hyperparameter or a fixed value, which limits the ability of the model to adapt to new data. Furthermore, discriminability metrics exhibit overconfidence for unknown samples, leading to misclassifications. This paper introduces UniJDOT, an optimal-transport-based method that accounts for the unknown target samples in the transport cost. Our method also proposes a joint decision space to improve the discriminability of the detection module. In addition, we use an auto-thresholding algorithm to reduce the dependence on fixed or fine-tuned thresholds. Finally, we rely on a Fourier transform-based layer inspired by the Fourier Neural Operator for better TS representation. Experiments on TS benchmarks demonstrate the discriminability, robustness, and state-of-the-art performance of UniJDOT.

No abstract available

In recent years, an increasing popularity of deep learning model for intelligent condition monitoring and diagnosis as well as prognostics used for mechanical systems and structures has been observed. In the previous studies, however, a major assumption accepted by default, is that the training and testing data are taking from same feature distribution. Unfortunately, this assumption is mostly invalid in real application, resulting in a certain lack of applicability for the traditional diagnosis approaches. Inspired by the idea of transfer learning that leverages the knowledge learnt from rich labeled data in source domain to facilitate diagnosing a new but similar target task, a new intelligent fault diagnosis framework, i.e., deep transfer network (DTN), which generalizes deep learning model to domain adaptation scenario, is proposed in this paper. By extending the marginal distribution adaptation (MDA) to joint distribution adaptation (JDA), the proposed framework can exploit the discrimination structures associated with the labeled data in source domain to adapt the conditional distribution of unlabeled target data, and thus guarantee a more accurate distribution matching. Extensive empirical evaluations on three fault datasets validate the applicability and practicability of DTN, while achieving many state-of-the-art transfer results in terms of diverse operating conditions, fault severities and fault types.

No abstract available

Electroencephalogram (EEG)-based emotion recognition is a feasible method to improve human-robot interaction (HRI) systems. However, most existing methods fail to generalize EEG data from existing users or from same user collected on different time, which limits the widespread use of HRI systems. To this end, we propose a joint distribution adaptation network (JDAN) model for multi-source EEG-based emotion recognition. Specifically, A deep neural network is firstly used to extract the deep features of EEG. We then propose two alignment stages: the joint distribution alignment and multi-classifier alignment. The former can reduce the joint feature and label distribution discrepancy between each pair of source and target domains, while the latter reduce the variance in the multi-source distribution. We evaluate our method on the public dataset SEED. Experiments prove that our JDAN outperforms support vector machine and a simple deep neural network on both cross-subject and cross-day settings, demonstrating its effectiveness in tackling multi-source EEG-based emotion recognition.

Microexpression recognition has been widely favored by researchers due to its many potential applications, such as business negotiation and lie detection. Cross-database microexpression recognition is more challenging and attractive than normal microexpression recognition because the training and testing samples come from different databases. The ensuing challenge is that the feature distributions between training and testing samples differ too much. As a result, the performance of current well-performing microexpression recognition methods often fails to achieve the desired effect. In this paper, we overcome this problem by introducing Subspace Learning and Joint Distribution Adaptation (SLJDA) by projecting the source and target domains into the subspace and later reducing the distance between them and then minimizing the distance between the marginal and conditional probability distributions of the data between the source domain and the target domain. To evaluate its performance, a large number of cross-database experiments are performed in the SMIC database and CASMEII database. The experimental results show the superiority of the method compared with existing microexpression recognition methods.

No abstract available

Cross-subject classification is of great practical value in the mental monitoring. The trained model on a person can be transferred to another person without retraining. To date, it is achieved by global domain adaptation without considering differences in subdomain distributions. In this case, there is a lack of sensitivity to specific information associated with each category. To solve this problem, we proposed a deep subdomain adaptation network (DSAN) to estimate mental workload levels across different persons. In the proposed DSAN, the first temporal and spatial layers were designed as a feature extractor. The features extracted by the feature extractor were aligned between the source samples and target samples in each subdomain separately. The alignment loss calculated by local maximum mean discrepancy (LMMD) was back-propagated to update the weights of the feature extractor to enhance the feature extraction performance. Subdomain adaptation was achieved over iterations during the model training. The proposed subdomain adaptation is not specialized for a particular feature extractor, as shown in this paper. It is universal and can be applied after any feature extractors. Two datasets (Dataset MATB and Dataset SFE) were used to evaluate the proposed DSAN. The results showed that the proposed DSAN outperformed the compared methods in terms of classification accuracy, showing an elevation of $3 \% \sim 7 \%$. This study provides an effective solution for the cross-subject mental workload classification and will promote practical applications of mental workload monitoring.

No abstract available

No abstract available

Indoor localization based on Wi-Fi Channel State Information (CSI) continues to gain prominence as a practical, infrastructure-free solution for position estimation in dynamic indoor environments. However, CSI fingerprints are highly sensitive to environmental variations such as furniture displacement, door and window states, device heterogeneity, and temporal drift, which introduce domain shifts that significantly degrade localization accuracy. Recent domain adaptation methods mitigate these discrepancies, yet most rely solely on aligning marginal or conditional distributions and neglect the geometric structure of feature manifolds, often resulting in class confusion and distorted local neighborhoods. This study proposes a Geometry-Aware Deep Joint Multi-Domain Adaptation Network (G-DJMDAN) that integrates a deep residual CSI encoder, multi-layer distribution alignment, and a novel graph-based geometry-preservation module to maintain spatial topology across domains. By jointly optimizing classification loss, MK-MMD, LMMD, and Laplacian regularization, the framework produces more stable, discriminative embeddings under diverse environmental changes. Comprehensive experiments conducted across multiple real-world domain-shift scenarios—including temporal variation, furniture layout changes, and mixed perturbations—show that G-DJMDAN reduces median localization error by up to 34.7% compared with DJMDAN and achieves statistically significant improvements over CNN, DANN, DAN, JAN, and DSAN baselines. Visualization using t-SNE further confirms that the proposed geometry module effectively preserves manifold structure and mitigates target-domain collapse. Overall, the results demonstrate that incorporating geometric constraints into domain adaptation provides a robust and scalable pathway for high-precision indoor localization in non-stationary environments.

Deep transfer learning has been widely used to improve the versatility of models. In the problem of cross-domain fault diagnosis in rolling bearings, most models require that the given data have a similar distribution, which limits the diagnostic effect and generalization of the model. This paper proposes a deep reconstruction transfer convolutional neural network (DRTCNN), which satisfies the domain adaptability of the model under cross-domain conditions. Firstly, the model uses a deep reconstruction convolutional automatic encoder for feature extraction and data reconstruction. Through sharing parameters and unsupervised training, the structural information of target domain samples is effectively used to extract domain-invariant features. Secondly, a new subdomain alignment loss function is introduced to align the subdomain distribution of the source domain and the target domain, which can improve the classification accuracy by reducing the intra-class distance and increasing the inter-class distance. In addition, a label smoothing algorithm considering the credibility of the sample is introduced to train the model classifier to avoid the impact of wrong labels on the training process. Three datasets are used to verify the versatility of the model, and the results show that the model has a high accuracy and stability.

A critical issue for data‐driven and machine learning‐based damage detection of engineering infrastructures is associated with unlabeled datasets and distribution shifts in cross‐domains. To overcome this challenge, this study develops an unsupervised cross‐domain method for bridge damage detection based on interclass alignment of time‐frequency features extracted from multichannel sensor data. The computational framework was developed based on a deep subdomain adaptation network integrating digital and physical information. Initially, a multichannel symmetric dot pattern was utilized to transform the structural acceleration signals into a comprehensive image. Subsequently, a convolutional block attention module‐enhanced ResNet34 (CBAM‐ResNet34) was constructed to extract discriminative time‐frequency features, where a local maximum mean discrepancy principle was introduced to perform class‐conditional alignment across subdomains. Compared with traditional global domain alignment methods, the proposed approach focuses on aligning class‐conditional distributions within subdomains to improve the generalization performance with unlabeled datasets. The proposed method was validated on both simulated and experimental datasets collected from a laboratory‐scaled steel truss bridge. Furthermore, a case study on the Old ADA Bridge in Japan was presented to demonstrate the robustness and practical applicability of the proposed approach, serving as a benchmark against classic unsupervised methods. The results show that the proposed framework has a substantial improvement in source‐to‐target transfer recognition performance. Discussions were conducted on the application prospects of the proposed framework for more in‐service infrastructures in complex conditions.

Radio frequency fingerprint identification (RFFI) utilizes nonideal hardware features present in the signal to identify different transmitters. However, existing RFFI models have poor generalization capabilities. When a model trained on one receiver is deployed on a new receiver, the identification performance of the model degrades due to the effect of different receiver characteristics, which can cause the signal distribution to be shifted. To address this problem, we propose a cross-receiver RFFI based on domain adaptation with dynamic distribution alignment. First, deep features are extracted using the ResNet18 network and the global distribution of the features is aligned using maximum mean discrepancy (MMD). Then multilevel features are extracted using a designed multiscale feature extraction module and the subdomain distribution is aligned using local MMD (LMMD). Finally, a dynamic parameter is introduced to adaptively adjust the relative importance between the global and subdomain distributions. Twelve sets of cross-receiver experiments are conducted on the WiSig dataset, and the algorithm in this article achieves an average identification rate of 92.52% in the target domain. Meanwhile, the experimental results under different signal-to-noise ratios (SNR) also verify the algorithm has strong robust performance. It shows that the algorithm can effectively alleviate the model performance degradation problem in the cross-receiver scenarios.

To solve the problem of feature distribution discrepancy in cross-corpus speech emotion recognition tasks, this paper proposed an emotion recognition model based on multi-task learning and subdomain adaptation, which alleviates the impact on emotion recognition. Existing methods have shortcomings in speech feature representation and cross-corpus feature distribution alignment. The proposed model uses a deep denoising auto-encoder as a shared feature extraction network for multi-task learning, and the fully connected layer and softmax layer are added before each recognition task as task-specific layers. Subsequently, the subdomain adaptation algorithm of emotion and gender features is added to the shared network to obtain the shared emotion features and gender features of the source domain and target domain, respectively. Multi-task learning effectively enhances the representation ability of features, a subdomain adaptive algorithm promotes the migrating ability of features and effectively alleviates the impact of feature distribution differences in emotional features. The average results of six cross-corpus speech emotion recognition experiments show that, compared with other models, the weighted average recall rate is increased by 1.89~10.07%, the experimental results verify the validity of the proposed model.

Recently, deep transfer learning (TL) has successfully addressed the problem of fault diagnosis under variable operating conditions. Existing methods default that the source and target domains have the same label space, and solve distribution discrepancy problem under different working conditions by aligning their feature distributions. However, in the practical industry, is unlikely to guarantee the health conditions of the target domain data are consistent with the source domain. Therefore, industrial applications usually face the challenge of more difficult partial domain diagnosis scenarios. In this paper, a deep partial domain adaptation network based on a balanced alignment constraint strategy is proposed to realize cross-domain diagnosis. The proposed method combines balanced augmentation and subdomain alignment, which can effectively facilitate the positive transfer of shared categories. Meanwhile, the conditional entropy minimization is introduced to encourage the predictions of target domain samples with high confidence. The experimental results on the rolling bearing dataset verify the effectiveness and feasibility of the proposed method in handling the actual partial domain fault diagnosis problem.

In recent years, intelligent condition monitoring and diagnosis based on deep learning have made great progress. However, traditional diagnostic methods mostly perform vibration analysis based on accelerometer signals, ignoring the influence of sensors on the mass load of the measured object. On the other hand, conventional transfer learning (TL) methods are mostly based on global distribution alignment to achieve intelligent diagnosis under variable working conditions. In this paper, a deep global subdomain adaptation network (DGSAN) is proposed to solve the intelligent diagnosis problem under variable working conditions based on vibration image and TL. First, visual measurement is introduced in vibration extraction. Based on the phase vibration extraction method, the vibration feature information is obtained from the visual vibration image to construct the vibration dataset. Then, the proposed DGSAN establishes a multi-layer domain adaptive network to minimize the difference in feature distribution and realize fine-grained feature distribution alignment of fault data under variable working conditions. Comparative experiments are carried out on the vibration image datasets of rotor-bearing systems, and the results show that the proposed method achieves high-precision transfer intelligent diagnosis.

The local climate zone (LCZ) classification system provides a standardized framework for presenting the morphological and functional characteristics of cities, especially for urban climate studies. The LCZ classification scheme is now widely used in urban heat island climate studies, urban planning, weather forecasting and other studies. Previous studies have generally been based on traditional machine learning and manual feature design for LCZ classification. Although deep learning methods perform well in LCZ classification, the generalization ability is still insufficient when facing large scale mapping, especially for cross-domain classification where the source and target domains have significant differences in feature distribution. In this paper, a subdomain adaptation feature alignment architecture (SAFA) is proposed to reduce inter-domain differences. In SAFA, convolutional neural network as a feature extractor in the framework, subdomain adaptive layers embedded in the network aligning features in the source and target domains. The proposed architecture is evaluated on a publicly available dataset. The experimental results show that the proposed architecture improves the overall classification performance of LCZ and NDVI helps to improve the accuracy of natural categories.

Unsupervised domain adaptation (UDA), which aims to transfer knowledge learned from a labeled source domain to an unlabeled target domain, is useful for various cross-domain image classification scenarios. A commonly used approach for UDA is to minimize the distribution differences between two domains, and subdomain alignment is found to be an effective method. However, most of the existing subdomain alignment methods are based on adversarial learning and focus on subdomain alignment procedures without considering the discriminability among individual subdomains, resulting in slow convergence and unsatisfactory adaptation results. To address these issues, we propose a novel deep subdomain alignment method for UDA in image classification, which consists of a Union Subdo-main Contrastive Learning (USCL) module and a Multi-view Subdomain Alignment (MvSA) strategy. USCL can create discriminative and dispersed subdomains by bringing samples from the same subdomain closer while pushing away samples from different subdomains. MvSA makes use of labeled source domain data and easy target domain data to perform target-to-source and target-to-target alignment. Experimental results on three image classifi-cation datasets (Office-31, Office-Home, Visda-17) demonstrate that our proposed method is effective for UDA and achieves promising results in several cross-domain image classification tasks. Our code will be available: https://github.com/zhaoyewei/DSACDIC.

Data augmentation (DA) has the potential to address the issue of imbalanced and insufficient datasets (I&ID) in pipeline fault diagnosis. However, the majority of existing DA methods for time series are inspired by computer vision techniques, ignoring the temporal dynamic properties and fine-grained fault features, which leads to limited performance of the augmentation. To tackle this problem, we introduce a novel DA approach called the subdomain-alignment adversarial self-attention network (SA-ASN), which takes into account both temporal association and semantic correlation. Our approach features a novel temporal association learning (TAL) mechanism, which transfers temporal information from the discriminator to the generator via a customized knowledge-sharing structure, improving the reliability of synthetic long-range associations. Additionally, we introduce a prototype-assisted subdomain alignment (PASA) strategy that forms a hierarchical structure in the synthetic dataset by incorporating local semantic correlation into the model training. With the support of TAL and PASA, our SA-ASN algorithm enhances the authenticity of temporal structure at the instance level and improves the discriminability of fault features at the category level. Our experimental results show that the SA-ASN algorithm provides a more diverse and accurate augmentation of pipeline data. The effectiveness of our SA-ASN algorithm encourages the use of data-driven diagnostic models in complex real-world oilfield pipeline networks.

No abstract available

No abstract available

EEG signals exhibit non-stationary characteristics, particularly across different subjects, which presents significant challenges in the precise classification of mental workload levels when applying a trained model to new subjects. Domain adaptation techniques have shown effectiveness in enhancing the accuracy of cross-subject classification. However, current state-of-the-art methods for cross-subject mental workload classification primarily focus on global domain adaptation, which may lack fine-grained information and result in ambiguous classification boundaries. We proposed a novel approach called deep subdomain adaptation network with class confusion loss (DSAN-CCL) to enhance the performance of cross-subject mental workload classification. DSAN-CCL utilizes the local maximum mean discrepancy to align the feature distributions between the source domain and the target domain for each mental workload category. Moreover, the class confusion matrix was constructed by the product of the weighted class probabilities (class probabilities predicted by the label classifier) and the transpose of the class probabilities. The loss for maximizing diagonal elements and minimizing non-diagonal elements of the class confusion matrix was added to increase the credibility of pseudo-labels, thus improving the transfer performance. The proposed DSAN-CCL method was validated on two datasets, and the results indicate a significant improvement of 3∼10 percentage points compared to state-of-the-art domain adaptation methods. In addition, our proposed method is not dependent on a specific feature extractor. It can be replaced by any other feature extractor to fit new applications. This makes our approach universal to cross-domain classification problems.

No abstract available

No abstract available

Most data-driven fault diagnosis methods for analog circuits achieve good results when the data satisfies the assumption of independent and equal distribution, which is difficult to achieve in real-world scenarios. To solve this problem, a fault diagnosis method for analog circuits based on Deep Subdomain Adaptation Network is presented. By incorporating the optimization of Local Maximum Mean Discrepancy loss into the training of One-dimensional Convolutional Neural Network, this method can adaptively align the feature representation of the source and target domains without labeling in the target domain. The simulation experiments of Sallen-Key band-pass filter and four-opamp biquad high-pass filter are designed. Two groups of different component parameters are selected as the data sources of source domain and target domain, noise and random offset are added to the target domain data to simulate the actual scene. Through comparative experiments, it is verified that the analog circuit fault diagnosis method presented in this paper has steady training and high accuracy.

Accurate and fast rolling bearing fault diagnosis is required for the normal operation of rotating machinery and equipment. Although deep learning methods have achieved excellent results for rolling bearing fault diagnosis, the performance of most methods declines sharply when the working conditions change. To address this issue, we propose a one-dimensional lightweight deep subdomain adaptation network (1D-LDSAN) for faster and more accurate rolling bearing fault diagnosis. The framework uses a one-dimensional lightweight convolutional neural network backbone for the rapid extraction of advanced features from raw vibration signals. The local maximum mean discrepancy (LMMD) is employed to match the probability distribution between the source domain and the target domain data, and a fully connected neural network is used to identify the fault classes. Bearing data from the Case Western Reserve University (CWRU) datasets were used to validate the performance of the proposed framework under different working conditions. The experimental results show that the classification accuracy for 12 tasks was higher for the 1D-LDSAN than for mainstream transfer learning methods. Moreover, the proposed framework provides satisfactory results when a small proportion of the unlabeled target domain data is used for training.

This study aims at improving fine-grained ship classification performance under the condition that there is no labeled samples available in SAR domain (target domain) by transferring the knowledge from optical remote sensing (ORS) domain (source domain), which has rich labeled samples. The proposed method improves the original deep subdomain adaptation network (DSAN) by designing a dual-branch network (DBN) embedding attention module to extract more discriminative deep transferable features, thereby improving the performance of the subdomain adaptation. Specifically, we utilized a deep base network (ResNet-50) and a shallow base network (ResNet-18) to build the DBN, and embedded the convolutional block attention module after the first and the last convolutional layer of each branch. Extensive experiments demonstrate that the proposed method, which is termed as DSAN++, is feasible and achieves remarkable improvement than the state-of-the-art methods on the task of fine-grained ship classification.

Sensor faults are non-negligible issues for soft sensor modeling. However, existing deep learning-based soft sensors are fragile and sensitive when considering sensor faults. To improve the robustness against sensor faults, this article proposes a deep subdomain learning adaptation network (DSLAN) to develop a sensor fault-tolerant soft sensor, which is capable of handling both sensor degradation and sensor failure simultaneously. Primarily, domain adaptation works for process data with sensor degradation in industrial processes. Being founded on the basic structure of deep domain adaptation, a novel subdomain learner is added to automatically learn the subdomain division, enabling DSLAN adaptable to multimode industrial processes. Notably, the subdomain structure of each sample follows a categorical distribution parameterized by output of the subdomain learner. Based on the designed subdomain learner, a new probabilistic local maximum mean discrepancy (PLMMD) is presented to measure the difference in distribution between source and target features. In addition, a generator for failure data imputation is integrated in the framework, making DSLAN handle sensor failure simultaneously. Finally, the Tennessee Eastman (TE) benchmark process and two real industrial processes are used to verify the effectiveness of the proposed method. With the fault tolerance ability, soft sensing technology will take a step toward practical applications.

This study presents a fault diagnosis method for rolling bearing based on multi-scale deep subdomain adaptation network (MSDSAN). The proposed MSDSAN, as improvement of deep subdomain adaptation network (DSAN), is an unsupervised transfer learning method. MSDSAN reduces the subdomain distribution discrepancy between domains rather than marginal distribution discrepancy, and so better domain invariant fault features are derived to avoid misalignment between domains. Aiming at avoiding fault information loss by fixed receptive fields feature extraction, selective kernel convolution module is introduced into feature extraction of MSDSAN, by which multiple receptive fields are applied to ensure an optimal receptive field for each working condition. Moreover, contribution rates are adaptively assigned to all receptive fields, and the disturbing information extracted by inappropriate receptive fields is further eliminated. As a result, more comprehensive and effective fault information is derived for bearing fault diagnosis. Fault diagnosis experiment of bearings is performed to verify the superiority of the proposed method, and the experimental results demonstrate that MSDSAN achieves better transfer effects and higher accuracy than SOTA methods under varying working conditions.

No abstract available

No abstract available

To improve the accuracy of fault diagnosis of multimodal chemical processes, an ensemble fault diagnosis method based on attention mechanisms and deep subdomain coping adaptive network (AM-DSCAN) is presented. First, a subdomain anti-migration learning network is created, and the error features are extracted through the fault-detection network reconstruction error along with the self-encoder and variation GRU. Then, the source domain and the target domain subdomains are aligned with a nonlinear transformation to minimize the difference in the distribution of the subdomains. Finally, the joint loss function is used to realize the fault diagnosis. The experimental results show that AM-DSCAN has good performance and good application prospects in the diagnosis of chemical process faults.

Rolling bearings in production practice usually serve in a healthy state. Some fault state labels are scarce or even no labels, resulting in unbalanced data categories. Meanwhile, frequent working condition switching results in significant differences in data distribution among working conditions, and labeled data in some working states cannot be fully utilized. To deal with the challenge of low fault identification accuracy caused by these practical factors, this paper proposed a novel adversarial unsupervised subdomain adaption multi-channel deep convolutional network (ASMDCN). Firstly, a parallel three-channel depth feature extraction module is built, and a multi-scale convolution kernel is used to fully extract the rich features of vibration signals under various working conditions. Secondly, a novel loss function is designed to adequately consider the classification difficulty of samples and the degree of class imbalance. Finally, the adversarial training strategy is used to force the feature extractor to extract the domain invariant features, and the Local Maximum Mean discrepancy (LMMD) is used to align the global and related subdomains of the source and target domains. The experimental results show that the designed feature extraction can fully extract the domain-invariant features of the rolling bearings under different working conditions. Under the proposed objective function optimization, the network model can fully align the features of multi-source and single-target domain under unbalanced data and has strong generalization performance.

No abstract available

No abstract available

Unsupervised domain adaptation (UDA) is a promising method for addressing the problem of SAR fine-grained ship classification in target domain with no labeled data available by leveraging a large number of labeled samples from source domains. This article proposes a novel framework, spherical metric refinement with deep subdomain adaptation, to address two crucial issues that are rarely recognized by existing UDA approaches, namely prioritizing adaptation over fine-grained classification and hindering cross-domain alignment and discrimination due to Euclidean feature norms. The proposed solution transforms features into spherical space to eliminate norm bias and introduces joint optimization of classification and adaptation, balancing discriminative feature learning and domain invariance. Experiments on GF-SAR and HR-SAR datasets demonstrate state-of-the-art performance, achieving 95.33% and 89.33% classification accuracy, respectively, outperforming the existing methods by 5.33–6.00%. Our GF-SAR and HR-SAR datasets have been released on GitHub.

No abstract available

No abstract available

No abstract available

Transfer learning has garnered significant interest in the field of bearing fault diagnosis under varying operational conditions due to its robust generalization capabilities. However, real-world diagnostic scenarios frequently encounter data imbalances, which complicates the learning of the classification boundary for the minority class within the diagnostic model. To address this challenge, we propose a normalization-guided and gradient-weighted unsupervised domain adaptation network (NG-UDAN) for intelligent bearing fault diagnosis, aimed at tackling inter-domain feature shifts and intra-domain category imbalances. Firstly, the proposed network integrates a residual feature extractor with the Domain Normalization (DN) module to enhance domain-invariant feature extraction. Subsequently, the Local Maximum Mean Discrepancy (LMMD) loss is utilized to minimize the conditional distributional differences between the source and target domains. Finally, the Gradient-Weighted Focal Loss (GWFL) is specifically designed to address the issue of class imbalance. Experiments conducted across three imbalanced scenarios using the Case Western Reserve University (CWRU) and Paderborn University (PU) datasets demonstrate that NG-UDAN is effective in both single-source and mixed-source domain adaptation. Furthermore, comparisons with alternative methods validate the superiority of this approach in managing class imbalances under varying working conditions.

Supervisory control and data acquisition (SCADA) data-based wind turbine blade icing detection has been widely studied due to its low cost and easy access. However, SCADA data often present severe class imbalance and thus challenge accurate icing detection. Moreover, since data distribution discrepancy exists in both spatio-temporal features of SCADA data from different wind turbines, the well-trained model has poor classification performance on new turbines. Building new models for different turbines is high-cost and time-consuming. Thus, model cross-turbine generalizability needs improvement. To solve these problems, a cross-turbine icing detection model is proposed based on the spatio-temporal alignment transfer learning method. Specifically, building an attention-based network to extract temporal and spatial features. Then, we apply maximum mean discrepancy (MMD) algorithms on shallow and deep networks to align spatio-temporal features of source and target domains. Besides, a self-adaptive weight (SAW) loss function is employed to address the class imbalance. Finally, we develop a loss weight assignment method based on analyzing the generated loss value variations with the number of training iterations for performance enhancement. The proposed method is evaluated on real SCADA datasets. Experiment results show our proposed transfer learning method significantly improves the model cross-turbine generalizability and classification performance.

Federated Learning (FL) enables collaborative model training across decentralized clients without sharing raw data, yet faces significant challenges in real-world settings where client data distributions evolve dynamically over time. This paper tackles the critical problem of covariate and label shifts in streaming FL environments, where non-stationary data distributions degrade model performance and necessitate a middleware layer that adapts FL to distributional shifts. We introduce ShiftEx a shift-aware mixture of experts framework that dynamically creates and trains specialized global models in response to detected distribution shifts using Maximum Mean Discrepancy for covariate shifts. The framework employs a latent memory mechanism for expert reuse and implements facility location-based optimization to jointly minimize covariate mismatch, expert creation costs, and label imbalance. Through theoretical analysis and comprehensive experiments on five datasets, we demonstrate 5.5–12.9 percentage point accuracy improvements and 22–95% faster adaptation compared to state-of-the-art FL baselines across diverse shift scenarios. The proposed approach offers a scalable, privacy-preserving middleware solution for FL systems operating in non-stationary, real-world conditions while minimizing communication and computational overhead.

Abstract. In remote sensing tree species datasets, there is a prevalent long-tail (LT) effect (class bias), which leads to significant performance differences between head and tail classes, with attribute bias (such as texture and color) further exacerbating this issue. To better address this challenge, we formally define a task: generalized tree species long-tail learning, which jointly considers class imbalance and attribute bias in real-world remote sensing scenarios. Instead of relying on traditional re-weighting or data augmentation strategies, we propose a model called GTLT-Net, which aims to extract domain-invariant features to improve the generalization performance under LT distributions. Specifically, GTLT-Net integrates a cross-intra domain–consistent contrastive learning module and an unsupervised domain adaptation module based on local maximum mean discrepancy to reduce distribution shifts between the source and target domains. We design three datasets (generalized long-tail, class-wise long-tail, and attribute-wise long-tail) to evaluate the proposed task and framework. Experimental results show that our method achieves the best performance across all datasets, with accuracy improvements of 3.02%, 2.82%, and 4.42% and precision improvements of 2.92%, 2.38%, and 4.52%, respectively.

The scarcity of fault samples limits cross-domain knowledge transfer, while training dominated by healthy samples leads to biased decision boundaries. In addition, distribution imbalance further aggravates the problem of negative transfer. To address these issues, a Multi-Source Contrastive Learning model with Pseudo-Label Self-Correction and Weight Adaptation (MSCL-PLWA) is proposed. Synthetic data are first generated using a Wasserstein Generative Adversarial Network (WGAN). Contrastive learning is then employed to train the model by evaluating the similarity between augmented instances and minimizing the distance between similar pairs of synthetic and real samples. A prototype-based pseudo-labeling algorithm is subsequently applied for pseudo-label self-correction, and a multi-pseudo-label-guided Local Maximum Mean Discrepancy (LMMD) strategy is incorporated to enhance subdomain alignment. Furthermore, an adaptive weighting mechanism is introduced to assign higher weights to source domains more relevant to the target domain, thereby reducing the adverse impact of less relevant domains and mitigating negative transfer. Experimental results on two bearing platforms demonstrate that MSCL-PLWA effectively suppresses negative transfer and exhibits strong cross-domain fault diagnosis performance.

Fraud detection in e-commerce remains a challenging task due to class imbalance and the high-dimensional nature of transactional data. This paper proposes CHEMTTL (Comprehensive Hybrid Ensemble with Multi-Task and Transfer Learning), a novel deep learning framework aimed at improving fraud detection performance. By integrating transfer learning for feature extraction, multiple ensemble models (LightGBM, XGBoost, CatBoost, and MLP), an attention-based fusion mechanism, and multi-task outputs, CHEMTTL addresses the challenges of fraud detection. The model combines task-specific losses and domain adaptation through Maximum Mean Discrepancy (MMD), enhancing both predictive accuracy and generalization across domains. Experimental results demonstrate the superiority of CHEMTTL over traditional models, such as RF, GBDT, CNN, and LSTM, in various performance metrics. Ablation studies validate the contribution of each component in enhancing the model’s overall performance.

Traditional cross-domain transfer learning faces significant challenges in rotating machinery fault diagnosis due to the general scarcity of fault samples. Furthermore, under complex operating conditions, differences in data distribution between the source domain and target domain can also cause negative migration. To overcome these limitations, this paper proposes a multi-source contrastive learning domain adaptation (MSCLDA) method. Firstly, the MSCLDA approach employs a Wasserstein generative adversarial network to synthesize minority-class fault samples. Subsequently, it integrates a supervised contrastive learning strategy to optimize feature consistency between real and synthetic samples, thereby effectively mitigating the training bias caused by category imbalance. Furthermore, we devise a prototype-based multi-pseudo label self-correction mechanism. By incorporating this mechanism with a multiple pseudo-label-guided local maximum mean discrepancy strategy, we achieve precise alignment of feature distributions between the source domains and target subdomains. Finally, an adaptive weighting mechanism is introduced to assign higher weights to source domains that are more relevant to the target domain, thereby reducing the adverse impact of less relevant domains and mitigating negative transfer. Cross-domain experiments on current signals from industrial robots and vibration signals from bearings show that MSCLDA achieves superior fault identification performance under various working conditions, with an average accuracy of up to 99.05%, and effectively suppresses negative transfer.

Hyperspectral image (HSI) classification is pivotal for remote sensing applications, including environmental monitoring, precision agriculture, and urban land-use analysis. However, its accuracy is often limited by scarce labeled data, class imbalance, and domain discrepancies between standard RGB and HSI imagery. Although recent deep learning approaches, such as 3D convolutional neural networks (3D-CNNs), transformers, and generative adversarial networks (GANs), show promise, they struggle with spectral fidelity, computational efficiency, and cross-domain adaptation in label-scarce scenarios. To address these challenges, we propose the Transformer–Graph Convolutional Network–Diffusion with Hybrid Domain Adaptation (TGDHTL) framework. This framework integrates domain-adaptive alignment of RGB and HSI data, efficient synthetic data generation, and multi-scale spectral–spatial modeling. Specifically, a lightweight transformer, guided by Maximum Mean Discrepancy (MMD) loss, aligns feature distributions across domains. A class-conditional diffusion model generates high-quality samples for underrepresented classes in only 15 inference steps, reducing labeled data needs by approximately 25% and computational costs by up to 80% compared to traditional 1000-step diffusion models. Additionally, a Multi-Scale Stripe Attention (MSSA) mechanism, combined with a Graph Convolutional Network (GCN), enhances pixel-level spatial coherence. Evaluated on six benchmark datasets including HJ-1A and WHU-OHS, TGDHTL consistently achieves high overall accuracy (e.g., 97.89% on University of Pavia) with just 11.9 GFLOPs, surpassing state-of-the-art methods. This framework provides a scalable, data-efficient solution for HSI classification under domain shifts and resource constraints.

The foremost challenge to causal inference with real-world data is to handle the imbalance in the covariates with respect to different treatment options, caused by treatment selection bias. To address this issue, recent literature has explored domain-invariant representation learning based on different domain divergence metrics (e.g., Wasserstein distance, maximum mean discrepancy, position-dependent metric, and domain overlap). In this paper, we reveal the weaknesses of these strategies, i.e., they lead to the loss of predictive information when enforcing the domain invariance; and the treatment effect estimation performance is unstable, which heavily relies on the characteristics of the domain distributions and the choice of domain divergence metrics. Motivated by information theory, we propose to learn the Infomax and Domain-Independent Representations to solve the above puzzles. Our method utilizes the mutual information between the global feature representations and individual feature representations, and the mutual information between feature representations and treatment assignment predictions, in order to maximally capture the common predictive information for both treatment and control groups. Moreover, our method filters out the influence of instrumental and irrelevant variables, and thus it effectively increases the predictive ability of potential outcomes. Experimental results on both the synthetic and real-world datasets show that our method achieves state-of-the-art performance on causal effect inference. Moreover, our method exhibits reliable prediction performances when facing data with different characteristics of data distributions, complicated variable types, and severe covariate imbalance.

No abstract available

No abstract available

This paper proposes a novel approach for unsupervised domain adaptation (UDA) with target shift. Target shift is a problem of mismatch in label distribution between source and target domains. Typically it appears as class-imbalance in target domain. In practice, this is an important problem in UDA; as we do not know labels in target domain datasets, we do not know whether or not its distribution is identical to that in the source domain dataset. Many traditional approaches achieve UDA with distribution matching by minimizing mean maximum discrepancy or adversarial training; however these approaches implicitly assume a coincidence in the distributions and do not work under situations with target shift. Some recent UDA approaches focus on class boundary and some of them are robust to target shift, but they are only applicable to classification and not to regression. To overcome the target shift problem in UDA, the proposed method, partially shared variational autoencoders (PS-VAEs), uses pair-wise feature alignment instead of feature distribution matching. PS-VAEs inter-convert domain of each sample by a CycleGAN-based architecture while preserving its label-related content. To evaluate the performance of PS-VAEs, we carried out two experiments: UDA with class-unbalanced digits datasets (classification), and UDA from synthesized data to real observation in human-pose-estimation (regression). The proposed method presented its robustness against the class-imbalance in the classification task, and outperformed the other methods in the regression task with a large margin.

No abstract available

Existing two-sample testing techniques, particularly those based on choosing a kernel for the Maximum Mean Discrepancy (MMD), often assume equal sample sizes from the two distributions. Applying these methods in practice can require discarding valuable data, unnecessarily reducing test power. We address this long-standing limitation by extending the theory of generalized U-statistics and applying it to the usual MMD estimator, resulting in new characterization of the asymptotic distributions of the MMD estimator with unequal sample sizes (particularly outside the proportional regimes required by previous partial results). This generalization also provides a new criterion for optimizing the power of an MMD test with unequal sample sizes. Our approach preserves all available data, enhancing test accuracy and applicability in realistic settings. Along the way, we give much cleaner characterizations of the variance of MMD estimators, revealing something that might be surprising to those in the area: while zero MMD implies a degenerate estimator, it is sometimes possible to have a degenerate estimator with nonzero MMD as well; we give a construction and a proof that it does not happen in common situations.

The intrusion detection system (IDS) ensures the safe and stable operation of the industrial control system (ICS). However, due to the lack of data in ICS and the influences of numerous communication protocols, the detection performance of the IDS constructed with the unbalanced dataset of ICS is limited. In this article, a causal inference-based adversarial adaptive approach is proposed to improve the detection performance. First, the data feature space mapping between cross-domain datasets is realized through causal inference. Second, the graph structure relationship and time series features contained in the data features are mined and two-dimensional. Finally, IDS is constructed through common domain-adversarial transfer learning based on high-impact features and fine-tuning based on remaining features. This method can not only construct a cross-application or cross-protocol IDS with a high F1-score for imbalanced data, but also detect some new attacks in the target domain. As for the problem of cross-domain data imbalance, the F1-scores of the trained ICS model in the two cross-domain tasks respectively reached 97.27% and 97.78%. In the detection of new attacks in the target domain, the trained ICS model achieved an average F1-score of 97% for known attacks and the best F1-scores of the two cross-domain tasks reached 90% and 56%.

No abstract available

Diagnosing compound faults in rotating machinery remains a major challenge due to the scarcity and imbalance of labeled training data, especially under varying operating conditions. To address this issue, this paper proposes a multi-source adversarial domain adaptation (MSADA) framework that integrates data augmentation, dynamic domain alignment, and multi-label learning. First, a deep convolutional generative adversarial network is employed to generate high-fidelity compound fault signals from single-fault data, thereby alleviating data imbalance and enriching label space. Second, the MSADA framework utilizes multiple heterogeneous source domains (SDs) and incorporates an adversarial weighting mechanism to dynamically assess the relevance between each SD and the target domain. This allows the model to suppress irrelevant information and enhance the contribution of shared fault categories during domain adaptation. Finally, a multi-label classifier is designed to decompose compound fault diagnosis into sequential binary classification tasks, explicitly modeling label dependencies.Experiments conducted on two datasets involving vibration and current signals demonstrate that MSADA consistently outperforms existing methods.

Skin lesion datasets used in the research are highly imbalanced; Generative Adversarial Networks can generate synthetic skin lesion images to solve the class imbalance problem, but it can result in bias and domain shift. Domain shifts in skin lesion datasets can also occur if different instruments or imaging resolutions are used to capture skin lesion images. The deep learning models may not perform well in the presence of bias and domain shift in skin lesion datasets. This work presents a domain adaptation algorithm-based methodology for mitigating the effects of domain shift and bias in skin lesion datasets. Six experiments were performed using two different domain adaptation architectures. The domain adversarial neural network with two gradient reversal layers and VGG13 as a feature extractor achieved the highest accuracy and F1 score of 0.7567 and 0.75, respectively, representing an 18.47% improvement in accuracy over the baseline model.

For safety, reliability, and uninterrupted output of gas turbines, aviation engines, power-generating equipment, pumps, gears, compressors etc, rotor mass imbalance must be detected and diagnosed to avoid catastrophic failure. Industry 4.0 relies on predictive digital maintenance and deep learning-based convolutional neural network (CNN), which predicts defects but fails if the operating conditions change. Research studies in various fields indicate that the domain shift issue occurs due to source and target samples being from different domains, which reduces prediction capability. Moreover, research studies are scarce in examining prediction capability under varying operating speeds for rotor mass imbalance. Hence, this research proposes the adversarial discriminative domain adaptation (ADDA) technique which predicts machine failures under various operational conditions. The efficacy of ADDA has been explored by introducing 1D-CNN as a source and a target encoder inside ADDA’s architecture to take advantage of CNN’s feature extraction capability. Further, this research effectively tackles CNN’s inherent issues of overfitting and hyperparameters value selection. Furthermore, The real-world scenario has more healthy samples than fault condition samples, causing a multiclass imbalance in sample data, which affects the classification decision boundary and causes biased prediction. Hence, the proposed methodology first addresses the class imbalance through synthetic minority oversampling (SMOTE), then genetic algorithm optimizes 1D-CNN’s hyperparameters, and the effective dropout layer positioning solves the overfitting. Finally, the deep learning-based SMOTE_ADDA_GO-1D-CNN decreases domain discrepancy with ADDA. The proposed methodology’s efficacy has been explored through F1-Score, which is used as multiclass evaluation metrics, and it has been benchmarked against standard machine learning and deep learning algorithms. The test results of the proposed methodology surpassed all of them with maximum prediction accuracy. Thus, this study contributes to rotor massimbalance detection and diagnosis for multiclass imbalanced data under varying operational conditions by successfully overcoming potential challenges during fault prediction.

In multi-floor indoor environments, the discrepancy of wireless feature distributions across different floors poses challenges to fingerprinting-based localization. This letter considers the multi-floor indoor localization problem with imbalanced fingerprint data among different floors, and aims to achieve accurate localization even on the floor with very limited fingerprint data. To address the issue of feature distribution discrepancy, this letter proposes a novel domain adaptation-based framework, termed Multi-source Domain Adversarial Adaptation for Localization (MDAALoc). MDAALoc consists of four key modules: weighted random sampling, hybrid feature extractor, floor discriminator, and location predictor. Through adversarial training, MDAALoc can effectively align cross-domain feature distributions while mitigating data imbalance. Experimental results demonstrate that the proposed MDAALoc significantly outperforms state-of-the-art methods in localization accuracy.

Single-cell sequencing technologies have enabled in-depth analysis of cellular heterogeneity across tissues and disease contexts. However, as datasets increase in size and complexity, characterizing diverse cellular populations, integrating data across multiple modalities, and correcting batch effects remain challenges. We present SAFAARI (Single-cell Annotation and Fusion with Adversarial Open-set Domain Adaptation Reliable for Data Integration), a unified deep learning framework designed for cell annotation, batch correction, and multi-omics integration. SAFAARI leverages supervised contrastive learning and adversarial domain adaptation to achieve domain-invariant embeddings and enables label transfer across datasets, addressing challenges posed by batch effects, biological domain shifts, and multi-omics modalities. SAFAARI identifies novel cell types and mitigates class imbalance to enhance the detection of rare cell types. Through comprehensive benchmarking, we evaluated SAFAARI against existing annotation and integration methods across real-world datasets exhibiting batch effects and domain shifts, as well as simulated and multi-omics data. SAFAARI demonstrated scalability and robust performance in cell annotation via label transfer across heterogeneous datasets, detection of unknown cell types, correction of batch effects, and cross-omics data integration while leveraging available annotations for improved integration. SAFAARI's innovative approach outperformed competing methods in both qualitative and quantitative metrics, offering a flexible, accurate, and scalable solution for single-cell analysis with broad applicability to diverse biological and clinical research questions. An open-source implementation of the SAFAARI algorithm is available at https://github.com/VafaeeLab/SAFAARI.

Recently, using deep learning to achieve WiFi-based human activity recognition (HAR) has drawn significant attention. While capable of achieving accurate identification in a single domain (i.e., training and testing in the same consistent WiFi environment), it would become extremely tough when WiFi environments change significantly. As such, domain adversarial neural networks-based approaches have been proposed to handle such diversities across domains, yet often found to share the same limitation in practice: the imbalance between high-capacity of feature extractors and data insufficiency of source domains. This article proposes i-Sample, an intermediate sample generation-based framework, striving to tackle this issue for WiFi-based HAR. i-Sample is mainly designed as two-stage training, where four data augmentation operations are proposed to train a coarse domain-invariant feature extractor in the first stage. In the second stage, we leverage the gradients of classification error to generate intermediate samples to refine the classifiers together with original samples, making i-Sample also capable to be integrated into most domain adversarial adaptation methods without neural network modification. We have implemented a prototype system to evaluate i-Sample, which shows that i-Sample can effectively augment the performance of nowadays mainstream domain adversarial adaptation models for WiFi-based HAR, especially when source domain data is insufficient.

To address the challenges posed by significant distribution divergence between source and target domains and the scarcity of target samples in industrial equipment fault diagnosis, particularly under cross-condition and cross-platform scenarios with limited data, this paper proposes a Metric-based Meta Domain Adaptation Network (MMDAN). The proposed method integrates a Multi-scale Attention Residual Network (MARN), a domain adversarial mechanism, and a Meta-Siamese Network (MSN) to achieve deep feature extraction, cross-domain feature alignment, and rapid adaptation for accurate classification in few-shot learning settings. By incorporating multi-scale convolutions and a dual-attention mechanism, the feature representation capability is significantly enhanced. The purpose of introducing the domain discriminator is to train the feature extractor adversarially and thus improve the transfer robustness. Additionally, a task-driven meta-learning classifier with a Siamese structure is designed to mitigate issues of class imbalance and label scarcity. Experimental results on multiple industrial fault diagnosis datasets, including CWRU and RM, demonstrate that MMDAN outperforms existing methods in diagnostic accuracy and stability across various cross-domain transfer tasks. Notably, it shows strong generalization and adaptation capabilities even with extremely limited target samples, validating its broad applicability and effectiveness in real-world industrial scenarios.

In this paper, we addressed the limitation of relying solely on distribution alignment and source-domain empirical risk minimization in Unsupervised Domain Adaptation (UDA). Our information-theoretic analysis showed that this standard adversarial-based framework neglects the discriminability of target-domain features, leading to suboptimal performance. To bridge this theoretical-practical gap, we defined "good representation learning" as guaranteeing both transferability and discriminability, and proved that an additional loss term targeting target-domain discriminability is necessary. Building on these insights, we proposed a novel adversarial-based UDA framework that explicitly integrates a domain alignment objective with a discriminability-enhancing constraint. Instantiated as Domain-Invariant Representation Learning with Global and Local Consistency (RLGLC), our method leverages Asymmetrically-Relaxed Wasserstein of Wasserstein Distance (AR-WWD) to address class imbalance and semantic dimension weighting, and employs a local consistency mechanism to preserve fine-grained target-domain discriminative information. Extensive experiments across multiple benchmark datasets demonstrate that RLGLC consistently surpasses state-of-the-art methods, confirming the value of our theoretical perspective and underscoring the necessity of enforcing both transferability and discriminability in adversarial-based UDA.

Social networks are now the primary channels for sharing information about events, especially during emergencies. Identifying crucial information in these situations is intricate due to the absence of specific event labels and the prevalent label imbalance in social network data. Prior efforts in disaster event management have applied unsupervised domain adaptation (UDA) techniques. However, these efforts encounter difficulties when focusing on specific events and dealing with class-imbalance domain adaptation (CDA), often leading to modifications in data distribution. This article introduces a solution named class-imbalanced adversarial neural network (CADNN) to address UDA challenges in emergency events across diverse domains. First, an adversarial domain adaptation model is developed to achieve cross-domain feature representation. Second, unlike existing methods that rely on data augmentation, a cross-domain interpolation model is designed to generate aligned pairwise samples. This ensures that the generated samples preserve the original data distribution by preserving the sample centroid. Third, the sample interpolation is integrated with the adversarial domain adaptation model, optimizing the overall loss function to enhance generalization performance. Experimental results on four open-source datasets demonstrate that CADNN outperforms state-of-the-art models in recognizing emergency event information.

We address the problem of severe class imbalance in unsupervised domain adaptation, when the class spaces in source and target domains diverge considerably. Till recently, domain adaptation methods assumed the aligned class spaces, such that reducing distribution divergence makes the transfer between domains easier. Such an alignment assumption is invalidated in real world scenarios where some source classes are often under-represented or simply absent in the target domain. We revise the current approaches to class imbalance and propose a new one that uses latent codes in the adversarial domain adaptation framework. We show how the latent codes can be used to disentangle the silent structure of the target domain and to identify under-represented classes. We show how to learn the latent code reconstruction jointly with the domain invariant representation and use them to accurately estimate the target labels.

This paper delves into the application of adversarial domain adaptation (ADA) for enhancing credit risk assessment in financial institutions. It addresses two critical challenges: the cold start problem, where historical lending data is scarce, and the data imbalance issue, where high-risk transactions are underrepresented. The paper introduces an improved ADA framework, the Wasserstein Distance Weighted Adversarial Domain Adaptation Network (WD-WADA), which leverages the Wasserstein distance to align source and target domains effectively. The proposed method includes an innovative weighted strategy to tackle data imbalance, adjusting for both the class distribution and the difficulty level of predictions. The paper demonstrates that WD-WADA not only mitigates the cold start problem but also provides a more accurate measure of domain differences, leading to improved cross-domain credit risk assessment. Extensive experiments on real-world credit datasets validate the model's effectiveness, showcasing superior performance in cross-domain learning, classification accuracy, and model stability compared to traditional methods.

Intelligent fault diagnosis encounters the challenges of varying working conditions and sample class imbalance individually, but very few approaches address both challenges simultaneously. This article proposes an improvement network model named ICDAN-F, which can deal with fault diagnosis scenarios with class imbalance and working condition variations in an integrated way. First, Focal Loss, which was originally designed for target detection, is introduced to alleviate the sample class imbalance problem of fault diagnosis and emphasize the key features. Second, the domain discriminator is improved by the default ReLU activation function being replaced with Tanh so that useful negative value information can help extract transferable fault features. Extensive transfer experiments dealing with varying working conditions are conducted on two bearing fault datasets with the effect of class imbalance. The results show that the fault diagnosis performance of ICDAN-F outperforms several other widely used domain adaptation methods, achieving 99.76% and 96.76% fault diagnosis accuracies in Case 1 and Case 2, respectively, which predicts that ICDAN-F can handle both challenges in a cohesive manner.

Entity Resolution (ER) is a fundamental task in data integration, aiming to identify data objects across different sources that refer to the same real-world entity. In recent years, deep learning solutions have become the mainstream approach. However, their effectiveness often depends on the availability of large amounts of labeled training data, which is often challenging to obtain in many scenarios. To address this challenge, we propose utilizing Domain Adversarial Neural Networks (DANN) for efficient domain adaptation, transferring knowledge from labeled source ER datasets to new target ER datasets. However, domain adaptation for ER faces unique challenges: (1) it requires transferring knowledge or parameters from individual records, not just considering the similarity of record pairs; (2) there is significant class imbalance between positive and negative samples, affecting the model's generalization ability and accuracy; (3) in DANN, adjusting the classifier solely using labeled data from the source domain leads to features biased towards the source domain. To address these challenges, we propose a model called the Dual-Module Feature Alignment Domain Adversarial Entity Resolution (DFA-ER). DFA-ER combines a feature extractor based on relationship reconstruction and a dual-classifier domain adversarial network optimized with the Maximize Classifier Difference (MCD) strategy, to enhance applicability to ER tasks. DFA-ER adopts a dual-module network architecture and uses adversarial training to align the feature distribution of entity record pairs in different domains from the perspectives of domain invariant features and domain discriminant features, thereby improving the performance of target domain tasks. Experimental results show that, compared to baseline methods, DFA-ER achieves better performance across various dataset settings.

Cross‐domain object detection aims to generalize the distribution of features extracted by an object detector from an annotated domain to an unknown and unlabelled domain. Although one‐stage cross‐domain object detectors have significant advantages in deployment than two‐stage ones, they suffer from two problems. First, neglect of category features and inaccurate alignment between multiple category features would lead to decreased domain adaptation efficiency. Second, one‐stage detectors are more sensitive to imbalance of samples and negative samples severely affect the alignment process of domain adaptation. To overcome these two problems, an innovative category‐related attention domain adaptive method that refines discrimination for each category's feature has been proposed in this paper. In the proposed method, a group of domain discriminators is assigned to each category to refine the fine‐grained features between categories. The discriminators are trained via an adversarial discriminant framework to align the fine‐grained distributions cross different domains. A category attention alignment (CAA) module is proposed to navigate more attention to the foreground regions in instance‐level, which effectively alleviates the negative migration problem caused by the positive and negative sample imbalance of the one‐stage detector. Specifically, two sub‐modules in the CAA module are developed: a local CAA module and a global CAA module. These modules aim to optimize the domain offsets in both the local and global dimensions. In addition, a progressive global alignment module is designed to align image‐level features, offering prior knowledge of migration for the CAA module. The progressive global alignment module and CAA module collaboratively engage in benign competition with the backbone network across various levels. Extensive transferring experiments are conducted among cityscapes, foggy cityscapes, SIM10K, and KITTI. Experimental results show that the proposed method has much superior performance than other one‐stage cross‐domain detectors.

Deep learning has significantly advanced in computer vision, but the performance of these well-trained models often degrades in cross-domain situations due to domain differences. To solve this challenge, adversarial learning is widely utilized in domain adaptation to reduce the differences between domains and explore domain-invariant information. However, as a result of the instability associated with adversarial learning and the neglect of domain-specific information, it often falls into the sub-optimal solution. In this paper, we propose a set of straightforward and powerful methods which do not use adversarial learning to reduce the performance degradation that occurs in cross-domain tasks. Specifically, we first propose GF2DA to reduce the distribution gap by shifting the source to the target. We also provide a method called KCW, which re-weights the training loss by calculating the class distribution of the inputted images, to solve the class imbalance of both domains. Compared with the adversarial-based DA methods, our methods are straightforward but powerful without additional feature-level operations. We provide a baseline for adversarial-free domain adaptive semantic segmentation. Extensive experiments prove that our methods can perform similarly or even better than adversarial-based methods on cross-domain semantic segmentation.

Unsupervised domain adaptation (UDA) for the semantic segmentation of remote sensing images is challenging since the same class of objects may have different spectra while the different class of objects may have the same spectrum. To address this issue, we propose a class-aware generative adversarial network (CaGAN) for UDA semantic segmentation of multisource remote sensing images, which explicitly models the discrepancies of intraclass and the interclass between the source domain images with labels and the target domain images without labels. Specifically, first, to enhance the global domain alignment (GDA), we propose a transferable attention alignment (TAA) procedure to add more fine-grained features into the adversarial learning framework. Then, we propose a novel class-aware domain alignment (CDA) approach in semantic segmentation. CDA mainly includes two parts: the first one is adaptive category selection, which is to alleviate the class imbalance and select the reliable per-category centers in the source and target domains; the second one is adaptive category alignment, which is to model the intraclass compactness and interclass separability from source-only, target-only, and joint source and target images. Finally, the CDA plays as a penalty of GDA to train GaGAN in an alternating and iterative manner. Experiments on domain adaptation of space to space, spectrum to spectrum, both space-to-space and spectrum-to-spectrum data sets demonstrate that CaGAN outperforms the current state-of-the-art methods, which may serve as a starting point and baseline for the comprehensive applications of semantic segmentation in cross-space and cross-spectrum remote sensing images.

Unsupervised domain adaptation for semantic segmentation aims to transfer knowledge from a labeled source domain to another unlabeled target domain. However, due to the label noise and domain mismatch, learning directly from source domain data tends to have poor performance. Though adversarial learning methods strive to reduce domain discrepancies by aligning feature distributions, traditional methods suffer from the training imbalance and feature distortion problems. Besides, due to the absence of target domain labels, the classifier is blind to features from the target domain during training. Consequently, the final classifier overfits the source domain features and usually fails to predict the structured outputs of the target domain. To alleviate these problems, we focus on enhancing the adversarial learning based feature alignment from three perspectives. First, a classification constrained discriminator is proposed to balance the adversarial training and alleviate the feature distortion problem. Next, to alleviate the classifier overfitting problem, self-training is collaboratively used to learn a domain robust classifier with target domain pseudo labels. Moreover, an efficient class centroid calculation module is proposed and the domain discrepancy is further reduced by aligning the feature centroids of the same class from different domains. Experimental evaluations on GTA5 <inline-formula><tex-math notation="LaTeX">$\rightarrow$</tex-math></inline-formula> Cityscapes and SYNTHIA <inline-formula><tex-math notation="LaTeX">$\rightarrow$</tex-math></inline-formula> Cityscapes demonstrate state-of-the-art results compared to other counterpart methods. The source code and models have been made available at.<xref ref-type="fn" rid="fn1">¹</xref><fn id="fn1"><label>¹</label><p>[Online]. Available: <uri>https://github.com/NUST-Machine-Intelligence-Laboratory/EFA</uri>.</p></fn>

Unsupervised domain adaptive object detection is a challenging vision task where object detectors are adapted from a label-rich source domain to an unlabeled target domain. Recent advances prove the efficacy of the adversarial based domain alignment where the adversarial training between the feature extractor and domain discriminator results in domain-invariance in the feature space. However, due to the domain shift, domain discrimination, especially on low-level features, is an easy task. This results in an imbalance of the adversarial training between the domain discriminator and the feature extractor. In this work, we achieve a better domain alignment by introducing an auxiliary regularization task to improve the training balance. Specifically, we propose Adversarial Image Reconstruction (AIR) as the regularizer to facilitate the adversarial training of the feature extractor. We further design a multi-level feature alignment module to enhance the adaptation performance. Our evaluations across several datasets of challenging domain shifts demonstrate that the proposed method outperforms all previous methods, of both one- and two-stage, in most settings.

Domain adaptation (DA) aims to transfer knowledge learned from a labeled source domain to an unlabeled or a less labeled but related target domain. Ideally, the source and target distributions should be aligned to each other equally to achieve unbiased knowledge transfer. However, due to the significant imbalance between the amount of annotated data in the source and target domains, usually only the target distribution is aligned to the source domain, leading to adapting unnecessary source specific knowledge to the target domain, i.e., biased domain adaptation. To resolve this problem, in this work, we delve into the transferability estimation problem in domain adaptation and propose a non-intrusive Unbiased Transferability Estimation Plug-in (UTEP) by modeling the uncertainty of a discriminator in adversarial-based DA methods to optimize unbiased transfer. We theoretically analyze the effectiveness of the proposed approach to unbiased transferability learning in DA. Furthermore, to alleviate the impact of imbalanced annotated data, we utilize the estimated uncertainty for pseudo label selection of unlabeled samples in the target domain, which helps achieve better marginal and conditional distribution alignments between domains. Extensive experimental results on a high variety of DA benchmark datasets show that the proposed approach can be readily incorporated into various adversarial-based DA methods, achieving state-of-the-art performance.

Many deep-learning (DL)-based, domain adaptation (DA) methods for remote sensing (RS) applications rely on adversarial training strategies to align features extracted from images of different domains in a shared latent space. However, the performance of such representation matching techniques is negatively impacted when class occurrences in the target domain, for which no labeled data are available during training, are highly imbalanced. In this work, we propose a DL-based representation matching approach for DA in the context of change detection tasks. We further evaluate the approach in a deforestation mapping application, characterized by a high-class imbalance between the deforestation and no-deforestation classes. The domains represent different sites in the Amazon and Brazilian Cerrado biomes. To mitigate the class imbalance problem, we devised an unsupervised pseudolabeling scheme based on change vector analysis (CVA) that prevents the feature alignment to be biased toward the overrepresented class. The experimental results indicate that the proposed approach can improve the accuracy of cross-domain deforestation detection.

We present an approach for unsupervised domain adaptation---with a strong focus on practical considerations of within-domain class imbalance and between-domain class distribution shift---from a class-conditioned domain alignment perspective. Current methods for class-conditioned domain alignment aim to explicitly minimize a loss function based on pseudo-label estimations of the target domain. However, these methods suffer from pseudo-label bias in the form of error accumulation. We propose a method that removes the need for explicit optimization of model parameters from pseudo-labels directly. Instead, we present a sampling-based implicit alignment approach, where the sample selection procedure is implicitly guided by the pseudo-labels. Theoretical analysis reveals the existence of a domain-discriminator shortcut in misaligned classes, which is addressed by the proposed implicit alignment approach to facilitate domain-adversarial learning. Empirical results and ablation studies confirm the effectiveness of the proposed approach, especially in the presence of within-domain class imbalance and between-domain class distribution shift.

Coastal land cover mapping is a significant yet challenging pixel-level segmentation task. Domain shift between optical remote sensing imagery will give rise to remarkable performance degradation for deep supervised methods. Besides, the ground objects characterized with interclass variance and class imbalance may further aggravate the adverse effect. Traditional adversary-based domain adaptation algorithms always leverage a binary discriminator to conduct global adaptation, ignoring the detailed class information. In this article, we develop a novel class-aware domain adaptation method to address these issues. Unlike the naive single one, we propose a joint local and global adversarial adaptation framework to separately execute class-specific and global domain alignment on feature and output spaces. For the former, the introduced classwise discriminator possesses different strategies to extract labels for both data domains. Meanwhile, we restore to entropy minimization to produce high-confident target prediction rather than using the early generated pseudo label with high confidence. Furthermore, we additionally adopt comprehensive reweighting on the supervised segmentation loss to track the class imbalance problem. This manner mainly comprises the sample-based median frequency balancing and the focal loss function for the minority and hard classes. We measure the proposed method on two typical coastal datasets and compare it with other state-of-the-art models. The experimental results confirm its excellent and competitive performance on cross-domain land cover mapping.

In a wide range of classification tasks, training data will produce class-imbalance due to collection difficulties in some classes, which leads to prediction biases on minority classes. For the class-imbalanced problem, existing researches are usually based on the assumption that the training dataset and the test dataset are from similar distributions. In reality, both of the datasets often come from domains with different distributions, which challenges generalization performances of models. In this paper, a decoupling deep domain adaptation method is proposed to overcome these problems. Based on the adversarial domain adaptation model, the method uses a two-stage training strategy which decouples representation learning and classifier adjustment. The results of experiments under scenarios of bearing fault diagnosis and digit images classification with class-imbalance and domain discrepancy show that the effect of the combination of domain adaptation method and specific decoupling strategy is better than that of one-stage training only using resampling or cost-sensitive methods in the domain adaptation model.

Data sparsity and data imbalance are practical and challenging issues in cross-domain recommender systems (RSs). This paper addresses those problems by leveraging the concepts which derive from representation learning, adversarial learning, and transfer learning (particularly, domain adaptation). Although various transfer learning methods have shown promising performance in this context, our proposed novel method RecSys-DAN focuses on alleviating the cross-domain and within-domain data sparsity and data imbalance and learns transferable latent representations for users, items, and their interactions. Different from the existing approaches, the proposed method transfers the latent representations from a source domain to a target domain in an adversarial way. The mapping functions in the target domain are learned by playing a min–max game with an adversarial loss, aiming to generate domain indistinguishable representations for a discriminator. Four neural architectural instances of ResSys-DAN are proposed and explored. Empirical results on real-world Amazon data show that, even without using labeled data (i.e., ratings) in the target domain, RecSys-DAN achieves competitive performance as compared to the state-of-the-art supervised methods. More importantly, RecSys-DAN is highly flexible to both unimodal and multimodal scenarios, and thus it is more robust to the cold-start recommendation which is difficult for the previous methods.

Current state-of-the-art object detectors can have significant performance drop when deployed in the wild due to domain gaps with training data. Unsupervised Domain Adaptation (UDA) is a promising approach to adapt models for new domains/environments without any expensive label cost. However, without ground truth labels, most prior works on UDA for object detection tasks can only perform coarse image-level and/or feature-level adaptation by using adversarial learning methods. In this work, we show that such adversarial-based methods can only reduce the domain style gap, but cannot address the domain content distribution gap that is shown to be important for object detectors. To overcome this limitation, we propose the Cross-Domain Semi-Supervised Learning (CDSSL) framework by leveraging high-quality pseudo labels to learn better representations from the target domain directly. To enable SSL for cross-domain object detection, we propose fine-grained domain transfer, progressive-confidence-based label sharpening and imbalanced sampling strategy to address two challenges: (i) non-identical distribution between source and target domain data, (ii) error amplification/accumulation due to noisy pseudo labeling on the target domain. Experiment results show that our proposed approach consistently achieves new state-of-the-art performance (2.2% - 9.5% better than prior best work on mAP) under various domain gap scenarios. The code will be released.

Unsupervised domain transfer is the task of transferring or translating samples from a source distribution to a different target distribution. Current solutions unsupervised domain transfer often operate on data on which the modes of the distribution are well-matched, for instance have the same frequencies of classes between source and target distributions. However, these models do not perform well when the modes are not well-matched, as would be the case when samples are drawn independently from two different, but related, domains. This mode imbalance is problematic as generative adversarial networks (GANs), a successful approach in this setting, are sensitive to mode frequency, which results in a mismatch of semantics between source samples and generated samples of the target distribution. We propose a principled method of re-weighting training samples to correct for such mass shift between the transferred distributions, which we call batch weight. We also provide rigorous probabilistic setting for domain transfer and new simplified objective for training transfer networks, an alternative to complex, multi-component loss functions used in the current state-of-the art image-to-image translation models. The new objective stems from the discrimination of joint distributions and enforces cycle-consistency in an abstract, high-level, rather than pixel-wise, sense. Lastly, we experimentally show the effectiveness of the proposed methods in several image-to-image translation tasks.

Improving ship classification performance in synthetic aperture radar (SAR) imagery by transferring knowledge from the related domain is a newly emerging research topic. Existing methods follow supervised or unsupervised homogeneous transfer learning (TL) techniques with certain restrictions on the use of features (homogeneous rather than heterogeneous) and data [ignoring to excavate the potential of unlabeled target domain (TD) data], which may hinder further performance improvements. To address these problems, this letter proposes a dynamic joint correlation alignment (DJ-CORAL) network to conduct semisupervised heterogeneous domain adaptation (HDA). Specifically, DJ-CORAL first transforms the heterogeneous features from the source and TDs into a common subspace to eliminate the heterogeneity and then simultaneously performs classifier adaptation and joint marginal, and conditional distribution alignment (CDA) to facilitate the domain shift minimization. Comprehensive experiments validate the superiority of the proposed DJ-CORAL network against state-of-the-art HDA methods. The codes are available at https://github.com/BUCT-RS-ML/DJ-CORAL.

In this chapter, we present CORrelation ALignment (CORAL), a simple yet effective method for unsupervised domain adaptation. CORAL minimizes domain shift by aligning the second-order statistics of source and target distributions, without requiring any target labels. In contrast to subspace manifold methods, it aligns the original feature distributions of the source and target domains, rather than the bases of lower-dimensional subspaces. It is also much simpler than other distribution matching methods. CORAL performs remarkably well in extensive evaluations on standard benchmark datasets. We first describe a solution that applies a linear transformation to source features to align them with target features before classifier training. For linear classifiers, we propose to equivalently apply CORAL to the classifier weights, leading to added efficiency when the number of classifiers is small but the number and dimensionality of target examples are very high. The resulting CORAL Linear Discriminant Analysis (CORAL-LDA) outperforms LDA by a large margin on standard domain adaptation benchmarks. Finally, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (DNNs). The resulting Deep CORAL approach works seamlessly with DNNs and achieves state-of-the-art performance on standard benchmark datasets. Our code is available at: https://github.com/VisionLearningGroup/CORAL.

Due to the well-known domain shift problem, directly deploying a trained multi-modal classifier to a new environment usually leads to poor performance. The existing multi-modal domain adaption methods not only lack the fine-grained information of cross-modal data distribution, but also lack the cross-modal correlation research. Therefore, this paper proposes a multi-modal domain adaption method based on parameter fusion and two-step alignment (PFTS) to solve the related problems. The consistency of network parameters is used to enhance the correlation among modalities, and a higher-order moment measurement is introduced to improve the alignment of data distribution at the fine-grained level. In addition, the weighting of each modality is further carried out to achieve focused transfer. Comprehensive experiments based on multi-modal datasets with different domain adaption settings have been conducted, the results show that the precision of PFTS is 5.38% higher than state-of-the-art multi-modal domain adaption methods.

Domain adaptation (DA) and domain generalization (DG) have emerged as a solution to the domain shift problem where the distribution of the source and target data is different. The task of DG is more challenging than DA as the target data is totally unseen during the training phase in DG scenarios. The current state-of-the-art employs adversarial techniques, however, these are rarely considered for the DG problem. Furthermore, these approaches do not consider correlation alignment which has been proven highly beneficial for minimizing domain discrepancy. In this paper, we propose a correlation-aware adversarial DA and DG framework where the features of the source and target data are minimized using correlation alignment along with adversarial learning. Incorporating the correlation alignment module along with adversarial learning helps to achieve a more domain agnostic model due to the improved ability to reduce domain discrepancy with unlabeled target data more effectively. Experiments on benchmark datasets serve as evidence that our proposed method yields improved state-of-the-art performance.

Domain shift caused by varying operating conditions severely limits the practical application of intelligent fault diagnosis in rotating machinery. To address this challenge, this paper proposes a domain generalization method based on structured moment matching with discriminative enhancement contrastive learning (SMM-DECL). The method designs a structured moment matching strategy that hierarchically aligns first-order moments and correlation structures of feature distributions, preserving structured dependencies across domains while avoiding information loss from traditional distribution alignment. A discriminative enhancement contrastive learning framework is constructed with a hard sample mining strategy to optimize intra-class aggregation and inter-class separation losses, enhancing model discriminability while maintaining cross-domain feature consistency. An end-to-end collaborative optimization framework integrates structured alignment and discriminative enhancement. Experimental validation on bearing and gearbox fault datasets demonstrates that SMM-DECL achieves the best performance across all transfer tasks, significantly outperforming existing domain generalization methods and providing an effective solution for cross-domain fault diagnosis in rotating machinery.

This paper proposes an interpretable two-stream transformer CORAL networks (TransCORALNet) for supply chain credit assessment under the segment industry and cold start problem. The model aims to provide accurate credit assessment prediction for new supply chain borrowers with limited historical data. Here, the two-stream domain adaptation architecture with correlation alignment (CORAL) loss is used as a core model and is equipped with transformer, which provides insights about the learned features and allow efficient parallelization during training. Thanks to the domain adaptation capability of the proposed model, the domain shift between the source and target domain is minimized. Therefore, the model exhibits good generalization where the source and target do not follow the same distribution, and a limited amount of target labeled instances exist. Furthermore, we employ Local Interpretable Model-agnostic Explanations (LIME) to provide more insight into the model prediction and identify the key features contributing to supply chain credit assessment decisions. The proposed model addresses four significant supply chain credit assessment challenges: domain shift, cold start, imbalanced-class and interpretability. Experimental results on a real-world data set demonstrate the superiority of TransCORALNet over a number of state-of-the-art baselines in terms of accuracy. The code is available on GitHub https://github.com/JieJieNiu/TransCORALN .

Transfer learning (TL) has been applied in seizure detection to deal with differences between different subjects or tasks. In this paper, we consider cross-subject seizure detection that does not rely on patient history records, that is, acquiring knowledge from other subjects through TL to improve seizure detection performance. We propose a novel domain adaptation method, named the Cluster Embedding Joint-Probability-Discrepancy Transfer (CEJT), for data distribution structure learning. Specifically, 1) The joint probability distribution discrepancy is minimized to reduce the distribution shift in the source and target domains, and strengthen the discriminative knowledge of classes. 2) A clustering is performed on the target domain, and the class centroids of sources is used as the clustering prototype of the target domain to enhance data structure. It is worth noting that the manifold regularization is used to improve the quality of clustering prototypes. In addition, a correlation-alignment-based source selection metric (SSC) is designed for most favorable subject selection, reducing the computational cost as well as avoiding some negative transfer. Experiments on 15 patients with focal epilepsy from the Children’s Hospital, Zhejiang University School of Medicine (CHZU) database shown that CEJT outperforms several state-of-the-art approaches, and can promote the application of seizure detection.

Domain shift, which occurs when there is a mismatch between the distributions of training (source) and testing (target) datasets, usually results in poor performance of the trained model on the target domain. Existing algorithms typically solve this issue by reducing the distribution discrepancy in the input spaces. However, for kernel-based learning machines, performance highly depends on the statistical properties of data in reproducing kernel Hilbert spaces (RKHS). Motivated by these considerations, we propose a novel strategy for matching distributions in RKHS, which is done by aligning the RKHS covariance matrices (descriptors) across domains. This strategy is a generalization of the correlation alignment problem in Euclidean spaces to (potentially) infinite-dimensional feature spaces. In this paper, we provide two alignment approaches, for both of which we obtain closed-form expressions via kernel matrices. Furthermore, our approaches are scalable to large datasets since they can naturally handle out-of-sample instances. We conduct extensive experiments (248 domain adaptation tasks) to evaluate our approaches. Experiment results show that our approaches outperform other state-of-the-art methods in both accuracy and computationally efficiency.

Deep learning-based AI models typically require a large amount of high-quality annotated data to achieve optimal performance. However, the label distribution shift caused by noisy annotations can lead to perturbations in the classification boundary, reducing the robustness and generalization capabilities of deep learning models. To mitigate this issue, we transform the problem of learning from noisy labels into a semi-supervised learning problem, and propose a novel Semi-Supervised Distribution Alignment (SSDA) framework that strategically integrates noise-robust distribution alignment within a unified semi-supervised learning paradigm for combating noisy labels. By leveraging the similarity distribution between historical predictions, the proposed SSDA approach benefits from a flexible multi-historical regression modeling strategy, which aims to identify high-confidence samples/pairs and recalibrate the label shift through pseudo-labels. Furthermore, our approach employs a comprehensive multi-granularity distribution adaptation strategy, incorporating both instance-wise and class-aware distribution alignment to quantitatively minimize semantic discrepancies across different mixed feature domains. In this way, our SSDA approach ultimately achieves more resilient and generalizable performance against label noise, even in the presence of substantial noise. Extensive experiments conducted on multiple simulated and real-world noisy benchmark datasets consistently demonstrate the superiority and effectiveness of our SSDA method compared to existing state-of-the-art baselines.

Large language models (LLMs) offer a promising way to simulate human survey responses, potentially reducing the cost of large-scale data collection. However, existing zero-shot methods suffer from prompt sensitivity and low accuracy, while conventional fine-tuning approaches mostly fit the training set distributions and struggle to produce results more accurate than the training set itself, which deviates from the original goal of using LLMs to simulate survey responses. Building on this observation, we introduce Distribution Shift Alignment (DSA), a two-stage fine-tuning method that aligns both the output distributions and the distribution shifts across different backgrounds. By learning how these distributions change rather than fitting training data, DSA can provide results substantially closer to the true distribution than the training data. Empirically, DSA consistently outperforms other methods on five public survey datasets. We further conduct a comprehensive comparison covering accuracy, robustness, and data savings. DSA reduces the required real data by 53.48-69.12%, demonstrating its effectiveness and efficiency in survey simulation.

No abstract available

No abstract available

Deep neural networks are able to learn powerful representations from large quantities of labeled input data, however they cannot always generalize well across changes in input distributions. Domain adaptation algorithms have been proposed to compensate for the degradation in performance due to domain shift. In this paper, we address the case when the target domain is unlabeled, requiring unsupervised adaptation. CORAL is a "frustratingly easy" unsupervised domain adaptation method that aligns the second-order statistics of the source and target distributions with a linear transformation. Here, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (Deep CORAL). Experiments on standard benchmark datasets show state-of-the-art performance.

Tabular learning transforms raw features into optimized spaces for downstream tasks, but its effectiveness deteriorates under distribution shifts between training and testing data. We formalize this challenge as the Distribution Shift Tabular Learning (DSTL) problem and propose a novel Shift-Aware Feature Transformation (SAFT) framework to address it. SAFT reframes tabular learning from a discrete search task into a continuous representation-generation paradigm, enabling differentiable optimization over transformed feature sets. SAFT integrates three mechanisms to ensure robustness: (i) shift-resistant representation via embedding decorrelation and sample reweighting, (ii) flatness-aware generation through suboptimal embedding averaging, and (iii) normalization-based alignment between training and test distributions. Extensive experiments show that SAFT consistently outperforms prior tabular learning methods in terms of robustness, effectiveness, and generalization ability under diverse real-world distribution shifts.

Frequency-domain channel extrapolation is effective in reducing pilot overhead for massive multiple-input multiple-output (MIMO) systems. Recently, deep learning (DL) based channel extrapolators have become promising candidates for modeling complex frequency-domain dependency. Nevertheless, current DL extrapolators fail to operate in unseen environments under distribution shift, which poses challenges for large-scale deployment. In this paper, environment generalizable learning for channel extrapolation is achieved by realizing distribution alignment from a physics perspective. Firstly, the distribution shift of wireless channels is rigorously analyzed, which comprises the distribution shift of multipath structure and single-path response. Secondly, a physics-based progressive distribution alignment strategy is proposed to address the distribution shift, which includes successive path-oriented design and path alignment. Path-oriented DL extrapolator decomposes multipath channel extrapolation into parallel extrapolations of the extracted paths, which can mitigate the distribution shift of multipath structure. Path alignment is proposed to address the distribution shift of single-path response in path-oriented DL extrapolators, which eventually enables generalizable learning for channel extrapolation. In the simulation, distinct wireless environments are generated using the precise ray-tracing tool. Based on extensive evaluations, the proposed path-oriented DL extrapolator with path alignment can reduce extrapolation error by more than 6 dB in unseen environments compared to the state-of-the-arts.