Cross-Domain Federated Data Modeling on Non-IID Data

Federated learning has received sustained attention in recent years for its distributed training model that fully satisfies the need for privacy concerns.

However, under the nonindependent identical distribution, the data heterogeneity of different parties with different data patterns significantly degrades the prediction performance of the federated model.

Additionally, the federated model adopts simple averaging in the model aggregation phase, which ignores the contributions of different parties and further limits the model performance.

To conquer the above challenges, we propose a new cross-domain federated data modeling (CDFDM) scheme by combining the attention mechanism.

Firstly, to mitigate the poor model performance caused by data heterogeneity, we propose a shared model that adjusts the number of shared data assigned to users according to their data size, which effectively alleviates data heterogeneity while avoiding shared data from overwriting the user's individual data features.

Then, we introduce the attention mechanism in the model aggregation phase, which assigns weights to users according to their contributions, thus improving the model performance.

Finally, we conducted a series of experiments on two real-world datasets (MNIST and CIFAR-10).

The results show that our CDFDM outperforms existing schemes in both nonindependent identical distribution conditions.

Furthermore, in terms of model prediction accuracy variation during the training phase, our approach is more stable.