Open Access

MITIGATING CONFIRMATION BIAS IN DEEP LEARNING WITH NOISY LABELS THROUGH COLLABORATIVE NETWORK TRAINING

PDF

Jianhong Wei ¹ , Aaliyah M. Farouk ¹ ,

⁴ Department of Computer Science, Tsinghua University, Beijing, China

⁴ School of Computer and Information Sciences, University of Cape Town, South Africa

Abstract

Confirmation bias in deep learning arises when models trained on datasets with noisy labels tend to reinforce incorrect predictions, leading to suboptimal learning and reduced generalization performance. This paper proposes a collaborative network training framework to mitigate confirmation bias in the presence of label noise. In the proposed method, two networks are trained simultaneously, each selecting clean samples for the other to learn from. This cross-training strategy prevents individual networks from overfitting to noisy labels and helps preserve model diversity. The framework also incorporates a sample agreement mechanism and consistency regularization to further stabilize training and improve robustness. Experimental evaluations on benchmark datasets including CIFAR-10, CIFAR-100, and Clothing1M show that the proposed approach outperforms existing noise-robust training methods, achieving higher accuracy and better noise tolerance. The results validate the effectiveness of collaborative learning in reducing confirmation bias and improving model reliability under label noise.

Keywords

Confirmation bias, noisy labels, collaborative training

References

Ahn, C., Kim, K., Baek, J., Lim, J., and Han, S. 2023. Sample-wise label confidence incorporation for learning with noisy labels. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1823–1832.

Arazo, E., Ortego, D., Albert, P., O’Connor, N. E., and McGuinness, K. 2019. Unsupervised label noise modeling and loss correction. In Proceedings of the International Conference on Machine Learning (ICML), 312–321.

Arpit, D., Jastrzebski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., and Lacoste-Julien, S. 2017. A closer look at memorization in deep networks. In Proceedings of the International Conference on Machine Learning (ICML), 233–242.

Berthelot, D., Carlini, N., Cubuk, E. D., Kurakin, A., Sohn, K., Zhang, H., and Raffel, C. 2019. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. arXiv:1911.09785.

Berthelot, D., Carlini, N., Goodfellow, I. J., Papernot, N., Oliver, A., and Raffel, C. 2019. MixMatch: A holistic approach to semi-supervised learning. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 5049–5059.

Chang, W., Shi, Y., and Wang, J. 2023. Csot: Curriculum and structure-aware optimal transport for learning with noisy labels. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 36, 8528–8541.

Chen, H., Tao, R., Fan, Y., Wang, Y., Wang, J., Schiele, B., Xie, X., Raj, B., and Savvides, M. 2023. SoftMatch: Addressing the quantity-quality tradeoff in semi-supervised learning. In Proceedings of the 11th International Conference on Learning Representations, 1–21.

Chen, J., Zhang, R., Yu, T., Sharma, R., Xu, Z., Sun, T., and Chen, C. 2023. Label-retrieval-augmented diffusion models for learning from noisy labels. In Proceedings of the Advances in Neural Information Processing Systems. A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, Curran Associates, Inc., 66499–66517.

Chen, M., Cheng, H., Du, Y., Xu, M., Jiang, W., and Wang, C. 2023. Two wrongs don’t make a right: Combating confirmation bias in learning with label noise. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, 14765–14773.

Chen, P., Liao, B. B., Chen, G., and Zhang, S. 2019. Understanding and utilizing KS trained with noisy labels. In Proceedings of the International Conference on Machine Learning. PMLR, 1062–1070.

Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 1597–1607.

Most read articles by the same author(s)

1 2 3 4 5 6 7 8 9 10 > >>

International Journal of Modern Computer Science and IT Innovations

MITIGATING CONFIRMATION BIAS IN DEEP LEARNING WITH NOISY LABELS THROUGH COLLABORATIVE NETWORK TRAINING

Abstract

Keywords

References

Most read articles by the same author(s)

Similar Articles