Dataset similarity to assess semi-supervised learning under distribution mismatch between the labelled and unlabelled datasets

Calderon-Ramirez, Saul; Oala, Luis; Torrents-Barrena, Jordina; Yang, Shengxiang; Elizondo, David; Moemeni, Armaghan; Colreavy-Donnelly, Simon; Samek, Wojciech; Molina-Cabello, Miguel; Lopez-Rubio, Ezequiel

Dataset similarity to assess semi-supervised learning under distribution mismatch between the labelled and unlabelled datasets

dc.cclicence	CC-BY-NC	en
dc.contributor.author	Calderon-Ramirez, Saul
dc.contributor.author	Oala, Luis
dc.contributor.author	Torrents-Barrena, Jordina
dc.contributor.author	Yang, Shengxiang
dc.contributor.author	Elizondo, David
dc.contributor.author	Moemeni, Armaghan
dc.contributor.author	Colreavy-Donnelly, Simon
dc.contributor.author	Samek, Wojciech
dc.contributor.author	Molina-Cabello, Miguel
dc.contributor.author	Lopez-Rubio, Ezequiel
dc.date.acceptance	2022-04-14
dc.date.accessioned	2022-04-25T12:56:31Z
dc.date.available	2022-04-25T12:56:31Z
dc.date.issued	2022-04-22
dc.description	The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.	en
dc.description.abstract	Semi-supervised deep learning (SSDL) is a popular strategy to leverage unlabelled data for machine learning when labelled data is not readily available. In real-world scenarios, different unlabelled data sources are usually available, with varying degrees of distribution mismatch regarding the labelled datasets. It begs the question which unlabelled dataset to choose for good SSDL outcomes. ftentimes, semantic heuristics are used to match unlabelled data with labelled data. However, a quantitative and systematic approach to this election problem would be preferable. In this work, we first test the SSDL MixMatch algorithm under various distribution mismatch configurations to study the impact on SSDL accuracy. Then, we propose a quantitative unlabelled dataset selection heuristic based on dataset dissimilarity measures. These are designed to systematically assess how distribution mismatch between the labelled and unlabelled datasets affects MixMatch performance. We refer to our proposed method as deep dataset dissimilarity measures (DeDiMs), designed to compare labelled and unlabelled datasets. They use the feature space of a generic Wide-ResNet, can be applied prior to learning, are quick to evaluate and model agnostic. The strong correlation in our tests between MixMatch accuracy and the proposed DeDiMs suggests that this approach can be a good fit for quantitatively ranking different unlabelled datasets prior to SSDL training.	en
dc.funder	Other external funder (please detail below)	en
dc.funder.other	Ministry of Science Innovation and Universities of Spain	en
dc.funder.other	Autonomous Government of Andalusia	en
dc.identifier.citation	S. Calderon-Ramirez, L. Oala, J. Torrents-Barrena, S. Yang, D. Elizondo, A. Moemeni, S. Colreavy-Donnelly, W. Samek, M. A. Molina-Cabello, and E. Lopez-Rubio. (2022) Dataset similarity to assess semi-supervised learning under distribution mismatch between the labelled and unlabelled datasets. IEEE Transactions on Artificial Intelligence, 4 (2), pp. 282-291	en
dc.identifier.doi	https://doi.org/10.1109/TAI.2022.3168804
dc.identifier.issn	2691-4581
dc.identifier.uri	https://hdl.handle.net/2086/21835
dc.language.iso	en_US	en
dc.peerreviewed	Yes	en
dc.projectid	RTI2018-094645-B-I00	en
dc.projectid	UMA18-FEDERJA-084	en
dc.publisher	IEEE	en
dc.researchinstitute	Institute of Artificial Intelligence (IAI)	en
dc.subject	Semi-supervised deep learning	en
dc.subject	MixMatch	en
dc.subject	Out of distribution data	en
dc.subject	Deep learning	en
dc.subject	Distribution mismatch	en
dc.subject	Dataset similarity	en
dc.title	Dataset similarity to assess semi-supervised learning under distribution mismatch between the labelled and unlabelled datasets	en
dc.type	Article	en