Fold-stratified cross-validation for unbiased and privacy-preserving federated learning - Université d'Angers Accéder directement au contenu
Article Dans Une Revue Journal of the American Medical Informatics Association Année : 2020

Fold-stratified cross-validation for unbiased and privacy-preserving federated learning

Résumé

Abstract Objective We introduce fold-stratified cross-validation, a validation methodology that is compatible with privacy-preserving federated learning and that prevents data leakage caused by duplicates of electronic health records (EHRs). Materials and Methods Fold-stratified cross-validation complements cross-validation with an initial stratification of EHRs in folds containing patients with similar characteristics, thus ensuring that duplicates of a record are jointly present either in training or in validation folds. Monte Carlo simulations are performed to investigate the properties of fold-stratified cross-validation in the case of a model data analysis using both synthetic data and MIMIC-III (Medical Information Mart for Intensive Care-III) medical records. Results In situations in which duplicated EHRs could induce overoptimistic estimations of accuracy, applying fold-stratified cross-validation prevented this bias, while not requiring full deduplication. However, a pessimistic bias might appear if the covariate used for the stratification was strongly associated with the outcome. Discussion Although fold-stratified cross-validation presents low computational overhead, to be efficient it requires the preliminary identification of a covariate that is both shared by duplicated records and weakly associated with the outcome. When available, the hash of a personal identifier or a patient’s date of birth provides such a covariate. On the contrary, pseudonymization interferes with fold-stratified cross-validation, as it may break the equality of the stratifying covariate among duplicates. Conclusion Fold-stratified cross-validation is an easy-to-implement methodology that prevents data leakage when a model is trained on distributed EHRs that contain duplicates, while preserving privacy.
Fichier principal
Vignette du fichier
2001.08090 (907.31 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03579136 , version 1 (08-01-2024)

Identifiants

Citer

Romain Bey, Romain Goussault, François Grolleau, Mehdi Benchoufi, Raphaël Porcher. Fold-stratified cross-validation for unbiased and privacy-preserving federated learning. Journal of the American Medical Informatics Association, 2020, 27 (8), pp.1244-1251. ⟨10.1093/jamia/ocaa096⟩. ⟨hal-03579136⟩
48 Consultations
6 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More