Membership inference vulnerability and the mitigating influence of regularization and differential privacy in dental radiograph classifiers

INTRODUCTION

Panoramic radiography is a routine examination in dental practice, and deep learning classifiers trained on these images now detect caries, restorations, impacted teeth, periapical lesions, root anomalies and other conditions with accuracy that approaches or matches experienced clinicians.(1-3) Systematic reviews of this literature report sensitivities and specificities that would have been difficult to imagine a decade ago, and the field has moved from proof-of-concept toward products that are integrated into imaging workstations and clinical decision support.(4) The clinical promise of these systems, however, rests on large collections of patient radiographs, which are concentrated in the training sets of hospital groups, software vendors and research consortia. Because a single panoramic radiograph can reveal the anatomy, age, prior treatment and oral health of an identifiable person, the training set itself has become a valuable and sensitive asset, and its confidentiality has not received the same scrutiny as the diagnostic performance of the models it produces.

Membership inference is the adversary's attempt to learn whether a specific record was part of a model's training data, and it is now a mature and reproducible privacy threat for machine learning systems.(5-7) An attacker who observes only the model's confidence scores, or in some settings only its hard labels, can often distinguish members from non-members with accuracy well above chance, and the leakage is stronger for models that overfit their training distribution.(8,9) The problem matters in health care in particular because the training records are clinical images or records of real patients, and because an inference that a person's radiograph was used to train a model can reveal a diagnosis, a treatment history or a data-sharing agreement that the patient never authorized.(10,11) Related attack families, including model inversion and attribute inference, extend the same principle and demonstrate that the release of a medical model can be tantamount to the release of information about the individuals behind it.(10)

The same techniques have been demonstrated in medical imaging, where classifiers for retinal, chest and other clinical images have been shown to reveal the presence of individual patients in their training sets.(12,13) Studies that audit collaboratively trained models and those that mount query-based attacks on medical data converge on the conclusion that medical deep learning is not immune to membership leakage.(12,13) Defenses have been proposed that add generative noise or synthesize protective images, but they typically trade diagnostic accuracy for privacy.(14) What remains underexplored is whether the transfer-learning classifiers that dominate dental radiograph analysis exhibit the same vulnerability, and whether the defenses proposed in the privacy literature actually reduce the risk in this specific setting. Dental imaging has distinct properties that could matter: panoramic radiographs are highly standardized, the diagnostic tasks are often unbalanced, and the prevailing pipeline reuses a frozen general-purpose backbone, all of which shape how much a model memorizes and how much an attacker can extract.

The regulatory environment raises the stakes of this question. Data protection frameworks that govern health data in Europe, North America and a growing number of jurisdictions in Africa, Asia and Latin America treat a patient radiograph as a biometric or health datum that cannot be repurposed for model training without a lawful basis, and they impose accountability obligations on whoever processes it. If a trained classifier leaks the identity of its training subjects, that leakage becomes a potential breach of those obligations, with consequences that differ across health systems. Privacy audits such as membership inference therefore serve a compliance function as well as a research one, and their results should inform the governance of dental AI wherever patient images are pooled.(11,15)

Two families of mitigation are available. The first is conventional regularization, such as L2 weight decay and dropout, which reduces overfitting and is therefore expected, under the overfitting hypothesis, to limit membership leakage.(5,16) Its effect has been inconsistent across studies, with some reporting only modest gains against a determined attacker and others finding that the reduction in leakage is too small to be practically meaningful.(17) The second is differential privacy, formalized as a bound on the influence of any single training example, and made practical for deep learning by the differentially private stochastic gradient descent (DP-SGD) algorithm.(18-20) Differential privacy carries a well-documented utility cost, and in medical imaging that cost has been large enough to discourage deployment in several benchmark studies.(21-23) Whether that cost is acceptable for a dental diagnostic classifier, and whether it buys a meaningful reduction in membership inference, has not been established.

This study audits the membership inference risk of a transfer-learning classifier for four common dental diagnoses in panoramic radiographs and compares the mitigating influence of regularization and differential privacy. The objective was to quantify membership inference risk in a dental radiograph classifier and to measure how much L2 weight decay, dropout and DP-SGD reduce that risk. The following hypotheses were tested: H1, a baseline classifier without privacy protection is vulnerable to membership inference, with attacker AUC above chance; H2, L2 weight decay and dropout reduce membership inference AUC relative to the baseline; and H3, DP-SGD reduces membership inference AUC to approximately chance.

METHODS

RESULTS

The study included 182 annotated teeth obtained from 50 panoramic radiographs (Table 1). The diagnosis classes were uneven: caries predominated with 101 teeth, followed by impacted teeth with 40, deep caries with 32 and periapical lesions with 9.

H2: Effect of regularization on membership inference risk

Regularization reduced membership inference risk only marginally (Table 2). The loss-based AUC fell from 0.648 in the baseline to 0.647 with L2 weight decay (p = 0.005) and to 0.633 with dropout (p < 0.001). The confidence-based AUC moved in the same direction, from 0.582 to 0.581 with L2 and to 0.563 with dropout. Neither form of regularization altered test accuracy, which remained at 70.4% in both configurations, with a slightly smaller standard deviation of 3.4% under dropout.

DISCUSSION

The main finding of this audit was that a transfer-learning dental panoramic radiograph classifier trained on an openly licensed dataset leaked membership information, and that the choice of mitigation determined whether that leakage could be brought to chance. The baseline loss-based attack AUC of 0.648 places dental radiograph classifiers among the machine learning models for which membership inference is a realistic threat.(5,27)

The magnitude is consistent with the modest but systematic leakage reported for overfitting deep classifiers in other domains, and with audits of clinical image models that found membership detectable from model outputs alone.(12,13) Dental images are no exception to a pattern that has been documented for retinal and chest imaging, and the result argues against the assumption that a medical classifier is private merely because its parameters are not released.

H1 was supported, and the finding is consistent with the theoretical account that ties membership leakage to overfitting. Models that memorize the idiosyncrasies of their training examples expose those examples to inference, and the gap between the loss distribution of members and non-members is the observable trace of that memorization.(8)

The confidence-based attack was weaker than the loss-based attack in this study, a pattern that matches prior work showing that loss is a more informative signal than confidence for threshold attacks, and that label-only attacks are weaker still.(6,9) The direction of these differences matters because it identifies which output a defender must protect: a model that publishes calibrated probabilities still leaks through its losses, and the loss signal is available to any party that can compute the training objective.

The finding also aligns with the observation that membership inference attacks are easier on difficult, poorly separated tasks, which describes a four-class diagnosis over an unbalanced and small training set.(17) In that respect the dental classifier is not an outlier but a representative case of the conditions under which leakage is most pronounced.

H2 was supported statistically, but the effect size was negligible for practice. L2 weight decay reduced the loss-based AUC by only 0.001 and dropout by 0.015, and neither changed test accuracy.

The result is in line with studies that found regularization to be a weak membership inference defense: because the attacker's decision boundary is fitted to the loss surface rather than to the model's generalization gap, a small reduction in overfitting does not erase the separation between members and non-members.(16,17)

The near-null effect of L2 is unsurprising under this account, since weight decay on a single linear head hardly constrains the memorization that drives leakage, and the modest effect of dropout is consistent with its role as a stochastic regularizer that reduces, but does not eliminate, the confidence gap between members and non-members.(5)

Regularization should therefore not be relied upon as a privacy control, even though it is already present in many dental AI pipelines for other reasons. Its real value is that it costs nothing in accuracy, which makes it a sensible default but not a safeguard. These results contrast with the more optimistic view that regularization is a sufficient membership inference defense, and they reinforce the conclusion of comprehensive analyses that no amount of weight decay or dropout, applied to a well-fitting model, closes the gap to chance.(16,26)

The reason is structural rather than parametric: a linear head trained to convergence on a small feature set reproduces the same separation between members and non-members regardless of how its norm is penalized.

H3 was largely supported, with an important qualification. DP-SGD reduced the loss-based AUC from 0.648 to 0.537 and brought the confidence-based attack to chance at 0.499, which is the expected outcome for a mechanism that bounds the influence of each individual training example.(18,20) The residual loss-based leakage above 0.5 is consistent with the small-sample setting, in which the noise required for a meaningful privacy guarantee also limits how far the model can be pushed toward a stable optimum, and with the known difficulty of reaching exactly the chance level on every attack score in practice.(23,28)

The qualification is the utility collapse: test accuracy fell from 70.4% to 54.5%, near the majority-class rate, and the zero variance across splits indicates that the differentially private model predicted a constant class in every run, effectively abandoning the minority classes. This trade-off mirrors the pattern reported in medical imaging, where differentially private training has often degraded performance enough to raise doubts about clinical viability, and where privacy-utility reconciliation remains an active research problem.(21,22,29,30)

For a dental classifier, the price of near-chance membership inference was a model that no longer distinguished periapical lesions from the majority class, which is not acceptable for a diagnostic tool.

The mechanism behind the collapse of the differentially private model is instructive. In a small, unbalanced dataset the gradient signal for the minority classes is weak, and per-sample clipping together with the additive noise masks that signal, so the optimizer settles on the majority class that minimizes the expected loss.

This is not a failure of the algorithm but a direct consequence of the privacy budget: at the chosen noise level the guarantee consumes the statistical power that a linear head needs to separate nine periapical lesions from the rest. The same mechanism explains why the loss-based attack retained a small advantage over chance even after the confidence-based attack was neutralized; the loss encodes a residual difference in the per-example objective that the clipped and noisy updates never fully erase, whereas the confidence collapses toward the constant output of a degenerate model.

This distinction between the two attack scores is not a curiosity but a practical warning, because it shows that a defense that appears to reach chance on one metric can still leave a measurable signal on another.(28,31)

These findings generalize across health systems in a specific sense. Membership inference is a property of the training algorithm and the geometry of the dataset rather than of the patient population, so a classifier trained in a high-income setting is, all else being equal, no more or less private than one trained elsewhere.

What varies across jurisdictions is the size and class balance of the available training data and the regulatory weight attached to the outcome. A dental center in a low- or middle-income country that can assemble only a few hundred labelled images faces the same privacy-utility frontier documented here, and the data protection law under which it operates, whether modeled on European rules or a sectoral health regulation, determines how much residual leakage is tolerable.

The utility collapse observed under DP-SGD is therefore a caution for any deployment that relies on small, class-imbalanced dental datasets, which are common across Africa, Latin America and parts of Asia where large annotated collections are scarce. The same principle applies in reverse: a well-resourced center with tens of thousands of labelled images can afford the accuracy loss that a smaller center cannot, so the optimal privacy strategy is a function of data abundance rather than of any universal rule.

The comparison between the three hypotheses resolves the question posed by the study in a way that should guide practitioners. Conventional regularization is insufficient, differential privacy is effective but costly at this dataset scale, and the residual risk under DP-SGD, though small, is not zero. The finding that the confidence-based attack reached chance while the loss-based attack did not further implies that even a privacy-conscious deployer who hides raw probabilities cannot assume safety, because the loss signal can often be reconstructed from repeated queries and is frequently exposed through auxiliary outputs such as uncertainty estimates.(9,16)

The trade-off observed here also bears on the broader governance of dental AI, in which federated and privacy-preserving training is increasingly proposed as a route to multi-institutional datasets; those approaches inherit the same tension between confidentiality and utility documented in the medical imaging literature, and they add the complication of heterogeneous participating sites.(15,32,33)

A dental institution that releases or shares a classifier should treat the training set as exposed by default, measure the actual membership inference risk with an audit of the kind performed here, and select a privacy budget explicitly rather than assuming that any single technique is sufficient.(31)

Concretely, a vendor that publishes a model card should report the attack AUC alongside accuracy, a research group that deposits a dental classifier should disclose whether differentially private training was used and at what budget, and a regulator assessing a dental AI submission should ask whether membership inference was tested before granting authorization. These are low-cost steps that convert a technical finding into an operational safeguard, and they are portable across the jurisdictions in which dental AI is deployed.

Several strengths support the validity of these results. The audit used a public dataset with an explicit licence, a fixed random seed and a script that was executed twice with identical output, so the findings are reproducible. The attack was evaluated over ten independent splits rather than a single partition, which stabilizes the AUC estimates and allowed the hypothesis tests to reflect run-to-run variability rather than a single lucky or unlucky partition. The use of a frozen pre-trained backbone matches the transfer-learning practice that dominates dental imaging, so the vulnerability measured here is the one that would confront a typical deployed system rather than an idealized end-to-end network.

The explicit mapping of each hypothesis to a test and a decision criterion, decided before the analysis, reduces the risk of post hoc interpretation. The methodology also transferred unchanged from the general privacy literature to a dental task, which means the audit can be repeated by any dental group with a labelled radiograph set and a pre-trained backbone, without specialized privacy infrastructure. Finally, reporting both loss- and confidence-based attacks avoided the single-metric optimism that can occur when a defense is evaluated against only one attack score.

CONCLUSIONS

A baseline dental panoramic radiograph classifier was vulnerable to membership inference, with a loss-based attack AUC substantially above chance.

L2 weight decay and dropout reduced membership inference risk only negligibly and did not alter accuracy.

Differentially private training reduced the risk to near chance but collapsed diagnostic accuracy, so the choice of privacy mechanism is a measured trade-off rather than a cost-free safeguard.

The findings support the routine inclusion of membership inference audits in the development and governance of dental artificial intelligence.

References

1. 1 Ayhan B, Ayan E, Atsü S. Detection of dental caries under fixed dental prostheses by analyzing digital panoramic radiographs with artificial intelligence algorithms based on deep learning methods. BMC Oral Health. 2025;25(1):216. doi: 10.1186/s12903-025-05577-3
2. 2 Pornprasertsuk-Damrongsri S, Vachmanus S, Papasratorn D, Kitisubkanchana J, Chaikantha S, Arayasantiparb R, et al. Clinical application of deep learning for enhanced multistage caries detection in panoramic radiographs. Sci Rep. 2025;15(1):33491. doi: 10.1038/s41598-025-16591-4
3. 3 Bonfanti-Gris M, Herrera A, Salido Rodríguez-Manzaneque MP, Martínez-Rus F, Pradíes G. Deep learning for tooth detection and segmentation in panoramic radiographs: a systematic review and meta-analysis. BMC Oral Health. 2025;25(1):1280. doi: 10.1186/s12903-025-06349-9
4. 4 Silva B, Fontinele J, Vieira CLZ, Tavares JMRS, Cury PR, Oliveira L. A holistic approach for classifying dental conditions from textual reports and panoramic radiographs. Medical Image Analysis. 2025;105:103709. doi: 10.1016/j.media.2025.103709
5. 5 Shokri R, Stronati M, Song C, Shmatikov V. Membership Inference Attacks Against Machine Learning Models. 2017 IEEE Symposium on Security and Privacy (SP). 2017:3-18. doi: 10.1109/sp.2017.41
6. 6 Salem A, Zhang Y, Humbert M, Berrang P, Fritz M, Backes M. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. Proceedings 2019 Network and Distributed System Security Symposium. 2019. doi: 10.14722/ndss.2019.23119
7. 7 Hu H, Salcic Z, Sun L, Dobbie G, Yu PS, Zhang X. Membership Inference Attacks on Machine Learning: A Survey. ACM Comput Surv. 2022;54(11s):1-37. doi: 10.1145/3523273
8. 8 Yeom S, Giacomelli I, Fredrikson M, Jha S. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. 2018 IEEE 31st Computer Security Foundations Symposium (CSF). 2018:268-282. doi: 10.1109/csf.2018.00027
9. 9 Li Z, Zhang Y. Membership Leakage in Label-Only Exposures. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021:880-895. doi: 10.1145/3460120.3484575
10. 10 Fredrikson M, Jha S, Ristenpart T. Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015:1322-1333. doi: 10.1145/2810103.2813677
11. 11 Krüger FP, Östman J, Mervin L, Tetko IV, Engkvist O. Publishing neural networks in drug discovery might compromise training data privacy. J Cheminform. 2025;17(1):38. doi: 10.1186/s13321-025-00982-w
12. 12 Huang Y, Huang CY, Li X, Li K. A Dataset Auditing Method for Collaboratively Trained Machine Learning Models. IEEE Trans Med Imaging. 2023;42(7):2081-2090. doi: 10.1109/tmi.2022.3220706
13. 13 Pedarla LP, Zhang X, Zhao L, Khan H. Evaluation of Query-Based Membership Inference Attack on the Medical Data. Proceedings of the 2023 ACM Southeast Conference. 2023:191-195. doi: 10.1145/3564746.3587027
14. 14 Paul W, Cao Y, Zhang M, Burlina P. Defending Medical Image Diagnostics Against Privacy Attacks Using Generative Methods: Application to Retinal Diagnostics. In: Lecture Notes in Computer Science. Springer International Publishing; 2021. p. 174-187. doi: 10.1007/978-3-030-90874-4_17
15. 15 Mothukuri V, Parizi RM, Pouriyeh S, Huang Y, Dehghantanha A, Srivastava G. A survey on security and privacy of federated learning. Future Generation Computer Systems. 2021;115:619-640. doi: 10.1016/j.future.2020.10.007
16. 16 Nasr M, Shokri R, Houmansadr A. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. 2019 IEEE Symposium on Security and Privacy (SP). 2019:739-753. doi: 10.1109/sp.2019.00065
17. 17 Shafran A, Peleg S, Hoshen Y. Membership Inference Attacks are Easier on Difficult Problems. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 2021:14800-14809. doi: 10.1109/iccv48922.2021.01455
18. 18 Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, et al. Deep Learning with Differential Privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 2016:308-318. doi: 10.1145/2976749.2978318
19. 19 Dwork C, Roth A. The Algorithmic Foundations of Differential Privacy. Foundations and Trends® in Theoretical Computer Science. 2014;9(3-4):211-487. doi: 10.1561/0400000042
20. 20 Dwork C. Differential Privacy: A Survey of Results. In: Lecture Notes in Computer Science. Springer Berlin Heidelberg; 2008. p. 1-19. doi: 10.1007/978-3-540-79228-4_1
21. 21 Kaissis GA, Makowski MR, Rückert D, Braren RF. Secure, privacy-preserving and federated machine learning in medical imaging. Nat Mach Intell. 2020;2(6):305-311. doi: 10.1038/s42256-020-0186-1
22. 22 Ziller A, Usynin D, Braren R, Makowski M, Rueckert D, Kaissis G. Medical imaging deep learning with differential privacy. Sci Rep. 2021;11(1):13524. doi: 10.1038/s41598-021-93030-0
23. 23 Ziller A, Mueller TT, Stieger S, Feiner LF, Brandt J, Braren R, et al. Reconciling privacy and accuracy in AI for medical imaging. Nat Mach Intell. 2024;6(7):764-774. doi: 10.1038/s42256-024-00858-y
24. 24 Ibrahimi Hamamci et al. DENTEX: an abnormal tooth detection with dental enumeration and diagnosis benchmark for panoramic X-rays [Internet]. 2023. Available from: https://huggingface.co/datasets/ibrahimhamamci/DENTEX
25. 25 He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:770-778. doi: 10.1109/cvpr.2016.90
26. 26 Truex S, Liu L, Gursoy ME, Yu L, Wei W. Demystifying Membership Inference Attacks in Machine Learning as a Service. IEEE Trans Serv Comput. 2021;14(6):2073-2089. doi: 10.1109/tsc.2019.2897554
27. 27 Carlini N, Chien S, Nasr M, Song S, Terzis A, Tramèr F. Membership Inference Attacks From First Principles. 2022 IEEE Symposium on Security and Privacy (SP). 2022:1897-1914. doi: 10.1109/sp46214.2022.9833649
28. 28 Benladgham R, Hadjila F, Belloum A. Efficient Privacy-Utility Optimization for Differentially Private Deep Learning. Int j electr comput eng syst (Online). 2025;16(5):377-395. doi: 10.32985/ijeces.16.5.3
29. 29 Zheng L, Cao Y, Yoshikawa M, Shen Y, Rashed EA, Taura K, et al. Sensitivity-Aware Differential Privacy for Federated Medical Imaging. Sensors. 2025;25(9):2847. doi: 10.3390/s25092847
30. 30 Adams T, Birkenbihl C, Otte K, Ng HG, Rieling JA, Näher AF, et al. On the fidelity versus privacy and utility trade-off of synthetic patient data. iScience. 2025;28(5):112382. doi: 10.1016/j.isci.2025.112382
31. 31 Wang Z, Zhang C, Chen Y, Baracaldo N, Kadhe SR, Yu L. Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble. Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. 2025:1724-1738. doi: 10.1145/3719027.3744818
32. 32 Raza A, Guzzo A, Ianni M, Lappano R, Zanolini A, Maggiolini M, et al. Federated Learning in radiomics: A comprehensive meta-survey on medical image analysis. Computer Methods and Programs in Biomedicine. 2025;267:108768. doi: 10.1016/j.cmpb.2025.108768
33. 33 Elabd E. Dynamic differential privacy technique for deep learning models. Sci Rep. 2025;15(1):39353. doi: 10.1038/s41598-025-27708-0