Hostname: page-component-7b7567c8c-z4zkx Total loading time: 0 Render date: 2026-11-17T22:48:43.907Z Has data issue: false hasContentIssue false

65 Integrating pathology reports into multimodal machine learning models to predict thyroid cancer recurrence

Published online by Cambridge University Press:  08 May 2026

Denise Lee
Affiliation:
Icahn School of Medicine at Mount Sinai
Steven Yen
Affiliation:
Department of Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
Grace Deng
Affiliation:
Department of Statistics and Data Science, Cornell University, Ithaca, NY, USA
Yehia Elkersh
Affiliation:
Department of Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
David S Matteson
Affiliation:
Department of Statistics and Data Science, Cornell University, Ithaca, NY, USA
Andrew Thomas
Affiliation:
Department of Statistics and Actuarial Science, University of Iowa, Iowa City, IA, USA
Michael L Marin
Affiliation:
Department of Surgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
Girish N Nadkarni
Affiliation:
Windreich Department of Artificial Intelligence, Icahn School of Medicine at Mount Sinai, New York, NY
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

Objectives/Goals: The objective of this study was to evaluate the performance of multimodal machine learning (ML) models trained to predict differentiated thyroid cancer (DTC) recurrence using clinical data combined with novel natural language processing (NLP) derived features extracted from patient cytopathology and surgical pathology reports. Methods/Study Population: This was a retrospective study of adult thyroid cancer patients treated at an academic medical center. Patients were classified as having cancer recurrence or no recurrence. NLP features were extracted from cytopathology and surgical pathology reports using Term Frequency–Inverse Document Frequency (TF-IDF), latent Dirichlet allocation (LDA), and a zero-shot large language model (LLM) classification. 5 multimodal ML models were trained to predict cancer recurrence utilizing a combination of NLP and LLM features and clinical variables. Model performance was evaluated using area under the receiver operating characteristic curve (ROC-AUC) and precision recall area under the curve (PR-AUC). The top performing model was optimized with a 5-fold cross-validation. Feature importance was calculated. Results/Anticipated Results: 480 patients with differentiated thyroid cancer diagnosed on surgical pathology were included in this study. The baseline model (clinical variables only) had a F1-score of 0.52 and an AUC of 0.53. The optimized gradient boosting model utilizing all features (EMR, LDA, TF-IDF, and LLM) had a F1-score of 0.87 and an AUC of 0.86. Topic words and themes from the patient cytopathology and surgical pathology reports were generated using LDA. Topic themes in cytopathology reports include malignancy, lymph node evaluation, and molecular testing. Topic themes in surgical pathology reports include histologic subtype, orientation of nodule, and intraoperative biopsy. The LDA themes of malignancy and histologic subtype ranked the highest in terms of feature importance. Discussion/Significance of Impact: Multimodal models utilizing novel NLP features derived from unstructured pathology reports may enable improved prediction of recurrence in patients with DTC. Our optimized model demonstrated that 4 of the top 6 highest features were LDA topics. Topic modeling may be a valuable tool to extract relevant information from unstructured clinical notes.

Information

Type
AI and Data Science
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Author(s), 2026. The Association for Clinical and Translational Science