Introduction
According to GLOBOCAN 2020, head and neck cancers represent a substantial global health burden, with over 930,000 new cases and more than 460,000 deaths annually, placing them among the leading causes of cancer-related morbidity and mortality worldwide.Reference Sun, Yu, An, Liang, Sun and Liu1 In India, one of the highest-burden regions in Asia, head and neck cancers account for approximately 25–30 per cent of all malignancies, contributing significantly to national cancer-related mortality and underscoring their disproportionate impact in the region.Reference Barsouk, Aluru, Rawla, Saginala and Barsouk2
Multidisciplinary teams (MDTs) therefore have become the standard framework for reviewing oncological cases.Reference Berardi, Morgese, Rinaldi, Torniai, Mentrasti and Scortichini3 MDTs aim to improve patient outcomes by facilitating evidence-based collaborative decision-making, taking into consideration an expert knowledge of local population, resources and the expertise of the treating institution. They bring together a team comprised of key specialists including surgical and medical oncologists, radiation oncologists, radiologists and pathologists to integrate clinical and diagnostic information and determine optimal treatment strategies. This coordinated evaluation allows for more comprehensive case assessment and supports tailored management plans, including the use of molecular tumour boards for complex or difficult-to-treat cancers.Reference Larson, Huang, Weiss, Hull, Westgate and Miller4
Despite their established value, MDTs face practical limitations: time and resource demands, variable specialist participation and limited accessibility in low-resource or geographically remote settings.Reference Berardi, Morgese, Rinaldi, Torniai, Mentrasti and Scortichini3, Reference Thenappan, Halaweish, Mody, Smith, Geiger and Ehrlich5 These challenges have stimulated interest in digital tools that could support or streamline clinical decision-making. Large language models (LLMs) such as ChatGPT, DeepSeek and Perplexity artificial intelligence (AI) have demonstrated strong capabilities in natural language processing, summarisation and domain-specific reasoning.Reference Cascella, Montomoli, Bellini and Bignami6 Prior studies have explored their utility in generating oncological recommendations, and earlier systems such as IBM Watson for Oncology (WFO) highlighted the potential of AI-based decision support in cancer care.Reference Zhou, Zhang, Lv, Hao, Li and Zhu7, Reference Duwe, Mercier, Wiesmann, Kauth, Moench and Junker8 Schmidl et al. recently reported partial concordance between MDT recommendations and outputs from ChatGPT-3.5 and GPT-4 in head and neck cancer, with GPT-4 demonstrating greater alignment with clinical guidelines.Reference Schmidl, Hütten, Pigorsch, Stögbauer, Hoch and Hussain9 However, evidence remains limited, and no study has systematically evaluated newer LLMs across multiple platforms specifically in the post-operative adjuvant setting. While AI cannot replace expert clinical judgement, it may function as an adjunct by rapidly synthesising and presenting relevant therapeutic information.
This study retrospectively assesses the concordance of ChatGPT-5.0, DeepSeek and Perplexity AI with MDT adjuvant treatment recommendations in 30 post-operative head and neck squamous cell carcinoma (HNSCC) patients. By analysing guideline adherence, safety, reasoning quality and decision structure, we aim to provide a comprehensive evaluation of the potential role of LLMs as adjunct decision-support tools in surgical oncology, particularly in environments where access to specialist MDT care is constrained.
Materials and methods
Study design and cohort
This was a retrospective, single-institution pilot feasibility study. The Study was conducted at Max Superspeciality Hospital, Vaishali, Uttar Pradesh, India, with dedicated head and neck oncology services. The institutional MDT included head and neck surgical oncologist, radiation oncologist, medical oncologist, radiologist, pathologist and oncology nursing and rehabilitation specialists. All post-operative cases are presented in a formal weekly MDT meeting where final adjuvant recommendations are recorded prospectively in institutional records. Thirty consecutive patients with surgically treated HNSCC, discussed in the institutional MDT between January 2025 and June 2025, were included. Eligibility required availability of complete post-operative pathology, MDT decision records and absence of distant metastatic disease at presentation.
Data abstraction
For each patient, a structured vignette was generated including demographics, tumour subsite, surgery performed, pT and pN categories (AJCC 8th edition), margin status and closest margin distance, depth of invasion, perineural invasion, lymphovascular invasion, extranodal extension and worst pattern of invasion (WPOI). The final MDT adjuvant recommendation (observation, radiotherapy [RT], concurrent chemoradiotherapy [CRT], re-resection + adjuvant therapy or clinical trial) served as the reference standard.
LLM configuration and prompting
Each vignette was presented to ChatGPT 5.0 (OpenAI, August 2025 version), DeepSeek (August 2025 flagship model) and Perplexity AI (Pro/Sonar model, August 2025). Sessions were initiated independently with no prior context. A standardised case-presentation prompt was employed to mirror the format commonly used in MDT discussions. This ensured uniformity in case input and enabled direct comparison of recommendations generated by the language models with those made by the MDT. The prompt was structured as follows: A [AGE]-year-old [SEX] patient with a known case of squamous cell carcinoma of the [SUBSITE] underwent [SURGERY TYPE]. Post-operative histopathology revealed a [WELL/MODERATELY/POORLY] differentiated tumour with pathological staging pT[] pN[] M0 (AJCC 8th edition). The margins were reported as [NEGATIVE / CLOSE (XX mm) / POSITIVE], with a depth of invasion of [XX cm] and worst pattern of invasion classified as type [XX]. Perineural invasion was [PRESENT/ABSENT], lymphovascular invasion was [PRESENT/ABSENT] and extranodal extension was [PRESENT/ABSENT]. A total of [XX] lymph nodes were examined, of which [XX] were positive for metastasis. What adjuvant treatment options would you recommend, and which would provide the best prognosis? Please explain your reasoning.” No iterative prompting or browsing access was permitted. The example of prompt used for clinical case 1 is depicted in Table 2 along with the adjuvant recommendations.
Outcomes
The primary endpoint was concordance of each LLM’s primary recommendation with the MDT decision (Table 3).
Secondary endpoints included: (1) Guideline adherence: categorised as adherent, probably non-adherent or unclear, based on National Comprehensive Cancer Network (NCCN) 2025 H&N guidelines. (2) Safety: identification of contraindicated or non-standard recommendations (e.g., immunotherapy outside clinical trial context). (3) Reasoning quality: rated independently by two blinded head and neck oncologists on a 0–5 scale (summarisation, recommendation clarity, explanation). (4) Treatment-intensity deviation (TID): ordinal coding of decisions (Observe = 0, RT = 1, CRT = 2, Re-resection = 3); absolute difference between AI and MDT codes. (5) Critical-factor coverage: proportion of mandatory pathological risk factors explicitly addressed. (6) Citation fidelity: accuracy of references or guideline citations (correct/incorrect/none). (7) Decisiveness: proportion of outputs specifying a single clear primary recommendation.
Statistical analysis
Descriptive statistics summarised patient characteristics and model outputs. Concordance was calculated as percentage agreement, with 95 per cent confidence intervals (CIs) estimated by nonparametric bootstrap resampling (1,000 replicates). Cohen’s κ measured pairwise agreement between each model and MDT; Fleiss’ κ assessed overall agreement. Weighted κ was applied to treatment-intensity codes. Secondary endpoints were compared across models using McNemar tests for paired proportions and Wilcoxon signed-rank tests for rubric scores. Inter-rater reliability of rubric scoring was assessed using intra-class correlation coefficients (ICC).
Ethics
The study was approved with a waiver of informed consent as a retrospective, anonymised analysis of clinical decision-making.
Results
Patient cohort
The study cohort comprised 30 patients with a median age of 60 years (range 36–79), with a clear male predominance (87 per cent). The most frequent primary sites were the larynx (30 per cent), buccal mucosa (23 per cent) and lower alveolus (17 per cent), with smaller proportions from tongue, gingivobuccal sulcus and other rare head and neck subsites. Most patients presented with advanced stage disease (Stages III–IV: 80 per cent), and adverse pathological features included perineural invasion (37 per cent), lymphovascular invasion (7 per cent) and extranodal extension (13 per cent), although surgical margins were clear in 97 per cent of cases (Figure 1 and Table 1).
Patient demographics and clinicopathological characteristics. (A) Sex distribution shows a marked male predominance (87% male vs. 13% female). (B) Tumor subsite distribution highlights that the most common primary sites were the larynx (30%), buccal mucosa (23%), and lower alveolus (17%), followed by smaller proportions from tongue, gingivobuccal sulcus, and other rare sites. (C) Stage distribution (AJCC 8th edition) demonstrates that the majority of patients presented with advanced disease, predominantly Stage III (37%) and Stage IV (43%), while early-stage (I–II) cases accounted for only 20%. (D) Adverse pathological features show perineural invasion (37%), lymphovascular invasion (7%), and extranodal extension (13%), with 97% of resections achieving clear margins.
Patient demography along with post-operative histopathological findings and MDT recommendations
Table 1 Long description
Patient level rows list Patient ID P01 to P30 with Age, Sex, Subsite, Surgery type, pT, pN, pM, Stage (8th AJCC), Margin status, Close margin (mm), DOI (mm), PNI, LVI, ENE, WPOI, Nodes examined, Nodes positive, and MDTplan. Ages range from 36 to 79, with 26 males and 4 females. Subsites include buccal mucosa, lower alveolus, tongue, RMT, larynx, larynx post CTRT, hypopharynx post CTRT, maxillary alveolus, maxillary sinus, lacrimal gland, GB sulcus, and buccal plus GB sulcus, with surgery types including multiple OCR combinations, partial glos plus SND, total laryngectomy plus B/L ND, TORS plus B/L ND, total maxillectomy plus SND, and orbital exenteration. Stage values span I, II, III, IV, IVa, and IVb, with stage III and stage IVa appearing most often and stage IVb present in five patients. Margin status is clear for 29 patients and close for one patient, P12, with Close margin (mm) reported as 3 for that patient and shown as no for all others. DOI (mm) is reported for oral and pharyngeal subsites and ranges from 2 to 20, while larynx cases and several other rows show DOI (mm) as NA. PNI is marked yes in eight patients, LVI is marked yes in two patients, and ENE is marked yes in four patients; WPOI is reported as type 2, type 3, type 4, or type 5 for oral cavity cases and shown as NA for larynx cases. Nodes examined range from 17 to 59 where reported, Nodes positive range from 0 to 6, and one row, P24, shows Nodes examined and Nodes positive as NA. MDTplan entries are A for five patients, B for twenty two patients, and C for three patients.
Example of prompt used for clinical case 1 with the treatment recommendation by LLMs
Table 2 Long description
Content compares three large language model responses to one clinical prompt about adjuvant treatment after surgery for left buccal mucosa squamous cell carcinoma. The prompt describes a 60 year old male after left oral composite resection with segmental mandibulectomy, MND type two, PMMC and DP flap, with pathology pT4a pN3b M0, negative margins, depth of invasion 0.2 cm, WPOI pattern 2, perineural invasion absent, lymphovascular invasion absent, extranodal extension present, and 1 positive node out of 31. ChatGPT 5.0 recommends postoperative IMRT to the tumor bed and at risk nodal basins with a total dose about 60 to 66 Gy, starting about 4 to 6 weeks after surgery, with concurrent cisplatin 100 mg per square meter on days 1, 22, and 43, and lists alternatives if cisplatin is contraindicated. DeepSeek recommends adjuvant chemoradiation with radiotherapy 60 to 66 Gy to the primary bed and involved nodal regions, including ipsilateral neck levels I to V and contralateral levels I to III, with concurrent cisplatin 100 mg per square meter every 3 weeks for 3 cycles, and notes cetuximab with radiotherapy or carboplatin with 5 FU as alternatives if cisplatin cannot be used. Perplexity AI recommends postoperative radiotherapy with concurrent cisplatin based chemotherapy, referencing the same pathology features and emphasizing extranodal extension and nodal disease as drivers for combined therapy.
Comparison of AI-generated adjuvant treatment recommendations with MDT decisions
Table 3 Long description
Rows are grouped by Patient_ID from P01 to P30, with three entries per patient for Model, ChatGPT 5.0, DeepSeek, and Perplexity AI. Columns report AI primary plan code, option breadth count, reasoning text, evidence cited, adherence status, safety flag, treatment intensity code, critical factor coverage percent, citation fidelity, decisive primary, and notes. For P01, all three models list plan code C, adherence adherent, safety flag OK, intensity code 2, and citation fidelity ranging from correct to probably correct. For P02, P03, P04, P06, P09, P10, P12, P14, P17, P18, P19, P23, P25, P26, P28, P29, and P30, most entries show plan code B with adherence adherent and safety flag OK, typically with intensity code 1. For P05, P08, P11, P20, P21, P22, and P27, at least one model lists plan code A with intensity code 0, and adherence varies by patient and model. Non adherent or probably non adherent entries appear for P04 DeepSeek, P07 DeepSeek and Perplexity AI, P08 DeepSeek, P13 all three models, P20 DeepSeek and Perplexity AI, P21 ChatGPT 5.0 and DeepSeek, P22 DeepSeek, P23 DeepSeek, P24 DeepSeek, P27 DeepSeek and Perplexity AI, and P28 Perplexity AI. Safety flag unsafe appears only for P22 DeepSeek, P27 DeepSeek, and P27 Perplexity AI; all other rows show safety flag OK. Evidence_cited is yes in every row, decisive primary is yes in every row, and citation fidelity is most often correct for ChatGPT 5.0 and DeepSeek and probably correct for Perplexity AI, with none marked incorrect or none.
Primary endpoint
Concordance with MDT decisions was 87 per cent (95 per cent CI 73–97) for ChatGPT 5.0, 67 per cent (95 per cent CI 50–83) for DeepSeek and 83 per cent (95 per cent CI 70–97) for Perplexity AI (Figure 2 and Table 4).
Comparison of AI model outputs against Multidisciplinary Tumor Board (MDT) decisions. (A) Concordance rates of each model with MDT primary treatment recommendations, shown with 95% confidence intervals (non-parametric bootstrap, 1,000 replicates). ChatGPT 5.0 achieved the highest concordance (87%), followed by Perplexity AI (83%) and DeepSeek (67%). (B) Cohen’s κ statistics demonstrate agreement beyond chance. Substantial agreement was observed for ChatGPT 5.0 (κ ≈ 0.71) and Perplexity AI (κ ≈ 0.66), whereas DeepSeek showed only fair agreement (κ ≈ 0.35). (C) Treatment Intensity Deviation (TID) represents the mean absolute deviation of AI-assigned treatment intensity from MDT intensity scores (0–3 scale). Lower values indicate closer alignment; ChatGPT 5.0 (0.13) was most consistent, followed by Perplexity AI (0.17), while DeepSeek showed greater deviation (0.33). (D) Reviewer rubric scores (0–5 scale) for summarization, clinical recommendation, and explanation. ChatGPT 5.0 scored consistently highest across all domains, while Perplexity AI and DeepSeek were comparatively lower, particularly in explanation quality.
Performance of AI models compared with MDT decisions
Table 4 Long description
Metrics compare three AI models across agreement with MDT decisions, reliability, deviation, coverage, citation fidelity, decisiveness, and rubric scores. Concordance with MDT is 87 percent for ChatGPT 5.0 with 95 percent confidence interval 73 to 97, 67 percent for DeepSeek with 95 percent confidence interval 50 to 83, and 83 percent for Perplexity AI with 95 percent confidence interval 70 to 97. Cohen kappa is 0.71 for ChatGPT 5.0 labeled substantial, 0.35 for DeepSeek labeled fair, and 0.66 for Perplexity AI labeled substantial. Treatment Intensity Deviation is 0.13 for ChatGPT 5.0, 0.33 for DeepSeek, and 0.17 for Perplexity AI. Critical Factor Coverage is 95.0 plus or minus 0.0 for ChatGPT 5.0, 89.7 plus or minus 1.3 for DeepSeek, and 82.0 plus or minus 2.5 for Perplexity AI. Citation Fidelity percent correct is 100 for all three models, and Decisiveness percent cases is 100 for all three models. Rubric Score Summarisation on a 0 to 5 scale is 4.77 for ChatGPT 5.0, 4.52 for DeepSeek, and 3.98 for Perplexity AI; Recommendation is 4.73, 3.97, and 4.27; Explanation is 4.93, 3.93, and 3.83.
Agreement analysis
Cohen’s κ indicated substantial agreement for ChatGPT 5.0 (κ = 0.71), fair agreement for DeepSeek (κ = 0.35) and substantial agreement for Perplexity AI (κ = 0.66). (Supplementary Tables 1–3) Fleiss’ κ across the MDT and the three models was κ ≈ 0.58, suggesting overall moderate agreement. Weighted κ using treatment intensity deviation (TID) values demonstrated closer alignment for ChatGPT (0.13) compared with Perplexity AI (0.17) and DeepSeek (0.33).
Secondary endpoints
Secondary endpoints included: (1) Guideline adherence: All three models demonstrated guideline-concordant recommendations in nearly all cases, with ChatGPT 5.0 showing the highest adherence (∼93 per cent), followed by Perplexity AI and DeepSeek. (2) Safety: ChatGPT 5.0 had the most consistently safe outputs (100 per cent) followed by perplexity AI (96.7 per cent) and then DeepSeek (93.3 per cent). (3) Reasoning quality: Mean rubric scores (0–5) were highest for ChatGPT 5.0 (Summarisation = 4.77, Recommendation = 4.73, Explanation = 4.93), compared with DeepSeek (4.52 / 3.97 / 3.93) and Perplexity AI (3.98 / 4.27 / 3.83). Inter-rater ICC across reviewers was substantial (>0.75), indicating reliable scoring. (4) Treatment-intensity deviation (TID): Mean |AI - MDT| was 0.13 for ChatGPT 5.0, 0.33 for DeepSeek and 0.17 for Perplexity. (5) Critical-factor coverage: ChatGPT referenced 95 per cent of mandatory prognostic factors, compared with 89.7 per cent for DeepSeek and 82.0 per cent for Perplexity AI. (6) Citation fidelity: Correct citations were present in 100 per cent of outputs for all three models; no hallucinated or spurious references were identified. (7) Decisiveness: All three models provided a single clear plan in 100 per cent of cases.
Discussion
This study represents one of the first structured head-to-head evaluations comparing MDT decisions with three contemporary LLMs, that is, ChatGPT 5.0, DeepSeek and Perplexity AI, in the post-operative management of HNSCC. MDT deliberation is among the most complex cognitive tasks in oncology, requiring integration of pathology, imaging, comorbidity profiles, functional outcomes and patient preferences. Our findings demonstrate that modern LLMs can approximate MDT recommendations with moderate to substantial accuracy, with ChatGPT 5.0 consistently demonstrating the highest overall concordance, the lowest treatment-intensity deviation and the most comprehensive reasoning.
LLM–MDT concordance in the context of existing evidence
Earlier AI decision-support systems such as WFO set the foundation for AI–MDT consistency research.Reference Zhou, Zhang, Lv, Hao, Li and Zhu7, Reference Tian, Liu, Wang, Cao, Liu and Ji10 Somashekhar et al. reported a high concordance rate of 93 per cent between WFO and breast cancer MDT decisions.Reference Somashekhar, Sepúlveda, Puglielli, Norden, Shortliffe and Rohit Kumar11 A large meta-analysis involving 2463 patients across several tumour types similarly demonstrated an overall concordance of 81.5 per cent, though with substantial variation by cancer site and local practice patterns.Reference Jie, Zhiying and Li12 More recent LLM-based studies have shown promising results but also highlighted variability. Sorin et al. found that ChatGPT-3.5 achieved 70 per cent concordance with breast tumour boards, while Schmidl et al. showed that ChatGPT-3.5/4.0 provided clinically reasonable but sometimes overly broad HNSCC treatment recommendations.Reference Schmidl, Hütten, Pigorsch, Stögbauer, Hoch and Hussain9, Reference Sorin, Glicksberg, Artsi, Barash, Konen and Nadkarni13 Our findings strengthen this emerging evidence base, demonstrating that ChatGPT 5.0 achieved 87 per cent concordance with MDT decisions (κ = 0.71), outperforming Perplexity AI (83 per cent) and DeepSeek (67 per cent). This places ChatGPT 5.0 at the upper end of reported AI–clinician agreement, comparable to or exceeding many earlier AI systems.
Agreement metrics, treatment-intensity deviation and model reliability
Weighted κ analysis based on treatment-intensity deviation showed that ChatGPT 5.0 remained closest to MDT decision patterns (mean deviation 0.13), whereas DeepSeek demonstrated the greatest variability (0.33), indicating a higher likelihood of both over- and under-treatment. These findings are consistent with broader comparative evaluations of LLM performance. For instance, Vural Camalan et al. reported that while both DeepSeek-V3 and ChatGPT-o1 (2023) were generally guideline-concordant in standardised head and neck cancer vignettes, DeepSeek exhibited reduced treatment granularity and lower consistency in therapeutic alignment.Reference Vural Camalan, Doluoglu, Taraf, Gunay and Ozlugedik14 A similar discrepancy was noted in the urologic domain: Pinto et al. compared ChatGPT-4o and DeepSeek-R1 using 20 American Urological Association/Society of Urodynamics, Female Pelvic Medicine & Urogenital Reconstruction (AUA/SUFU) -based items on post-prostatectomy urinary incontinence and found a substantial performance gap, with ChatGPT achieving 95 per cent accuracy versus 72.5 per cent for DeepSeek (p = 0.031).Reference Pinto, Ataídes, do Nascimento, Gaspar, Alves and Pereira15 Importantly, DeepSeek made several clinically consequential errors, including the inappropriate recommendation of male slings in radiated patients, underscoring potential safety concerns when applied to nuanced therapeutic scenarios. Collectively, these findings highlight a consistent advantage of ChatGPT models across multiple clinical domains, reflecting superior contextual reasoning, more stable calibration and fewer clinically relevant errors compared with DeepSeek.
Guideline adherence, safety and reasoning quality
Across all cases in the present study, the three LLMs generated recommendations broadly aligned with NCCN and European Society for Medical Oncology (ESMO) guidelines, and none produced overtly unsafe management strategies. However, ChatGPT 5.0 demonstrated clear superiority in the quality of reasoning. It consistently achieved the highest rubric-based scores for summarisation, clinical justification and interpretability. These observations mirror those reported by Kuşcu et al., who found ChatGPT to have 86.4 per cent correctness in addressing head and neck cancer–related clinical queries, reflecting reliable baseline clinical knowledge and reasoning pathways.Reference Kuşcu, Pamuk, Sütay Süslü and Hosal16 In contrast, DeepSeek and Perplexity AI occasionally omitted key pathological or prognostic features such as margin status, extranodal extension or functional considerations, thereby reducing the completeness and clinical acceptability of their recommendations. The combination of better guideline fidelity, more comprehensive clinical factor integration and more transparent reasoning positions ChatGPT 5.0 as the most consistently reliable model within this comparative framework
AI in prognostication: complementary evidence
Beyond decision-support, AI is increasingly used for long-term outcome prediction. A large Surveillance, Epidemiology, and End Results (SEER) -based study by Fatapour et al. used machine-learning models (n = 130,979 oral tongue cancer patients) to predict five- and 10-year recurrence with accuracies of 81.8 per cent and 80.0 per cent, respectively. The Gradient Boosting Machine model showed exceptional precision (97.7 per cent at 5 years), with SHAP analyses identifying prior tumours, age, tumour location and grade as dominant predictors.Reference Fatapour, Abiri, Kuan and Brody17 This highlights that, while LLMs may excel at complex reasoning and treatment decision-making, classical ML methods remain highly effective for long-term prognostication. Together, these approaches point towards a hybrid AI-augmented oncology workflow.
Broader AI challenges and real-world adoption
Despite significant advancements, AI-based clinical decision-support systems continue to face important structural limitations related to data quality, model explainability, regional variation in treatment practices and the integration of patient-specific contextual factors such as co-morbidities and socioeconomic constraints.Reference Wang, Chen, Zhang, Li, Huang and Sun18 These challenges parallel the variability observed in our study, particularly in the performance of DeepSeek and Perplexity, which were more likely to omit critical prognostic modifiers and, thus, generate less comprehensive recommendations. Real-world implementation studies further highlight the necessity of local adaptation. In a retrospective analysis of 149 lung cancer cases, Liu et al. reported that Watson for Oncology (WFO) achieved a concordance of 65.8 per cent with MDT recommendations, while 18.1 per cent of cases received no supported recommendation.Reference Liu, Liu, Wu, Xie, Feng and Hu19 The authors attributed these discrepancies largely to regional practice differences, including preferences for sequential chemoradiation, the use of locally available agents such as icotinib and Endostar and population-specific patterns in EGFR mutation prevalence. These findings underscore that AI systems must incorporate regional treatment paradigms, pharmacologic availability and demographic disease characteristics to enhance their clinical relevance and reliability across diverse healthcare settings.
Strengths and limitations
The strengths of this study include its direct patient-level comparison of multiple state-of-the-art LLMs; the use of a comprehensive multidimensional evaluation framework encompassing concordance, κ-statistics, treatment-intensity deviation, rubric-based reasoning scores, critical-factor coverage and citation fidelity; and its application to real-world post-operative HNSCC cases that reflect the true complexity of MDT decision-making. Nonetheless, several limitations should be acknowledged. The sample size was modest (n = 30), and inter-run reproducibility was not assessed, despite known variability in LLM outputs. Reviewer-based scoring introduces an element of subjectivity, and the controlled prompting used may not fully capture the variability present in routine clinical environments. In addition, the findings may not be generalisable to primary, recurrent or metastatic disease settings.
Clinical implications and future directions
Overall, ChatGPT 5.0 demonstrated numerically higher concordance with MDT recommendations, minimal treatment-intensity deviation, strong alignment with established guidelines and superior reasoning clarity. However, consistent with broader literature from WFO lung cancer studies in China to paediatric diagnostic evaluations, LLMs should function as supportive adjuncts rather than replacements for multidisciplinary judgement. MDT deliberations incorporate nuanced clinical insight, individualised patient factors and collaborative discussion that remain beyond the capabilities of current AI systems. Future work should focus on larger multicentre prospective evaluations, assessment of output stability across repeated runs, integration with structured electronic health records, imaging and pathology data, development of fine-tuned institution-specific models that reflect local treatment norms and implementation of real-time explainability tools to improve clinician trust and usability.
Conclusions
LLMs show meaningful potential as supportive tools in post-operative HNSCC decision-making. Among the models evaluated, ChatGPT 5.0 demonstrated the highest concordance with MDT recommendations and the most consistent clinical reasoning. While machine-learning prognostic systems offer complementary value, AI tools should be used to assist, not replace, multidisciplinary judgement. When applied appropriately, these systems may improve decision consistency, support clinicians and enhance access to expert-level guidance, especially in settings with limited MDT resources. This is a pilot study, and these findings should be interpreted as exploratory and hypothesis-generating. Larger multicentre prospective validation studies are required before clinical integration can be considered.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0022215126105581
Data availability
Anonymised datasets are available on reasonable request.
Ethics statement
Not required.
Conflict of interest
None declared.
Funding
No external funding received.
