Large Language Model Superiority in Guideline-Based Neurointerventional Multiple-Choice Questions: Performance Analysis of Radiologists vs. LLMs Across Experience Levels

Objective: To assess large language models’ (LLMs) mastery of contemporary consensus documents in guideline-based neurointerventional decision-making domains and to compare their performance with radiologists of different experience levels.

Methods: In this cross-sectional study, 100 single-best-answer, five-option MCQs were derived from three CIRSE guideline sources. A non-participating general radiologist authored the items (33 carotid stenting, 33 thrombectomy, 34 acute stroke EVT). One neurointerventional radiologist, two general radiologists, and one resident answered independently, blinded to the key. The same set was posed in January 2026 to Claude 4.5 Opus, ChatGPT 5.2 Instant, and Gemini 3 Pro using default settings and a standardized prompt. Accuracy was compared with McNemar’s test and Bonferroni correction (P≤0.002).

Results: The highest accuracy was observed with Claude 4.5 Opus (0.99±0.10); ChatGPT 5.2 Instant (0.96±0.19) and Gemini 3 Pro (0.95±0.21) showed comparable performance. Among human participants, the neurointerventional radiologist achieved the highest accuracy (0.85±0.35), while general radiologists scored 0.67–0.70 and the resident achieved 0.57. Each of the three LLMs significantly outperformed all human groups. Moreover, Claude 4.5 Opus also demonstrated significantly higher performance than the neurointerventional radiologist (P=0.001).

Conclusion: LLMs have reached a competitive level with neurointerventional radiologists in recalling and interpreting neurointerventional guideline knowledge. In particular, Claude 4.5 Opus demonstrated strong performance and appears to be a promising candidate for “real-time decision support” applications. Multimodel validation and verification studies using real clinical scenario–based designs are warranted.