Automated radiation therapy treatment planning using a context-aware foundation model

Stanford researchers in the Xing Lab have developed GPT-RadPlan, a large language model (LLM) and vision-language model (VLM) based radiation therapy treatment planning automation tool that reduces treatment planning time and lowers costs. Laborious, costly, and subject to substantial variability, radiation therapy (RT) treatment planning requires precise imaging, target identification, dose calculation, and optimization by human planners often using trial and error to balance conflicting objectives. Deep learning and reinforcement learning methods require large, diverse datasets and effective reward functions.

GPT-RadPlan uses multi-modal large language models, such as GPT-4Vision (GPT-4V) from OpenAI, to leverage prior radiation oncology knowledge and incorporate additional knowledge and clinic-specific preference (via in-context learning prompt) in a fully automated treatment planning framework where the user need not train the model.
GPT-RadPlan has three modules:

• Evaluation Module - where vision and language models analyze dose distribution images and dose volume histogram (DVH) tables, to provide focused feedback.
• Memory Module - a crucial information repository that stores historical plans and corresponding optimization parameters to facilitate understanding of treatment plan change and adjustments in the optimization parameters.
• Planner Module - learning from historical plan trajectories and few reference approved plans, this module uses feedback from the evaluation module to modify treatment parameters.

When integrated into Stanford Medicine's in-house treatment planning system, GPT-RadPlan either outperformed or matched clinical plans, demonstrating superior target coverage and organ-at-risk sparing. Using multimodal large language models to mimic human planners in radiation oncology clinics, GPT-RadPlan successfully automates and streamlines treatment planning without additional training or computational resources.

Stage of Development – Prototype
The Xing Lab tested the prototype software, integrating it with Stanford Medicine in-house clinical planning software, where it either outperformed or matched the clinical plans for multiple prostate, and head & neck cancer cases. Research continues across multiple diseases and protocols with clinical practice benchmarking.