AI Model Aids Clinical Decision-Making and Precision Medicine Predictions

Stanford researchers have developed a vision-language foundation model designed to leverage large-scale, unlabeled, unpaired image and text data. The model, coined "MUSK", shows strong performance in outcome prediction, including melanoma relapse prediction, pan-cancer prognosis prediction, and immunotherapy response prediction in lung and gastro-esophageal cancers.

Clinical decision-making is driven by multimodal data, including clinical notes and pathologic characteristics. Artificial intelligence approaches that can effectively integrate multimodal data hold significant promise to advance clinical care. Foundation AI models in particular represent a new frontier of medical AI R&D. These models are pretrained on massive, diverse datasets and can be applied to numerous downstream tasks with minimal or no further training. A major hurdle for the development of multi-modal AI models has been the scarcity of well-annotated datasets, especially in the clinical setting. Recent efforts to develop vision-language foundation models for medicine have fallen short due to insufficient data scale, inefficient training approach (i.e. using contrastive learning) and focusing on primarily simple tasks such as image classification or retrieval.

To advance the clinical relevance and impact of foundation AI models, Stanford researchers developed MUSK. MUSK can be used for a wide range of downstream tasks including image and text retrieval, visual question answering, image classification, and molecular biomarker prediction. MUSK outperforms comparable state-of-the-art foundation models on multiple benchmarks. Additionally, not only can MUSK be used to develop more reliable and robust foundation AI models but the method can be utilized to improve the prediction of treatment response and patient outcomes for precision medicine. MUSK has commercial potential as clinical diagnostic tests, as a provided service and more.

Stage of research
In vitro data