AI-assisted Systematic Review of the Scientific Literature

Systematic reviews of medical research are resource- and labor-intensive, and a cornerstone of the scientific enterprise. These comprehensive studies, which typically take months to complete, can now potentially be carried out in mere hours without compromising on their precision and rigor.

Stanford researchers are developing an LLM-assisted pipeline for systematic reviews that could save significant time and resources for researchers, policy makers, healthcare and commercial entities.

Systematic reviews currently depend on scientifically literate human resources to screen articles and extract data. These tasks, however, are particularly well-suited for large language models (LLMs). To streamline this process, they have broken down the PRISMA guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) into smaller, resource-intensive tasks that are particularly well-suited for LLMs. Techniques like fine-tuning, prompt engineering, and Retrieval Augmented Generation (RAG) all play a role in ensuring the process is rigorous, grounded, and trustworthy.

The Stanford team's prototype introduces a comprehensive pipeline that automates the conversion of a scientific query into a detailed evidence table, integrating LLM-assisted screening and content extraction. These advanced features, not available in existing tools, have the potential to make systematic reviews much more accessible and affordable for researchers, policymakers, healthcare providers, and biotech companies interested in entering novel scientific landscapes.

[Pipeline architecture in progress]

Stage of Development:
Prototype; Optimization and Validation still under development