LitLLM: An AI-powered tool to supercharge scientific literature reviews

Image generated by AI; authors: Shubham Agarwal, Gaurav Sahu, Abhay Puri, Issam H. Laradji, Krishnamurthy DJ Dvijotham, Jason Stanley, Laurent Charlin, and Christopher Pal

Researchers have all been there—staring at a blank document, trying to craft a comprehensive literature review while thousands of potentially relevant papers await their attention. With more than 4,000 machine learning papers submitted to arXiv each month, staying current has become nearly impossible using traditional methods. This is where LitLLM comes in.

What is LitLLM?

LitLLM is an intelligent AI assistant designed specifically to streamline the literature review process for scientific papers. Unlike traditional large language models (LLMs), which often hallucinate information or miss recent publications, LitLLM:

• Retrieves real papers from academic search engines based on your research focus
• Accurately ranks results by relevance to your specific topic
• Generates concise, factual literature reviews grounded in actual publications
• Provides full control through optional sentence-planning features

How does LitLLM work?

The modular pipeline in LitLLM breaks down the complex literature review process into four intuitive stages:

1. Smart keyword extraction: Rather than forcing you to come up with the perfect search terms, LitLLM uses an advanced LLM to analyze your abstract and automatically extract the most relevant keywords. LitLLM supports any natural language query.

2. Multisource paper retrieval: Using these extracted keywords, LitLLM:

• Queries academic search engines such as Semantic Scholar to find relevant papers
• Optionally uses document-embedding-based search for greater coverage
• Combines results to ensure comprehensive coverage of your research area

3. AI-powered re-ranking

Not all papers that match keywords are truly relevant. LitLLM employs a prompt-based re-ranking mechanism that:

• Evaluates each paper's relevance to your specific research focus
• Provides attribution explaining why each paper was selected
• Eliminates irrelevant results to focus only on the most pertinent publications

4. Controlled literature review generation

The final stage produces a coherent, well-structured literature review that:

• Cites only real, relevant papers
• Minimizes hallucinations through retrieval-augmented generation (RAG) principles
• Allows optional sentence planning for precise structural control
• Produces ready-to-use text for your paper or proposal

10x faster research with better quality

Our empirical evaluations demonstrate that LitLLM delivers:

• 30% improvement in recall compared to standard keyword search methods
• 18% to 26% reduction in hallucinations compared to traditional LLM approaches
• Significant time savings—complete literature reviews in minutes rather than days
• Enhanced relevance through precise keyword targeting and intelligent ranking

Real-world applications beyond academia

Although LitLLM is primarily designed for academics, its capabilities extend well beyond research institutions. Product managers can quickly assess competitive landscapes. Industry analysts can produce market reports more efficiently. And research and development teams can track emerging trends with minimal effort.

Enterprise agents and healthcare professionals can also use the tool to research solutions and treatment options, respectively. LitLLM can prove valuable across any knowledge-intensive field.

Get started with LitLLM

See LitLLM in action:

You can also visit the project website for more details: https://litllm.github.io/.

LitLLM is an open research project. We welcome contributions from the community to help us:

• Integrate new academic search engines
• Improve the ranking algorithms
• Enhance the user interface and user experience
• Add support for specialized domains

We invite you to join us in making literature reviews faster, more accurate, and more accessible to researchers worldwide.

If you find this work useful, please cite:

@article{agarwal2024llms,
  title={LitLLMs, LLMs for Literature Review: Are we there yet?},
  author={Agarwal, Shubham and Sahu, Gaurav and Puri, Abhay and Laradji, Issam H and Dvijotham, Krishnamurthy DJ and Stanley, Jason and Charlin, Laurent and Pal, Christopher},
  journal={arXiv preprint arXiv:2412.15249},
  year={2024}
}
@article{agarwal2024litllm,
  title={Litllm: A toolkit for scientific literature review},
  author={Agarwal, Shubham and Sahu, Gaurav and Puri, Abhay and Laradji, Issam H and Dvijotham, Krishnamurthy DJ and Stanley, Jason and Charlin, Laurent and Pal, Christopher}
  journal={arXiv preprint arXiv:2402.01788},
  year={2024}
}

Find out more about ServiceNow Research.