UMEM Educational Pearls

Title: Searching – Answers and Sources

Category: Administration

Keywords: AI, LLM, large language model, artificial intelligence, search (PubMed Search)

Posted: 8/27/2025 by Steve Schenkel, MPP, MD (Updated: 12/5/2025)
Click here to contact Steve Schenkel, MPP, MD

Have you tried using Large Language Model (LLM) driven searches in clinical practice, for example, OpenEvidence, ChatGPT, or Claude?

A recent paper, far from medicine, argues that LLM searching has changed the way we think about search and what we expect from search.

We have moved from searching for sources to searching for the information contained within sources. With this, our expectations have changed – we expect search to provide answers, not documents.

With this shift, the foundations of trust have shifted. Rather than finding a document that provides the answer, and which is supported by the integrity of the authors and publishers – and which the reader can evaluate – LLMs provide an answer, often with little evidentiary base to support that answer. 

Some LLMs do a better job of referencing sources in support of their answers than others. This argument suggests how important such transparency can be.

Find the full conversation here, https://publicera.kb.se/ir/article/view/52258, in Sundin O, Theorising notions of searching, (re)sources and evaluation in the light of generative AI, Information Research 2025, vol 30.

References

Sundin O, Theorising notions of searching, (re)sources and evaluation in the light of generative AI, Information Research 2025, vol 30. https://publicera.kb.se/ir/article/view/52258