In Fact, Retrieval Augmented Generation Is Also Not Enough to Meet User Requirements in Healthcare.

Because Sometimes You Want Exhaustive Answers

In the previous blog, we discussed how LLMs by themselves are insufficient for satisfying user requirements in healthcare. We suggested, at the minimum, we needed frameworks like RAG to provide a natural language interface for retrieval-oriented tasks in healthcare. We also discussed ways to mitigate hallucinations in retrieval-oriented applications in healthcare where there is ground truth like data in the EHR system. However, dealing with hallucinations only aims to ensure that the assertions in the text generated by the LLM are true. We have two additional requirements from healthcare users. These are: exhaustive enumeration, and semantics of not finding anything. Let us look at each in turn and see why these requirements cannot be met reliably even with RAG like frameworks.   

Exhaustive Enumeration 

Our users often want information like what allergies the patient has, or the entire list of medications. In many such situations, leaving out any piece of information can be problematic. Imagine giving a medication that interacts poorly with the one left out of the enumeration or triggering an allergic reaction because we were not aware of an allergy. It is in these situations where LLMs, even RAG-like approaches struggle. To illustrate this problem, below are a couple of screenshots of sample interactions with ChatGPT. I was trying to see what happens when I type in two natural language queries that are semantically equivalent to most humans and what the response is. I have also truncated the output to the first bullet point, because it was sufficient to make my point.