Recently I uploaded a lab report to chatGPT and asked it to summarize it.
It hallucinated serious cancer, along with all the associated details you’d normally find on a lab report. It had an answer to every question I had pre-asked about the report.
Sigh. This is a point in favor of not allowing free access to ChatGPT at all given that people are getting mad at GPT-4o-mini which is complete garbage for anything remotely complex... and garbage for most other things, too.
Just give 5 free queries of 4o/o3 or whatever and call it good.
Or a non-normie. Even while logged in, I had no idea what ChatGPT model it was using, since it doesn't label it. All the label says is "great for everyday tasks".
And as a non-normie, I obviously didn't take its analysis seriously, and compared it to Grok and Gemini 2.5. The latter was the best.
I wonder if it was unable to read your report, and just answered as if role-playing?
I gave it a pdf of an engine manual recently and asked some questions, which it answered reasonably. It even pulled a schematic out for me, though it was the wrong one (it gave me a schematic for the CDI ignition variant that we first talked about, rather than the DSAI one we settled on later.)
Other Key Metrics
Absolute Neutrophils, Lymphocytes, Monocytes, Eosinophils, Basophils: All within normal ranges, suggesting no significant abnormalities in immune cell populations.
Anisocytosis (1+): Indicates variation in red blood cell size.
Hypochromasia (1+): Suggests pale red blood cells, supporting iron-deficiency anemia.
Summary
These results suggest iron-deficiency anemia, characterized by low hemoglobin, low MCV, low MCH, and high RDW. The elevated red blood cell count may reflect compensatory mechanisms or mild dehydration.
Your provider recommends a heart-healthy, low-fat diet, which is great for overall health. However, addressing the anemia might also involve increasing dietary iron or taking iron supplements.
The diagnosis is wrong, btw, I don't have iron deficiency. The anemia is caused by a genetic condition called thalassemia, which has been verified by genetic tests. You can use the Mentzer Index to differentiate the two on a simple CBC - https://www.mdcalc.com/calc/10534/mentzer-index
Mine numbers return a "probable diagnosis."
I was wondering if chatgpt would catch it, nope, it didn't. It did say that it was a possibility once I suggested it though.
It's also disappointing that the LLM would falsely claim that a low-fat diet is "great for overall health". That claim has been thoroughly debunked in the scientific literature (or at least not supported by evidence-based medicine), and yet the recommendation still keeps popping up. While there are certain specific types of fats that can be problematic, in general a moderate amount of fat is fine as a macronutrient.
LLMs don't have a magical ability to arbitrate complex truths. If the "low-fat diet" narrative is prominent in the training literature, I would expect the model to say that more often that it said the opposite.
I don't ever really use the term "thoroughly debunked" when referring to nutrition science; as you noted, a better term is that the claim is not supported by the evidence. I've seen enough things debunked and then rebunked to know that nutrition science is not really that accurate.
What was the outcome (did the subject of the lab report go on to present cancer)?
(sometimes what we think is a false positive ends up being a true positive later, and cancer is a late-arriving signal).
I dunno, cancer is complicated. I'd have to see the details to make that judgement. It really depends on a lot of factors- what the nature of the sample was, what the lab report did, etc.
It hallucinated serious cancer, along with all the associated details you’d normally find on a lab report. It had an answer to every question I had pre-asked about the report.
The report said the opposite: no cancer detected.