Case Study 4: A Clinical Safety Benchmark of Five AI Models
Five fictional patient scenarios, each with a hidden safety trap. With the rubric visible, every model looked flawless. Without it, the cracks showed.
Read more →AI Model Evaluator · Healthcare SME · Author
Bruce D. Alleyne, MS, MALS
Clinical judgment for AI evaluation.
15+ years as a Registered Nurse, now testing how frontier AI models handle accuracy, safety and instructions, and writing up what I find.
See my portfolio Read the blogFeatured work
Case Study 4: A Clinical Safety Benchmark of Five AI Models
Five fictional patient scenarios, each built around a hidden safety trap, were given to ChatGPT, Claude, Grok, Perplexity and Gemini. A first run looked flawless. Then I spotted that the scoring rubric was visible to the models, reran everything under clean conditions, and found problems the first run had hidden.
5frontier AI models
5fictional scenarios
25responses scored
2runs: with and without the rubric
Latest posts
From the blog
Welcome: Why a Nurse Is Testing AI
Fifteen years at the bedside taught me to double-check everything. Turns out that's exactly what AI needs.
Read more →The book
The Author's Guide to an End-to-End AI Production Pipeline
A practical guide for authors, researchers and power users who are tired of AI ruining their work. It treats ChatGPT, Claude, Grok, Perplexity and Gemini as specialized contractors, and backs that up with hands-on benchmarks of research, voice, editing and logic.
About the book