Generalist annotators cannot catch what a doctor can.
Crowd labelers rate fluent, confident answers as safe. A warfarin patient asking about ibuprofen gets a green check. A 78-year-old with confusion and fever gets routed to a GP visit next week. The model looks good in eval. The harm shows up in production.
- Annotation — generalists misclassify lesions, miss subtle radiological findings, and under-label brown and South Asian skin tones.
- Evaluation — model outputs are scored against rubrics no clinician wrote, with no adversarial safety probing by the relevant specialist.
- Reasoning — training data is scraped from the web instead of authored by physicians who know how to construct a differential diagnosis.