AI about to replace ER doctors: A new study by Harvard has revealed that artificial intelligence (AI) does better than human doctors when triaging patients in an emergency room (ER) and completing complicated diagnoses. The study published in Science evaluated OpenAI‘s o1 reasoning model against human physicians across multiple clinical scenarios, with striking results.
How the triages were conducted
When analyzing initial triage data from 76 patients seen in an emergency department in Boston, the AI was able to determine the correct or almost correct diagnosis in 67 percent of cases, while human doctors were accurate only 50 to 55 percent. When there was little or no clinician input (scarce information) regarding what the possible issues may be, which is essential during times of high stress in an emergency setting. On the other hand, with more complete data, AI accuracy increased to 82 percent, compared to 70 to 79 percent for experts, meaning there was no statistically significant difference between them.
In the area of management reasoning tasks (for example, coming up with an antibiotic treatment plan or an end-of-life care plan), AI performed better (scoring 89 percent) than humans (34 percent) using traditional tools such as search engines. The AI model also excelled at rare disease diagnosis, correctly identifying a lupus-related lung inflammation that human doctors missed.
Yet, according to the lead author, Dr. Arjun Manrai, “these results do not suggest AI replaces physicians.” The study only tested text-based data; AI cannot currently provide information/assess about patient distress, visual appearance, or bedside manner. “Ultimately, humans want humans to guide them through life-or-death decisions,” Manrai said.
Key findings
- Triage accuracy (limited info): AI 67 percent vs. humans 50 to 55 percent
- Full-detail diagnosis: AI 82 percent vs. humans 70 to 79 percent (not statistically significant)
- Management reasoning: AI 89 percent vs. humans 34 percent
- Rare disease detection: AI identified subtle lupus complication missed by physicians
What this means for healthcare
The research findings indicate that AI could be a strong option to provide physicians with a second opinion tool in emergency rooms, helping doctors avoid diagnostic errors and missed opportunities. Already, more than 20 percent of physicians in the U.S. have used large language models (LLMs) for clinical decision support. But researchers envision a “triadic care model,” meaning: doctor, patient, and AI working together, not replacement.