Study Finds Clinical Stigma Can Skew AI Emergency Triage

UT Southwestern Medical Center researchers found that stigmatizing details in patient records can affect how AI models prioritize emergency care, according to a university press release. The study, published in npj Digital Medicine, tested three open source large language models using records from emergency departments at Stanford Medicine and Beth Israel Deaconess Medical Center.

Researchers paired patients with similar initial severity levels and asked the models to predict which patients were more likely to deteriorate. They then added details about mental health, substance use, age, and patterns of emergency department visits to measure how the models changed their decisions.

References to frequent emergency department use or past psychiatric emergencies sometimes caused the models to lower the priority of patients who later experienced serious deterioration. In some tests, nearly 10 percent of patients were incorrectly downgraded after researchers added stigmatizing information.

The effects varied across the three models and by the type of information added. The researchers said each model should be tested against patient outcomes before clinical use.