AMA urges safeguards for AI mental health tools

The American Medical Association is urging Congress to create safety guardrails for artificial chatbots in mental healthcare, as Americans increasingly turn to the technology for health information and advice (Source: “AMA urges lawmakers to implement safeguards on AI chatbots,” Health Care Dive, April 23). 
 
In letters sent Wednesday to the chairs of three congressional committees on digital health and AI, the major physician lobby said “well-designed, purpose-built” tools could help patients who would otherwise struggle to access mental healthcare, but that the lack of safety protocols poses serious risks.
 
Proponents argue the tools could help Americans navigate the nation’s complex healthcare ecosystem and find quick answers to questions — a particular challenge in behavioral healthcare, where many communities face a shortage of mental health professionals. 
 
But other experts worry the chatbots could give misleading or inaccurate information that could lead to patient harm. There have been multiple cases in recent years of young users dying by suicide after confiding in AI chatbots. Family members have reported that AI didn’t urge them to seek help and sometimes appeared to encourage self-harm.