911吃瓜

Reprinted with permission from 911吃瓜 News

by Robyn Stubbs

As voice artificial intelligence (AI) speeds toward use in clinical settings, a researcher from 911吃瓜 is highlighting the urgent need for ethical, legal, and social oversight鈥攅specially in therapeutic care.

Voice AI analyzes vocal patterns to detect signs of physical, cognitive, and mental health conditions based on vocal qualities like pitch and jitter or fluency and specific words people use. Some tech companies have even dubbed it 鈥渢he new blood鈥� of healthcare because of its potential to act as a biomarker, but 911吃瓜 health sciences researcher Zoha Khawaja urges caution.

In her , Khawaja, a member of the  explores the potential and the perils of voice-based AI apps in the mental health field. 

Khawaja鈥檚 study used structured, multi-round surveys to gather insights from 13 stakeholders, including clinicians, ethicists, and patients. While 77 per cent of participants supported using voice AI to improve patient outcomes, 92 per cent agreed that governance models should be established by healthcare or governmental organizations to oversee its integration. 

鈥淰oice AI holds real promise as an objective tool in the mental health field, which has always relied on subjective diagnostics like self-reporting and interviews,鈥� says Khawaja. 鈥淏ut the entrepreneurial speed of the tech is outpacing regulatory oversight in such a high-stakes environment like healthcare.鈥�

Some companies already offer apps that analyze short voice samples to assess mental fitness. However, Khawaja warns that these tools often operate in a 鈥渨ellness鈥� gray zone鈥攁voiding classification as medical devices and sidestepping privacy protections.

鈥淭here鈥檚 a real risk of therapeutic misconception, where people may believe these apps are providing clinical diagnoses or treatment, when in fact they鈥檙e not," Khawaja explains. 鈥淭hat鈥檚 particularly dangerous for vulnerable users who may not have access to traditional care.鈥�

Key concerns raised by participants included algorithmic bias, lack of transparency, erosion of human connection in care, and unclear accountability. The study advocates for a digital compassionate care approach, where AI tools support鈥攏ot replace鈥攈uman relationships in therapy.

鈥淧atients might feel safer talking to a chatbot than a person,鈥� Khawaja says. 鈥淏ut that can lead to overreliance and isolation. These tools should strengthen the clinician-patient bond, not undermine it.鈥�

She also recommends a shared responsibility model among developers, regulators, and healthcare providers to prevent ethics dumping鈥攖he unfair shifting of ethical burdens onto clinicians. Notably, 83 per cent of participants agreed that healthcare practitioners should be held accountable for adverse events resulting from the use of voice AI tools.  

鈥淏ut clinicians are already overburdened,鈥� Khawaja says. 鈥淓xpecting them to bear the ultimate responsibility of these technologies is unrealistic.鈥�

Clinical trials to validate voice as a biomarker are currently underway in the U.S., where regulatory sandboxes鈥攃ontrolled environments for testing new technologies鈥攁re being proposed to anticipate ethical challenges and inform policy before voice AI enters clinical practice.