For millions of Americans struggling to afford health care, a new source of medical advice is only a few keystrokes away.
As use of AI tools such as ChatGPT, Claude and Google Gemini becomes more widespread, national surveys suggest that uninsured people — particularly Black Americans and young adults — are among the groups most likely to use them for physical health information and mental health guidance.
Some users appear to be turning to AI because they can’t afford a doctor’s visit, can’t schedule an appointment or lack a regular health care provider. While instant, free advice is hard to resist, researchers and patient advocates warn that AI can produce inaccurate, misleading or potentially dangerous recommendations.
The stakes may be even higher for Black patients. Researchers have found that chatbots can reproduce racial biases, generalizations and stereotypes. Using one for medical advice, experts say, could recreate some of the same problems Black Americans have long faced within the health care system.
Higher use among the young, uninsured
A new study published in JAMA Pediatrics earlier this month illustrates the trend. It found that Black youth are around five times more likely than white youth to seek mental health advice from a chatbot at least once a month.
The finding appears to reflect a broader pattern. Black and Hispanic adults are also more likely to use AI for mental health information than white adults and people with health insurance, according to KFF Health Tracking Poll results published in March.
The findings come amid growing concerns about mental health in Black communities, including suicide rates that have risen over the past two decades.
Kamal Grewal, a health technology entrepreneur, worries that AI therapists can give young people in crisis the illusion of support even when the technology may do more harm than good. Several major AI companies, including OpenAI and Google, are facing lawsuits alleging their chatbots contributed to or encouraged user suicides.
“Consumer AI tools are optimized to keep users engaged: they validate, they agree, they make you feel heard,” said Grewal, creator of Therapy Companion, an AI tool designed to help therapists with administrative and non-patient services. “That’s great for retention metrics, but dangerous for someone in a mental health crisis, where what you need isn’t validation but a trained clinician who can challenge your thinking and hold you accountable.”
It’s no surprise that AI use is higher among younger people and is gaining traction among adults.
Roughly one-third of adults responding to a nationwide poll said they used AI chatbots last year to get health information. Of those who used AI for health information, about one-third were looking for advice about physical health matters and just over 15% searched for mental health advice.
Most respondents said they were looking for immediate advice. But there are indications that problems affording or finding health care also played a role for some respondents, especially younger adults.
The RAND study published in JAMA estimates that roughly 8.2 million young people nationwide reported using artificial intelligence for mental health advice this year, an increase of 40% compared with last year.
The study sampled 1,009 youth, including 95 Black respondents. The authors noted the sample size for that analysis was small and said the issue requires further investigation.
Among youth who sought mental health advice from AI chatbots, 42.8% did so at least once a month and 91.7% said the advice ranged from “somewhat” to “very” helpful. More than two-thirds of adolescents said they had not told anyone they used AI chatbots for mental health support.
Chatbots used included ChatGPT, Gemini, Character.AI and Meta AI. Young people tended to consult them when they felt angry, nervous, sad or stressed.
Bias hidden in the code
A 2024 study published in Nature found that when large language models were given prompts written in African American English, they produced stereotypes as negative as — or worse than — those recorded from humans during the Jim Crow era, without users’ race being mentioned.
The AI systems assigned speakers of African American English to lower-prestige jobs, convicted them more often in hypothetical criminal cases and more frequently recommended the death penalty when that punishment was an option.
Sharese King, assistant professor of linguistics at the University of Chicago, said the findings did not make her optimistic.
“If we continue to ignore the field of AI as a space where racism can emerge, then we’ll continue to perpetuate the stereotypes and the harms against African Americans,” she said.
The implications for teen mental health could be significant because many Black youth communicate in some form of African American English. If a chatbot they trust for advice about depression, anxiety or a personal crisis is unknowingly responding differently because of how they speak, the help being offered may itself be biased.
Among Black youth, use of AI for mental health support appears to vary by gender.
Black teenage girls are more likely to use chatbots to address anxiety, depression and relationship stress. Researchers say Black teenage boys are generally less likely to disclose emotional struggles, but when they do turn to a chatbot, they may already be experiencing acute distress.
Grewal said the growing reliance on AI for mental health support among Black Americans is troubling.
“Black adults are already turning to these tools at nearly double the rate of white adults, which means the people most underserved by the mental health system are also the most exposed to AI that isn’t built to help them,” he said.
This article originally appeared here.