It is hard to find any part of daily life that is not impacted by the growth of Artificial Intelligence (AI). That includes healthcare. Many organizations, governments, and businesses have promised the benefits of AI, especially as an assistive tool for people with disabilities. It’s important to assess and understand how AI can benefit our communities, and where it may cause harm.
Kaiser Family Foundation has reported on the growth of AI in the consumer health market. In the last year new consumer tools have emerged like ChatGPT Health. Consumers looking for health information may view chatbot tools as a supplement to professional care providers, especially with the growing cost of insurance and accessing care. Knowing how to navigate the resources and limitations of an AI chatbot is an important public health concern for people with disabilities. The Urban Institute conducted a study of AI tools, finding inaccurate data retrieval in over 60% of cases.
Individuals are increasingly turning to generative AI tools to answer their own health questions without the assistance of a provider, a trend that is likely to grow as many people are unable to access insurance and afford medical care. Data from OpenAI showed that in 2026, over 40 million people worldwide were accessing health information daily on ChatGPT. People use consumer AI tools for medical advice and diagnosis assistance. A KFF survey found one-third of adults were using AI chatbots to understand their health insurance. The OpenAI data showed users were asking up to 1.9 million questions each week about their health coverage options, including comparing plans, discussing claims process and billing. We encourage people to follow up with a trusted provider to discuss health information they retrieve from AI tools.
For people with cognitive or psychosocial disabilities, relying on AI chatbots for mental health support can be a dangerous alternative to the support of a licensed counselor or peer professional. In a recent study, researchers investigated the risks of increased psychosis and other concerns of AI chatbot use on individuals with mental health conditions. They found that AI chatbots often provide false narratives that may increase the risk of psychosis in vulnerable individuals. In particular, these chatbots can reinforce the social withdrawal and isolation of people who are having difficult thoughts. They are also designed to tell users what they want to hear and can reinforce false perceptions or share false information. For someone experiencing a mental health crisis, they may be even less likely to critically examine the information they retrieve from a chatbot, and may even view the tool as a sentient being. AI psychosis has been identified as a triggering event for some individuals without a pre-existing mental health condition as well. These findings have led regulatory and professional organizations like the WHO to call for more oversight and regulations around AI.
Those of us living with disabilities require more personalized, and accurate care to meet our needs. Unfortunately, AI resources may not always be able to offer that support when working off limited, ableist datasets. For people with disabilities and chronic illness, navigating health services and resources that rely on AI can bring up unique challenges. When any person seeks accurate information from an AI program, they may not know how to determine what is true from what is not. For people with intellectual and cognitive disabilities, navigating this environment may be even more challenging. In this blog, we will talk about how AI is being used behind the scenes in health care systems and services, and the impacts this has on people with disabilities.
Artificial Intelligence in Health Systems
AI is increasingly being used in healthcare settings for a variety of uses. AI has applications in everything from diagnostics to creating new drug protocols and helping out with administrative tasks. AI is also used to assess large data sets to study environmental and social drivers of health, or to examine trends in electronic health records. Through deep learning models, AI has is used as a tool for disease surveillance, illness outbreak predictions, and risk assessments for patients and providers alike. Healthcare providers are increasingly using AI to support the efficiency and accuracy of their work. Administrative tasks like paperwork management and scheduling are increasingly being performed by AI for medical offices and hospitals. These models are being used to support with tasks like scheduling procedures, and taking notes during a patient appointment.
Health insurance companies are also seeing AI as a tool for cutting costs, which could decrease provision of care. AI can be used to analyze claims, process denials, and automate prior authorization determinations. According to a survey from the American Medical Association (AMA), over 60% of doctors say AI tools regularly deny patients coverage for necessary care. With AI tools in prior authorization, denials have been shown to increase by 16%, creating challenges for patients seeking requested care. Without regulation of AI in things like prior authorization, health care access for disabled people faces another barrier.
Disability, Healthcare & AI
We agree that for AI to benefit disabled people, especially our health, it needs to be both anti-ableist and accessible. While AI-based tech holds promise for improving accessibility, it can reproduce harmful, ableist, and discriminatory data that inform healthcare decision-making and care. It is often limited to ableist datasets that already lack meaningful disability data. AI is trained on models that do not fully represent those with disabilities. Bias becomes systemic if these models cannot address data disparities.
Disabled AI users may also have additional privacy risks when interacting with digital tools, including AI data analysis. Digital data may be analyzed for signs of disability, which could impact access to employment, housing, or healthcare. Without disabled users involved with AI development, current systems may continue to reproduce biases and harmful outcomes.
Similar to the concern around AI usage in prior authorization, advocates are also concerned about the potential for AI to automate determinations using quality-adjusted life years (QALY) for people with disabilities. QALYs are a discriminatory way to make treatment decisions, which can reduce drug and treatment access. We at AAHD oppose QALYs , as they directly disadvantage and devalue people with disabilities. QALYs can be used to limit access to care, as more expensive treatment may be labelled as less cost-effective. This can take away care from those who need treatment most. If AI utilizes datasets that include QALYs, it will dangerously increase the use of QALYs in health care and undermine the laws disability advocates have fought for which prohibit the use of QALYs.
Artificial intelligence can deepen disparities for people with disabilities. Data from KFF shows that AI models are reproducing systemic racism, leading to racial disparities in care. These impacts are compounded for disabled people of color. AI models trained off large data sets are also at risk of reproducing a medical model of disability, which views disability as a condition to be cured. This ableist notion undermines the needs and concerns of disabled individuals who are trying to achieve their highest health.
Even though 67% of adults report mistrust in AI being able to provide reliable services and information and ethical and environmental issues abound with its widespread use, it seems for many providers AI is not only here to stay, it is the way of the future for their practice. For AI systems to be more accessible, those living with disabilities need to be at the table, advocating for what they need and anticipating the needs of the community in the future. Without those voices, the future of AI continues to look discriminatory, inaccessible, and potentially dangerous for those who stand to benefit from the potential it promises the most. To ensure AI supports positive health outcomes, regulations should require that prior authorization and claims decisions involving medical necessity include meaningful human clinical review and cannot rely solely on automated algorithms. Policymakers should also mandate transparency, regular audits for bias and accuracy, protections for patient privacy, and requirements that AI systems consider each patient’s individual clinical circumstances rather than making decisions based only on historical data patterns, especially when that data comes from a place of known bias.