Ahead of the recent Super Bowl, online discourse was focused not only on what would take place on the field, but also on the advertisements that often act as markers of where the Western zeitgeist is headed. One campaign that sparked widespread debate came from Anthropic, the AI company behind the Claude chatbot. Its ads presented exaggerated scenarios designed to highlight how advertising could reshape the way people use chatbots. In one advert, a young man exercising and hoping for a six-pack is told by a muscular older figure – meant to represent a chatbot – that “confidence isn’t just built in the gym” and advised to buy insoles that “help short kings stand tall”. Each advert ends with the same tagline: “Ads are coming to AI. But not to Claude.”

The campaign arrived just weeks after OpenAI, the maker of ChatGPT, announced plans to introduce adverts for users on its free service, alongside a new subscription tier called ChatGPT Go. It also followed the launch of ChatGPT Health for US audiences, a feature that allows users to analyse medical records in order to receive more tailored responses.

OpenAI CEO Sam Altman found humour in Anthropic’s campaign but described it as “dishonest” and “deceptive”, clarifying that OpenAI would never introduce ads directly into chatbot responses in the way portrayed. In a post on X, he rationalised that advertising is necessary to provide free AI access at scale, suggesting that Anthropic focuses on selling premium products to wealthier users while OpenAI is trying to reach a broader audience. The company, he said, feels a responsibility to make AI accessible to “billions of people who can’t pay for subscriptions”.

A week on from the Super Bowl, daily active users of Anthropic’s Claude chatbot increased by 11% and visits to its website jumped 6.5%. The spike in engagement pushed Claude into the top 10 free apps on the Apple App Store, surpassing competitors Meta, Google Gemini, and, notably, OpenAI. 

Beyond the public disagreement sits a broader question about how AI is funded, who it is built for, and what compromises users are being asked to accept. When the product is free, the age-old question resurfaces: does the user ultimately become the commodity?

Dr AI will see you now

Many of us are familiar with how difficult it can be to secure a doctor’s appointment, with the ordeal of securing one sometimes seeming more painful than the health issue that prompted it.

In the UK, the NHS has begun incorporating AI in targeted areas, from radiology tools that help flag early signs of cancer to systems that transcribe and summarise clinical consultations to ease the administrative load on doctors. Private providers such as Bupa and Nuffield Health have moved more quickly, embedding AI-driven triage and symptom checkers into their offerings. While the NHS has largely positioned AI as a support for clinicians rather than a substitute for them, consumer-facing platforms have raced ahead with far less oversight.

Therefore, it is unsurprising that some of us are turning to AI with questions about our health. OpenAI says more than 230 million people seek health and wellness advice from ChatGPT each week, presenting the chatbot as an ally that helps users navigate insurance, manage paperwork, and advocate for themselves.

In reality, this usually means sharing highly sensitive personal information, including diagnoses, medications, test results, and medical histories. While these exchanges can resemble a conversation with a physician, they take place under very different conditions. Crucially, tech companies are not bound by the same legal or ethical responsibilities as healthcare providers – there is no equivalent of doctor–patient confidentiality, no duty of care, and no regulator overseeing how advice is framed or how vulnerable users are treated.

When benchmarks meet real people

Much of the optimism around AI in healthcare comes from benchmark tests, where large language models (LLMs) appear to perform well, recalling symptoms, suggesting diagnoses, and explaining treatments with confidence.

However, a study from the University of Oxford shows how quickly that promise breaks down in real-world use. Outside of controlled settings, people do not describe their health in neat prompts, and medical evaluations rarely resemble multiple-choice exams. Once these systems meet real users, performance drops sharply, not because the models lack medical knowledge, but because people leave out key details and the models misinterpret what they are given.

Tested in isolation, models correctly identified medical conditions nearly 95% of the time. Paired with human users, that figure fell below 35%, and performance was often worse than when participants used traditional online search.

Communication sits at the centre of that decline. Users struggle to explain what they are experiencing, and models misread vague prompts or offer inconsistent guidance. In several cases, the correct diagnosis appears somewhere in the conversation, but the user does not recognise it in their final assessment. In one instance, two people describing similar symptoms associated with subarachnoid haemorrhage, a rare type of stroke, received opposite advice. One was told to lie down and the other was urged to seek emergency care.

A study published in Nature Medicine echoes those concerns. Researchers at the Icahn School of Medicine at Mount Sinai assessed ChatGPT Health using 60 realistic patient scenarios and found that, in more than half of cases requiring immediate emergency care, the system advised users to stay home or book routine appointments instead. The tool performed well in obvious emergencies but was less reliable when symptoms were less clearly defined. In simulations involving suicidal ideation, crisis safeguards appeared inconsistently and, in some cases, disappeared when unrelated clinical details were introduced.

Benchmarks measure recall and pattern recognition, but they do not capture uncertainty, anxiety, or the way people actually talk about their health.

Mental health: high uptake, uneven outcomes

These limitations become even more pronounced in mental health, where chatbots have emerged as one of the most common ways people seek support, particularly as access to other forms of support becomes harder to secure. Polling by Mental Health UK suggests that more than one in three UK adults now use AI tools to support their mental health or wellbeing, with many drawn to a system that, amid long waiting lists and overstretched services, offers something that feels immediate, personalised, anonymous, and accessible.

Many users report positive experiences, with some finding practical coping mechanisms and reassurance during difficult moments.

At the same time, the same research points to more troubling outcomes, with a minority reporting worsened psychosis symptoms, exposure to harmful information about suicide, or being triggered to think about self-harm.

Mental Health UK called for independent safety testing, clearer guidance for users, and greater transparency around data handling, stressing that AI should complement professional care rather than replace it. Taken together, these mixed mental health outcomes expose a broader tension between accessibility and safety.

The privacy paradox

Alongside questions of accuracy and safe use sits a quieter but still important concern about what happens to the data people share.

OpenAI says health-related uploads are stored separately and not used to train its models. Legal experts note that these assurances rest on company policy rather than binding regulation. Unlike hospitals or doctors’ surgeries, most AI platforms are not formally subject to healthcare privacy enforcement.

Products are often marketed as ‘HIPAA ready’, a US standard for handling medical data. In practice, this is a marketing claim and not evidence of clinical oversight; the label is self-applied, with no official certification body behind it, and most consumer-facing AI platforms fall outside HIPAA’s enforcement scope altogether. At the same time, companies sidestep medical device regulation by insisting their tools are not intended for diagnosis or treatment, even as people use them to interpret test results and make health decisions.

When healthcare meets advertising

Once advertising arrives, the debate shifts again.

OpenAI has begun testing sponsored links inside chatbot conversations as it searches for sustainable revenue models, amid growing concern that the AI bubble may be close to bursting. Even if ads remain technically separate from responses, monetisation brings new priorities around engagement, retention, and conversion.

In health-related contexts, this shift carries real consequences, as moments of vulnerability can quietly be reframed as promotional opportunities. Consumer surveys suggest many users are open to ad-supported services in exchange for free access, but most also worry about their data being used to train AI systems and overwhelmingly want greater transparency.

A system still finding its footing

Today’s LLMs appear impressively knowledgeable on paper, but they still struggle with the messy, unpredictable reality of real conversations. This makes them hard to rely on safely at scale.

The Super Bowl ads frame this as a clash of business models, reflecting a sector still negotiating its boundaries between access and accuracy, convenience and privacy, innovation and responsibility. Move fast and break things.

AI already plays a meaningful role in helping many people manage their health, often filling gaps left by overstretched systems. For some, it offers reassurance when no professional is available, but adoption has raced ahead of regulation, and monetisation is arriving before clear guardrails are in place.

If AI is to become a genuine support system rather than a high-risk substitute for care, evaluation will need to move beyond benchmark tests and towards real-world user studies. Privacy protections will need to be enforceable, not optional. And commercial incentives will also require closer scrutiny, especially in sensitive domains such as health.

The Super Bowl ads were funny, but the stakes are not.