Health care's emerging risk: unauthorized AI agents

Stunning stat: 72% of health industry leaders said AI tools or agents are being deployed without formal IT approval, according to a new survey for digital security company Imprivata.

• That's creating a trust gap for some of the executives, who say their oversight is struggling to keep up as agents shift from fetching information to helping coordinate care.
• The survey of 250 health security and AI strategy officials found 28% of health organizations have agentic AI in production today, with another 44% piloting or testing new uses.

• 88% expect AI agents to operate with at least some autonomy doing clinical and administrative work.

Threat level: Many health systems are trying to establish guardrails. But it's a unique challenge since the agents, unlike traditional software, can make plans, launch workflows and act independently.

• Giving them broad access to electronic health records, billing systems and communications platforms could lead to situations where information is removed, passwords are changed and doctors' orders get modified.
• It's also possible that a bot under a physician's supervision could be commandeered by an outside actor who injects it with prompts to look up sensitive information and move across clinical systems, experts say.
• "What we're advocating for is a paradigm shift, a plea to get out ahead of this before it's too late," said Sean Kelly, Imprivata's chief medical and growth officer and an emergency physician.

Between the lines: Health care has lagged other industries in adopting new technology. AI is changing that, giving providers a way to streamline administrative tasks, ease clinician burnout and improve the coordination of care.

• But unapproved deployments and fragmented security strategies can leave holes. There's particular concern about agents initiating tasks entirely on their own, or taking unauthorized actions that their human operators didn't request.
• In 2024, an Otter meeting agent joined a meeting of hospital physicians in Ontario, Canada, via a recurring calendar invitation, transcribed discussions about seven patients' personal health information, then emailed a summary to the 65 people on the invite, some of whom were no longer at the hospital, authorities said.
• This summer, an OpenAI agent researching public health spending data in Australia breached non-public parts of a government network containing information on the country's universal health program and appeared to try to conceal its tracks, according to the Cloud Security Alliance.

"As these agents start accessing systems and taking action, health care organizations need to know what they can access, what they're allowed to do, and how they're being monitored. That's going to become a much bigger part of the security conversation," said Jaren Day, group director of cybersecurity at KLAS Research.

The Imprivata survey says health care organizations will have to go beyond verifying who or what can access information to defining what an agent is authorized to do after it's granted access.

• Kelly said health systems should consider creating a "moat of credentials" so that an agent can only do a task for a specified time. "It's almost a case of guilty until proven innocent," he said.

What we're watching: Rural hospitals and clinics with tight budgets and overworked staff may need AI solutions the most, but could be especially vulnerable.

• States are using rural health funding from last year's GOP reconciliation law for more AI deployment. Some like Utah and Missouri are also focusing on risk management and security training.
• Providers also could face major lawsuits if an agent contributes to a bad health outcome.