State efforts in regulating AI and ensuring its safety

As an emergency physician in Allegheny County, Venkat shares in the excitement around AI and its potential, especially to relieve health care workers with time-consuming tasks like patient charting, medical documentation and assessing staffing needs.

But it also requires great scrutiny, he said. Could this evolving technology one day supersede humans’ roles in clinical decision making? Could it deny health insurance coverage for lifesaving treatments without any human intervention? Or leave the door open to new cybersecurity risks for sensitive patient information?

Pennsylvania has existing laws on health care ethics, patient privacy and data collection, transparency and informed consent, as well as consumer protections for health insurance, but they don’t specifically speak to AI.

“Right now, it is the Wild West when it comes to artificial intelligence, as to whether in the deployment of artificial intelligence, those laws are being followed,” he said.

The proposed bill would create a rule book for how health providers and companies could apply AI in clinical settings, the health insurance sector and in data collection, “without creating an onerous burden that would prevent them from continuing to innovate and apply artificial intelligence where it may be appropriate,” Venkat said.

Patients should be told when AI is involved in their care, he said, and a human should be responsible for any final decisions on treatment and health insurance coverage.

The bill also calls for AI tools and software that prevent bias and discrimination in health care settings, not reinforce or add to it.

Without legislation at the federal level to build on, Venkat said it’s up to individual states to address AI sooner rather than later.

“I think we have no choice but to move forward in this regard,” he said.

Independent, nonregulatory groups like the Patient Safety Authority are just beginning to monitor and assess AI in Pennsylvania health care systems and its impact on patients.

The authority looks for new and emerging issues that affect patient safety. Hospitals, nursing homes and health offices are required to report misdiagnoses, fall injuries, medication errors and other kinds of adverse events.

The number of safety reports that specifically mention the involvement of AI is small right now, Jones said, but she expects it will grow — not necessarily because the technology is becoming unsafe, but rather in the hopes that health care workers will become more aware of how and when AI is contributing to care.

But so far, early data show that AI is having more positive effects on patient care rather than negative.

“We don’t want to only focus on the negative. We want to see where it is performing well for patient safety, as well,” Jones said. “In those cases where there was an event that actually did occur, but the AI came along and somehow helped to identify it sooner, we want to know that.”