Former ONC Head of Policy: AI Executive Order Is Surprisingly Comprehensive

The Biden Administration’s executive order on Artificial Intelligence covers a wide range of AI uses, but its tasks for the Department of Health and Human Services will affect the medtech industry. The “Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence” was published 30 October.

Crowell partner and former head of policy for the Office of the National Coordinator (ONC) for Health IT Jodi Daniel told Medtech Insight she was surprised at how extensive the order was.

“We have seen HHS take some actions with regard to health care and AI and the [US Food and Drug administration] has put out some guidance with regard to AI-enabled medical devices… but there isn’t really this comprehensive thinking about what HHS should be doing,” she said.

While many of the requirements of the order build on work already underway at the department, it provides a bigger picture of how HHS should think about AI so that each agency isn’t looking at its regulatory authority with “blinders” on, she said.

Jodi Daniel, former ONC director and current Crowell partner.

Daniel, who bears no relation to the author, singled out three pieces of the order she believes will affect the medtech space the most: the creation of an HHS AI Task Force, the creation of a health care AI safety program and the discussion of data privacy.

The order establishes an AI Task Force, which will be required to publish a strategic plan comprised of policies, frameworks and regulatory actions for the safe use and deployment of AI technology in health care within 365 days.

The health care AI safety program, which will be enacted in collaboration with the Departments of Defense and Veterans Affairs, will also work with patient safety organizations to enable adverse event reporting from AI devices in the healthcare space.

“If we want to build trust in the use of AI in health care, it’s important to have a mechanism for looking at trends and potentials for adverse events,” Daniel said.

The order discusses data privacy across different governmental sectors, but Daniel singled out grants from the National Science Foundation to support the development of privacy-enhancing technologies.

“I thought it was interesting that, as a part of this AI push for innovation, [the Biden Administration] is trying to promote that development of better privacy-enabled technology,” she said.

With innovation comes questions of intellectual property, so the US Patent and Trademark Office is tasked to publish guidance for its reviewers about the inventorship and use of AI within 120 days and a guidance on the intersection of AI and intellectual property within 270 days.

A Peek Behind The Curtain

Having worked at the HHS for 15 years, Daniel had some insight into the HHS’s next steps.

She expects the department to begin by forming the AI Task Force with representatives from all the involved agencies, including the Center for Medicare and Medicaid Services (CMS), the Office of Civil Rights (OCR), the FDA and likely someone from the White House as well.

Additionally, there will be a working group formed to write the text of the strategic plan.

Most of the actions laid out in the executive order are, unsurprisingly, government-focused, so Daniel advised device manufacturers and other industry stakeholders to keep an eye on the HHS.

“We’re going to have to wait and see, in 90 days, what the [HHS’s] priorities are,” she said.

Daniel, who helped draft the Health Insurance Portability and Accountability Act of 1996 (HIPAA), left HHS in 2015. While she couldn’t have predicted how AI would be used in 2023, she said HHS was having discussions about clinical decision support software and software as a medical device, both of which were precursors to AI and ML medical devices.

Possible Sore Spot: Dual-Action Foundation Models

One area that may receive some pushback from the medtech industry is section 4.2 of the order, which invokes the Defense Production Act to ensure the safety of AI products.

The policy would require many new safety features from companies that manufacture “dual-use foundation models.” These are defined in the order as self-supervising AI models that are trained on “broad data” that could be modified to perform tasks that could pose a risk to security to national public health or safety.

An example of a dual use foundation model is one that could “substantially” lower the barrier to entry for non-experts to create a biological weapon. It is unclear if this could include medical device manufacturers.

If it does apply, within 90 days of the order’s date, companies will be required to report intent to develop dua-use foundation models, planned or ongoing activities of training, ownership and possession of model weights, and performance results of relevant red-team tests.