Feedback from front-line clinicians about the benefits of artificial intelligence ambient listening tools typically solicits the same confident response: "It's game-changing technology."
Physicians often share how the use of AI scribes has saved them "pajama time," or after-hours work in the electronic health record.
Research has shown that the use of AI for clinical documentation improves clinicians' professional satisfaction while reducing cognitive load and burnout. Physicians praise AI scribes for reducing their stress and enhancing their relationships with patients.
While physician satisfaction is a key benefit to deploying AI scribes, health system leaders need more than unstructured feedback and anecdotal reports as they make decisions about scaling AI. Executives say they need precise outcome measurements in order to evaluate the return on investment.
Generative AI company Abridge has contracts with more than 150 health systems and saw an opportunity to build a measurement framework for its partners. The company developed a methodology to assess the ROI impact of ambient AI using data from partner health systems and "rigorous measurement frameworks," the company said.
Abridge developed custom SQL queries (statements written in structured query language), which were validated by early-adopter health systems, to evaluate the impact of AI on documentation-related and operational metrics. Those metrics include work outside work (WOW), percent utilization, work relative value units (wRVUs) per encounter, clean claims, net collection ratio, CDI queries, time spent in the EHR, percentage of notes closed the same day, patient access, clinician retention and other criteria.
"These custom queries retrieve encounter-level data, which more accurately enable health systems to analyze AI impact across diverse workflows, specialties and patient complexities, controlling for differences in AI utilization," Abridge report in its ROI report, which published today.
This level of granularity, Abridge executives said, arms health systems with actionable insights across AI adoption levels, specialties and visit types.
The company shared evaluation results from four health systems that reported notable time savings, financial outcomes and burnout reduction.
San Diego-based Sharp HealthCare, a system with 3,000 clinicians serving more than 3 million patients, reported a 3.5% to 6% increase in wRVUs per encounter.
The use of ambient AI technology enables clinicians to document more details about the care they are providing patients, executives said.
"If you didn't write it down, it didn't happen," said Brian Lichtenstein, M.D., chief medical informatics officer at Sharp HealthCare, as quoted in the ROI report. "Even when you're trying to be exhaustive in your notes, you may document eight or 10 things that you did, which is good, but getting all 10 is great and helps ensure our documentation fully reflects the care we deliver. In addition, our clinicians are reporting that they feel more able to see patients in need of urgent attention."
Using Abridge's evaluation methodology, MaineHealth, a health system based in Portland, Maine, with 2,000 clinicians serving 600,000 patients, reported a 23% drop in the amount of time clinicians spend in clinical notes per patient encounter, based on an assessment of average minutes in notes. Clinicians are also spending less time on clinical notes outside of work hours, with a 9% drop in WOW average minutes per patient encounter.
Of Abridge users at MaineHealth, 70.3% of encounters were documented using Abridge's AI scribe. Rebecca Hemphill, M.D., chief medical information officer at MaineHealth, said it's important to accurately measure what health system leaders are hearing from clinicians. Working with Abridge, the health system is developing the tools to do that measurement easily and efficiently, Hemphill said.
Reid Health, a Richmond, Indiana-based health system with 300 clinicians and 400,000 patients, reported a 4% increase in wRVUs per patient encounter.
Samaritan Health Services, based in Corvallis, Oregon, reported an 18% increase in patients seen by clinicians using Abridge's AI scribe, which signifies that the technology could help improve access to care. Of note, after deploying Abridge, the health system did not change its policy encouraging clinicians who used the technology to see more patients, according to the report.
The health system, which has more than 600 clinicians serving 290,000 patients, reported a 7.8% increase in wRVUs per patient encounter. Among clinicians using Abridge, the time they spent in clinical notes dropped a significant amount, by 38%.
"Abridge's ambient listening isn't just about capturing conversations; it's about actionable intelligence that saves time in real time. By using Abridge we have providers spending less time in their notes documenting and more time focusing on the patient's well-being. The ROI is not theoretical, it is measurable and meaningful," Sonny Sapra, senior vice president and chief digital and information officer at Samaritan Health Services, said.
Abridge says its approach to ROI evaluation gives health systems access to advanced analysis tools and reliable data while also developing industrywide standards to establish benchmarks. It also builds a foundation for more rigorous measurement tools and lays the groundwork for standardized data, which will become critical as AI becomes more ubiquitous in clinical workflows, executives said.
As AI scribing tools are more widely deployed, many organizations are developing ways to assess their impact as there currently isn't a shared framework. Health systems lack a standard way to evaluate and perform oversight for the technologies once deployed.
Researchers at Duke University recently proposed a new framework to evaluate AI scribing tools by using a combination of human review and technological evaluation. The Duke researchers created an evaluation and governance method called SCRIBE. The researchers hope it will allow hospitals and health systems to more easily compare commercial AI scribe tools and effectively evaluate their performances over time, staff writer Emma Beavins reported.
As AI solutions gain traction at provider organizations, ambient documentation adoption is the furthest along, with roughly 1 in 5 providers at full rollout and another 2 in 5 in pilot, according to a recent Bain & Company and KLAS Research report.
As AI moves from pilot to production, providers, who remain capacity-constrained are prioritizing AI technologies with demonstrable ROI, the survey found.
“Healthcare AI is moving out of the pilot stage and being leveraged to meet profitability needs of both providers and payers,” said Aaron Feinberg, a partner in Bain & Company’s healthcare and life sciences and private equity practices. “Executives want quickly scalable solutions that address key business challenges and pay for themselves with tangible results and short time-to-value windows. This is all about the bottom line now.”
According to the report, 70% of providers and 80% of payers now have an AI strategy in place or in development—significantly up from 60% for both groups in last year’s Bain/KLAS survey.
Providers cite revenue cycle management as one of their highest IT investment priorities, the survey found. RCM represents the four most common AI use cases currently: ambient documentation, clinical documentation improvement, coding and prior authorization.