Business
Commentary: AI isn’t ready to be your doctor yet — but will it ever be?
As almost everybody knows, the AI gold rush is upon us. And in few fields is it happening as fast and furiously as in healthcare.
That points to an important corollary: Beware.
Artificial intelligence technology has helped radiologists identify anomalies in images that human users have missed. It has some evident benefits in relieving doctors of the back-office routines that consume hours better spent treating patients, such as filing insurance claims and scheduling appointments.
Eventually, a lot of this stuff is going to be great, but we’re not there yet.
— Eric Topol, Scripps Research
But it has also been accused of providing erroneous information to surgeons during operations that placed their patients at grave risk of injury, and fomenting panic among users who take its offhand responses as serious diagnoses.
The commercial direct-to-consumer applications being promoted by AI firms, such as OpenAI’s ChatGPT Health and Anthropic’s Claude for Healthcare — both of which were introduced in January — raise special concerns among medical professionals. That’s because they’ve been pitched to users who may not appreciate their tendency to output erroneous information errors and offer inappropriate advice.
“Eventually, a lot of this stuff is going to be great, but we’re not there yet,” says Eric Topol, a cardiologist associated with Scripps Research Institute in La Jolla.
“The fact that they’re putting these out without enough anchoring in safety and quality and consistency concerns me,” Topol says. “They need much tighter testing. The problem I have is that these efforts are largely stemming from commercial interests — there’s furious competition to be the first to come out with an app for patients, even if it’s not quite ready yet.”
That was the experience reported by Washington Post technology columnist Geoffrey A. Fowler, who provided ChatGPT with 10 years of health data compiled by his Apple Watch — and received a warning about his cardiac health so dire that it sent him to his cardiologist, who told him he was in the bloom of health.
Fowler also sought out Topol, who reviewed the data and found the Chatbot’s warning to be “baseless.” Anthropic’s chatbot also provided Fowler with a health grade that Topol deemed dubious.
“Claude is designed to help users understand and organize their health information, framing responses as general health information rather than medical advice,” an Anthropic spokesman told me by email. “It can provide clinical context—for example, explaining how a lab value compares to diagnostic thresholds—while clearly stating that formal diagnosis requires professional evaluation.”
OpenAI didn’t respond to my questions about the safety and reliability of its consumer app.
Topol, who has written extensively about advanced technology in medicine, is nothing like an AI skeptic. He calls himself an AI optimist, citing numerous studies showing that artificial intelligence can help doctors treat patients more effectively and even to improve their bedside manners.
But he cautions that “healthcare can’t tolerate significant errors. We have to minimize the errors, the hallucinations, the confabulations, the BS and the sycophancy” that AI technology commonly displays.
In medicine, as in many other fields, AI looks to have been oversold as a labor-saving technology. According to a study of AI-equipped stethoscopes provided to about 100 British medical groups published earlier this month in the Lancet, the British medical journal, the high-tech stethoscopes effectively identified some (but not all) indications of heart failure better than conventional stethoscopes. But 40% of the groups abandoned the new devices during the 12-month period of the study.
The main complaint was the “additional workflow burden” experienced by the users — an indication that whatever the virtues of the new technology, they didn’t outweigh the time and effort needed to use them.
Other studies have found that AI can augment physicians’ skills — when the doctors have learned to trust their AI tools and when they’re used in relatively uncomplicated, even generic, conditions.
The most notable benefits have been found in radiology; according to a Dutch study published last year, radiologists using AI to help interpret breast X-rays did as well in finding cancers as two radiologists working together. That suggested that judicious use of AI could free up time for one of the two radiologists. But in this case as in others, the AI helper didn’t do consistently well.
“AI misses some breast cancers that are recalled by human assessment,” a study author said, “but detects a similar number of breast cancers otherwise missed by the interpreting radiologists.”
AI’s incursion into healthcare even has become something of a cultural touchstone: In HBO’s up-to-the-minute emergency room series “The Pitt,” beleaguered ER doctors discover that an AI app pushed on them as a time-saving charting tool has “hallucinated” a history of appendicitis for a patient, endangering the patient’s treatment.
“Generative AI is not perfect,” the app’s sponsor responds. “We still need to proofread every chart it creates” — thus acknowledging, accurately, that AI can increase, not relieve, users’ workloads.
A future in which robots perform surgical operations or make accurate diagnoses remains the stuff of science fiction. In medicine, as elsewhere, AI technology has been shown to be useful to take over automatable tasks from humans, but not in situations requiring human ingenuity or creativity — or precision. And attempts to use AI-related algorithms to make healthcare judgments have been challenged in court.
In a class-action lawsuit filed in Minnesota federal court in 2023, five Medicare patients and survivors of three others allege that UnitedHealth Group, the nation’s largest medical insurer, relied on an AI algorithm to deny coverage for their care, “overriding their treating physicians’ determinations as to medically necessary care based on an AI model” with a 90% error rate.
The case is pending. In its defense, UnitedHealth has asserted that decisions on whether to approve or deny coverage remain entirely in the hands of physicians and other clinical professionals the company employs, and their decisions on coverage and care comply with Medicare standards.
The AI algorithm cited by the plaintiffs, UnitedHealth says, is not used “to deny care to members or to make adverse medical necessity coverage determinations,” but rather to help physicians and patients “anticipate and plan for future care needs.” The company didn’t address the plaintiffs’ assertion about the algorithm’s error rate.
“We shouldn’t be complacent about accepting errors” from AI tools, Topol told me. But it’s proper to wonder whether that message has been absorbed by promoters of AI health applications.
Disclaimers warning that AI responses “are not professionally vetted or a substitute for medical advice” have all but disappeared from AI platforms, according to a survey by researchers at Stanford and UC Berkeley.
The issue becomes more urgent as the language of chatbots becomes more sophisticated and fluent, inspiring unwarranted confidence in their conclusions, the researchers cautioned. “Users may misinterpret AI-generated content as expert guidance,” they wrote, “potentially resulting in delayed treatment, inappropriate self-care, or misplaced trust in non-validated information.”
Typically, state laws require that medical diagnoses and clinical decisions proceed from physical examinations by licensed doctors and after a full workup of a patient’s medical and family history. They don’t necessarily rule out doctors’ use of AI to help them develop diagnoses or treatment plans, but the doctors must remain in control.
The Food and Drug Administration exempts medical devices from government licensing if they’re “intended generally for patient education, and … not intended for use in the diagnosis of disease or other conditions. That may cover AI bots if they’re not issuing diagnoses.
But that may not help users who have willingly uploaded their medical histories and test results to AI bots, unaware of concerns, including whether their information will be kept private or used against them in insurance decisions. Gaps in their uploaded data my affect the advice they receive from bots. And because the bots know nothing except the content they’ve been fed, their healthcare outputs may reflect cultural biases in the basic data, such as ethnic disparities in disease incidence and treatment.
“If there’s a mistake with all your data, you could get into a pretty severe anxiety attack,” Topol says. “Patients should verify, not just trust” what they’ve heard from a bot.
Topol warns that the negative effect of misleading AI information may not only fall on patients, but on the AI field itself. “The public doesn’t really differentiate between individual bots,” he told me. “All we need are some horror stories” about misdiagnoses or dangerous advice, “and that whole area is tarred.”
In his view, that would limit the promise of technologies that could improve the effectiveness of medical practice in many ways. The remedy is for AI applications to be subjected to the same clinical standards applied to “a drug, a device, a diagnostic. We can’t lower the threshold because it’s something new, or different, with some broad appeal.”
Dear Mr. Pelley:
I meant what I said in my letter last week to the 60 Minutes team: joining 60 Minutes is the honor of my career and I am grateful to be working alongside the people who have contributed to the most important television journalism brand this country has ever produced. While I’m new to 60 Minutes, I’ve devoted my career to investigative journalism and storytelling. I started this job excited to collaborate and to benefit from the wisdom and experience of the 60 Minutes veterans, with you among them. For that reason, one of the first things I did in my new role was call you to talk and invite you to dinner. It is a profound disappointment that you rejected that overture and chose ambush instead. Yesterday, you hijacked my first meeting with staff to disparage me, my qualifications, and my intentions with remarkable incivility and contempt. I welcome a diversity of viewpoints and respectful debate among the team, but this was nothing of the sort. Yesterday’s performative display of hostility enacted in front of the staff instead of in a civil, private conversation-demonstrated that you have no interest in contributing to the future success of the show, or approaching my new tenure with a mind open to collaboration and progress. I am here to deliver first-in-class news programming, not to make headlines about newsroom drama. I am eager to work alongside those who share this goal.
Despite yesterday’s misconduct, I had hoped that in sitting down with you today we could find a path forward together. You made clear that you are not interested in such a path.
Your antipathy to the future of the show has come through loud and clear. And I have heard you. I therefore write on behalf of CBS News, Inc. (“CBS”) to inform you that your employment with CBS is terminated for cause effective immediately. Enclosed is your formal termination letter.
Sincerely,
Nick Bilton
Executive Producer, 60 Minutes
The co-founder of Aspiration, Joseph Sanberg, was sentenced to 14 years in prison on Monday after defrauding investors and lenders of over $248 million.
The startup, an eco-friendly digital banking company boasting fossil fuel-free investments, carbon offsets for gas purchases, and a debit card with cash-back benefits for shopping at clean companies, was founded by Sanberg and Andrei Cherny. Cherny left the company in 2022 and has not been charged.
Sanberg, an Orange County native, pleaded guilty to wire fraud in October after being arrested in March last year. Aspiration subsequently filed for bankruptcy and liquidated all of its assets by July.
Sanberg and venture capitalist Ibrahim AlHusseini, who also faces charges, together forged a series of bank statements in order to obtain loans. From 2020 to 2021, the pair forged AlHusseini’s bank statements to show millions of dollars in assets in order to obtain millions of dollars from lenders.
Additionally, they forged a letter from their audit committee stating that $250 million in funds were available, when in reality Aspiration had less than $1 million. The amount of loans defrauded exceeded $248 million.
In 2021, Sanberg artificially inflated Aspiration’s 2021 revenue by $44 million by recruiting 27 fake customers to sign letters of intent pledging tens of thousands of dollars per month for tree planting services. Sanberg himself funded the contracts and used the inflated revenue numbers to obtain more loans.
The charges sparked an NBA investigation into salary cap allegations due to Aspiration’s connections with Clippers owner Steve Ballmer.
Ballmer personally invested $60 million in Aspiration, all of which was lost. He is now the target of a civil lawsuit alleging his participation in the scheme. Ballmer denies the allegations.
The team announced a $300-million sponsorship deal with Aspiration, and Clippers player Kawhi Leonard signed a four-year, $28-million marketing contract with the company, which reportedly performed no duties. The issue has raised concerns about how players are circumventing the NBA’s salary cap.
The team lost the $300-million sponsorship deal and an additional $20 million paid for carbon offset purchases.
Residents in the city of Monterey Park will be the first in the nation to vote on a permanent ban on data centers Tuesday.
If approved, Measure NDC would prohibit data centers within the city limits and could only be overturned by another vote.
Yard signs saying “No Data Center” in English and Chinese with images of dragons line sidewalks in the San Gabriel Valley city.
As a wave of data center opposition sweeps the country, numerous towns and counties across the U.S. have instituted temporary moratoria and other restrictions on the facilities. But only a handful have instituted indefinite bans, and just four other towns have sent related matters to the ballot.
Supporters are hoping the vote will set a precedent for the rest of the region, where residents are fighting proposals in Vernon and City of Industry.
“This is about as permanent a ban as we can get,” said Steven Kung, co-founder of the group No Data Center Monterey Park. “Winning Measure NDC would send a huge message to the rest of the San Gabriel Valley about how residents don’t want data centers.”
The ballot measure emerged from the fight against a 247,000-square-foot center proposed in 2024 by the Australian-owned investment firm HMC StratCap for a residential area in Monterey Park.
The facility would have sat less than 500 feet away from the nearest home and used three times the electricity of the 60,000-person, predominantly Asian American city.
While the developer touted the potential for jobs and tax revenue, residents expressed concerns about noise and air pollution, rising electricity rates and a potential to lower property values.
The company pulled its plans in late March following public outcry and a March 4 city council vote to extend a temporary data center moratorium and place a ban on Tuesday’s ballot.
In a letter to the city council, HMC StratCap said it would pursue a different use for the land and would not engage in a ballot measure fight.
The city council later banned data centers indefinitely, the first in California to do so, said Mayor Elizabeth Yang. But she’s still been out campaigning for the measure with all four other council members.
“If a council puts in an ordinance, a future council can reverse it too,” said Yang. “With the ballot measure, unbanning it is a lot harder because you need the entire city to vote on it.”
The measure proposes the ban “to protect air quality, drinking water resources, and public health” and “prevent impacts to electricity and water rates.”
While California places third in the country for existing data centers with about 300 facilities, it hasn’t been a hot spot in the recent AI-driven data center boom. High electricity rates, expensive land and regulatory hurdles mean that fewer, and smaller, facilities are currently planned than in Virginia, Texas, Georgia, Illinois or Arizona.
“Most of California’s data centers are small by today’s standards,” said Shaolei Ren, an engineering professor at UC Riverside who studies how to reduce the environmental impacts of data centers. “Ten years ago, they would be medium-sized, but the power demand for new AI data centers has increased a lot.”
The average operating data center demands 45 megawatts, according to the Washington Post, while the average planned one would draw 430 MW. The one proposed for Monterey Park would have required about 50 MW at peak demand.
As proposals crop up in SoCal, they’re met with fierce opposition. Montebello, El Monte and Baldwin Park have all enacted temporary moratoria, and Alhambra recently banned data centers as part of a zoning code update. City of Industry, Vernon, City of Commerce and Santa Fe Springs are moving in the other direction, trying to court developers and streamline data center approvals. Community groups are fighting that.
Outside the San Gabriel Valley, residents of Coachella and Imperial County are showing up in droves to protest local proposals.
Matthew Shaw, a volunteer with the Coalition for Responsible Data Center Development, who recently published a report on opposition to AI data centers, said a vote to ban them in Monterey Park “would lead to copycats, partially because so many groups are just opposed to any data center development at all.”
While there is no formal opposition to Measure NDC, some building trades like Ironworker Local 433 supported the Monterey Park data center when it was still live before city council. Those in the data center industry are lamenting the state of public opinion.
“These are multi-billion-dollar assets that are built by multi-trillion-dollar companies. These things will get done,” said Mehdi Paryavi, chairman of the International Data Center Authority. “My biggest problem is that our industry does not invest enough in community engagement.”
Paryavi said towns that seek to limit data centers are missing out on thousands of jobs generated by data center construction, operations and customers, as well as faster artificial intelligence speeds and better performance.
Kung said local community organizers are “looking at the empirical evidence” and seeing a ban as a win.
“We’ve never seen a city that embraces a data center and is like, ‘Look how our quality of life has increased, look how all the revenue has gone into citywide improvements,’” he said. “That just doesn’t exist.”
-
News8 minutes agoCalifornia’s primary for governor is undecided as candidates vie to be in the top two
-
Los Angeles, Ca2 hours agoCalifornia primary election results: governor and L.A. mayor races
-
Detroit, MI2 hours agoAnother bribery scandal hits Detroit. It involves the People Mover
-
San Francisco, CA2 hours agoWhat’s Worth More Than Cash in San Francisco Real Estate? Anthropic Stock
-
Dallas, TX2 hours agoDallas weighs $500 million‑plus repair plans as City Hall’s future comes up for debate
-
Miami, FL2 hours agoMiami biotech executive was followed into his condo by man who allegedly threw him from 25th floor
-
Boston, MA2 hours agoWhat a World Cup ‘fan zone’ is and what Boston fans can expect in 2026
-
Denver, CO2 hours agoDefensive lineman Jordan Miller has a tough battle to make the Broncos’ final 53-man roster
