Science
Contributor: The crucial medical question that AI can’t ever answer
One of us got a call last spring from a longtime friend. The story was familiar: two doctors, an MRI, an online AI tool, a stack of articles — and one anxious question. “Everything tells me something different. The AI says I might need surgery. What should I do?”
We believe there’s one key response to anyone in this all-too-common conundrum: “What matters most to you?”
There was a long pause.
That pause is one of the most important moments in modern healthcare — and it is exactly the question artificial intelligence is unable to address.
In our careers as physicians and researchers, we have found, clearly and repeatedly, that for many common conditions the medical evidence does not point to a single “right” answer. The biology is often close. What determines the success of an outcome is whether the choice fits the person making it.
Some patients with back pain want the fastest possible return to physically demanding work, even if it means surgery. Others want to avoid an operation at almost any cost, even if recovery takes longer. The scan may look the same. The lives behind the scan are not.
That insight is becoming critically important as artificial intelligence moves deeper into everyday health decisions.
In our research on AI and clinical decision-making, we’ve studied what happens when systems are trained to optimize medical outcomes but are blind to human values. In plain English, today’s AI is very good at telling you what usually works for people like you with similar demographics and medical histories. It is far less capable of understanding what you are trying to protect, avoid or prioritize.
This matters because some of the most common and most expensive medical decisions are not purely biological. Should someone with low-risk prostate cancer choose surgery, radiation or careful monitoring? Should a person with atrial fibrillation undergo a procedure or manage the condition with medication? Should a patient with chronic knee or back pain operate now or try months of physical therapy to see whether surgery can be avoided?
In these situations, the medical differences between options are often small or uncertain. What makes the biggest difference is whether the treatment aligns with the patient’s goals: tolerance for risk, willingness to undergo recovery, ability to adhere to long-term therapy or simply what kind of life they want to live.
AI systems can calculate probabilities. They cannot determine what those probabilities mean to a particular person.
In some respects, artificial intelligence may know more medicine than any individual physician. It can synthesize millions of scientific papers, clinical studies and patient records in seconds. Yet it knows remarkably little about the person sitting across from it. AI does not know a patient’s goals, fears, obligations, tolerance for risk or personal definition of a good outcome. And because it knows little about either the patient or the physician, it knows even less about the conversation between them — the place where facts, values and trust come together to produce the right decision for a particular person.
A second patient story brought this home. A retired teacher was referred after an AI-based symptom checker flagged a heart rhythm abnormality and “favored” an invasive procedure. The patient arrived frightened, convinced there was one correct path. When we talked, it became clear that what mattered most was avoiding a long recovery and staying healthy enough to travel to see grandchildren.
Medication and monitoring — less dramatic, but well-supported by evidence — fit those goals better. The AI wasn’t wrong. It just didn’t know what mattered.
This blind spot is not trivial. Roughly a quarter of U.S. healthcare spending flows through decisions in which patient preferences meaningfully affect outcomes. When those preferences are ignored — by people or by algorithms — care becomes misaligned. That can mean unnecessary procedures, poor adherence, regret and rising costs without better health.
So what should consumers do when an app, portal or “smart” tool recommends a course of action?
Start with three questions.
First: “Best for whom?” If a tool says one option is best, ask whether it means best on average — or best for someone with your priorities.
Second: “What does this system not know about me?”
AI can see lab values and imaging results. It cannot see your job, your family responsibilities, your fears or what you are trying to get back to.
Third: “What happens if I wait or choose differently?”
Many important medical decisions are not emergencies. When options are close, taking time to reflect is often part of good care.
Artificial intelligence is becoming a powerful partner in medicine. It can help explain options, surface evidence and reduce confusion. But it should inform human decisions, not replace them.
AI may know more medicine than any physician.
It knows far less about any patient.
And it knows least about the conversation between them.
The most important variable in your healthcare is not in any algorithm. It is you.
James N. Weinstein is a surgeon and former chief executive of Dartmouth Health. He is a clinical professor at Northwestern University’s Kellogg School of Management and global head of Health Futures at Microsoft, which develops AI systems. Ogan Gurel is a physician and assistant professor at the University of Texas at Arlington, where he researches AI, causal inference and patient decision-making.
Science
China Launches Reusable Rocket in Race With SpaceX
Video released by Chinese state media shows a state-owned aerospace company launching a rocket and recovering part of it on Friday. The successful launch of a reusable rocket was a major step for China toward challenging SpaceX’s satellite internet dominance.
Science
Nobel Prize winner leaving UC Berkeley for new role in China
Nobel Prize recipient Omar Yaghi is leaving his role at UC Berkeley to lead the development of a new artificial intelligence institute at Tsinghua University in Beijing, the Chinese university announced.
Yaghi will head the AI Chemistry and Materials Research Institute at Tsinghua, where he was appointed an honorary professor in 2022. Known as AIMATRY (AI × Materials × Chemistry), the new center will focus on material design and synthesis through artificial intelligence, according to a statement from the university.
In 2025, Yaghi shared the Nobel Prize in chemistry with Susumu Kitagawa of Kyoto University and Richard Robson of the University of Melbourne for their development of metal-organic frameworks, a type of super-porous material in which metal ions and carbon-based molecules combine to form crystals with exceptionally large surface areas.
The material has the potential to combat climate change by capturing and storing carbon or other pollutants, and by extracting water from the atmosphere in water-scarce areas. Upon awarding the prize, a member of the Nobel committee likened the technology’s ability to store enormous amounts of stuff in seemingly compact spaces to Hermione Granger’s enchanted handbag in the Harry Potter series.
Yaghi’s Irvine-based company, Atoco, has said it will start taking orders later this year for its technology that harvests water from the air.
A representative for Yaghi said he was not yet available to respond to questions.
China is one of several countries that has been actively recruiting scientists from the U.S., where the Trump administration has slashed science funding, suspended research grants, fired science advisors and tightened immigration restrictions.
“For many, many years, our funding was very competitive; if you worked hard and you were doing good research, you would get funding,” Yaghi said of the U.S. in an interview with Scientific American earlier this year. “The current state is not so encouraging because of the cutting back on grants and support of science by the very agencies that many university researchers rely on.”
Yaghi was born in Jordan to Palestinian refugees, and immigrated to the U.S. when he was 15 to study.
“We’ve learned over and over in human civilization that scholars can move across borders,” Yaghi told the New York Times last year. “This is how knowledge spread and how vast regions of the world lifted themselves out of poverty.”
Science
Trump administration seeks to limit federal funding that doesn’t ‘advance’ presidential policies
A new rule proposed by the White House Office of Management and Budget would fundamentally overhaul the way federal grants are awarded and overseen — a sweeping change that one scientific society said “would all but end the use of scientific merit in the selection of grants and programs across the government.”
Proposed in late May, the rule would give political appointees unprecedented control over federal grants for research, education and infrastructure, and specifies that government funds can only be spent on projects “aligned with administration policies and priorities,” according to a copy of the proposed rule.
The rule would also restrict research topics, limit U.S. scientists’ ability to collaborate with colleagues in other countries and make it easier for the government to suspend or cancel grants at any time.
The changes are intended to improve “transparency, accountability, and oversight for Federal awards” while “ensuring that American tax dollars are not wasted or misused,” according to the White House office.
But critics say that if the rule is implemented, the final sign-off for grants will no longer be in the hands of subject-matter experts within individual agencies, but in those of political appointees.
“This touches all parts of American life,” said Dr. Eric Rafla-Yuan, a psychiatrist who practices at the Veterans Administration and San Diego County’s psychiatric hospital.
“Control of how all of the federal grants and programs are funded will fall under a small group of highly partisan individuals who would have very few limits on how they spend these billions of taxpayer dollars,” said Rafla-Yuan, who also chairs the Committee to Protect Public Mental Health advocacy group. “This touches everyone’s life, even if they don’t realize it.”
OMB published the proposed rule May 29, opening a 45-day comment period that closes July 13.
Opposition to the proposed rule has mobilized multiple sectors of society. Professional groups representing cancer researchers, civil engineers, county governments, medical schools, housing agencies, city and municipal governments, nonprofits and others have publicly expressed concerns about potential consequences.
By midday Thursday, the Federal Register logged nearly 100,000 comments about the proposal, many of them expressing concern.
“I understand the need for oversight, fiscal responsibility, and accountability. That is not the issue,” wrote Jack Feldman, a neuroscientist who holds the David Geffen School of Medicine Chair in Neuroscience at UCLA. “The issue is whether scientific research is to be judged by scientific merit, or whether it can be approved, denied, or terminated according to broad political criteria that may change from one administration to the next.”
Crucially, the rule converts policies governing federal grants from “guidance” into binding regulations that all agencies would be required to follow. It would give political appointees power to override federal agencies’ merit-based reviews and mandate that a political appointee review decisions to ensure that all awards “demonstrably advance the President’s policy priorities.”
The elevation of political appointees in what were previously merit-based decisions has alarmed many scientists.
“The proposed rule changes would all but end the use of scientific merit in the selection of grants and programs across the government,” read a statement from the Planetary Society, a nonprofit dedicated to space research.
Researchers and science groups have also expressed concern about a section of the rule prohibiting the promotion of “theories of disparate-impact liability” — a legal concept that refers to policies that appear neutral but cause disproportionate harm to certain groups.
The section’s vague language and many loopholes could have a chilling effect on any research that studies the effects of a disease, policy or public health intervention on any specific group of people, Rafla-Yuan said.
As an example, he said, “if there’s a specific age range that is at higher risk for suicide, and we want to figure out, well, what’s going on with people that are aged 14 to 19 … we can’t do that under the wording in this rule.”
New restrictions on collaborations with scientists in other countries would hinder opportunities for U.S. researchers and limit innovation, said Joanne Padrón Carney, chief government relations officer for the American Assn. for the Advancement of Science.
“Science is a global enterprise. Especially in biomedical and public health fields, diseases don’t care about borders or government policies,” she said.
California’s congressional delegation sent a letter Wednesday asking OMB to rescind the proposal, outlining concerns about its impact on scientific innovation, U.S. competitiveness and the fiscal stability of local governments, many of which rely on federal grants for local services.
The proposed rule grants the federal government broad powers to suspend or cancel grants for any reason, introducing “unprecedented unpredictability into local governance,” the lawmakers wrote, “leaving vital infrastructure projects unfinished and abandoning vulnerable populations who rely on these services.”
Republican Sen. Susan Collins has also asked the White House to withdraw certain parts of the letter and extend the public comment period, saying the proposed rule as written would “harm small and rural communities, undermine scientific and biomedical research, and conflict with Congress’ control over the federal funding process.”
-
Los Angeles, Ca44 minutes agoRosemead High School student identified as teen stabbed to death in El Monte
-
Detroit, MI1 hour agoThree dead in Metro Detroit mall shootings in eight days: What we know
-
San Francisco, CA1 hour ago
I’m a writer who left LA for an AI startup in San Francisco. It was like stepping into a whole new world.
-
Dallas, TX1 hour agoDallas weather: Widespread thunderstorms bring flash flood risks and brief heat relief
-
Miami, FL1 hour agoWhy I’m Not Worried About Giannis in Miami
-
Boston, MA1 hour agoGallery: Tall ships display their splendor at Sail Boston
-
Seattle, WA2 hours agoSeattle Weather: Cooler Sunday to close out weekend
-
Denver, CO2 hours agoColorado Court of Appeals reverses sanction against Denver DA for pattern of discovery violations