Health
AI chatbots fall short when giving cancer treatment recommendations: ‘Remain cautious’
OpenAI’s ChatGPT has become a popular go-to for quick responses to questions of all types — but a new study in JAMA Oncology suggests that the artificial intelligence chatbot might have some serious shortcomings when it comes to doling out medical advice for cancer treatment.
Researchers from Mass General Brigham, Sloan Kettering and Boston Children’s Hospital put ChatGPT to the test by compiling 104 different prompts and asking the chatbot for recommendations on cancer treatments.
Next, they had a team of four board-certified oncologists review and score the responses using five criteria.
Overall, ChatGPT scored an underwhelming 61.9%.
WHAT IS ARTIFICIAL INTELLIGENCE?
Although language learning models (LLMs) have successfully passed the U.S. Medical Licensing Examination, the chatbot underperformed when it came to providing accurate cancer treatment recommendations that align with National Comprehensive Cancer Network (NCCN) guidelines.
In many cases, the responses were unclear or mixed inaccurate and accurate information.
Nearly 13% of the responses were “hallucinated,” which means they might have sounded factual, but were completely inaccurate or unrelated to the prompt, according to the researchers’ findings.
“This is a significant concern, as it could lead to misinformation and potentially harmful patient decisions,” said Dr. Harvey Castro, an emergency medicine physician and AI expert in Coppell, Texas.
NEW AI TECH AIMS TO DETECT THE ORIGIN OF CANCERS FOR OPTIMAL TREATMENTS: ‘AN IMPORTANT STEP’
Castro was not involved in the study but commented on the findings.
“For example, a patient with advanced lung cancer may receive a recommendation for a treatment not recognized by the NCCN guidelines, which could lead to delays in receiving appropriate care.”
Danielle Bitterman, study co-author and assistant professor of radiation oncology at Harvard Medical School, said that overall, the results met expectations.
“ChatGPT and many of the similar large language models are trained primarily to function as chatbots, but they are not specifically trained to reliably provide factually correct information,” she told Fox News Digital.
“Our results showed that the model is good at speaking fluently and mimicking human language,” she noted. “But a challenging aspect for health advice is that it makes it hard to detect correct versus incorrect information.”
AI CHATBOT AIMS TO PROVIDE SUPPORT FOR WOMEN WITH POSTPARTUM DEPRESSION: ‘A TOOL, NOT A REPLACEMENT’
She went on, “When reading the responses, I was struck by how correct treatment options were seamlessly mixed in with wrong ones. Also, I was encouraged that almost all responses did contain some correct information — this shows the future potential of models to communicate information in collaboration with physician input, even if we aren’t there yet,” she added.
Study limitations
The study’s key limitation was that the researchers evaluated only one LLM in one “snapshot in time”; but they believe the findings highlight legitimate concerns and the need for future research.
ChatGPT 3.5 was used for this study, but OpenAI released a newer model, GPT 4, after the research concluded.
“Nevertheless, the model we tested is the one that is publicly available and the most accessible by a wide population of patients,” Bitterman said.
The researchers also did not do intensive investigations into prompt engineering, which may have improved results, she added.
ASK A DOC: 25 BURNING QUESTIONS ABOUT AI AND HEALTH CARE ANSWERED BY AN EXPERT
“Instead, we designed our prompts (questions) from the perspective of a general member of the population asking general questions about cancer treatment.”
Also, the study does not discuss the ethical considerations of using AI chatbots for providing cancer treatment recommendations, noted Dr. Castro.
“While AI chatbots can be a valuable tool, they should be used as a supplement, not a replacement, for professional medical advice.”
“It is important to consider the potential risks and benefits of using AI chatbots in this context and have safeguards to ensure that patients receive accurate and appropriate recommendations,” he told Fox News Digital.
Castro said he sees promise in the use of AI chatbots for providing cancer treatment information — but significant challenges still need to be addressed.
“While AI chatbots can be a valuable tool, they should be used as a supplement, not a replacement, for professional medical advice,” he said.
“As a physician, it is important to remain cautious and continue relying on established guidelines and clinical expertise when making treatment recommendations,” Castro went on.
“There is too much at stake if we get this wrong.”
“Future research must assess AI chatbots’ long-term impact and generalizability in cancer treatment and patient self-education.”
NEW AI ‘CANCER CHATBOT’ PROVIDES PATIENTS AND FAMILIES WITH 24/7 SUPPORT: ‘EMPATHETIC APPROACH’
Also, Castro would like to see future studies assess more types of cancer.
“The study assessed the chatbot’s performance in providing breast, prostate and lung cancer treatment recommendations,” he noted. “It is unknown how the chatbot would perform in giving suggestions for other types of cancer or other medical conditions.”
While generalist models like ChatGPT are not trained to provide medical advice — and the quality of the information “doesn’t meet the bar for medicine” — Bitterman said they do show potential for synthesizing information in accessible language.
“There is much excitement and potential of AI in health care, but we need to carefully evaluate our models at each step and optimize them for the high-stakes clinical domain,” she told Fox News Digital.
CLICK HERE TO SIGN UP FOR OUR HEALTH NEWSLETTER
With medicine and standards of care constantly evolving, Bitterman noted that if a model were developed for clinical use, it would have to provide up-to-date guidelines.
“This will require that developers provide transparency about what data the models were trained on and re-evaluate their performance over time,” she said.
“There is too much at stake if we get this wrong — and patient safety is paramount,” Bitterman added.
“If there are early errors due to hasty uptake without sufficient testing, it could ultimately set the field back and slow the potential gains.”
Health
State Department Permits Distribution of H.I.V. Medications to Resume — for Now
The Trump administration on Tuesday issued a waiver for lifesaving medicines and medical services, offering a reprieve for a worldwide H.I.V. treatment program that was halted last week.
The waiver, announced by Secretary of State Marco Rubio, seemed to allow for the distribution of H.I.V. medications, but whether the waiver extended to preventive drugs or other services offered by the program, the President’s Emergency Plan for AIDS Relief, was not immediately clear.
Still, PEPFAR’s future remains in jeopardy, with potential consequences for more than 20 million people — including 500,000 children — who could lose access to lifesaving medications. Without treatment, millions of people with H.I.V. in low-income countries would be at risk of full-blown AIDS and of premature death.
“We can very rapidly return to where the pandemic is exploding, like it was back in the 1980s,” said Dr. Steve Deeks, an H.I.V. expert at the University of California, San Francisco.
“This really cannot happen,” he said.
On Monday, the Trump administration ordered health organizations in other countries to immediately stop distributing H.I.V. medications purchased with U.S. aid. The directive stemmed from a freeze — which may become permanent — in the activities of PEPFAR, a $7.5 billion program overseen by the State Department.
Since it started in 2003, PEPFAR is estimated to have saved more than 25 million lives; more than 5.5 million children have been born free of H.I.V. who otherwise would have been infected.
In South Africa alone, PEPFAR’s shutdown would add more than a half million new H.I.V. infections and more than 600,000 related deaths over the next decade, according to one estimate.
The organization employs 270,000 doctors, nurses, pharmacists and other health workers. They had been told not to report to work or to serve patients.
PEPFAR’s end would “create instability and potentially collapse several countries’s AIDS programs that will be difficult to repair, if and when PEPFAR funding becomes available again,” said Dr. Salim Abdool Karim, an infectious disease epidemiologist at the University of KwaZulu-Natal in Durban, South Africa.
Dr. Abdool Karim said countries should stop relying on PEPFAR and support their own citizens, a goal that the program’s staff and partners had been working toward. But ideally that shift would happen gradually, over years during which PEPFAR would train local health workers and prepare them for the transition, he said.
“This is not a bad opportunity for countries to take greater responsibility,” he said. “But I think they can’t do it if it’s done in this kind of haphazard and unplanned way.”
Here’s what he and others expect from PEPFAR’s unexpected pause.
Sudden stops to H.I.V. treatment can quickly turn dangerous.
Every day, more than 220,000 people pick up H.I.V. medications at clinics funded by PEPFAR; the number included more than 7,400 children under 15, according to data published on Tuesday by AMFAR, The Foundation for AIDS Research.
The drugs work by suppressing H.I.V. in the body. When patients go off the drugs, the virus grabs the opportunity to rebound — and quickly. Within a week, H.I.V. levels will skyrocket from undetectable levels to more than 100,000 copies per milliliter of blood.
“That may be a time where you are very much at risk of passing the virus on to others,” Dr. Sallie Permar, a pediatrician and H.I.V. expert at Weill Cornell Medicine, said.
Then, the virus will start attacking a certain type of immune cell, crippling the body’s ability to fend off other infections, including tuberculosis, which frequently accompanies H.I.V. infection.
Spiking H.I.V. levels at first may cause flulike symptoms, including sore throat, swollen glands and fatigue. The immune system will likely marshal enough force to suppress the virus temporarily, but H.I.V. is adept at hiding until it finds the right opportunity to re-emerge.
When that occasion arises, “they can develop AIDS and progress,” Dr. Deeks said.
Children may be among the hardest hit.
PEPFAR is best known for financing H.I.V. treatment programs, but its funds also go to drugs for prevention, outreach and testing, and to support for orphans and women experiencing gender-based violence.
The loss of resources for each of these efforts will derail the fight against AIDS, said Dr. Glenda Gray, a pediatric H.I.V. expert at Wits University in South Africa.
“If H.I.V. testing falls by the wayside, it’s unlikely that we will be able to even diagnose people who need to go into treatment,” she said.
If a pregnant or breastfeeding woman has H.IV. but is not tested and not treated, she may pass the virus to her child. The higher her viral load, the more likely this is to occur.
Children with H.I.V. are less likely to be diagnosed than adults, and may not be treated till the virus makes them visibly very sick. This progression can be much more rapid in children than in adults, Dr. Gray said, “and obviously, children who are untreated are likely to die.”
Inconsistent treatment drives drug resistance.
As people lose access to medications, they may try to spread out their supplies by alternating days or to share their pills with others. If the virus replicates in people with only partial protection, it can learn to evade those defenses and become resistant to the medications.
People living with the virus may then pass the resistant virus on to others.
“That becomes a big problem, because now, suddenly, our cheap first-line drugs might not work when we have to restart them on treatment,” Dr. Abdool Karim said.
A virus that is resistant to treatments will also be better at evading vaccine candidates being tested.
“Not only are we looking at more drug resistance, but we’re looking at losing whatever ability we had to make an effective vaccine,” Dr. Permar said.
PEPFAR’s end may affect Americans, too.
More than one million Americans are living with the virus, and more than 30,000 become infected each year. If H.I.V. becomes resistant to available medications, it isn’t likely to remain in low-income countries. Americans, too, will be at risk.
They may also face indirect harms from ending PEPFAR. Creating huge populations of immunocompromised people may mean that other pathogens have an opportunity to spread. For example, dangerous Covid variants, including Omicron, are thought to have evolved in immunocompromised people with H.I.V.
At the same time, people worldwide have benefited from trials conducted under PEPFAR’s auspices, showing the importance of treating H.I.V. early, demonstrating that pregnant women can safely breastfeed as long as they are treated and that H.I.V. infections can be prevented with long-acting drugs.
“America has gotten an amazing amount of love around the world because of what it’s done,” Dr. Deeks said.
“From a humanitarian perspective, I can’t imagine anyone really wants to go along this pathway,” he added. “This doesn’t make any sense on any level.”
Health
This disease kills more people than all cancers and accidents combined
Heart disease remains the top killer of Americans as risk factors continue to grow.
The latest statistics were revealed in the American Heart Association’s annual report, 2025 Heart Disease and Stroke Statistics.
The report, which was published on Jan. 27 in the AHA’s journal Circulation, revealed that cardiovascular disease kills more people than all types of cancer and accidental deaths combined.
7 BLOOD PRESSURE MISTAKES THAT COULD BE THROWING OFF YOUR READINGS
In 2022 — the most recent year for which mortality data is available — a total of 941,652 people died of heart disease-related conditions, according to the report. This was a little more than 10,000 more deaths than the prior year.
The age-adjusted death rate decreased slightly from last year, however.
“Overall, we are seeing cardiovascular-related deaths leveling out compared to 2022,” said Dr. Bradley Serwer, a Maryland-based cardiologist and chief medical officer at VitalSolution, an Ingenovis Health company that offers cardiovascular and anesthesiology services to hospitals.
“We are making some improvements in regard to improved prevalence of hyperlipidemia (high cholesterol) and lower incidence of smoking, but have lost ground in regard to diabetes, obesity and hypertension,” Serwer, who was not involved in the report, told Fox News Digital.
DIABETES, HEART DISEASE CASES SKYROCKET — AND SCIENTISTS SUGGEST A KEY REASON
Someone dies of heart disease every 34 seconds in the U.S., and a total of 2,500 die per day, according to Keith Churchwell, M.D., the volunteer president of the American Heart Association.
“It’s alarming to note that excess weight now costs us even more lives than smoking.”
“Those are alarming statistics to me – and they should be alarming for all of us,” Churchwell said in a press release.
“Too many people are dying from heart disease and from stroke, which remains the fifth leading cause of death.”
Risk factors
The report also detailed the prevalence of specific risk factors for heart disease.
More than 72% of U.S. adults have “unhealthy weight,” which is defined as a body mass index of at least 25, the report stated.
Nearly 42% of those adults meet the criteria for obesity (a body mass index of 30 or more).
The report also revealed that more than half of U.S. adults (57%) have type 2 diabetes or prediabetes.
“Although we have made a lot of progress against cardiovascular disease in the past few decades, there is a lot more work that remains to be done,” American Heart Association volunteer Dhruv S. Kazi, M.D., wrote in an editorial that accompanied the report.
FRIENDS, FAMILY MAY PROTECT AGAINST HEART ATTACK, STROKE AND TYPE 2 DIABETES, STUDY SUGGESTS
“If recent trends continue, hypertension and obesity will each affect more than 180 million U.S. adults by 2050, whereas the prevalence of diabetes will climb to more than 80 million.”
Experts predict a 300% increase cardiovascular-related healthcare costs, added Kazi, who is also head of health economics and associate director of the Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology at Beth Israel Deaconess Medical Center.
The report also called out different rates of risk factors among different races and ethnic groups.
Black women were found to have the highest rate of obesity (57.9%) and Asian women had the lowest rate (14.5%).
Black women also had the highest rate of blood pressure (58.4%) and Hispanic women, at 35.3%, had the lowest rate.
“Just because we are seeing improvements in smoking and high cholesterol doesn’t mean we can back off.”
“We are also seeing a rise in obesity in our youth, with as many as 40% having unhealthy weight,” Serwer warned.
“This trend continues into adulthood, when we see nearly 60% of adults having unhealthy weight.”
Excess weight is a contributing factor in as many as 1,300 additional deaths per day in the U.S., or nearly 500,000 per year, according to the report.
“It lowers life expectancy by as much as 2.4 years compared to a healthy weight,” said American Heart Association volunteer Latha P. Palaniappan, M.D., a professor of cardiovascular medicine at Stanford University in Palo Alto, California.
“It’s alarming to note that excess weight now costs us even more lives than smoking – as smoking rates have actually fallen in recent years. Being overweight is the new smoking when it comes to health threats.”
CLICK HERE TO SIGN UP FOR OUR HEALTH NEWSLETTER
One positive finding in the report is that rates of high cholesterol have dropped, which is attributed to improved dietary and lifestyle factors, availability of medications and “better clinical control.”
“Recent clinical research has identified a number of new medication therapies to address the growing burden of obesity, as well, and we look forward to learning more about those advances as the body of science builds,” Churchwell noted.
What needs to change?
The researchers called for interventions to help reduce heart disease risk factors.
“Any medical or clinical therapies that can treat the risk factors that contribute to CVD are essential,” Churchwell wrote. “…We need to stop these risk factors in their tracks, keep people healthy throughout their lifespan.”
Serwer agreed that the best way to combat this top killer is to aggressively attack the risk factors.
“Just because we are seeing improvements in smoking and high cholesterol doesn’t mean we can back off,” he said.
For more Health articles, visit www.foxnews.com/health
“We need to continue to aggressively attack tobacco use and treat hyperlipidemia while increasing efforts to treat obesity, high blood pressure and diabetes.”
Health
Caroline Kennedy Calls RFK Jr. a ‘Predator’ in Letter to Senators
Caroline Kennedy wrote a scathing letter to key senators on Tuesday, calling her cousin, Robert F. Kennedy Jr., a “predator” addicted to attention from airing dangerous views on vaccinations and someone who is unfit to be the nation’s health secretary.
She urged lawmakers, who will be questioning Mr. Kennedy at his confirmation hearings Wednesday and Thursday, to reject his nomination. She cited his lack of experience, misinformed views on vaccines and personal attributes. In the letter, she described how he led other family members “down the path of drug addiction.”
“His basement, his garage, and his dorm room were the centers of the action where drugs were available, and he enjoyed showing off how he put baby chickens and mice in the blender to feed his hawks,” Ms. Kennedy wrote. “It was often a perverse scene of despair and violence.”
Her letter was first reported in The Washington Post.
Ms. Kennedy expressed particular outrage over the new disclosures in his ethics agreement filed with the Senate, which she described as outlining how his “crusade against vaccination has benefited him in other ways.”
She cited Mr. Kennedy’s decision to keep a financial stake in litigation against Merck, which makes a key vaccine against the human papillomavirus (HPV) that is administered to protect against cervical cancer.
“In other words, he is willing to enrich himself by denying access to a vaccine that can prevent almost all forms of cervical cancer and which has been safely administered to millions of boys and girls,” Ms. Kennedy wrote.
As President Joseph R. Biden Jr.’s ambassador to Australia, Ms. Kennedy was actively involved in promoting the HPV vaccine, which has put Australia on a path to eliminate cervical cancer. She was instrumental in persuading Mr. Biden to expand his “cancer moonshot” initiative to the Indo-Pacific region.
In her role as ambassador, Ms. Kennedy said, she was reluctant to make public comments against Mr. Kennedy, who launched his presidential campaign in 2023 as a primary challenger to Mr. Biden before running as an independent candidate. When Mr. Kennedy dropped his presidential bid, he endorsed Mr. Trump, who, after winning the election, named Mr. Kennedy as his choice for health secretary.
After that, she broke with her cousin, saying his views about vaccination were dangerous.
Her letter painted Mr. Kennedy as a charismatic figure, “willing to take risks and break the rules,” and able to attract others through the strength of his magnetic personality. Then she traced a tragic history of Mr. Kennedy’s influence over other family members.
“But siblings and cousins who Bobby encouraged down the path of substance abuse suffered addiction, illness and death,” she wrote, “while Bobby has gone on to misrepresent, lie and cheat his way through life.”
Mr. Kennedy’s younger brother David died in Palm Beach County in May of 1984 of “multiple ingestion” of three drugs found in his body fluids, authorities said at the time.
Other relatives have also spoken out against Mr. Kennedy, including his brother Joseph Kennedy II and his sister Kerry Kennedy, who described his comments on race and vaccines as “deplorable and untruthful.”
On Tuesday, Jack Schlossberg, Ms. Kennedy’s son, who has also been critical of Mr. Kennedy, posted a video on social media of his mother reading the letter she had written.
“I’m so proud of my courageous mother, who’s lived a life of dignity, integrity and service,” Mr. Schlossberg wrote.
Ms. Kennedy, in the letter sent Tuesday, gave her cousin credit for overcoming his drug addiction, which Mr. Kennedy has discussed extensively. By his own account, Mr. Kennedy became addicted to heroin when he was 14, in 1968, as he struggled to cope with the assassination of his father. In 1984, he pleaded guilty to a felony charge of possessing heroin, and entered treatment.
But Ms. Kennedy was harsh in criticism of her cousin’s advocacy against vaccines, describing it as part of an addiction to attention and power.
“Bobby preys on the desperation of parents of sick children — vaccinating his own children while building a following by hypocritically discouraging other parents from vaccinating theirs,” she wrote.
Ms. Kennedy also highlighted “the conspiratorial half-truths he has told about vaccines,” in connection with the 2019 measles outbreak in Samoa, which she said “cost lives.”
The letter was addressed to senators who lead the committees that will be reviewing his nomination this week, including Mike Crapo, a Republican from Idaho; Ron Wyden, a Democrat from Oregon; Bill Cassidy, a Republican of Louisiana and Bernie Sanders, an Independent of Vermont.
She noted that the family is close and that speaking out was difficult. Still, she faulted her cousin for using the family’s legacy of tragedy for political gain. Mr. Kennedy’s father, Robert F. Kennedy, was assassinated during a campaign for president in 1968. Her father and his uncle, President John F. Kennedy, was fatally shot in Dallas in 1963.
Robert F. Kennedy Jr. “distorted President Kennedy’s legacy to advance his own failed presidential campaign — and then groveled to Donald Trump for a job,” the letter said. “Bobby continues to grandstand off my father’s assassination, and that of his own father.”
She suggested that her father John F. Kennedy, her uncle Robert F. Kennedy and another uncle, the long-serving lawmaker Ted Kennedy, “would be disgusted.”
She closed the letter with a plea for the senators to reject her cousin’s nomination on behalf of the doctors, nurses, scientists and caregivers who fuel the American health care system.
“They deserve a secretary committed to advancing cutting-edge medicine to save lives, not rejecting the advances we have already made,” Ms. Kennedy wrote. “They deserve a stable, moral and ethical person at the helm of this crucial agency. They deserve better than Bobby Kennedy — and so do the rest of us.”
-
Culture1 week ago
Book Review: ‘Somewhere Toward Freedom,’ by Bennett Parten
-
News1 week ago
Judges Begin Freeing Jan. 6 Defendants After Trump’s Clemency Order
-
News4 days ago
Hamas releases four female Israeli soldiers as 200 Palestinians set free
-
Business5 days ago
Instagram and Facebook Blocked and Hid Abortion Pill Providers’ Posts
-
Politics5 days ago
Oklahoma Sen Mullin confident Hegseth will be confirmed, predicts who Democrats will try to sink next
-
World4 days ago
Israel Frees 200 Palestinian Prisoners in Second Cease-Fire Exchange
-
News1 week ago
A Heavy Favorite Emerges in the Race to Lead the Democratic Party
-
World3 days ago
How to Watch Jannik Sinner vs. Alexander Zverev Australian Open Men’s Tennis Final Online Free