Connect with us

Science

A Stroke Stole Her Ability to Speak at 30. A.I. Is Helping to Restore It Years Later.

Published

on

A Stroke Stole Her Ability to Speak at 30. A.I. Is Helping to Restore It Years Later.

At Ann Johnson’s wedding reception 20 years ago, her gift for speech was vividly evident. In an ebullient 15-minute toast, she joked that she had run down the aisle, wondered if the ceremony program should have said “flutist” or “flautist” and acknowledged that she was “hogging the mic.”

Just two years later, Mrs. Johnson — then a 30-year-old teacher, volleyball coach and mother of an infant — had a cataclysmic stroke that paralyzed her and left her unable to talk.

On Wednesday, scientists reported a remarkable advance toward helping her, and other patients, speak again. In a milestone of neuroscience and artificial intelligence, implanted electrodes decoded Mrs. Johnson’s brain signals as she silently tried to say sentences. Technology converted her brain signals into written and vocalized language, and enabled an avatar on a computer screen to speak the words and display smiles, pursed lips and other expressions.

The research, published in the journal Nature, demonstrates the first time spoken words and facial expressions have been directly synthesized from brain signals, experts say. Mrs. Johnson chose the avatar, a face resembling hers, and researchers used her wedding toast to develop the avatar’s voice.

“We’re just trying to restore who people are,” said the team’s leader, Dr. Edward Chang, the chairman of neurological surgery at the University of California, San Francisco.

Advertisement

“It let me feel like I was a whole person again,” Mrs. Johnson, now 48, wrote to me.

The goal is to help people who cannot speak because of strokes or conditions like cerebral palsy and amyotrophic lateral sclerosis. To work, Mrs. Johnson’s implant must be connected by cable from her head to a computer, but her team and others are developing wireless versions. Eventually, researchers hope, people who have lost speech may converse in real time through computerized pictures of themselves that convey tone, inflection and emotions like joy and anger.

“What’s quite exciting is that just from the surface of the brain, the investigators were able to get out pretty good information about these different features of communication,” said Dr. Parag Patil, a neurosurgeon and biomedical engineer at the University of Michigan, who was asked by Nature to review the study before publication.

Mrs. Johnson’s experience reflects the field’s fast-paced progress. Just two years ago, the same team published research in which a paralyzed man, who went by the nickname Pancho, used a simpler implant and algorithm to produce 50 basic words like “hello” and “hungry” that were displayed as text on a computer after he tried to say them.

Mrs. Johnson’s implant has nearly twice as many electrodes, increasing its ability to detect brain signals from speech-related sensory and motor processes linked to the mouth, lips, jaw, tongue and larynx. Researchers trained the sophisticated artificial intelligence to recognize not individual words, but phonemes, or sound units like “ow” and “ah” that can ultimately form any word.

Advertisement

“It’s like an alphabet of speech sounds,” David Moses, the project manager, said.

While Pancho’s system produced 15 to 18 words per minute, Mrs. Johnson’s rate was 78 using a much larger vocabulary list. Typical conversational speech is about 160 words per minute.

When researchers began working with her, they didn’t expect to try the avatar or audio. But the promising results were “a huge green light to say, ‘OK, let’s try the harder stuff, let’s just go for it,’” Dr. Moses said.

They programmed an algorithm to decode brain activity into audio waveforms, producing vocalized speech, said Kaylo Littlejohn, a graduate student at the University of California, Berkeley, and one of the study’s lead authors, along with Dr. Moses, Sean Metzger, Alex Silva and Margaret Seaton.

“Speech has a lot of information that is not well preserved by just text, like intonation, pitch, expression,” Mr. Littlejohn said.

Advertisement

Working with a company that produces facial animation, researchers programmed the avatar with data on muscle movements. Mrs. Johnson then tried to make facial expressions for happy, sad and surprised, each at high, medium and low intensity. She also tried to make various jaw, tongue and lip movements. Her decoded brain signals were conveyed on the avatar’s face.

Through the avatar, she said, “I think you are wonderful” and “What do you think of my artificial voice?”

“Hearing a voice similar to your own is emotional,” Mrs. Johnson told the researchers.

She and her husband, William, a postal worker, even engaged in conversation. She said through the avatar: “Do not make me laugh.” He asked how she was feeling about the Toronto Blue Jays’ chances. “Anything is possible,” she replied.

The field is moving so quickly that experts believe federally approved wireless versions might be available within the next decade. Different methods might be optimal for certain patients.

Advertisement

On Wednesday, Nature also published another team’s study involving electrodes implanted deeper in the brain, detecting activity of individual neurons, said Dr. Jaimie Henderson, a professor of neurosurgery at Stanford and the team’s leader, who was motivated by his childhood experience of watching his father lose speech after an accident. He said their method might be more precise but less stable because specific neurons’ firing patterns can shift.

Their system decoded sentences at 62 words per minute that the participant, Pat Bennett, 68, who has A.L.S., tried to say from a large vocabulary. That study didn’t include an avatar or sound decoding.

Both studies used predictive language models to help guess words in sentences. The systems don’t just match words but are “figuring out new language patterns” as they improve their recognition of participants’ neural activity, said Melanie Fried-Oken, an expert in speech-language assistive technology at Oregon Health & Science University, who consulted on the Stanford study.

Neither approach was completely accurate. When using large vocabulary sets, they incorrectly decoded individual words about a quarter of the time.

For example, when Mrs. Johnson tried to say, “Maybe we lost them,” the system decoded, “Maybe we that name.” But in nearly half of her sentences, it correctly deciphered every word.

Advertisement

Researchers found that people on a crowdsourcing platform could correctly interpret the avatar’s facial expressions most of the time. Interpreting what the voice said was harder, so the team is developing a prediction algorithm to improve that. “Our speaking avatar is just at the starting point,” Dr. Chang said.

Experts emphasize that these systems aren’t reading people’s minds or thoughts. Rather, Dr. Patil said, they resemble baseball batters who “are not reading the mind of the pitcher but are kind of interpreting what they see the pitcher doing” to predict pitches.

Still, mind reading may ultimately be possible, raising ethical and privacy issues, Dr. Fried-Oken said.

Mrs. Johnson contacted Dr. Chang in 2021, the day after her husband showed her my article about Pancho, the paralyzed man the researchers had helped. Dr. Chang said he initially discouraged her because she lived in Saskatchewan, Canada, far from his lab in San Francisco, but “she was persistent.”

Mr. Johnson, 48, arranged to work part time. “Ann’s always supported me to do what I’ve wanted,” including leading his postal union local, he said. “So I just thought it was important to be able to support her in this.”

Advertisement

She started participating last September. Traveling to California takes them three days in a van packed with equipment, including a lift to transfer her between wheelchair and bed. They rent an apartment there, where researchers conduct their experiments to make it easier for her. The Johnsons, who raise money online and in their community to pay for travel and rent for the multiyear study, spend weeks in California, returning home between research phases.

“If she could have done it for 10 hours a day, seven days a week, she would have,” Mr. Johnson said.

Determination has always been part of her nature. When they began dating, Mrs. Johnson gave Mr. Johnson 18 months to propose, which he said he did “on the exact day of the 18th month,” after she had “already gone and picked out her engagement ring.”

Mrs. Johnson communicated with me in emails composed with the more rudimentary assistive system she uses at home. She wears eyeglasses affixed with a reflective dot that she aims at letters and words on a computer screen.

It’s slow, allowing her to generate only 14 words per minute. But it’s faster than the only other way she can communicate at home: using a plastic letter board, a method Mr. Johnson described as “her just trying to show me which letter she’s trying to try to look at and then me trying to figure out what she’s trying to say.”

Advertisement

The inability to have free-flowing conversations frustrates them. When discussing detailed matters, Mr. Johnson sometimes says something and receives her response by email the next day.

“Ann’s always been a big talker in life, an outgoing, social individual who loves talking, and I don’t,” he said, but her stroke “made the roles reverse, and now I’m supposed to be the talker.”

Mrs. Johnson was teaching high school math, health and physical education, and coaching volleyball and basketball when she had her brainstem stroke while warming up to play volleyball. After a year in a hospital and a rehabilitation facility, she came home to her 10-year-old stepson and her 23-month-old daughter, who has now grown up without any memory of hearing her mother speak, Mr. Johnson said.

“Not being able to hug and kiss my children hurt so bad, but it was my reality,” Mrs. Johnson wrote. “The real nail in the coffin was being told I couldn’t have more children.”

For five years after the stroke, she was terrified. “I thought I would die at any moment,” she wrote, adding, “The part of my brain that wasn’t frozen knew I needed help, but how would I communicate?”

Advertisement

Gradually, her doggedness resurfaced. Initially, “my face muscles didn’t work at all,” she wrote, but after about five years, she could smile at will.

She was entirely tube-fed for about a decade, but decided she wanted to taste solid food. “If I die, so be it,” she told herself. “I started sucking on chocolate.” She took swallowing therapy and now eats finely chopped or soft foods. “My daughter and I love cupcakes,” she wrote.

When Mrs. Johnson learned that trauma counselors were needed after a fatal bus crash in Saskatchewan in 2018, she decided to take a university counseling course online.

“I had minimal computer skills and, being a math and science person, the thought of writing papers scared me,” she wrote in a class report. “At the same time, my daughter was in grade 9 and being diagnosed with a processing disability. I decided to push through my fears and show her that disabilities don’t need to stop us or slow us down.”

Helping trauma survivors remains her goal. “My shot at the moon was that I would become a counselor and use this technology to talk to my clients,” she told Dr. Chang’s team.

Advertisement

At first when she started making emotional expressions with the avatar, “I felt silly, but I like feeling like I have an expressive face again,” she wrote, adding that the exercises also enabled her to move the left side of her forehead for the first time.

She has gained something else, too. After the stroke, “it hurt so bad when I lost everything,” she wrote. “I told myself that I was never again going to put myself in line for that disappointment again.”

Now, “I feel like I have a job again,” she wrote.

Besides, the technology makes her imagine being in “Star Wars”: “I have kind of gotten used to having my mind blown.”

Advertisement

Science

Cluster of farmworkers diagnosed with rare animal-borne disease in Ventura County

Published

on

Cluster of farmworkers diagnosed with rare animal-borne disease in Ventura County

A cluster of workers at Ventura County berry farms have been diagnosed with a rare disease often transmitted through sick animals’ urine, according to a public health advisory distributed to local doctors by county health officials Tuesday.

The bacterial infection, leptospirosis, has resulted in severe symptoms for some workers, including meningitis, an inflammation of the brain lining and spinal cord. Symptoms for mild cases included headaches and fevers.

The disease, which can be fatal, rarely spreads from human to human, according to the U.S. Centers for Disease Control and Prevention.

Ventura County Public Health has not given an official case count but said it had not identified any cases outside of the agriculture sector. The county’s agriculture commissioner was aware of 18 cases, the Ventura County Star reported.

Advertisement

The health department said it was first contacted by a local physician in October, who reported an unusual trend in symptoms among hospital patients.

After launching an investigation, the department identified leptospirosis as a probable cause of the illness and found most patients worked on caneberry farms that utilize hoop houses — greenhouse structures to shelter the crops.

As the investigation to identify any additional cases and the exact sources of exposure continues, Ventura County Public Health has asked healthcare providers to consider a leptospirosis diagnosis for sick agricultural workers, particularly berry harvesters.

Rodents are a common source and transmitter of disease, though other mammals — including livestock, cats and dogs — can transmit it as well.

The disease is spread through bodily fluids, such as urine, and is often contracted through cuts and abrasions that contact contaminated water and soil, where the bacteria can survive for months.

Advertisement

Humans can also contract the illness through contaminated food; however, the county health agency has found no known health risks to the general public, including through the contact or consumption of caneberries such as raspberries and blackberries.

Symptom onset typically occurs between two and 30 days after exposure, and symptoms can last for months if untreated, according to the CDC.

The illness often begins with mild symptoms, with fevers, chills, vomiting and headaches. Some cases can then enter a second, more severe phase that can result in kidney or liver failure.

Ventura County Public Health recommends agriculture and berry harvesters regularly rinse any cuts with soap and water and cover them with bandages. They also recommend wearing waterproof clothing and protection while working outdoors, including gloves and long-sleeve shirts and pants.

While there is no evidence of spread to the larger community, according to the department, residents should wash hands frequently and work to control rodents around their property if possible.

Advertisement

Pet owners can consult a veterinarian about leptospirosis vaccinations and should keep pets away from ponds, lakes and other natural bodies of water.

Continue Reading

Science

Political stress: Can you stay engaged without sacrificing your mental health?

Published

on

Political stress: Can you stay engaged without sacrificing your mental health?

It’s been two weeks since Donald Trump won the presidential election, but Stacey Lamirand’s brain hasn’t stopped churning.

“I still think about the election all the time,” said the 60-year-old Bay Area resident, who wanted a Kamala Harris victory so badly that she flew to Pennsylvania and knocked on voters’ doors in the final days of the campaign. “I honestly don’t know what to do about that.”

Neither do the psychologists and political scientists who have been tracking the country’s slide toward toxic levels of partisanship.

Fully 69% of U.S. adults found the presidential election a significant source of stress in their lives, the American Psychological Assn. said in its latest Stress in America report.

The distress was present across the political spectrum, with 80% of Republicans, 79% of Democrats and 73% of independents surveyed saying they were stressed about the country’s future.

Advertisement

That’s unhealthy for the body politic — and for voters themselves. Stress can cause muscle tension, headaches, sleep problems and loss of appetite. Chronic stress can inflict more serious damage to the immune system and make people more vulnerable to heart attacks, strokes, diabetes, infertility, clinical anxiety, depression and other ailments.

In most circumstances, the sound medical advice is to disengage from the source of stress, therapists said. But when stress is coming from politics, that prescription pits the health of the individual against the health of the nation.

“I’m worried about people totally withdrawing from politics because it’s unpleasant,” said Aaron Weinschenk, a political scientist at the University of Wisconsin–Green Bay who studies political behavior and elections. “We don’t want them to do that. But we also don’t want them to feel sick.”

Modern life is full of stressors of all kinds: paying bills, pleasing difficult bosses, getting along with frenemies, caring for children or aging parents (or both).

The stress that stems from politics isn’t fundamentally different from other kinds of stress. What’s unique about it is the way it encompasses and enhances other sources of stress, said Brett Ford, a social psychologist at the University of Toronto who studies the link between emotions and political engagement.

Advertisement

For instance, she said, elections have the potential to make everyday stressors like money and health concerns more difficult to manage as candidates debate policies that could raise the price of gas or cut off access to certain kinds of medical care.

Layered on top of that is the fact that political disagreements have morphed into moral conflicts that are perceived as pitting good against evil.

“When someone comes into power who is not on the same page as you morally, that can hit very deeply,” Ford said.

Partisanship and polarization have raised the stakes as well. Voters who feel a strong connection to a political party become more invested in its success. That can make a loss at the ballot box feel like a personal defeat, she said.

There’s also the fact that we have limited control over the outcome of an election. A patient with heart disease can improve their prognosis by taking medicine, changing their diet, getting more exercise or quitting smoking. But a person with political stress is largely at the mercy of others.

Advertisement

“Politics is many forms of stress all rolled into one,” Ford said.

Weinschenk observed this firsthand the day after the election.

“I could feel it when I went into my classroom,” said the professor, whose research has found that people with political anxiety aren’t necessarily anxious in general. “I have a student who’s transgender and a couple of students who are gay. Their emotional state was so closed down.”

That’s almost to be expected in a place like Wisconsin, whose swing-state status caused residents to be bombarded with political messages. The more campaign ads a person is exposed to, the greater the risk of being diagnosed with anxiety, depression or another psychological ailment, according to a 2022 study in the journal PLOS One.

Political messages seem designed to keep voters “emotionally on edge,” said Vaile Wright, a licensed psychologist in Villa Park, Ill., and a member of the APA’s Stress in America team.

Advertisement

“It encourages emotion to drive our decision-making behavior, as opposed to logic,” Wright said. “When we’re really emotionally stimulated, it makes it so much more challenging to have civil conversation. For politicians, I think that’s powerful, because emotions can be very easily manipulated.”

Making voters feel anxious is a tried-and-true way to grab their attention, said Christopher Ojeda, a political scientist at UC Merced who studies mental health and politics.

“Feelings of anxiety can be mobilizing, definitely,” he said. “That’s why politicians make fear appeals — they want people to get engaged.”

On the other hand, “feelings of depression are demobilizing and take you out of the political system,” said Ojeda, author of “The Sad Citizen: How Politics is Depressing and Why it Matters.”

“What [these feelings] can tell you is, ‘Things aren’t going the way I want them to. Maybe I need to step back,’” he said.

Advertisement

Genessa Krasnow has been seeing a lot of that since the election.

The Seattle entrepreneur, who also campaigned for Harris, said it grates on her to see people laughing in restaurants “as if nothing had happened.” At a recent book club meeting, her fellow group members were willing to let her vent about politics for five minutes, but they weren’t interested in discussing ways they could counteract the incoming president.

“They’re in a state of disengagement,” said Krasnow, who is 56. She, meanwhile, is looking for new ways to reach young voters.

“I am exhausted. I am so sad,” she said. “But I don’t believe that disengaging is the answer.”

That’s the fundamental trade-off, Ojeda said, and there’s no one-size-fits-all solution.

Advertisement

“Everyone has to make a decision about how much engagement they can tolerate without undermining their psychological well-being,” he said.

Lamirand took steps to protect her mental health by cutting social media ties with people whose values aren’t aligned with hers. But she will remain politically active and expects to volunteer for phone-banking duty soon.

“Doing something is the only thing that allows me to feel better,” Lamirand said. “It allows me to feel some level of control.”

Ideally, Ford said, people would not have to choose between being politically active and preserving their mental health. She is investigating ways to help people feel hopeful, inspired and compassionate about political challenges, since these emotions can motivate action without triggering stress and anxiety.

“We want to counteract this pattern where the more involved you are, the worse you are,” Ford said.

Advertisement

The benefits would be felt across the political spectrum. In the APA survey, similar shares of Democrats, Republicans and independents agreed with statements like, “It causes me stress that politicians aren’t talking about the things that are most important to me,” and, “The political climate has caused strain between my family members and me.”

“Both sides are very invested in this country, and that is a good thing,” Wright said. “Antipathy and hopelessness really doesn’t serve us in the long run.”

Continue Reading

Science

Video: SpaceX Unable to Recover Booster Stage During Sixth Test Flight

Published

on

Video: SpaceX Unable to Recover Booster Stage During Sixth Test Flight

President-elect Donald Trump joined Elon Musk in Texas and watched the launch from a nearby location on Tuesday. While the Starship’s giant booster stage was unable to repeat a “chopsticks” landing, the vehicle’s upper stage successfully splashed down in the Indian Ocean.

Continue Reading
Advertisement

Trending