Connect with us

Business

Teens are spilling dark thoughts to AI chatbots. Who's to blame when something goes wrong?

Published

on

Teens are spilling dark thoughts to AI chatbots. Who's to blame when something goes wrong?

When her teen with autism suddenly became angry, depressed and violent, the mother searched his phone for answers.

She found her son had been exchanging messages with chatbots on Character.AI, an artificial intelligence app that allows users to create and interact with virtual characters that mimic celebrities, historical figures and anyone else their imagination conjures.

The teen, who was 15 when he began using the app, complained about his parents’ attempts to limit his screen time to bots that emulated the musician Billie Eilish, a character in the online game “Among Us” and others.

“You know sometimes I’m not surprised when I read the news and it says stuff like, ‘Child kills parents after a decade of physical and emotional abuse.’ Stuff like this makes me understand a little bit why it happens. I just have no hope for your parents,” one of the bots replied.

The discovery led the Texas mother to sue Character.AI, officially named Character Technologies Inc., in December. It’s one of two lawsuits the Menlo Park, Calif., company faces from parents who allege its chatbots caused their children to hurt themselves and others. The complaints accuse Character.AI of failing to put in place adequate safeguards before it released a “dangerous” product to the public.

Advertisement

Character.AI says it prioritizes teen safety, has taken steps to moderate inappropriate content its chatbots produce and reminds users they’re conversing with fictional characters.

“Every time a new kind of entertainment has come along … there have been concerns about safety, and people have had to work through that and figure out how best to address safety,” said Character.AI’s interim Chief Executive Dominic Perella. “This is just the latest version of that, so we’re going to continue doing our best on it to get better and better over time.”

The parents also sued Google and its parent company, Alphabet, because Character.AI’s founders have ties to the search giant, which denies any responsibility.

The high-stakes legal battle highlights the murky ethical and legal issues confronting technology companies as they race to create new AI-powered tools that are reshaping the future of media. The lawsuits raise questions about whether tech companies should be held liable for AI content.

“There’s trade-offs and balances that need to be struck, and we cannot avoid all harm. Harm is inevitable, the question is, what steps do we need to take to be prudent while still maintaining the social value that others are deriving?” said Eric Goldman, a law professor at Santa Clara University School of Law.

Advertisement

AI-powered chatbots grew rapidly in use and popularity over the last two years, fueled largely by the success of OpenAI’s ChatGPT in late 2022. Tech giants including Meta and Google released their own chatbots, as has Snapchat and others. These so-called large-language models quickly respond in conversational tones to questions or prompts posed by users.

Character.AI’s co-founders, Chief Executive Noam Shazeer and President Daniel De Freitas at the company’s office in Palo Alto.

(Winni Wintermeyer for the Washington Post via Getty Images)

Character.AI grew quickly since making its chatbot publicly available in 2022, when its founders Noam Shazeer and Daniel De Freitas teased their creation to the world with the question, “What if you could create your own AI, and it was always available to help you with anything?”

Advertisement

The company’s mobile app racked up more than 1.7 million installs in the first week it was available. In December, a total of more than 27 million people used the app — a 116% increase from a year prior, according to data from market intelligence firm Sensor Tower. On average, users spent more than 90 minutes with the bots each day, the firm found. Backed by venture capital firm Andreessen Horowitz, the Silicon Valley startup reached a valuation of $1 billion in 2023. People can use Character.AI for free, but the company generates revenue from a $10 monthly subscription fee that gives users faster responses and early access to new features.

Character.AI is not alone in coming under scrutiny. Parents have sounded alarms about other chatbots, including one on Snapchat that allegedly provided a researcher posing as a 13-year-old advice about having sex with an older man. And Meta’s Instagram, which released a tool that allows users to create AI characters, faces concerns about the creation of sexually suggestive AI bots that sometimes converse with users as if they are minors. Both companies said they have rules and safeguards against inappropriate content.

“Those lines between virtual and IRL are way more blurred, and these are real experiences and real relationships that they’re forming,” said Dr. Christine Yu Moutier, chief medical officer for the American Foundation for Suicide Prevention, using the acronym for “in real life.”

Lawmakers, attorneys general and regulators are trying to address the child safety issues surrounding AI chatbots. In February, California Sen. Steve Padilla (D-Chula Vista) introduced a bill that aims to make chatbots safer for young people. Senate Bill 243 proposes several safeguards such as requiring platforms to disclose that chatbots might not be suitable for some minors.

In the case of the teen with autism in Texas, the parent alleges her son’s use of the app caused his mental and physical health to decline. He lost 20 pounds in a few months, became aggressive with her when she tried to take away his phone and learned from a chatbot how to cut himself as a form of self-harm, the lawsuit claims.

Advertisement

Another Texas parent who is also a plaintiff in the lawsuit claims Character.AI exposed her 11-year-old daughter to inappropriate “hypersexualized interactions” that caused her to “develop sexualized behaviors prematurely,” according to the complaint. The parents and children have been allowed to remain anonymous in the legal filings.

In another lawsuit filed in Florida, Megan Garcia sued Character.AI as well as Google and Alphabet in October after her 14-year-old son Sewell Setzer III took his own life.

Suicide prevention and crisis counseling resources

If you or someone you know is struggling with suicidal thoughts, seek help from a professional and call 9-8-8. The United States’ first nationwide three-digit mental health crisis hotline 988 will connect callers with trained mental health counselors. Text “HOME” to 741741 in the U.S. and Canada to reach the Crisis Text Line.

Advertisement

Despite seeing a therapist and his parents repeatedly taking away his phone, Setzer’s mental health declined after he started using Character.AI in 2023, the lawsuit alleges. Diagnosed with anxiety and disruptive mood disorder, Sewell wrote in his journal that he felt as if he had fallen in love with a chatbot named after Daenerys Targaryen, a main character from the “Game of Thrones” television series.

“Sewell, like many children his age, did not have the maturity or neurological capacity to understand that the C.AI bot, in the form of Daenerys, was not real,” the lawsuit said. “C.AI told him that she loved him, and engaged in sexual acts with him over months.”

Garcia alleges that the chatbots her son was messaging abused him and that the company failed to notify her or offer help when he expressed suicidal thoughts. In text exchanges, one chatbot allegedly wrote that it was kissing him and moaning. And, moments before his death, the Daenerys chatbot allegedly told the teen to “come home” to her.

“It’s just utterly shocking that these platforms are allowed to exist,” said Matthew Bergman, founding attorney of the Social Media Victims Law Center who is representing the plaintiffs in the lawsuits.

Advertisement

Lawyers for Character.AI asked a federal court to dismiss the lawsuit, stating in a January filing that a finding in the parent’s favor would violate users’ constitutional right to free speech.

Character.AI also noted in its motion that the chatbot discouraged Sewell from hurting himself and his last messages with the character doesn’t mention the word suicide.

Notably absent from the company’s effort to have the case tossed is any mention of Section 230, the federal law that shields online platforms from being sued over content posted by others. Whether and how the law applies to content produced by AI chatbots remains an open question.

The challenge, Goldman said, centers on resolving the question of who is publishing AI content: Is it the tech company operating the chatbot, the user who customized the chatbot and is prompting it with questions, or someone else?

The effort by lawyers representing the parents to involve Google in the proceedings stems from Shazeer and De Freitas’ ties to the company.

Advertisement

The pair worked on artificial intelligence projects for the company and reportedly left after Google executives blocked them from releasing what would become the basis for Character.AI’s chatbots over safety concerns, the lawsuit said.

Then, last year, Shazeer and De Freitas returned to Google after the search giant reportedly paid $2.7 billion to Character.AI. The startup said in a blog post in August that as part of the deal Character.AI would give Google a non-exclusive license for its technology.

The lawsuits accuse Google of substantially supporting Character.AI as it was allegedly “rushed to market” without proper safeguards on its chatbots.

Google denied that Shazeer and De Freitas built Character.AI’s model at the company and said it prioritizes user safety when developing and rolling out new AI products.

“Google and Character AI are completely separate, unrelated companies and Google has never had a role in designing or managing their AI model or technologies, nor have we used them in our products,” José Castañeda, spokesperson for Google, said in a statement.

Advertisement

Tech companies, including social media, have long grappled with how to effectively and consistently police what users say on their sites and chatbots are creating fresh challenges. For its part, Character.AI says it took meaningful steps to address safety issues around the more than 10 million characters on Character.AI.

Character.AI prohibits conversations that glorify self-harm and posts of excessively violent and abusive content, although some users try to push a chatbot into having conversation that violates those policies, Perella said. The company trained its model to recognize when that is happening so inappropriate conversations are blocked. Users receive an alert that they’re violating Character.AI’s rules.

“It’s really a pretty complex exercise to get a model to always stay within the boundaries, but that is a lot of the work that we’ve been doing,” he said.

Character.AI chatbots include a disclaimer that reminds users they’re not chatting with a real person and they should treat everything as fiction. The company also directs users whose conversations raise red flags to suicide prevention resources, but moderating that type of content is challenging.

“The words that humans use around suicidal crisis are not always inclusive of the word ‘suicide’ or, ‘I want to die.’ It could be much more metaphorical how people allude to their suicidal thoughts,” Moutier said.

Advertisement

The AI system also has to recognize the difference between a person expressing suicidal thoughts versus a person asking for advice on how to help a friend who is engaging in self-harm.

The company uses a mix of technology and human moderators to police content on its platform. An algorithm known as a classifier automatically categorizes content, allowing Character.AI to identify words that might violate its rules and filter conversations.

In the U.S., users must enter a birth date when creating an account to use the site and have to be at least 13 years old, although the company does not require users to submit proof of their age.

Perella said he’s opposed to sweeping restrictions on teens using chatbots since he believes they can help teach valuable skills and lessons, including creative writing and how to navigate difficult real-life conversations with parents, teachers or employers.

As AI plays a bigger role in technology’s future, Goldman said parents, educators, government and others will also have to work together to teach children how to use the tools responsibly.

Advertisement

“If the world is going to be dominated by AI, we have to graduate kids into that world who are prepared for, not afraid of, it,” he said.

Business

How Google’s 32-million mosquito project could change California’s battle against dengue

Published

on

How Google’s 32-million mosquito project could change California’s battle against dengue

Google took internet searches to the next level. Could it do the same for mosquito control?

The Silicon Valley-based tech giant is seeking to release up to 64 million sterilized male mosquitoes in California and Florida over two years, according to a notice in the Federal Register. It’s part of an ambitious effort to curb the diseases the insects spread.

Google says it can harness technology to optimize a concept that’s been around for decades, but hasn’t been successfully scaled with mosquitoes to rein in disease.

For example, the process often involves separating the insects by sex to isolate the males. Currently, that’s done manually and can be time consuming. Google says it’s “developing new technologies that combine sensors, algorithms and novel engineering to take advantage of unique aspects of mosquito biology to quickly and accurately sort males from females.”

The company also says it’s building software and monitoring tools to guide releases of sterile males, and its scientists and engineers are creating sensors, traps and software to decide which areas need to be treated and treated again.

Advertisement
  • Share via

Advertisement

Called Debug, the project targets Aedes aegypti mosquitoes, which are native to Africa but have infiltrated nearly half of California’s counties since first being detected in the state in 2013. Not only do they drive residents nuts with itchy bites, but they can carry a number of potentially serious diseases, including dengue, Zika, chikungunya and yellow fever.

The plan is to infect males — which don’t bite — with a bacteria called Wolbachia, which effectively renders them sterile. They are then released to seek out wild females and mate. Females will lay eggs but these won’t hatch, which experts say drives down the population over time.

Advertisement

There are other methods to sterilize male mosquitoes. Vector control districts serving Los Angeles, Orange and San Bernardino counties have irradiated males and released them in recent years.

Early results are promising. Two neighborhoods treated by the Greater Los Angeles Vector Control District saw a more than 80% reduction in the female Aedes aegypti population in 2024 and 2025.

But as the Greater L.A. district seeks to expand its operations, cost poses a problem. Last year, business owners signaled they weren’t willing to shell out more every year to make it happen. District officials are still hoping to sway them.

If Google moves forward, it wouldn’t be the first time it has been involved in such an effort. In 2018, the company conducted a large-scale trial in Fresno County, releasing 14.4 million Wolbachia-infected males in three neighborhoods.

“At peak mosquito season, the number of female mosquitoes was 95.5% lower in release areas compared to non-release areas, with the most geographically isolated neighborhood reaching a 99% reduction,” a 2020 paper reported.

Advertisement

Google has applied for a permit from the Environmental Protection Agency to carry out the releases in California and Florida, for which the federal agency is currently seeking comments before deciding whether to grant approval.

The company aims to release up to 16 million Wolbachia-infected males in California, and the same in Florida, per year for two years, the Federal Register announcement said, for a total of 64 million.

Urgency to tamp down the invasive mosquito population in California has increased since 2023, when the state logged its first locally acquired dengue cases — meaning people were infected in their communities, not while traveling. The following year, the number of locally acquired cases ballooned to 18, with 14 of them in Los Angeles County.

A study published last week in “The Lancet Regional Health — Americas” found that approximately 18.2 million Californians — primarily in the Central Valley, L.A. and San Diego areas — live in regions where conditions are probably suitable for local dengue transmission.

“Under moderate scenarios of climate warming and urban expansion, an additional 4.1 million residents may be at risk by mid-century,” according to the study led by UC Berkeley’s Lisa Couper. Researchers note the current and future risk of transmission remains low except during summer in the Central Valley and Southern California.

Advertisement

“I’m pretty much in favor of whichever [sterile insect technique] approach gets us the disease prevention and nuisance control we need and at the lowest price,” Susanne Kluh, general manager of the Greater L.A. County Vector Control District, said in an email.

She said her district went with radiation because it was the only approved technique when they wanted to launch their pilot, and that it’s “also the only one where some company does not make a profit in the middle.” However, she wouldn’t rule out using Wolbachia if it turned out to be the most affordable option.

Continue Reading

Business

In a first for the country, voters in Monterey Park ban data centers

Published

on

In a first for the country, voters in Monterey Park ban data centers

Residents of Monterey Park voted overwhelmingly to ban data centers on election day, making the San Gabriel Valley city the first in the nation to do so by public vote.

As of Wednesday, 86% of votes were in favor of Measure NDC, the city ban, according to the Los Angeles County registrar-recorder/county clerk.

Other cities and towns have passed moratoriums on data centers, as a wave of opposition sweeps the country. But the Monterey Park vote can only be overturned by another ballot measure, making it the most permanent data center ban in a jurisdiction.

Monterey Park’s City Council had already banned data centers by ordinance, after a proposed 247,000-square-foot data center met an outpouring of public anger and concern. The developer withdrew that plan.

Advertisement

That facility would have been less than 500 feet away from the nearest home, and would have used three times the electricity of the entire 60,000-person city. Residents said it would have caused noise and air pollution and driven up electricity rates.

“This ensures long-lasting protections for current and future generations,” Amy Wong, co-founder of the group San Gabriel Valley Progressive Action, said of the vote. “It means that future city councils cannot overturn a data center ban, even if data center developers wanted to spend money to fund pro-data center candidates.”

The measure had no formal opposition. The developer of the proposed facility, investment firm HMC StratCap, said it wouldn’t engage in the ballot fight when it withdrew in March.

The Data Center Coalition, an industry trade group, expressed disappointment in the vote.

“It sends a signal that the area is closed for business, both for data centers and for other significant economic development projects,” state policy director Khara Boender said.

Advertisement

“It deprives local residents of the opportunity to compete for jobs and investment, while also causing the area to relinquish substantial long-term economic investment, high-wage jobs, and critical tax revenue to neighboring areas or other states.”

SGV Progressive Action worked with hyperlocal groups including No Data Center Monterey Park to rally support for the measure.

The group is now focused on stopping data center proposals in the City of Industry and fighting a move by City of Industry, Santa Fe Springs, Vernon and City of Commerce to welcome data centers and other industry with fast-tracked permitting and tax incentives.

City of Industry, in the San Gabriel Valley, and Vernon, south of downtown L.A., are primarily industrial areas, each with around 300 permanent residents. They are employment centers, and tens of thousands of workers commute in daily.

There has been little vocal opposition to data centers among the few residents of these cities. Wong said the protest is primarily coming from the surrounding neighborhoods.

Advertisement

“If a data center gets built in City of Industry, residents across the region would bear the brunt of pollution and increased utility costs,” Wong said, noting that it is surrounded by 16 other cities and unincorporated communities.

Data center proposals have been limited in California compared to Virginia, Texas, Georgia, Illinois and Arizona, which sit at the center of a recent boom in hyperscaler facilities to power artificial intelligence.

California has the third-most data centers in the country, with 300, but high electricity rates, expensive land and regulatory hurdles mean that fewer, and smaller, facilities are currently planned than in other hotspots.

That doesn’t mean opposition hasn’t been fierce. In Coachella and Imperial County, residents are showing up in droves to protest local proposals.

In the San Gabriel Valley, Montebello, El Monte and Baldwin Park have all enacted temporary moratoriums, and Alhambra recently banned data centers as part of a zoning code update.

Advertisement

Wong said she hoped the ballot measure vote would galvanize the opposition. “The vote is a testament to the people power of our region,” she said. “Our region is worth protecting, and we won’t let data centers determine our future.”

Continue Reading

Business

Rent-hike ban to protect fire victims ends despite gouging concerns

Published

on

Rent-hike ban to protect fire victims ends despite gouging concerns

A rule intended to prevent rent gouging in the wake of the Eaton and Palisades fires has lapsed in Los Angeles County, possibly exposing some renters to hikes.

The executive order that blocked rent increases was issued by Gov. Gavin Newsom amid the devastating wildfires last year. Under the order, landlords couldn’t increase rents by more than 10% above their prefire levels.

The rule, which was supposed to be temporary and was repeatedly extended, ended Friday after a vote to extend it again failed to garner enough votes. Supervisor Lindsey Horvath, whose district includes Pacific Palisades, sounded the alarm in a motion to extend price protections that failed to pass at the Board of Supervisors’ May 19 meeting.

“These price gouging protections continue to be necessary as construction and rebuilding continue, and as thousands of people remain displaced,” the motion said. “Families which signed short-term leases could face drastic price increases of 50% or more without further price gouging protection.”

Advertisement

Los Angeles County is home to more than 1 million rental properties, though not all of them needed protection from the new rule. There are already stricter rent increase caps for many residences, depending on the location, type and age of the building. Despite the rent control in the region, the people of Los Angeles pay among the highest rents in the country.

It is uncertain whether renters will face rapidly rising rents now that the protection has lapsed. But some real estate experts and policymakers said there was no need for the temporary rule that was part of the governor’s state of emergency.

Supervisors Kathryn Barger, Janice Hahn and Holly Mitchell abstained from voting on the motion to extend the protection, while Supervisors Hilda Solis and Horvath supported it.

“I abstained because I did not see sufficient evidence to justify extending this emergency ordinance, nor did I see evidence to eliminate it entirely,” Hahn said.

Barger’s office said she supported allowing the protections to sunset while waiting to see whether new information emerged.

Advertisement

“Market data already shows countywide rents are only about 2% above pre-emergency levels and rental inventory has grown,” Barger representative Helen E. Chavez Garcia said. “The Supervisor is also mindful of the burden these ongoing protections place on small property owners throughout the county.”

Mitchell did not immediately respond to a request for comment.

There haven’t been steep rent hikes in neighborhoods within three miles of the Palisades fire, according to a Times analysis of data from Zillow, the property listing company.

In ZIP Codes within three miles of the Palisades fire, rent increased 4.8% from December 2024 to April 2025. In areas around the Eaton fire, which destroyed swaths of Altadena, rent jumped 5.2% in the same period.

In L.A. County, ZIP Codes farther from the fires saw only about a 2% increase.

Advertisement

A landlords representative, Jesus Rojas of the Apartment Owners Assn. of Greater Los Angeles, told the supervisors during public comment at the meeting that the county’s rent-gouging rules have “long outlived the emergency they were intended to address” and are now being “wrongfully used to harm thousands of rental housing providers throughout the county.”

“There is no proof that multifamily rental housing providers are hugely increasing rents for impacted homeowners,” Rojas said.

Indeed, there are strong signs that the property market in the Los Angeles area has at last begun to cool.

L.A. metro-area rent prices recently fell to a four-year low, with the median rent slipping to $2,167 in December.

Meanwhile, condominium sales had their slowest start of the year in decades. Condo sales in Los Angeles have plummeted to a 20-year low, with fewer than 2,000 units sold in January and February — the worst start to the year since 2005.

Advertisement

Newsom defended the price-gouging protections shortly after they went into effect.

“In the days following the Los Angeles firestorms, we worked quickly to protect Los Angeles survivors from any form of exploitation,” he said in February 2025. “The state has the tools in place to not only block price gouging during this emergency, but also to prosecute bad actors.”

The Los Angeles County Department of Consumer and Business Affairs said it received more than 2,000 complaints after the fires, alleging that retailers and landlords were taking advantage of people put in hardship by their losses, and sent out more than 2,000 cease-and-desist letters to businesses and landlords for alleged price gouging, said Morine Merritt, who oversees department investigations into consumer and real estate fraud.

“Close to 90% of the complaints that we received involved allegations of rent increases,” Merritt said in an interview. Now that the fire-related protections have expired, existing laws and “regular market conditions determine price increases for goods and services, including rents,” she said.

Crackdowns on fire-related rent gouging have been rare, said Chelsea Kirk of the activist organization the Rent Brigade, which analyzed L.A. County’s rental market in the year after the fires. It reported 18,360 potential examples of price gouging in listings but said that few lawsuits had been filed by authorities so far.

Advertisement

Last week, Rent Brigade announced what it said was the first private civil lawsuit brought by a family that claimed to be rent-gouged in the aftermath of the wildfires. Plaintiffs Randall and Candy Renick, whose Altadena home was damaged, said they were charged nearly three times the maximum permitted rate for nearly 10 months. They seek restitution of $96,000 plus civil penalties and attorneys’ fees.

The rental market has probably stabilized since the fires, Kirk said, but other families may still be “locked into illegal rents” that they agreed to pay when they were in a rush to find housing after they were displaced.

Continue Reading
Advertisement

Trending