Unlock the Editor’s Digest for free

Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.

Rivals of Nvidia, which dominates the market for AI chips, have long hoped that an inflection point would help them make up lost ground.

That point may be at hand. So far, however, there is little sign of Nvidia ceding its lead — though it is still an open question as to whether the AI market will develop in ways that eventually erode its dominance.

The key issue is when the main focus in AI moves from training the large “foundation” models that underpin modern AI systems, to putting those models into widespread use in the applications used by large numbers of consumers and businesses.

With their ability to handle multiple computations in parallel, Nvidia’s powerful graphical processing units, or GPUs, have maintained their dominance of data-intensive AI training. By contrast, running queries against these AI models — known as inference — is a less demanding activity that could provide an opening for makers of less powerful — and cheaper — chips.

Anyone expecting a quick shift will have been disappointed. Nvidia’s lead in this newer market already looks formidable. Announcing its latest earnings on Thursday, it said more than 40 per cent of its data centre sales over the past 12 months were already tied to inference, accounting for more than $33bn in revenue. That is more than two and a half times the entire sales of Intel’s data centre division over the same period.

But how the inference market will develop from here is uncertain. Two questions will determine the outcome: whether the AI business continues to be dominated by a race to build ever larger AI models, and where most of the inference will take place.

Nvidia’s fortunes have been heavily tied to the race for scale. Chief executive Jensen Huang said this week that it takes “10, 20, 40 times the compute” to train each new generation of large AI models, guaranteeing huge demand for Nvidia’s forthcoming Blackwell chips. These new processors will also provide the most efficient way run inferences against these “multitrillion parameter models”, he added.

Yet it is not clear whether ever-larger models will continue to dominate the market, or whether these will eventually hit a point of diminishing returns. At the same time, smaller models that promise many of the same benefits, as well as less capable models designed for narrower tasks, are already coming into vogue. Meta, for instance, recently claimed that its new Llama 3.1 could match the performance of the advanced models such as OpenAI’s GPT-4, despite being far smaller.

Improved training techniques, often relying on larger amounts of high-quality data, have helped. Once trained, the biggest models can also be “distilled” in smaller versions. Such developments promise to bring more of the work of AI inference to smaller, or “edge”, data centres, and on to smartphones and PCs. “AI workloads will go closer to where the data is, or where the users are,” says Arun Chandrasekaran, an analyst at Gartner.

The range of competitors with an eye on this nascent market has been growing rapidly. Mobile chip company Qualcomm, for instance, has been the first to produce chips capable of powering a new class of AI-capable PCs, matching a design laid out by Microsoft — a development that throws down a direct challenge to longtime PC chip leader Intel.

The data centre market, meanwhile, has attracted a wide array of would-be competitors, from start-ups like Cerebras and Groq to tech giants like Meta and Amazon, which have developed their own inference chips.

It is inevitable that Nvidia will lose market share as AI inference moves to devices where it does not yet have a presence, and to the data centres of cloud companies that favour in-house chip designs. But to defend its turf, it is leaning heavily on the software strategy that has long acted as a moat around its hardware, with tools that make it easier for developers to put its chips to use.

This time, it is working on a wider range of enterprise software to help companies build applications that make best use of AI — something that would also guarantee demand for its chips. Nvidia disclosed this week that it expects its revenue from this software to reach an annual run-rate of $2bn by the end of this year. The figure is small for a company expected to produce total revenue of more than $100bn, but points to the increasing take-up of technologies that should increase the “stickiness” of products. The AI chip market may be entering a new phase, but Nvidia’s grip shows no signs of being loosened.

richard.waters@ft.com

Unlock the Editor’s Digest for free

Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.

The UK is seeking to accelerate returns of migrants to 11 countries including Iraq, Ethiopia and Vietnam in a bid to reduce the number of people residing in the country without the right to work or study.

The government has posted a contract opportunity worth £15mn over three years for a commercial partner to support the “reintegration” of people returning from Britain to their home countries.

The Home Office intends to deliver the support in Albania, Bangladesh, Ethiopia, Ghana, India, Iraq, Jamaica, Nigeria, Pakistan, Vietnam and Zimbabwe, according to official documents.

Activists have highlighted human rights violations in Iraq where the UK government acknowledges that returnees’ without certain documentation are at real risk of serious harm at security checkpoints when attempting to travel internally.

Prime Minister Sir Keir Starmer has pledged to “smash the gangs” that traffic asylum seekers across the Channel on small boats. 

He has also vowed to significantly accelerate the return of irregular migrants to their home countries in a bid to reduce the increasing cost of supporting asylum seekers and foreign national offenders. 

The government has identified a £6.4bn overspend in the asylum budget for this year, which it blamed for decisions to cut public spending elsewhere and raise taxes in the autumn budget.

According to a report released on Thursday by the Institute for Fiscal Studies think-tank, the previous Conservative government overspent billions each year on average on the asylum system between 2021 and 2023.

The contract released by the Home Office this month indicates where the government intends to focus its resources on migrant returns, including through bilateral agreements.

Previous administrations have struggled to significantly increase returns in part because of claims brought by migrants on human rights grounds and the large costs involved.

Home secretary Yvette Cooper announced last week that the government aimed to increase returns of migrants — which have dropped precipitously over the past decade — to levels last seen in 2018. 

She has set a target of returning 14,500 migrants over the next six months, which can include asylum seekers, foreign national offenders, and people living and working in the UK illegally.

Labour has formed a “returns unit” inside the Home Office to fast track cases of people from priority countries and hired around 300 of the planned 1,000 people to staff this unit.

One issue facing the government as it seeks to accelerate returns is that many of the people arriving in the UK to claim asylum are not from the country’s list of “safe states”.

Under international law, people seeking asylum cannot be returned to a country if it puts their safety in jeopardy. The return of people from countries not listed as safe are examined on a case-by-case basis.

The Home Office said the government was planning to deliver “a major surge in immigration enforcement and returns activity to remove people with no right to be in the UK and ensure the rules are respected and enforced”.

It added that continued international co-operation with partner nations played “a critical role”, and that it would be “working closely with a number of countries across the globe”.

Laura Barron-Lopez:

In their first joint visit to Georgia, Vice President Kamala Harris and her running mate, Governor Tim Walz, embarked on a bus tour through the Peach State. The goal, hold on to the battleground state that Biden narrowly won in 2020.

Before joining Harris in Georgia, Walz spoke to the International Association of Firefighters in Boston.

Gov. Tim Walz (D-MN), Vice Presidential Candidate: When Republicans used to talk about freedom, they meant it. They meant it. Not anymore. These guys over there, they want government to have the freedom to invade every corner of your life, from our union halls to our kids’ schools, even our doctor’s office.