A South Korean startup captures workers’ techniques to develop AI brains for robots

SEOUL, South Korea (AP) — His head, chest and hands strapped with body cameras, David Park deftly folded a banquet napkin the way he has thousands of times during his nine years at the five-star Lotte Hotel Seoul. Each of his motions is fed into a database that will one day teach a robot to do the same.

The hotel chain is one of many companies South Korean artificial-intelligence startup RLWRLD (pronounced “real world”) is working with to create an extensive library of human expertise, harvested from skilled workers across industries, to develop AI brains for robots that could be coming to industrial sites and homes.

It collects similar data from logistics workers at CJ, capturing how they grip, lift and handle goods in warehouses, and from staff at a Japanese convenience store chain Lawson, tracking how they organize food displays.

The goal is to build an AI software layer that can work in robots across a range of factories and other work sites in coming years, before potentially expanding into homes. RLWRLD’s engineers say replicating the dexterity of human hands is a key priority, reflecting their views that humanlike machines, or humanoids, will drive the field.

“I’ve been doing this about once a month,” said Park, one of about 10 members of Lotte Hotel’s food and beverages team being wired up to capture their techniques.

After folding the napkin into a tight, layered square, Park wiped wine glasses, knives and forks in a corner of a banquet hall as colleagues prepared for real services nearby. He complained lightly to an engineer that the cameras on his hands felt too tight.

South Korea focuses on physical AI

RLWRLD is among a wave of South Korean high-tech firms and manufacturers competing in the unproven yet fiercely contested global market for “physical AI.” The term refers to machines equipped with AI and sensors that can perceive, decide and act in real-world environments with some degree of autonomy, moving beyond conventional factory robots designed for repetitive tasks.

While it remains unclear whether these machines will fully meet expectations of transforming industries, they are central to South Korea’s ambitions to leverage its semiconductor and manufacturing strengths to become an AI powerhouse. The competition is tough, with U.S. tech giants like Tesla and a flood of Chinese firms pouring billions into humanoids and other AI robots.

Just as chatbots such as ChatGPT and Gemini train on vast troves of internet text, AI robots likewise require extensive data on human action to handle advanced physical tasks. South Koreans may struggle to compete in chatbots, where English language proficiency gives U.S. firms major advantages, but they see a better chance in physical AI, given their deep base of skilled workers in manufacturing and other sectors that could help train robot systems.

Robots are central to South Korea’s AI ambitions

The government last month announced a $33 million project to capture the “instinctive know-how and skills” of “master technicians” into a database for AI-powered manufacturing, hoping robots will boost productivity and offset an aging, shrinking workforce.

RLWRLD, which last week unveiled its robotics foundation model, an AI system for robots, expects industrial AI robots to be deployed at scale sometime around 2028, a timeline shared by major businesses.

Hyundai Motor plans to introduce humanoids built by its robotics unit, Boston Dynamics, at its global factories in coming years, starting with its Georgia plant in 2028. Chip giant Samsung Electronics plans to convert all manufacturing sites into “AI-driven factories” by 2030, with humanoids and task-specific robots across production lines.

“South Korea has a highly developed manufacturing sector and the focus is squarely on humanoids tailored specifically for those industries,” said Billy Choi, a professor at Korea University’s center for Human-Inspired AI Research.

South Korea’s AI push has unsettled labor groups, who fear robots could possibly take jobs and hollow out the skilled workforce long seen as the nation’s competitive edge, the very asset it’s now counting on for its AI transition.

After Hyundai’s union warned in January that robots could trigger an “employment shock,” President Lee Jae Myung issued a rare rebuke, describing AI as an unstoppable “massive cart” and calling for unionists to adapt to changes “coming faster than expected.”

“Mastery of skills is ultimately a human achievement — even if AI can replicate existing abilities, the continuous development of craft will remain fundamentally human,” said Kim Seok, policy director at the Korean Confederation of Trade Unions. He said widespread robot deployments would risk “severing the pipeline” for skilled labor and urged the government and employers to engage with workers over AI to win their buy-in and ease job concerns.

Robots are trained on human behavior

Humanoids developed by U.S. and Chinese companies have displayed impressive physical feats, even long-distance running. But Hyemin Cho, who handles business strategies at RLWRLD, said the ability to perform delicate tasks with hands will determine whether humanoids can be used in diverse industrial settings and homes.

“Capturing motion data in real-world settings is extremely important and the quality of that data matters greatly,” she said.

After converting worker footage into machine-readable data, RLWRLD’s engineers add another layer by repeating those tasks wearing cameras, VR headsets and motion-tracking gloves. That data is used to train test robots, often guided by RLWRLD “pilots” using wearable devices. The process captures fine details such as joint angles and the amount of force applied, said Song Hyun-ji of the company’s robotics team.

One of RLWRLD’s labs occupies a cluttered, 34th-floor suite at Lotte Hotel. Scratched carpets are buried under tangles of wires and computing gear. Poles fitted with infrared laser readers stand in the corners. Beneath a chandelier, a rare trace of the room’s former luxury, a wheeled robot with black, humanlike metal hands moves back and forth with a low mechanical whir.

During a recent demonstration, the robot, guided by engineers, gingerly lifted and placed cups at a minibar, at one point knocking over a dish. The company’s latest test footage shows a more advanced system: a humanoid carefully opening a box, placing a computer mouse inside, closing it and setting it on a conveyor belt.

Most robots, including Boston Dynamics’ Atlas, use task-specific hands, like two or three-fingered “grippers.” RLWRLD is among a smaller group of companies developing AI for five-fingered hands that mimic human touch.

While five-fingered designs may not always suit factory needs, they could prove crucial as robots move into homes, where closer interaction with humans will be required, said Choi, the professor.

Hospitality workers provide valuable training data for machines learning precise or nuanced tasks — skills that could also expand their use in industrial settings, Cho said.

Although current humanoids would need several hours to clean a guest room that human workers finish in about 40 minutes, Lotte Hotel hopes robots will be ready for cleaning and other behind-the-scenes tasks by 2029. It also plans robot rental services for the hospitality and other service industries, with a potential expansion to homes.

“If you look at the entire process of preparing for an event in back-of-house areas, we think humanoids might be able to take over about 30% to 40% of that workload,” Park said. “It will be difficult for them to replace the remaining 50%, 60% and 70%, which involves actual human-to-human interaction.”