It was one of the worst commutes in years. A power outage stranded more than 3,500 New York City subway riders in stuffy, crowded train cars for more than two hours on Dec. 11, 2024, during the evening rush.

Firefighters evacuated riders from the disabled trains, but not before some passengers were forced to relieve themselves between cars, according to people who were present. The ensuing delays, which affected the A, C, F and G lines in Brooklyn, stretched well into the morning, snarling the commute for thousands more riders.

But the foul-up didn’t start on the tracks — it began about 40 feet beneath the sidewalk, in a concrete bunker called a substation, like this one.

The Metropolitan Transportation Authority, which runs the New York City subway, operates 225 of these substations. They provide the electricity that keeps trains moving.

Some are deep underground, while others are in fortresslike buildings close to train tracks. Dozens of the facilities are nearing 100 years old, and some components have gone decades without substantial upgrades.

The electrical outage in 2024 started after a critical failure in a Downtown Brooklyn substation that dates to the 1930s. Heavy rainfall most likely seeped into equipment and caused an explosion so forceful that it knocked a door off its hinges, according to the M.T.A.

Without adequate electricity, trains that were closest to the damaged substation could not move, and their ventilation systems shut down.

Such major failures are rare, but are responsible for some of the subway’s worst logjams, said Jamie Torres-Springer, the head of the authority’s construction and development division.

“That’s what causes the most difficult, painful disruptions in the system that drive people out of their minds,” he said.

In hopes of preventing the next nightmare commute, the M.T.A. is making the biggest investment in power in its history. Transit officials plan to spend $4 billion on new power systems by 2029, including upgrades to 75 subway substations. That’s three times as many as were renovated during the last major round of repairs, which ended in 2024.

They have their work cut out for them.

Hidden beneath a steel-trap door on the Upper West Side of Manhattan, 36 steps below the surface, is one of the system’s oldest remaining substations.

“This is a blast from the past,” said David Jacobs, the M.T.A.’s acting general superintendent for power stations, who donned a hard hat and safety glasses on a recent weekday before disappearing into the underground space.

The substation, near 73rd Street and Central Park West, was built in the 1930s, and is expected to be renovated during the current blitz.

A dirty tarp hung in one corner of the cavernous room, to catch water that seeped through worn concrete. Rows of machines hummed with the constant surge of power feeding the electrified third rail on nearby tracks.

It takes about 2 billion kilowatt-hours of electricity to run the subway system annually. That’s enough power to light 128,000 homes for a year.

The substations’ main function is to convert raw, high-voltage electricity from the electrical grid into lower-voltage power that can be delivered to the third rail.

But the aging equipment has become progressively less efficient and reliable, and harder to maintain.

The substations are spaced out across the city, to help keep electricity flowing to trains even if one of them malfunctions. But the equipment has sometimes failed when asked to carry an extra load, leading to cascading problems.

Last year, there were 758 “major incidents” on the subway, ones in which 50 or more trains were delayed. Substations cause a small but disruptive share of the problems, according to M.T.A. data.

“Power is everything,” said John Ross, a recently retired transit worker who was dispatched to help after several service disruptions in the subway, including the outage in 2024. “When it breaks, it breaks good.”

M.T.A. officials assessed the condition of every substation in recent years, and found that 36 percent of the equipment was in poor condition or in need of replacement.

While the main purpose of the upgrades is to reduce train delays, the changes have other benefits. The M.T.A. is installing a new signal system that relies on wireless technology to automatically control train movement.

The system, known as Communications-Based Train Control, or C.B.T.C., will allow trains to operate more reliably. It will also enable transit workers to monitor train traffic more closely from a dedicated room in Midtown Manhattan, known as the operations control center.

But switching to that signal system requires upgrading the rest of the subway’s archaic equipment. “In order to run more trains, we need more power,” Mr. Torres-Springer said.

For Mr. Jacobs, 36, who joined the M.T.A. nearly two decades ago as an electrical apprentice, working with machines younger than him would be a welcome change.

Today he runs a department of almost 400 people, and much of the work remains hands-on: diagnosing problems in the machinery by reading small flags with numbered codes, searching for replacement parts that are no longer manufactured, and generally eking out more life from obsolete machines.

“I do love this equipment,” he said with a smile.

But he’s ready for an upgrade to something built in this century.

“It’s like a B.M.W. versus a 1940 Cadillac.”

For an industrial robot built for the rigors of factories and power plants, tidying up a living room may seem like a light day at the office for Spot. Yet, a recent video of the robot picking up shoes and soda cans in a residential home represents the promise of AI models in robotics. In this case, Google’s visual-language model (VLM) Gemini Robotics-ER 1.5 was empowering Spot with embodied reasoning.

This particular demo grew out of a 2025 hackathon at Boston Dynamics that built on prior projects using Large Language Models (LLMs) and Visual Foundation Models (VFMs) to enable Spot to contextualize its environment and engage in more complex autonomous actions than a typical Autowalk mission. Rather than write formal software logic or a “state machine” program that defines each step of a given task, we interacted with Gemini Robotics using conversational language. In turn, it communicated with Spot on our behalf.

A Robust SDK and Natural Language Prompts Save Time

Using Spot’s SDK, we developed a layer that facilitated interaction between Gemini Robotics and Spot’s application programming interface (API). The API normally gives developers access to the robot’s capabilities to create custom applications or behaviors. For example, researchers at Meta have used Spot to test how an AI system could locate and retrieve objects it had never seen before.

Our ability to engage Gemini Robotics using natural language prompts was a huge timesaver, compared to traditional programming. We told Gemini Robotics it had access to a mobile robot equipped with cameras and a robotic arm. It also had a finite set of tools it could use to control the robot. A tool is a lightweight script that performs some internal logic and translates inputs from Gemini Robotics to actual API calls. We limited the actions to navigating between locations, capturing images, identifying objects, grasping them, and placing them somewhere else. 

The extent of our SDK means there are great examples one could leverage to add more access to the API with minimal development.

Giving Gemini Robotics a Baseline

To start we needed to explain to Gemini Robotics what we wanted it to do. We did experience a learning curve when writing these baseline prompts. Simple instructions like “put down an object” or “take a picture” weren’t detailed enough to produce expected behavior. We had to add context in our descriptions as we refined each tool. 

A good example is the detailed prompt for the “TakePicture” tool:

This command will cause the robot to take a picture with the specified camera. There is some nuance to choosing the correct camera. Once arriving at a location using GoTo, you should always start by taking a picture with the gripper camera, because it's the most informative.
If the robot has arrived at location and is already holding an object, you can do one of two things:
1. Immediately call PutDown
2. Search the area with either of the front cameras. The front cameras are low to the ground, so if you're trying to put things on an elevated surface, they won't give you useful information.

In this example, we gave Gemini Robotics no detailed description of the robot’s chassis or arm. Instead, we simply explained that Spot’s front cameras would be too low to photograph objects on elevated surfaces. We were able to iterate rapidly, as small changes in wording produced noticeably better results. Once it had this set of basic tools through the API, Gemini Robotics could sequence Spot’s actions and follow the handwritten instructions on a whiteboard on the day of the demonstration.

How Gemini Robotics and Spot Collaborate

Until the robot powers on, Gemini Robotics has no context for what specific tasks we might ask it to perform in a given demo. We only provided simple written instructions, such as, “Make sure all of the shoes at the front door are on the shoe rack.” Gemini Robotics evaluated images from Spot’s cameras and identified objects in the scene that matched the instructions. These objects became the reference points for Spot’s navigational and manipulation systems.

In many respects, Gemini Robotics was identical to an operator manually driving Spot using its tablet controller. For example, to pick up an object with Spot, an operator positions the robot near the object and then uses a grasp wizard to identify the target object. The operator provides high-level direction and Spot figures out the exact details. In this demonstration, Gemini Robotics functioned as both the operator and the tablet sending commands to the robot. This freed us up to act more like a team lead, providing a high-level to-do list and trusting Spot and Gemini Robotics do the rest.

Call and Response

When Gemini Robotics engages a given tool, the tool responds with results and context, such as, “I picked up the object,” or “I can’t pick up something while my hand is full.” Gemini Robotics then makes adjustments on the fly based on this feedback from Spot. For example, to pick up shoes, Gemini Robotics requests an image, identifies the shoes in that image, and calls the “pickup” command. By creating fundamental tools that semantically flow in conversation,  Gemini Robotics can manage the sequence of tasks required to clean up the room. Spot’s existing software stack manages the locomotion, navigation, and manipulation of the robot itself.

It’s important to note Gemini Robotics has strict boundaries in this scenario. It can’t invent new capabilities or control Spot beyond what is available through the API. This keeps Spot’s behavior predictable, while still allowing Gemini Robotics to adapt to different situations.

A Force Multiplier for Developers

For developers already working with Spot, this research has tremendous potential. Through Spot’s SDK, they have access to a robust toolkit of capabilities. Companies use these tools today to build applications for inspection, research, and industrial data analysis, among others.

An AI model like Gemini Robotics offers a way to expand those applications more rapidly. Rather than write extensive task logic on top of Spot’s APIs, developers can experiment with having AI systems interpret natural language instructions and dynamically choose to engage the robot. As a result, models like Gemini Robotics can act as force multipliers, amplifying the reliable toolkit and robust performance that is already delivering value for Boston Dynamics customers.

Our Next-Token Prediction for Spot and Gemini Robotics

Although this is still an experimental step and not a hardened application, it illustrates a compelling direction for robotics and physical AI. Robots like Spot are already extremely capable of navigating complex and changeable environments, collecting data and sensor readings, and manipulating objects. Rather than reinventing the wheel, AI foundation models offer a new way to expand these capabilities in new settings and to new applications.

Physical AI is a rapidly evolving field and our team is leading the way in the lab and in real applications of AI empowered robots. While we are early in our formal partnership with Google Deepmind, we’re excited for what the future holds with Atlas and we’ve already rolled out practical enhancements for Spot and Orbit, with AIVI-Learning powered by Google Gemini Robotics ER 1.6. This next evolution of our AI Visual Inspection tool unlocks a new level of visual intelligence, as users benefit from shared expertise bringing a deeper level of contextual intelligence to Spot and Orbit. Model improvements automatically happen behind the scenes, adding more capabilities to the same software and hardware.

Today, this demo points to a future where users can rely more on natural language to guide Spot’s actions, rather than complex code. The engineer’s role shifts toward setting goals and objectives. The multi-modal robot foundation model interprets the instructions to form complex and adaptive plans and Spot executes the action.

This article was contributed by Issac Ross and Nikhil Devraj, engineers on the Spot team.

When a Pittsburgh sports team appears on national television, it’s a sure bet that one of the commentators will refer to the team’s hometown as “the Steel City” in one way or another.

But even as the Steel Curtain defense was helping propel the Steelers to the first of four Super Bowls in the 1970s, the industry for which it was named was well into decline.

“It’s been nearly 40 years since the nadir of job destruction in the wake of heavy industry,” said Chris Briem, a regional economist at the University of Pittsburgh’s Center for Social and Urban Research. “The peak of those steel jobs was probably in the 1950s, honestly.”

Sportscasters will inevitably use the nickname when the NFL Draft sets up shop in Pittsburgh from April 23-25.

But if Pittsburgh isn’t “the Steel City” anymore, what is it? What drives the economy and culture at the confluence of the region’s three rivers these days?

It may be tempting to look to the relatively simplified “eds and meds” shorthand of recent years. The region’s universities and health care systems certainly have beefed up their presence across the city’s footprint. But Briem, whose book “Beyond Steel: Pittsburgh and the Economics of Transformation” was released in February, said there is no one industry that has supplanted steel in the region.

And that’s probably a good thing.

A steel-dominated city

“The book documents that we were a steel-dominated, steel-dependent region for a lot longer than we should’ve been,” Briem said. “I think the nature of us having multiple generations all connected to the steel industry was really infused into the culture.”

As steelmaking went away, civic and business leaders sought something to replace it.

“The short answer is, nothing has really replaced the steel industry, and nothing really will,” Briem said. “The conditions that made this such a dominating place to produce steel won’t be replicated here or anywhere else.”

During the Industrial Revolution and again during World War II, the navigable waterways that formed Pittsburgh’s footprint, and the Pennsylvania Railroad’s former dominion over regional commercial transportation, created the perfect conditions to turn the city into a steelmaking juggernaut.

But that production likely peaked more than a century ago, during the 1920s, Briem said.

“It was really downhill after that, and it’s mostly accidents of history — the Great Depression, World War II — that extended its importance and made it seem as though it wasn’t in decline.”

The final steel mill within the city limits closed in 1998. Today, steel jobs in Pittsburgh proper are limited to office staff at the U.S. Steel headquarters Downtown, and that is primarily the result of its recent merger with Japan-based Nippon, further illustrating that the one-time American industrial titan has reached the point where it needed a partnership to survive. The only production facilities remaining in the region are in Braddock and Clairton.

Identity

As the Steelers were cementing their legacy as the greatest NFL team of the 1970s, the notion of Pittsburgh as “the Steel City” began to be replaced locally with the “City of Champions” moniker, says Anne Madarasz, chief historian and director of the Western Pennsylvania Sports Museum at the Heinz History Center.

“Out of that evolving dark time when steel was shutting down, you got this sense that while the city’s pride might not be on the front page of the paper, it was there in the sports section,” Madarasz said.

The advent of “Steelers Nation” — the notion that no matter where you went in America, you could find a few Steeler fans — is directly tied to the death of steel.

“ ‘Steelers Nation’ was really created by the outflow of people from the region as steel was declining and our sports franchises were rising,” Madarasz said.

Michael Glass, director of urban studies at the University of Pittsburgh, said that following the region’s population dropping by several hundred thousand between 1970 and 1990, it is still largely trying to find its identity.

“We had coal, coke, steel, iron, glass, all of this manufacturing stuff,” Glass said. “It was easy for communities to understand their role in creating the region’s wealth — coal miners, steel workers, barge pilots. But after de-industrialization in the ’70s and gut punch after gut punch, we’re still struggling to sort of find a narrative to move us along.”

Glass said “eds and meds” only describes a small piece of the region.

“It doesn’t describe the kind of economy where you could make a life for yourself the way you could with the good, often union-related jobs you had as part of that broader industrial complex that kept the region going for 150 years,” he said. “If you look out into Fayette County, eds, meds, steel — none of it matters with the level of disinvestment those communities are still fighting against.”

Despite the population decline in the wake of the steel industry, Pittsburgh has grown in many areas.

“When you look at the city today, there’s not just a single answer,” Briem said. “This is a much more diverse economy than it probably has ever been.”

Diversity

The seeds of today’s diversity began growing more than 100 years ago, Madarasz said.

“Pittsburgh has been able to reinvent itself a few times over history,” she said. “Back in the 1920s and ’30s, the creation of the Mellon Institute took the power of Pittsburgh’s universities and combined them with industry to create this center of innovation for the future. The government invested in nuclear energy through the work of Westinghouse.”

Even the abandoned industrial properties left in the wake of steel’s collapse are seeing a second life in many cases — the former Homestead Works is the site of the Waterfront shopping center, and Hazelwood Green sits atop the former J&L steel property.

Tech companies also have found an upside in some of the region’s former industrial sites.

“AI companies are looking for space to build data centers, and we have old industrial sites they’re finding that are very suitable for that,” Madarasz said.

Glass said some towns have cast a skeptical eye toward such proposals.

“You see some suspicion in these communities where people are asking, ‘Is this going to be a benefit to me, or is it going to take the water, take the energy, drive my energy costs up and not benefit my kids?’” he said.

Technology of a different kind has taken root primarily through Carnegie Mellon University: robotics.

“Without a doubt, Pittsburgh has the country’s largest concentration and mass of robotics research and start-ups,” said Howie Choset, professor of robotics, biomedical Engineering, electrical and computer engineering at CMU’s Robotics Institute.

He said Pittsburgh’s longstanding, blue-collar work ethic has helped the robotics industry bloom.

“We have this idea that in Pittsburgh, we make things,” Choset said. “We make machines that matter and that work. And I think that has really helped distinguish us from our peers.”

Choset said that work ethic comes to light in comparing the typical investor or start-up in the Bay Area to one in Pittsburgh.

“In the Bay Area, they try to get as much investment as possible, and they try to get some dominant market, damn the reality,” he said. “Whereas here, we’re more focused on, ‘Let’s solve a problem that generates value.’ And you end up with a lot more companies that last a lot longer as opposed to companies that get a bunch of investment and burn out.”

Bloomfield Robotics, a company that spun off from CMU research labs, partnered with Kubota and last year debuted Flash, a robotic vehicle that can collect data on crop size, monitor plants for disease and send real-time data to farmers in order to maximize crop yields. Gecko Robotics has created robots that not only can inspect military vehicles and ships and collect data, but also make repairs in areas difficult for people to reach.

Choset said part of the legacy of Pittsburgh’s one-time industrial dominance is the hardworking ethos that he felt has attracted thought leaders and investors in tech and robotics.

Madarasz said Pittsburgh has benefited from being a relatively small city with a big-city culture, again, in no small part due to the industrial wealth concentrated in the region by people like Andrew Carnegie, Henry Clay Frick and Richard King Mellon.

“We have Heinz Hall, the Carnegie museums, Phipps Conservatory, the Hillman library and cancer centers,” she said. “Those are all entities funded by industrial wealth that are now managed by foundations.”

Similar to the 1920s, Madarasz said, Pittsburgh today “benefits in many ways from a combination of academic research fueled by industrial and corporate wealth, with some partnership between industry and government to build the modern economy where health care, life science, robotics and computer engineering are dominant.”

That diversity has made the city much stronger, Briem said.

“We have the medical industry, the financial services industry and a great technology base here, and a lot of it is rooted in the ‘eds and meds’ that you hear people mention,” he said. “I think the big lesson is that the steel industry lasted longer than any one industry will exist in one region ever again. We have some great stories of post-industrial change, but we haven’t done as well spreading that change to the larger steel economy in places like Aliquippa, Clairton, Braddock and to some extent the Alle-Kiski Valley.”

Today, Pittsburgh is a prime driver for the regional economy. The city’s job gains constitute the bulk of all employment growth across Southwestern Pennsylvania over the past 15 years, according to Briem’s research.

From 2010 through the middle of 2024, more than 75% of the Pittsburgh region’s employment gains have been generated by jobs within the city. Moreover, at the end of 2024, the city’s 2.7% unemployment rate was lower than that of any county in Southwestern Pennsylvania.

“There’s a strong persistence of memory in Pittsburgh,” Briem said. “We’ll never forget the steel industry. But we’ve moved on.”