Today, San Francisco’s Golden Gate Bridge may be the world’s most photographed. Upon its completion, it became Earth’s longest suspension bridge and the Bay Area’s most famous attraction.

But in the early 20th century, it was just an impossible dream—and when construction workers broke ground on January 5, 1933, work started inauspiciously as they began moving three million cubic feet of dirt.

The idea for a bridge across the Golden Gate Strait, where the Pacific Ocean flows into the bay in Northern California, was first floated in 1872 by railroad mogul Charles Crocker. But most dismissed Crocker’s idea. A bridge stretching almost two miles across open ocean? Unfeasible.

Nearly five decades later, in 1916, San Francisco engineer James H. Wilkins re-proposed the bridge, and by 1919, officials tasked city engineer Michael M. O’Shaughnessy with exploring the idea. When O’Shaughnessy consulted with engineers from across the country, most estimated such a project would cost more than $100 million, if it could be done at all.

One engineer, though, believed in the project from the start: Joseph B. Strauss, who told O’Shaughnessy it could be built for around $27 million.

Strauss’ original design was a dud, so he recruited other players who steered the project onto a successful course. Charles Ellis, an Illinois engineer, and Leon Moisseiff, designer of New York City’s Manhattan Bridge, drew up a new, $35 million plan. Architect Irving Morrow contributed the Gate’s famous aesthetics, like its Art Deco lines, dramatic lighting and iconic reddish color—called “industry orange.”

Construction began in January 1933. In 1934, the north tower was raised, and in 1935, the south pier. By 1936, workers had built a precarious catwalk between them so they could build suspension cables in situ.

Facing Pacific winds atop the towers, workers insulated their jackets with crumpled newspaper. “You put all the clothes on you had and worked, worked hard, or you’d freeze,” worker Martin Adams told KQED. He called the Golden Gate Strait “the coldest place I’ve ever worked.”

Still, it was the 1930s—the middle of the Great Depression—and people were desperate for work. Hopeful men lined up, waiting for construction jobs that would open when laborers inevitably died on the job.

Loss of life was expected with big projects like this one, but Strauss took a special interest in protecting the bridge’s builders. Workers wore special hard hats and glare-free goggles, and Strauss insisted on an unheard-of construction feature: a $130,000 safety net. It ended up catching 19 men, who called themselves the “Halfway to Hell Club.” But it didn’t catch all who fell.

On February 17, 1936, construction workers were tasked with removing wooden scaffolding, working from a temporary catwalk. Adams watched as the catwalk broke away, ripped through the safety net and fell into the ocean, taking 12 men with it—220 feet down.

“The only thing that went through my mind was survival,” said Slim Lambert, one of the falling men. “I knew that to have a prayer, I had to hit the water feet first.”

When Lambert plunged into the Pacific, his legs became tangled in the sinking net. He was pulled so deep that his ears bled before he untangled himself and swam to the surface. He and two others were plucked from the waves by a crab fisherman, but only Lambert and colleague Oscar Osberg survived.

Construction continued. By May 1936, the cable compression was finished, In November, two main span sections were joined, marked by a blessing with holy water. In the first half of 1937, the roadway was paved.

Finally, on the morning of May 27, 1937, 18,000 people gathered on each side of the finished Golden Gate Bridge as it opened to pedestrians. San Franciscans had fun with it, marking historic firsts: The San Francisco Chronicle recorded the first person to walk across the bridge on stilts, pushing a stroller, on roller skates, on a unicycle and while playing a tuba. A week’s worth of celebrations became known as the Golden Gate Bridge Fiesta.

The bridge has since become a symbol of architectural ingenuity and Bay Area style. After all, its construction was championed by citizens who voted to spend a fortune building a structure once deemed impossible in a time of economic strife.