Bug causes widespread internet outages in North Dakota

BISMARCK — Around 60% of North Dakota residents and parts of state government experienced internet outages on Thursday, Aug. 17.

The outages impacted internet providers that are part of the Dakota Carrier Network, a Bismarck-based communications network company.

A software bug on DCN’s core routers was to blame for the outages, according to CEO Seth Arndorfer. The bug was not malicious.

“If you think of a flood of traffic to a port, an internet port, it shuts the port down, and so that’s what ended up happening out of this bug,” he said.

DCN received the first indications of an issue Thursday morning, and the company was getting reports of widespread outages by early afternoon.

Most traffic was back up after three hours. All services were restored just before 7 p.m., according to Arndorfer.

He said it was difficult to solve the issue since it was not due to human error. The network had to enlist the help of its software manufacturer to fix the problem.

Arndorfer said the company is working with its software manufacturer to identify and patch any further vulnerabilities in its programs to ensure this does not happen again.

This was the first time an outage of this size and scale has happened to DCN, according to Arndorfer. Over 400,000 residents were estimated to have been affected.

State Information Technology representatives sent out a notice regarding connectivity issues on Thursday. The agency on Friday did not immediately have an update on the extent of the state government problems.

“There’s so much of the economy and our lives that depend on the internet. … It’s just like the kitchen faucet: we just expect it to be on and it typically is, but this is one of those rare instances where when it’s down we all feel the pain,” Arndorfer said.

DCN received a $19.7 million grant from the National Telecommunications and Information Administration in June as part of its $43.8 million Middle Mile Backbone project to fill broadband gaps in North Dakota.

The outages were not connected to the expansion program, which is yet to begin, Arndorfer said.

“DCN understands the impact and the inconvenience that this created for all of our customers and stakeholders, and we apologize for the inconvenience,” he said.