The day the internet died (after deleting 11 lines of code) 

March 2016, headline news 

The Artificial Intelligence revolution begins in strategy game win
AlphaGo, Google’s DeepMind AI wins Go challenge against Lee Se-dol, 4-1 

US college student sentenced to 15 years hard labour
Otto Warmbier is found guilty of trying to steal a political poster, in Pyongyang, North Korea 

France beaten 31-21 by England at Stade de France, Paris
England claim Six Nations Rugby Championship; securing their first Grand Slam since 2003. 

President Obama and his family visited Cuba for 3 day tour
Barack Obama becomes the first US President to visit Cuba since 1928.

And the Internet was “killed”
28-year-old Azer Koçulu crippled the internet by deleting 11 lines of code 

What? You don’t remember that last one? It wasn’t in the news? Well, it happened…and in the most unexpected way. 

 

The man who killed the internet

This is the story of how 28-year-old, Azer Koçulu of Oakland, California broke the internet. He did it by simply deleting 11 lines of code called ‘left-pad’. Now this wasn’t vital programming lines on some mystic master server. Just 11 lines of code that pretty much anyone could write. All it does is add characters to the beginning of a string, or line, of text. Think of the zero at the beginning of a phone number for example. It’s a simple, single-purpose function, in fact here it is in its entirety:  The key thing is the left-pad code was open-source. As such it can be picked up and used by others to save time and simplify projects. As a result, open-source code can end up being embedded in other projects. And if that is open-source too, that could be embedded as well… 

 

What is open-source? 

This term refers to something people can modify and share due to the design being publicly accessible. It originated in the context of software development to identify a specific approach to creating computer programs. Today ‘open source’ designates a broader set of values – “the open-source way”. This implies projects, products or initiatives that embrace the concept of open exchange and collaborative participation. This paves the way for rapid prototyping, transparency, meritocracy and community-oriented development – a truly altruistic approach. 
 

But let’s get back to Azer… He had been publishing code he had written to npm. NPM is a widely used service to find and install open-source software written in JavaScript. It had become an essential tool for web-development utilised millions of times a month. It’s appeal? Simple UX and enormous library of free code amassed from the open-source community.

One of the open-source, JavaScript packages Azer had written was kik. This helped programmers set up templates for their projects. Now remember, the code that actually broke the internet was left-pad and we are talking about kik, both written by Azer. Both pieces were open-source but performed different functions and that’s when we diverge to a slightly different story. 

 

Trademark tribulations 

On the 11th of March 2016, Azer received an email from patent and trademark agent Bob Stratton. Bob was representing a little-known Canadian messaging app called Kik. Bob asked Azer to rename his kik package as the Canadian Kik (who had registered Kik as a trademark globally) was about to release its own package and as you must actively defend your trademarks or lose them, it would cause Azer a lot of problem if he didn’t rename his projects. Azer staunchly believed in the principles of the open-source creed and its extended virtues. Consequently, he was diametrically opposed to the corporate principles of trademarks and told Bob in no uncertain terms* and asked him not to contact him again. 

But Bob did contact Azer again and offered to pay Azer $30,000 for the trouble. Azer was not having it. **  Bob, having a job to do for kik had to go further.  

*The language used by Azer in the email is not suitable to be quoted here, but can be found on the internet! 

** This reply was even worse! 

 

npm betrayal 

Bob contacted npm directly (where Azer had placed kik for use) explained the situation, again citing the trademark application and potential confusion. npm’s CEO, Isaac Schuerer agreed to turn the name over to the Canadian Kik. 

“In this case, we believe that most users who would come across a kik package, would reasonably expect it to be related to kik.com,” Isaac wrote to Bob and Azer on March 18. “In this context, transferring ownership of these two package names achieves that goal.” 

Azer was not happy. He felt betrayed by npm and replied to Isaac, 

“I know (sic) you for years,” Azer replied, “and would never imagine you siding with corporate patent lawyers threatening open-source contributors.” 

In an email that was later made public, Azer wrote, “I think I have the right of deleting all my stuff”.   

So, he did. 

 

A matter of principle 

It is a common stance amongst programmers, particularly those active in the open-source community, to be critical of intellectual property law in the US, and patent/trademark holders who seek to enforce it. It is common for software companies to be hounded by lawsuits driven by patents that cover common-place technologies (displaying images for example). Many see these lawsuits as being opposed to the innovation and cooperation of open-source community. 

With that in mind you can appreciate how npm’s decision to favour Kik in this disagreement sat in countenance to the values of the community Azer felt it was intended to serve. Azer’s response showed his aggrievance as he asked for all his packages to be taken down.

 

Deleting the 11 lines of code

By March 22nd, two days after Azer’s last message to npm, JavaScript programmers started receiving an unusual error message when they tried to run their code. The error resulted in a few lines, but one stood out… 

…meaning the code called ‘left-pad” was needed for the script to run but it was no longer in the npm registry. Most programmers had never heard of left-pad (at least on the 22nd of March 2016!) but their code wouldn’t run without it.  

How could that be? Well, if you remember earlier, we said code could get embedded in other projects then that was embedded in another and so on – Like a Russian Doll – well that’s what happened. Loading your own app required other packages from npm, those required their own set of packages, and so on. That’s the reason npm was so popular, it was like a friendly librarian making sure you could find your favourite book by keeping them all in the right place – until one went missing. 

Some of the largest, most widely used npm packages were suddenly broken – React for example. This was used by Meta (then Facebook), Netflix, Uber, Airbnb, and The New York Times. In the previous month over 1 million people had downloaded React from npm but React didn’t require these 11 lines of code but used another package, and so on, until one program did use left-pad and now it had gone. Another fundamental package that used it was Babel, the JavaScript “compiler”—a tool that cleans up and updates JavaScript code to match current standards, another example where left-pad was code within code but when missing had a devastating impact. 

 

‘Un-un-publishing’ 

The effect was global – commenters on left-pad’s GitHub page were writing from every corner of the globe and ironically even the Canadian messaging app kik were running into left-pad problems! Within an hour of the first instance being noted Azer posted and article “I’ve just liberated my articles’. He explained the dispute and said he’d deleted his packages – all 273, still not realising the significance of left-pad! 

Faced with the crisis of so many applications being broken and potential loss of faith in its repository npm did the unprecedented and restored the 11 lines of code.  

“Un-un-publishing is an unprecedented action that we’re taking given the severity and widespread nature of breakage, and isn’t done lightly,” wrote Laurie Voss, npm’s CTO. “This action puts the wider interests of the community of npm users at odds with the wishes of one author; we picked the needs of the many.” 

And just like that, within 2 hours the problem was fixed. 

 

Building on borrowed land (well, 11 lines of code at least)

Seeing the impact, if only for a few hours, that these missing 11 lines of code caused highlights just how fragile modern software is developed. Web services of seemingly vital importance like Meta being dependant on obscure lines of code written by an anonymous league of programmers is not ideal but is the state in which we live.  

The resultant pointing of fingers was telling all looking to shift the blame or take some moral high ground. Where as the truth is that systems work until someone messes with the foundations. 

 

There is a popular thought that ponders why an 11-line npm package existed at all. Surely programmers should be able to write those 11 lines for themselves and there is too much of a reliance on the work of others. But we all know the adage of “Why reinvent the wheel”. So if the code exists, is tested and robust, why not use it? 

What have we learnt? 

Other than perhaps being able to answer an obscure question on Eggheads what we can take from this story. Well let’s leave out the morality of Open-source philosophy and the corporate inclination toward copyright ownership vs the protection of work by its originator and look at the facts. 

• Essential and non-essential services were impacted by the loss of 11 lines of code, FACT. 
• Open-source principles and code had allowed successful collaborative progression, FACT. 
• Applications stopped working suddenly due to the actions on one individual. Someone who was not (fully) aware of the implications of altering one small part of a much bigger system, FACT. 
• Systems were working correctly until someone changed the data assigned to one aspect. Made worse by not informing the rest of the system users, (Azer removing the rights of use), FACT. 
• Normal operation was able to be restored swiftly with clear diagnosis and a full understanding of the problem, FACT. 

 

These facts show the importance of every element of a system talking and working together to provide a cohesive whole. Changing one element, irrespective of size or importance can have a critical impact on the system as a whole. Even critical failures can be reversed if a thorough understanding of the problem can be obtained. 

If you liked this article  and the connections between code and code ownership you may also like “The President, The Biscuit and Zoho Oneauth” where we look at the connections between identity and identity ownership.