Infrastructure—we should all own it

Here are three disturbing ideas:

• There is more infrastructure in the world than you think;
• The infrastructure you rely on is more vulnerable than you think;
• The people in charge of managing that infrastructure either don’t realise that it’s infrastructure, or don’t care.

What do I mean by ‘infrastructure’? The Oxford English Dictionary defines it as:

infrastructure (noun) The basic physical and organizational structures and facilities (e.g. buildings, roads, power supplies) needed for the operation of a society or enterprise.

Traditionally this is applied to the physical stuff that we rely on every day. Water. Electricity. Roads. And most of the time, it’s invisible to us: if we notice that infrastructure is there, it’s because something has gone wrong. Low water pressure? Dangerous road junction? Long wait for a train? We don’t typically talk about infrastructure unless it’s to complain—planning, building, and maintaining it is often a thankless task.

We do have things in the tech industry that we understand as infrastructure. Plenty of development teams maintain their own continuous integration rig, for example. When people ship their service, and let paying customers use it, (one hopes) they have some kind of operations and production support system, and service quality monitoring.

What’s often missing, though, is an understanding of how our internal infrastructure dovetails with public infrastructure: stuff that’s outside our control, that we often don’t pay for at the point of use, but that we nonetheless rely on to do business. And the fact we don’t seem to acknowledge our reliance on this stuff is a problem—we depend on it, but we don’t care about it, until it breaks.

Something to rely on

Take, for instance, npm. Npm is the default package manager for Node.js; while there are open-source alternatives to the client (e.g. yarn, which Facebook open-sourced in 2016) the most important part of npm is the package registry. This is owned by npm Inc., a Silicon Valley startup. Most JavaScript developers simply run npm install on a regular basis, installing hundreds of dependencies without thinking about it. That’s fine—it’s perfectly natural to want to use npm as if it was a public resource, like the air we breathe or the water we drink (we’ll come to that later.)

Meanwhile, not everything has been rosy at npm Inc. In 2018, they hired a new CEO, who, according to The Register, was brought in to take the company from $3m annual revenue to 10-20× that amount. C J Silverio, the company’s CTO, was fired by text, amongst others who were laid off suddenly after a dramatic culture change. C J spoke about her experience recently at JSConf EU: her talk, The Economics of Open Source, is excellent and I implore you to watch it.

The key thing that sticks out here is this: npm Inc. is now a company backed by venture capitalists, whose modus operandi is to fund startups, and expect them to go big or go home. This is how VC works: they want hypergrowth and an enormous return on investment, or they want the company to go bust quickly so they can move onto other endeavours. (Npm’s new CEO resigned in September.)

“Build it, and they will come…” whether you like it or not

Npm is one example of a single point of failure, and it is no mystery as to why C J Silverio and others are now working on Entropic, a federated alternative. But it’s worth looking at how we got here, to a point where the de facto JavaScript language commons are in the control of a private entity.

The biggest reason is probably convenience. It is as close to frictionless as it can be to import other people’s work and incorporate it into your own. The result is that people, naturally, use it: a fresh project created by the Angular CLI, using the default settings, downloads no less than 1,460 modules using npm. A cursive search of GitHub suggests there are in excess of 111 million projects containing at least one file called package.json.

Npm makes it very easy to depend on packages maintained by others. This is an example of a phenomenon known as the Jevons paradox—namely, where the efficiency of using a resource increases (via technical progress or by policy), demand increases to a point where overall consumption is higher than it originally was.

This is easily understood using the maxim, “build it, and they will come,” which is often applied to public infrastructure. For instance, new and upgraded transport infrastructure (railways, cycleways, etc.) is, when well-designed and convenient to use, often a victim of its own success.

It’s easy to imagine that this means people behave like rainwater, and any usage will expand to fill the available space—but this isn’t quite the case. In urbanist circles, the phenomenon is known as induced demand. Many policy-makers now understand that this is why, for instance, widening a motorway will often make congestion worse. (Long story short: extra lanes make driving faster and more convenient; more cars on the road; more congestion.)

What does this mean from a technical perspective? Well—no matter how much you scale, you can be pretty certain there will be someone willing to take that capacity. This is why rate limiting exists.

But if we start treating public APIs like infrastructure, the ultimate lesson is that people start to rely on them. This makes changing them, and deprecating them, hard—because it has consequences.

A case for public (not government) ownership

Here is the inherent problem with npm’s predicament: its status as, in C J Silverio’s words, “a financial instrument intended to turn money into more money for a handful of people you don’t know,” is directly in conflict with the fact it is treated as a public utility.

What would happen if npm’s package registry were to disappear tomorrow? The economic damage would be hard to quantify. It probably wouldn’t be disastrous (most packages exist elsewhere) but not having a registry to make dependency resolution easy would considerably slow things down. Big firms would probably weather the storm; smaller developers may have trouble competing. How long would it take for the web development industry to adapt to a world without npm? It might not kill the sector, but it would be painful.

So, what’s the answer? I don’t know. There is certainly a case you could make for public ownership of key resources such as the npm registry. This has happened before, where private assets emerged as infrastructure: it’s why many countries nationalised their highways, railways, fire brigades, and telecoms networks.

But for the tech industry, this model is unlikely to work neatly. For a start, which public should take ownership of something like npm? I certainly wouldn’t trust, for instance, the US (or indeed any) government to have control over what is and isn’t accepted into the package registry; nor would I trust them not to inject some kind of malware to be used for espionage and mass surveillance.

Maybe the answer is to put such infrastructure into the hands of a not-for-profit foundation. But hosting heavily-used infrastructure is expensive: most centralised package repositories depend on volunteers and donations.

An alternative solution might be co-operative control. As recently as 2006, Mastercard was a co-operative between the financial institutions that issued it; this isn’t ancient history.

Many tout federation as a solution. In some cases, this might work. Entropic (the federated package registry to replace npm) is still a young project, and I certainly hope that people embrace it; the JavaScript commons is too precious to be owned by go-big-or-go-home venture capitalists.

But if we can make the case that npm is infrastructure, then we should examine other assets too—particularly those funded by venture capitalists, and those struggling to turn a profit.

Infra is everywhere

There are many startup darlings that you could argue qualify as ‘infrastructure.’ If we go by the definition above, you could argue that, for instance, an app like Citymapper is public infrastructure—a large section of the public relies on it for their personal mobility and independence, despite the fact it is owned by private investors. The same goes for Uber (although its existence and business model, in many cases, causes problems for people using publicly-owned transit infrastructure.)

But if we look further, we can see emergent infrastructure everywhere. Is Facebook public infrastructure? Many people rely on it to organise events, stay in touch with their friends, and organise their lives. If so, should its policies be driven by a desire to turn a profit at all costs? (Elizabeth Warren, a candidate for the US presidential election next year, wants to break up Facebook. To prove her point, her campaign ran an advertisement with a blatantly false opening statement: Facebook has scurried to defend its policies, but has not demonstrated how it will effectively combat misinformation and political interference on its platform.)

Is Twitter public infrastructure? Donald Trump, a racist landlord who is also the President of the United States, routinely uses it to deliver his racist tirades, cutting out the middleman of a press secretary. It’s not all bad news: when a false alarm for an incoming ballistic missile was delivered to every mobile phone in Hawai’i, many officials used Twitter to calm the panic.

It’s clear from this that we have come to rely on Twitter to run a large amount of our society. Many businesses rely on it. Most news organisations depend on immediate delivery of Tweets. And yet, who’s paying for Twitter? Its revenue comes mainly from advertising, and it has only just started turning a profit. Twitter’s key purpose, of course, is not healthy conversations, nor public safety, nor service reliability: much like npm, Inc., Twitter’s key purpose is to turn money into more money to be paid out to its shareholders.

Co-operation over corporation?

I’m not convinced federation is going to save us from stuff like this. Mastodon, a broadly similar federated alternative to Twitter, has a significantly higher barrier to entry: having to choose an instance to create an account on is a decision that many people are not equipped to deal with.

I think part of the solution must be for platforms such as Twitter and Facebook (or whatever the next big one is) to be run differently, preferably by a co-operative of entities rather than by a Silicon Valley tech bro-infused startup with angel investors on their backs. How this actually happens, I don’t know. I don’t claim to be an expert.

But it’s clear, from the problems at npm, Facebook, Twitter, and others, that the status quo is unacceptable. This is an inherently political problem, and it’s not worth trying to claim that tech or services such as these are politically neutral: the public relies on infrastructure, and utilities, provided by a small number of private companies whose overriding purpose is to make a small number of very rich people, even richer. In some ways, this is privatisation by stealth.

Now is the time to work out how our industry is going to solve this. I’m not sure how, but I’m not comfortable—and I doubt anyone else is—with our industry and our society relying on a continued stream of funding from venture capitalists. Recent events have shown us that things have to change.