one character, one night
Entry five of ten in “the end of the wire,” a series on the terminal. Each one stands alone.
The pipe is the most consequential character in computing, and it is consequential because it knows nothing.
On 11 October 1964, Doug McIlroy wrote an internal memo at Bell Labs containing this sentence:
We should have some ways of coupling programs like garden hose — screw in another segment when it becomes necessary to massage data in another way.
Ritchie kept the yellowed typewritten sheet on his office wall.
Then almost nothing happened for eight years. McIlroy explained the idea one afternoon on a blackboard, proposing a notation with no punctuation at all — input sort paginate offprint, verbs in a row. Ritchie’s account of how the group received it is the most useful sentence in the whole history: “The idea, explained one afternoon on a blackboard, intrigued us but failed to ignite any immediate action… What a failure of imagination!”
That is the honest shape of it. Not a suppressed insight, not a visionary ignored. Four or five of the best systems programmers alive looked directly at the idea, found it interesting, and went back to work.
Somewhere between June 1972, when the Second Edition shipped without pipes, and 15 January 1973, when McIlroy gave a talk describing a Unix that had them, McIlroy finally wore Thompson down. McIlroy’s own account, written years later in the annotations to a Bell Labs technical report:
In one feverish night Ken wrote and installed the pipe system call, added pipes to the shell, and modified several utilities, such as
prandov, to be usable as filters. The next day saw an unforgettable orgy of one-liners as everybody joined in the excitement of plumbing.
The feature is documented in the Third Edition manual, February 1973. Ritchie remembered the year as 1972 and he was in the room, so the honest version is a window, not a date.
McIlroy, who caused it, was clear about what it turned out to be worth: “Pipes ultimately affected our outlook on program design far more profoundly than had the original idea of redirectable standard input and output.”
Here is the mechanism, and it is almost embarrassingly small.
A pipe connects one program’s output to another program’s input. That’s the entire feature. Neither side is told. sort does not know it is being read by uniq; uniq does not know a file, a keyboard, or a program on the other end of a network wrote what it is reading. There is no negotiation, no registration, no shared type, no version handshake. Two programs cooperate without either one containing a single line of code about the other.
The 1978 Bell System Technical Journal issue on Unix put the design rule in one clause: expect the output of every program to become the input to another, as yet unknown, program.
As yet unknown is the whole thing. A program that knew its consumer would have to be updated to gain a new one.
That mutual ignorance has an economic consequence, and it is the reason this matters outside of Unix nostalgia.
Take fifty tools that compose. You do not have fifty capabilities. You have every ordered sequence you can build out of them, which is a number nobody has counted because nobody needs to — it is large. Add the fifty-first tool and you have not added one capability. You have added every pipeline that tool can participate in, including the ones nobody will think of for another decade. Value grows superlinearly in the number of parts.
Now take an application with fifty features. Its value is fifty features. Adding the fifty-first adds one, because nothing inside it composes with anything else inside it. Every capability had to be anticipated, designed, and shipped by someone who was paid to imagine you.
This is the actual difference, and it isn’t power or speed or aesthetics. One of these systems can do things its authors never conceived of. The other can do exactly what its authors conceived of, very well.
The character came later, and the way it arrived proves the point on itself.
The first pipe notation reused the redirection characters. sort input | pr | opr was originally written sort input >pr>opr>. It was ugly in ways that compounded — you had to quote to pass arguments, sort input >"pr -2">opr>, and < was accepted symmetrically, so the same pipeline could be written backwards. It took most of a page to describe in the Third Edition manual.
Then Thompson had to explain it in a big public talk. McIlroy: “The syntax was reformed when, to avoid the embarrassment of describing it in a big public talk, Thompson proposed the appealing infix |. Thus naturalized, pipelines became describable in just four sentences in v4.”
A page, to four sentences. The notation that took a page was the one that carried opinions about direction and quoting. The one that took four sentences carried none.
The cost is real and it gets paid every day.
If text is the universal interface, then every program that reads text is a parser, and every parser is a bug waiting for a filename with a space in it. awk exists because the interface is a string. So does every fragile pipeline that worked for two years and then broke because a column shifted. The tax on composition is that nothing is checked, by anyone, ever.
The trade was still correct, and the reason is in the 1978 sentence. A checked interface requires both sides to agree in advance. Agreement in advance is precisely what as yet unknown forbids. You can have safety or you can have strangers.
One character, and it holds no opinion about what is on either side of it. Fifty-three years later, nothing has replaced it, because there is nothing there to replace.