Interview with Ken Thompson, 9-6-89
MSM: Is it six? Is it six? Pass says seven. Okay, I Ďll be here tomorrow. Okay. I was going by the pass.
MSM: Various accounts Iíve read of UNIX, Ritchieís retrospective on it, and even an interview you did with some people for a video back in 1981 talk about the system as being, or UNIX as being, sort of culling all the best ideas in operating systems that emerged during the Ď60ís. What were those ideas and how did you first encounter them, how did you encounter them as ideas?
Thompson: My background for obtaining these ideas was uh, there was a I went to the school at Berkeley and there was a thing called Project Genie at Berkeley. As project Genie it was never very heavily advertised. But, what they is brought a SDS930 through an ARPA grant and cannibalized it, put paging in it and it became what SDS later marketed as SDS940,which was a time sharing system. In it there were some Mel Butler, Lampson, Peter Deutsche and Mel Hurdle were there. They were the chief people there, who went on to do other things. But, they essentially made a cleaned up version of MITís operating system. Time sharing system.
MSM: The CTSS?
Thompson: Actually, no PB1060. (not clear) Itís a TDA. Itís a three-letter acronym. IDN or ISN or I something. I canít remember. Anyway they had a lot of fun ideas in there and there and a nice clean file system. Then when I went to Bell Laboratories, I worked on CTSS, I used CTSS per say. I used CTSS and did some, a lot of programming on CTSS and I worked on MULTICS.
MSM: What did you promote?
Thompson: We were involved with the file system, which never really came to exist, because um the addressing is built into the paging system. -- the whole process is seen by paging -- and what we did was try to develop read and write calls that were sequential calls that turned around and ended just reading sequentially out of pages, itís sort of upside down notion. And the um there were problems with the segments, things called files, and that they were fairly short and maximum size. Through the eighteen words of addressing, max. So, if you want big files you had to concatenate segments and walk across to the two-dimensional address --one dimension being the segment number and the other dimension being the word within the segment. Anyway, it was to try to clean up some of those problems with paging. But, it... so anyway... Thatís were most of the ideas came from was the combination of those three systems. The 940 system... what became the 940 system. CTSS and Multics, you know a couple of new ideas.
MSM: Which ones were they?
Thompson: Um Pipes. There were a lot of things that were talked about but werenít really done. Like treating files and devices the same, you know having the same read calls. Typically during those days there were special calls for the terminal and then the file system itself. Those calls werenít the same. Confusing them and redirecting IO was just not done in those days. So, that was... I think everyone sort of viewed that as a clean concept and the right thing to do but for some reason it just wasnít done. It was just the right time to actually install the feedback; and, uh, the things we stole: We stole a shell out of a MULTICS, the concept of a shell. We stole per process execution. You know create a process -execute the command. From a combination of the two, although, neither of them really did it, MULTICS wanted to do it. But, it was so expensive creating a process that it ended up creating a few processes and then using them and putting them back on the shelf, then picking them up and reinitializing them. So, they never really created a process for command because it was just too expensive. The ION direction and the stuff like that and later in fact streams came from um the IO switch, that we worked on in MULTICS. Having everything work the same and just directing, you know, changing what it really pointed to. Hard to think. I remember at the time that there was a discussion on whether we should go to six or eight bytes. Seems like silly discussion now. Wasting all that space, you know, going to eight bit bytes when there was only six bits of information there. (Laughing) It doesnít seem like a grave decision, but it really was. In higher level language which was still (not clear), we had always wanted to do that. The original wasnít, was written in a simple language. But,
MSM: Go you wanted to go to high level from the star t?
Thompson: Right from the start. Knew we had to.
MSM: Was that MULTICS influence?
Thompson: That was MULTICS influence. And just the complexity of maintaining the thing, we just knew that, you canít maintain something. Even write it, get it going. But, it will evolve.
MSM: Did the choice seem obvious to you? As to which high-level language to use?
Thompson: No. Not at all. Because none was really good. PL/1 was too high for us. Which was what MULTICS did. Or even the simpler versions of PL/1that we used in MULTICS. Like there was a thing called EPL. The 360 was around, although it was IBM-proprietary. After UNIX was up, or, simultaneous with UNIX coming out, BCPL was just emerging and that was a clear winner with both of us. Both of us were really taken by the language and did a lot of work with it.
MSM: How did you come up with it, itís an English language wasnít it?
Thompson: Ah yes, but the guy who did, Martin Richards, actually developed it at MIT. It was available in a very informal way, on CTSS and we pulled it off of CTSS and got a version running on GECOS here and did system programming there. It was too big a language to run on the UNIX machines that were 4K machines. Thatís when B was developed. Which was ...
MSM: Did you develop B?
Thompson: I did B.
MSM: As a subset of BCPL
Thompson: It wasnít a subset. It was almost exactly the same. It was a interpreter instead of a compiler. It had two passes. One went into intermediate language and which one was the interpreter of the intermediate language. Dennis wrote a compiler for B, that worked out of the intermediate language. It was very portable and in less than a day you could get very versatile (not clear). Typically the interpreter was a set macros for your interpreter, they were very field orientated and you just define these macros with these fields and then write a little interpreter that would switch the set routines, and you had to write about twenty three-line routines, and it would run. And it was very small, very clean. It was the same language as BCPL, it looked completely different, syntactically it was, you know, a redo. The semantics was exactly the same as BCPL. And in fact the syntax of it was, if you looked at, you didnít look too close, you would say it was C. Because in fact it was C, without types. Thereís no word like interchar or struct or anything like that. The word for... There was a word for extern, which means to declare an external thing. There was a word auto, which declared an auto thing. So, it would be like auto XYZ, instead int XYZ and it meant "word". Which was the only time.
MSM: So it operated really at the machine level.
Thompson: Yeah. It was used to a very small extent. It was written in its own language. Thatís why itís so portable. Because you just pull it through and itís up real quickly. Um... But, the interpreters, the interpreter for the 11 was having some trouble. It wasnít a word machine, and this thing had a word notion, and so on almost every operator you had shift left and shift right, shift left and shift right. It was just not a good match at all and part of this is we didnít have a good -on top of the interpreter problem-, it wasnít even a good interpreter on the 11, because of the mismatch of the machine and that we wanted something better as the systems language is what prompted Dennis to slowly permute it into C.
MSM: So C essentially contains B?
Thompson: Well, some of the anachronisms of C, that are now gone, or, at least are not or are unpublished to the point that no one knows theyíre there, are B anachronisms. Like auto. Thereís a word called auto. No one knows, I think itís actually ANSI finally. The word oriented parts of C, as C emerged were in fact the basic routines. And in fact one of the major, at least in my view (not clear) with C is that a arrays to are promoted to the address of the base of the array every time you touch them and thatís one of the fundamental things of the NBCPL. That thereís no such thing as an array but thereís these things called vectors. A vector is a list of words and declaration of a vector is a word containing a pointer to a list of words. If you say auto x of 5, thereís no such thing as x of five, you know, thatís a type, and there is no types in this language. So, what it is, itís a single word called x and then five words that are unnamed and a pointer, initialization of a pointer into x to the base of the five words. To keep that semantics and develop a notion of an array, which we want to promote. The name of an array into the address to do it at run time. Anyway,
MSM: It always seems to be one of the neat features. The way you could step through an array with arithmetic.
Thompson: Oh yeah, yeah.
MSM: You prompted a question when you talked about portability of B and of course one makes a great deal of the portability of UNIX itself and itís a portability, if I understand you correctly, based on self-reference, or almost self-modification, which was the theme you were pursuing in your Turing Award talk, largely to suggest the dangers of doing it. Is that a theme of continuing interest to you?
Thompson: Have I got it right to start with? I guess itís wrapped up. Von Neumann machines in the real sense. Thereís a lot of power in executing data --generating data and executing data. In fact, thatís how languages work and in college I worked for the comp center and it was thrown upon me to maintain a language called NELIAC and itís no longer wanted. Then later on then another language called Smalgol as a subset of Algol. Which were compilers both written in their own language. You get a sense of, I donít know, bootstrapping and of self-modifying programs and of self-replicated programs when you are in a position of maintaining a language written in its own language. Even if itís written in a simple language, you know, you get this feeling of bootstrapping and moving on and I used to do a lot of that stuff, earlier. In fact, the Turing talk was about work I did a long, long time ago. Iím really sure I referenced the date that it was done in the talk.
MSM: You talked about the game of writing the shortest program that writes itself. You said, "I imagine people programmed in Fortran for the same reason they took three-legged races."
Thompson: (Laughing) I shouldnít say such things.
MSM: Well, all right. Itís a great remark.
Thompson: Last year I taught at University of Sydney I gave that to my class, the shortest self-reproducing program in C, and I got a surprise. I didnít think there was a surprise there to be had. But, I got somebody who has the shortest one Iíve ever seen, which is a record breaker, by about four characters of what I had proved to myself was the shortest program, and they did it by a totally different mechanism which of course nullified the proof.
MSM: Did you spend any time up at MIT during MULTICS ? Did you come...
Thompson: I just went in and out for a day at a time. Maybe for ten times. Something like that. Yeah. Iíd go up there for... just ran through the halls and did work and go to meetings and stuff like that. I spent no time, I didnít teach and I didnít stay there for more than a day at a time .
MSM: ĎCause some of these things were very much a part of that environment: Minsky and then LISP, which essentially is a language written in itself.
Thompson: Well, LISP , least the original LISP, you know, the book, 1.5 is... you know I think itís a horrible language. I really do. But, I was struck with that book and the idea of defining very, very low level semantics, you know cons and (not clear). Essentially thatís all thatís defined, maybe a few more. From that, developing a... itís not so much written in itself that it defines its own interpreter, in a way that gets into the what I think is the whole semantics for (hearing?) languages. Itís always been a problem when you write a language or describe a language to say what constructs it recognizes and what they mean and what they actually do and that was the cleanest, simplest, most recursive, beautiful semantics of a language Iíve ever seen. Probably even to this day. But, unfortunately, what it describes I think is just a horrible language. I agree. Thatís really striking, 1.5. I did a lot of that. I did a lot of compiling. Even in college and out of college I did a lot of on-the-fly compilers. Ah. ah. I wrote a GREP-like program. It would... You type in Ö, youíd say what you wanted it to look for, and a sed-like thing also. That youíd say, I want to do a substitute of A for B or some block of text. What it would do is compile a program that would look for A and substitute in B and then run the compiled program so that one level removed from it do I direct my (unclear) and the early languages, the regular expression searching stuff in ED and its predecessors on CTSS and those things were in fact compilers for searches. They in fact compiled regular...
MSM: Does this reflect itself in UNIX as it was developed?
Thompson: Not a whole lot. Outside of operating systems in general tend to operate on programs and they have to somehow turn the notion of data and programs inside out. Theyíre operating on what they think are data, and that data are running programs. The whole (unclear) is encapsulating processes as not variables just data comes into it. But, no itís nothing real fancy in terms of.... Do you know this kid Henry Heslin (?) Heís a PHD student at Columbia. Heís the doing a lot of weird stuff very similar to this now. He has a UNIX mailbox that does 68,000. But, when he issues a open on a file. Itís the same semantics as UNIX. He compiles into what would be the open file table. Build the subroutines to getchar, putchar, read and rrite and getchar, putchar, that are just amazingly fast with all the checking built in. You know the files open, you know the descriptors here. You know all of this so that.... A read call traps right into this pre-compiled code for that at one character per time in a system that he gets faster than most systems get and are doing 8K at a time. He does a lot of that stuff.
MSM: I see. Does he work from here?
Thompson: No, no. Heís...
MSM: How do you know about him?
Thompson: Um ... he wrote a paper that some people hate and some people love. I was struck by it. Itís called Super Optimizer. What he does, he defines a function he wants to write, and see, and then he by trial and error, he builds that machine language that will implement the function, he uses the function to check the machine language. So, heíll try essentially all programs and then see if that program equals that program, but semantically.
MSM: (Laughing) I like to see what Iím getting... . Itís a difference of opinion.(Laughing)
Thompson: And um, it generates shortest possible program for small functions, you canít do big things. It generates code that is absolutely inhuman. Itís, itís indescribable, to be honest. Um, and um code you,... itís easy...thereís no way to describe it except that it proves it. Iíve use that idea, since I read that paper, Iíve used that idea around four or five times. On one case I used it for a compiler Iím writing for 68000 um, multiply takes thirty-two seconds no matter what. So, if you multiply something by three, thirty-two cycles. Those same thirty-two cycles, thirty-two adds, on this machine. So, what a combination if you change a multiply into shifts and adds. Multiply by a constant with shifts and adds of, you know, the original thing. Youíre going to always beat the multiply because, the multiply is implemented so badly on this chip and so what I did is write super optimizer, which tries all combinations of shifts and adds to generate, to simulate a multiply by constants between one and ten thousand or something like that, and put them into tables and take the C the compiler and generate explicit code which is the best shift and adds and subtract.
MSM: So the multiply go to the table, look up at....
Thompson: No, no it doesnít go to the table, the compiler goes to the table. You say multiply by five and the compiler goes to the table and does shift left of four and add. Itís four plus one.
MSM: So it picks up a particular combination of shifts and adds that will work for that particular multiplication.
Thompson: Right, shifts, adds, and multiplies.
MSM: By doing table lookup and then imbedding code.
Thompson: Right. And itís optimal because thatís how the table was generated, by trial and error, all shifts and adds that can generate all multiplies of all things. Another one, is that the bit blit on this thing is um... itís read a pixel, a block of pixels, perform some operation on a block of pixels. Itís like plus equals. You know, pixel block plus equals pixel block. Where a plus is a pixel is an XOR an man / you know all these operators. So, you put in any of the arbitrary binary sixteen, any of the sixteen binary operators in this opcode for the bit blit. And the whole thingís compiled. When you do a bit blit you compile the code and run it, and you want the best compiled code for these operations. You had to find the best compiled code for these operations. Put it into tables that are up and generated by trying all of the programs.
MSM: What machines?
MSM: Is that the NeXT symbol of the NeXT machine? Or is that just the (unclear)right there?
Thompson: No itís just a joke.
Thompson: Thatís the machine sitting on the table. Itís a terminal in.... Itís...tactically, if you want to use the word, itís almost a Sun3. Itís the 68020 floating point, four-meg memory. Network interface which is an ether - itís a (unclear)
MSM: Did you design that yourself?
Thompson: No, no. It was designed here. Not by me.
MSM: Let me pull you back. ...Talked about this distillation of all the good ideas. Were there ideas that you particularly wanted to avoid, or features you wanted to avoid, or that you had in mind as representative of what was bad with operating systems?
Thompson: Yeah. There were lots of them. I wanted to avoid, special IO for terminals. I wanted to have virtual memory, at least as itís coupled with file systems. I wanted to keep file systems really exclusive and separate from virtual memory -as not be read and be write. There were lots of things. Ah. I wanted to avoid this thing called an "executive". The word has lost its meaning now. What it was is a pseudo-shell, built into the kernel, that somehow controlled the console and execute the commands for you and to drag that out and make it a process of any process could execute any command.
MSM: Just to make sure that every processor...it sets every processor the same as a user processor and having this privileged.
Thompson: Yeah and also to make the thing that became known as the shell itís handled in like any other program. There was no Ďthe shellí that came with the system that you were stuck with for life. In fact, we started off with two or three different shells and the shell had life of its own. A new shell would come in and supplant the old one and there were... shells performed different functions. Like, there was a shell for a video interface. A shell for a voice synthesizer. You know, what you would do is bring in a touch tone phone and put another shell in its place. So, anyway, the idea that there was no built in known level of command, that, that was just a replacement program that we could avoid like any other. We tried to avoid, you know, records. We were told over and over that was probably the most serious mistake and the reason was the system would never catch on, because we didnít have records. Essentially, the record manager was images imbedded in disk images. Having just this uniform sequence of lights, they said over and over to us that was a serious mistake, but we stuck by it.
MSM: Iíve just been having exchange systems programmer down at Princeton on the IBM mainframe. The problem, default record formats and what happens when you try and do a get on a file thatís got a different format, losing (unclear) records when...do you have a code?
Thompson: I have um. I have um...in that era, we werenít trying to promote this idea. Iíd give talks, we always come up, you know, why you didnít do records and Iíd have some extra slides, cause I knew Iíd be asked this. You know, you know, you jumped and said, " Well, I just happen to have a couple of these laying around. (Laughing) There, the best slide and best story is McIlroyís test. You ever heard of it?
MSM: Sort of finding a....
Thompson: Yeah itís a Fortran program that works ....
MSM: ... the first Ďeí in the in the eighth column, and you do it on the Fortran program itself.
Thompson: Right. To ask them to go through the steps is just priceless-- to see what happens in these systems when you do that. It all has to do with record formats. Confusing program and data, in a file format. You know, that things... You know, thereís text files that are data formatted. You know, on and on and on. (not clear) Itís just.... That was one of the things. The other thing is that, thereís a series of thunderbolts out of manuals. Describing um. (not clear) By chanceÖ
(Shuffling of papers)
Thompson: I think this might be it...No, this isnít it... .This is very similar.
Thompson: You know about the Yeah. Itís putting it there . In our UNIX paper. Um, um. Dennis wrote this. It offers a number of features (not clear) it was actually a joke. It means nothing in a sense. Itís those kind of things that people write in papers. People picked it up. Some people picked it as a joke, and some people didnít really understand it as a joke.(Laughing) Variations. Variations on that and thatís how it gets picked up. It was a joke. (Laughing) ... four Princeton (unclear) larger bankrupt packages. Can you picture (not clear) ever wearing this thing.(Laughing) BT52, DECís newest version of (not clear) (Laughing) So, and the other is, the one I was trying to find is, it, it, it was one of the HP3000, brand new operating system, post-UNIX. Talks about the editor and says that, "The editor edits um um binary um card images 84columns -whatever, some magic number- in a variable format. Itís a variable column format, with columns set to 84 and this is what their editor edits and thereís about ten computations like that, describing, you know, formats. Record formats and what this has do to convert this to this, you know.
MSM: Is the absence of those kinds record formats the reason why the UNIX editors had no concept of column? That is, if I am in a CMS editor, I can do column substitution, column searches, it not only has the notion of a line, but it has a column and I can go directly to it, do column locates and column changes.
Thompson: I donít know, I think mostly, it has no notion of columns, because none of the languages have notions. Grant, if we edited Fortran, I assume that youíd put a column thing in there. There are regular expressions that you can use to get your (not clear) at things.
MSM: Just no one found that feature particularly useful.
Thompson: If itís a notion that you need or want then Iím sure it will easily. I donít think it has anything to do with records, because, in fact, you know, the only thing important about lines there is newlines. There is a notion of lines and a notion of columns. Itís just that we never (unclear)
MSM: The story, as I gather, is that behind UNIX stood MULTICS. All of you been working on MULTICS, then the word came down: no more MULTICS. How did you feel about that?
Thompson: Um. Mixed. Um. Technically, I thought it was a good idea that we were getting out of MULTICS. That it was too big, too expensive, too over-designed. It was just clear it was an exercise in building monstrosities. Efficiency would never come back to the point of where it was .... was cost effective and useful. Most of the efficiencies were dumped into features that were there because they were ... I donít know how to describe them. They werenít there because they were good features, they were there because they were neat technical acts. Um. So, on one hand, I thought it was the right decision. Even then. On the other hand, we, meaning essentially Dennis and I, two or three others, had a ten million dollar personal computer. It was clear that this decision was aimed at getting rid of that. You know, the side effects of this decisions was that this thing was going out the door. Our personal way of life was going to go much more spartan. So, in that sense, we didnít want this decision to go. Um. Thereís a deeper decision in it than just MULTICS, the crew wanted to work on MULTICS in Bell Laboratories, and thatís that computer science research shouldnít work on operating systems. Operating systems were dead. This was the whole.... There was a whole change in thought at this point that, operating system research was dead. Um. Manufacturers, in a laboratory environment, you couldnít build a workable operating system. It really required a development kind of mentality and you know, grind it out. That we provide insight, but we couldnít build one. They were too big, too expensive to build or maintain. The whole computer science research was going to go back to theoretical, paper and pencil kind... There was a signaling of a very strange change here, at that point. Essentially getting out of the computing type of computer science as opposed to the theoretical type. When we persisted we were almost outlaws. We had to beg and borrow machines from weird places and weird sources.
MSM: Why did you persist?
Thompson: Itís what I do.
MSM: You wanted to work on operating systems?
Thompson: Well, no not per say. I just wanted to work on computing and programs and it wasnít in essence no operating system. I never really viewed almost anything I did, as what I worked on. Itís what I wanted to do next, for some other goal. After MULTICS went away and things settled after this, our computing environment.... Computer science and the Computer Center were one, and it split and the Computer Center went off to the services area, you know, like people who do the air conditioning. Computer science went into research. Um. We then had to go over a fence to talk to the computers. The computers were not with us anymore. We were a service organization where, you know, you had input boxes and output boxes and submitted cards over a counter. Very, very different approach what we were used to up until this point when we, since it controlled everything. The operating systems became vendor-supplied. In particular we went GECOS, and a word on top of GECOS, called TSS. Which was their time sharing system. Which was nothing but batch card entry. Quick turnaround batch card entry into the batch world. Thatís what they call it: TSS with a partition for editing. It was horrible just to use that. The operating system, I think was just a (not clear) to get into a environment (not clear).
MSM: I was thinking about this yesterday. As you may know from what Doug has said, Iím doing a history of software in general, during the Ď50ís andĎ60ís, working up to the roots and the kind of thinking that was going on in the late Ď60ís as the software crisis emerged. Basically the question: how did the industry get itself into that, that situation? I was thinking about the operating systems of the Ď60ís, which, if I understood them correctly, as someone who, when he was programming in the late Ď50ís for a small company, still on the machine at night and worked at the console that to debug a program. I never went through that: hand on your cards and wait until the next day to get your output, first stage of program computing. That the notion of the operating system was to make the machine efficient. That the notion of a system like UNIX is more making the program more efficiently. As itís a programmers system rather than Ö
Thompson: It was a combination of both. I mean thereís reason they would be mutually exclusive. Um, um. The talk of the day in the conventional wisdom, which I never really bought, was that they were mutual exclusive.
MSM: That was the feeling at the time?
Thompson: Yes, and that um, time sharing would never survive because, youíre spending all of your time on this big mainframe, you know, all floating point hardware and all of this stuff, you know, fielding these ratty little interims from people typing on flex-o-writers. And it was self-fulfilling prophecy, they believed that, in the systems they that built. Believing that, demonstrated that. Because, it makes it what it had to be. I think that in time sharing you can do better than batch. That you have a better mix of things to do and you can do scheduling in such a way that you just canít get in batch.
MSM: Follow that up, because, again, there is the lore that has grown up around that. On one hand, I gather that what you all felt most strongly about at the end of the MULTICS project, other than the fact is that you have a ten-million dollar personal computer, was that notion of communal or convivial computing, that is that you have been able to share files with one another, become a medium of communication among you and that you felt - Doug at least said he felt that that really hurt to lose out. Then the other story is that UNIX started off as a personal system. A one-person system. Those two stories (People laughing) arenít entirely compatible. That is...did you have a notion when you started UNIX of restoring that sharing? Was that in from the beginning? It was going to be multi-user system from the start?
Thompson: Not, explicitly, I think. I was more interested myself. Just selfish notions of trying to get a environment to work in.
MSM: Were you trying to build a programming environment for yourself?
Thompson: We always wanted to expand it and turn it back into communal things. We were always trying to get machines that we could take home, you know and share among wider groups of people. Thereís massive amounts of software that had to be developed, languages and all applications and all sorts of things. You just canít sit there with that Model 33, you know, wired right into a computer and do it all yourself. You can get your own work done, but you can really work faster if thereís a community of ideas, a community of help. Application programs you can use and rely on. (People talking in the background)
MSM: As you were the developing the system did you have it in mind to keep that option open at all times, with decisions informed?
Thompson: Well, it was always time sharing in that sense. Sometimes it was a single-user system, but, it was always a time sharing system. I think it was implicit. It was never voiced, but it was always meant to be a shared system with lots of users.
MSM: When you and Canaday and Ritchie, I think that's the three, settled down to find a file system, what were you looking for? Because, the file system you designed looked like, if I understand it correctly, looked a lot like the MULTICS file system.
Thompson: Up to the point of writing simulators. Ah. The idea of the file system was to um, to have the activity locus of manipulation of data for user one and user two.... to be disjoint, so that in fact, wouldnít be locking common tables. Wouldnít be going through anything common unless we in fact shared files. To try to keep up real high, efficient access to disks. In fact, interleave accesses in a way. If two users... one user would expect some sort of response call or whatever it is and that at least with the disks of the disks of the day, two users will be able to command it and interleave seek times on the disks, and would not degrade each other. That was the idea behind the file system and the design; to move the addresses to the point where things could be cached and that your working and my working wouldnít interfere with each other in a locking sense or in a real sense in any way that (not clear). Um. I had built this system in a high level simulation of the whole file system and had gotten as results that these common disks of the day that in fact, you could enter(not clear) of your requests and get lots and lots of users happy at the same time.
MSM: Was this the one you were doing on the 645?
Thompson: Yeah. I was doing it on the 635 at the time. Yeah . I got these exponential curves where before it would get into trouble it would go way out and get lots and lots of simultaneous accesses going... I was playing with a disk sorting algorithms and caching algorithms at the time. All of those actually went into UNIX. Um.
MSM: This would be the research aspect of the work?
Thompson: Yeah. Then in the actual design. At that point, it just went to... There was a model of a user and a model of this, and they generated activities, and the activity went into the disks that were sorted and things like that. Um, um. It was never down to a design to the point of where you put the addresses, how you expand files and things like that. It was never down to that level. It was always at some higher level. I think it was just like one or two meetings, Dennis and Canaday and myself. Was just discussing these ideas of the general nature of keeping the files out of each otherís hair and the nitty-gritty of expanding. Of the real implementation, where you put the block addresses, where you put this and this. I remember, um, we did it in Canadayís office. At the end of this discussion Canaday picked up the phone, and there was a new service in Bell Laboratories, dictation, where you call up essentially a tape recorder and you give notes, and then the next morning notes are typed and sent to you. The next day, these notes came back and the acronyms were butchered, like "inode" was "eyenÖ" (Laughing)
MSM: You should see the transcripts of your Ď81 interview. (Laughing)
Thompson: So, we get back these, (unclear) description and they were copied and we each got copies of them. They became the working document for the file system, which was just built in a day or two on the PDP-7.
MSM: But to the user it would look roughly the same as a hierarchy of directories.
Thompson: No, the first one was a DG. In fact, it wasnít even a acyclic. If you understand the UNIX file system, it was.... there was the I-list, which is a definition of all the files on the system. And then some of those files, were directories which just contained name and I-number. Thereís nothing in there that constrains it to a tree. So it was not in fact, not hierarchical at all.
MSM: I see.
Thompson: And we did not restrain it to a tree. We were experimenting with various topologies. What we ended up doing is turning into concrete and forcing the topologies that in fact were the topologies that came by convention from that system. The... Every time we made a directory, by convention we put it in another directory called directory - directory, which was dd. Its name was dd and that all the users directories and in fact most other directories, users maintain their own directory systems, had pointers back to dd, and dd got shortened into Ďdot-dot,í and dd was for directory-directory. It was the place back to where you could to get to all the other directories in the system to maintain this spaghetti bowl. So, I mean this tuff in various forms, which was strictly convention in this DG implementation of just random set of directories and files got forced into a typology that we maintained. When we started writing things like file systems checking programs and stuff, the locking of the spaghetti bowl directories and finding of disjointed things, I mean youíd dissever something and never get it back, because you know youíd lost it. Those problems became close to insurmountable, and so in the next implementation we forced a typology stronger than that.
MSM: The PDP-7, you used the famous graphics machines you found. Um you went to when you found out you had in mind to just put the file system on there or ...?
Thompson: At first, yes, we used it for other things, the famous space travel game, and it was a natural candidate of a place to put the file system. When we hacked out this rough design of a file system on the dictation that day in Canadayís office, um I went off and implemented it on the PDP-7.
MSM: Ok the PDP-7 was already around at that point.
Thompson: Yeah. We had already done uh...weíd spent a lot of the summer doing it... space travel...we had a lot of the pads worked out, we had assemblers and...the assemblers were actually on GECOS, and theyíd generate paper tape and weíd carry the paper tape down the hall and...
MSM: Were you looking for a graphics machine, was that...because of the space travel game?
Thompson: No, no, we used it because it was there, it was a graphics machine before. It was designed to be a circuit design system where youíd lay out resistors and transistors and things.
MSM: So originally you grabbed that from doing the s pace travel, worked up a certain number of tools on that in order to implement space travel, and then came the file system and you went to implement that.
Thompson: The file system didnít exist by itself very long. What we did was... to run the file system you had to create files and delete files, re-unite files to see how well it performed. To do that you needed a script of what kind of traffic you wanted on the file system, and the script we had was, you know, paper tapes, that said, you know, read a file, read a file, write a file, this kind of stuff. And youíd run the script through the paper tape and it would rattle the disk a little bit...you wouldnít know what happened. You just couldnít look at it, you couldnít see it, you couldnít do anything. Um and um we built a couple of tools on the file system...we used this paper tape to load the file system with these tools, and then we would run the tools out of the file system, thatís called an "exec" by the way (laughter), and type at these tools that was called a "shell", by the way, to drive the file system into the contortions that we wanted it to uh, measure how it worked and reacted. So uh it only lasted by itself for maybe a day or two before we started developing the things that we needed to load it.
MSM: At what point did you feel you had something here?
Thompson: Um, well, the first one was not at all multiprogrammed, and was almost like subroutines on the file system. The read call, the system read call, was in fact the call "read" of the file system and it was very synchronous, just subroutine call to the file systems for these applications. And um there was a very quick rewrite that admitted it was an operating system, and it had a kernel user interface that you trapped across. I really canít remember what the realization was, I mean, the whole time span, from initially starting with...walking downstairs, down there with the idea that we were going to build a file system.
MSM: When was this, do you remember the time?
Thompson: Yeah, it was the summer of Ď69.
MSM: Summer of Ď69 ok
Thompson: In fact um my wife went on vacation to my familyís place in California to visit my parents -weíd just had a new son in August Ď68- and uh they hadnít seen the kid so (unclear) took te kid to visit my family and she was gone a month to California and I allocated a week each to the shell, to the operating system, the shell, the editor, and the assembler, to reproduce itself. During the month she was gone, which was in the summer of Ď69, it was totally rewritten in a form that looked like an operating system, with tool that were sort of known, you know assembler an editor and a shell. If not maintaining itself, right on the verge of maintaining itself, to totally sever the GECOS connection.
MSM: So that you could work directly on it.
Thompson: Yeah. And from then on it kept pulling up files.
MSM: So weíre talking about a monthís development.
Thompson: Essentially one person for a month, it was just my self.
MSM: Howíd the others get involved?
Thompson: um Doug got involved (unclear). Uh it was multiprogrammed...processes from the beginning, but it was just one console. And with just a little bit of work we turned the graphics scope into just another typewriter. You know, you print on the screen by inking characters all by hand as well. And so then it became two users, and it was constantly full, it was constantly at two users on it. (Unclear) got involved he was doing TMG, which was a compiler compiler language predecessory to the yacc kind of languages. Dennis got involved with ...during his language work. Uh (unclear) who was doing, he didnít do too much. I mean when it was in the PDP-7 form he didnít do too much. He did some... a number of theoretical kind of things.
MSM: Who else was involved?
Thompson: There were a lot of people involved, in a political sense, you know, trying to keep the machine for us, and get us the next machine, and that kind of stuff that werenít doing programming.
MSM: Who were these people?
Thompson: Uh Peter Neumann, Lee McMahon, (unclear) Matthews.
MSM: They were running interference with managers?
Thompson: Yeah. Joe Osanna did for ... at that point also. Uh he later became very involved in the thing. His pet thing was to develop a um text processing system, to um, which you call a desktop publishing or whatever word processing. A text processor for... he had ideas about secretaries and typing pools. And he was constantly on the lookout for good typewriters, in the sense that secretaries would use, and that could use a touch type, you know, IBM Selectric type typewriters, that were computer interfaced. And weíd go to almost every toy show, with this in mind. Looking for...I mean the industry was in sad shape at that point in trying to (unclear)things that we needed to accomplish some of these goals. But um everything that connected to computers was upper-case only, six bit generated and are very, very expensive. Thereís the 1050 - you know what that is?- itís an IBM Selectric but itís about this big, and was loud and was about this tall on a console. So anyway they were interested in typesetting equation ... not typesetting but ... typewriter setting equations and doing TMS and documentation.
MSM: So was it Joe figured this was something he could sell to management? Keep your system (unclear) going?
Thompson: I donít know his motives. I wonít guess his motives. Um I know he was genuinely interested in it, uh he in fact got a commercial type-setter and uh got it interfaced with the computer. Essentially ripped out the paper...it was meant to be driven by paper tape, and the paper tapes were to be, you know these typewriter to paper tape things, and what we did was we cannibalized the paper tape interface and just ran it over to a parallel interface on a computer, and we just punched paper tape...logical paper tape over to it over a wire. And uh it really had the first typesetting of this sort, so it was all way ahead of itís time and that was all Osannaís work. And it was all towards this ultimate goal of computer text processing.
MSM: I know the um when I asked Doug about pipes in (unclear), the story that I was telling him when I was coming up to talk to him - my daughter who is a computer science/music major up at Harvard - said uh "Well, what did he do?í, and I said "Well, heís (unclear) who had the idea of pipes. And she said, "Oh, well, you ought to call this project Ďpipe dreamsí." Uh I asked Doug about pipes and he talked about what the background to it had been, but he also told me that you were able to implement that overnight.
Thompson: Yeah,well, Doug had was for years and years, well it seemed like years, I donít know the actual span was probably one year, Doug had uh, and he talked to us continually about it, a notion of interconnecting computers in grids, and arrays, you know very complex, you know, and there were always problems in his proposals. That what you would type would be linear and what he wanted was three-dimensional...n-dimensional...I mean he wanted just topological connection of programs and to build programs with loops and and you know horrid things. I mean he had such grandiose ideas and we were just saying, you know, ĎGod, itís worthless, the complexity youíre generating just canít be fathomed. You donít sit down and you donít type these kind of connections together.í And he persisted with his the grandiose ideas where you get into Kirchoffís law problems, where you get into you know, what happens if you have a feedback loop and every program doubles the number of characters, you know, it reads one and writes two? You know, what happens to...itís got to go somewhere you know. And you get these synchronization just, I mean thereís just no way to implement his ideas and we kept trying to pare him down and weed him down and get him down, you know, and get something useful and distill it. What was going on, what was needed, what was real ideas, what was the fantasy of his ...and we there were constant discussions all through this period, and it hit just one night, it just hit, and they went in instantly, I mean they are utterly trivial.
MSM: And that a reflection of the basic structure of the system or was it just coincidence?
Thompson: No it was just we had control over it, it was our system. We could....it wasnít a big system, it wasnít a big thing to put in it was just, it just took minutes to do because we knew what...
MSM: Is pipes the sort of thing...pipelines the sort of thing that can be implemented in any system or are there certain system requirements?
Thompson: Uh well you really have to have real processes, and some places (unclear)data. Um for them to work in a um for them to actually work you have to have the notion of reading and writing streams, or whatever you read and write, and that the I/O cannot be different from the files.
MSM: Oh so that goes back to that idea, that thatís a prerequisite.
Thompson: Yeah. Because if you have programs that sit there and read terminals, and then manipulate files back and forth thereís just no way to connect them. Because what they read and what they write have to be the same thing.
MSM: How long was this the skunk works when did you . .. well you said in the beginning you were building an operating system in spite of what was supposed to be going on. Uh when did you go public with it, within the company?
Thompson: Uh we never really did. Um every step was painful . (Unclear) We couldnít have...uh it was an obsolete machine at the time the company the the, we didnít own it, uh it was another department that owned it, and when it would break, it would be a hassle over who maintained it, and we didnít maintain it because we couldnít get our department to pay for maintenance. There was, you know, no money at all but they just didnít want us to do this. We not only had to buoy this company, this department that owned the machine and wanted to throw it away, but to keep it, on their space and maintain it for us. That was a precarious situation, and that persisted. Then when it became clear that these machines were nearing the end of their life, uh we either tried to get them officially ours, which failed, our manager wouldnít pick up these machines, at zero cost, you know, they didnít want...the cost of the space. Then we started on a set of proposals for getting a new machine, through our management to replace it, to get a new machine, and they were all um... There was no explicit policy that we werenít going to get back into the computer business, (unclear) we really know the rules, you know, but the rules were in effect. So what would happen was that we would take these proposals for these machines and do all the research on them and get the vendors in and waste everybodyís time, and get these proposals up and theyíd be thought about our management for a extended periods of time and theyíd say Ďnoí for some funny reason, you know, never for a real one (unclear) computing anymore. There were several, really several, of these big rounds of trying to get a vendor and a machine and, to get....to do this work. Most of it was carried on by Osanna and me, and the interference type people. Ultimately what happened was um we found a PDP-11, it was in fact not announced yet, but uh it was right on the edge of being announced. We would like the for the idea of text-processing we liked the expandible IO of the Unibus, where we could build our own interfaces, they had general purpose interfaces, they had lots of com gear, uh that we could do for...I mean it had everything. It looked like it was going to expand to any machine we wanted to make it. Um and these people, Osanna and I put together a proposal to buy a PDP-11 to do text-processing, research in text processing, (unclear) and document preparation, this type of stuff. Uh it was the first of the goals that were specific, uh we want to do this for this machine, um for this purpose. The other ones were...we wanted to play with computers and operating systems, and they were unspecific, and the our management went off and thought about it, and rejected it again. But in the meantime going up and down the hierarchy a sister department, 122, psychology research, uh came over and said Ďwell weíll fund it out of our area,í embarrassed the hell out of our management. And they bought it, gave it to us, and...
MSM: What were you looking for?
Thompson: Well it was interesting, they thought it was interesting. They just had insight, and inspiration and unfortunately our management didnít. They were suffering from wounds, our management was suffering from wounds from the MULTICS days and you know ...
MSM: MULTICS really hurt?
Thompson: Yes. Lots of promises to lots of people to develop software that would be everything to everybody. It was sold, company wide, to be the computing utility. Thereíd be, there would be a plug for 110 volts, and right next to it there would be a MULTICS plug, and youíd just plug it in and suck out whatever cycles you wanted for anything you wanted. You know it just, it would just be the utility, you know just like a power utility or a phone utility, it was just the computing utility. For everybody. Everybody would have all the cycles they wanted. It was sold big all the way down the company and...
MSM: So it was a notion of once burned, twice wary .
Thompson: Anyway the 11 came in, um it say for a month in Osannaís office, because uh, it had no disk, the disk was delayed, it didnít come with it. And weíd type stand-alone type things, this stuff, um. Then when the disk was just ready to ... you know the disk was just ordered and was on its way, they were actually starting to manufacture and we were on the waiting list, we brought the machine up next to the PDP-11, which was what we were working on, and we started writing all the cross stuff. In B. We wrote a PDP-11 assembler in B, and ran on the PDP-7 the PDP-11 assembly code and punched paper tape out of the 7, you know out of the...across the floor into the 11 and had fake file systems that were done in memory, and we got it almost running, the disk came in and in probably another week we got UNIX running on it...At that point a lot of things came into being...the topology of the directory structure was fixed on more convention than convention by that point...
MSM: When was this, 1971
Thompson: Yeah, let me see, 71? Maybe Ď2? I donít know that. Itís mentioned in one of the UNIX manuals, the dates rolled up....(unclear) And we moved over...Osanna...I was just interested in operating systems, I went along with the text processing (unclear). It didnít interfere with my plans so I ...went along with it. The editing, and the stuff around the text processing, and Osanna did the... went on to the nroff, troff stuff or the text processing. We instantly put all our secretaries on it. They did our mail messages and our documents and did that so we did text processing for that, then we uh as part of the demonstration Osanna got the patent people, patent application to come over - they were just about to buy a horrible little commercial type setting package- called, we shouldnít mention this...AstroText was the name of it...and it was just truly bad, and we could cobble together something that would be vastly superior to what they were about to pay real hard...you know a lot of hard money for. Um, and so we put their stuff together and developed a package for them that was specific to their applications, you know, they have very different kind of formatting (unclear). Um and in fact they liked it and they still had this money in the budget for this AstroText thing, and we talked them into buying our system, physically, you know, the hardware, move it out, and we took their money and bought an 11/45 with it. (Unclear)interim machine was a um. Before the 11/45 was available we bought a PDP-11 that had PDP-10 memory management, KS-1, it was a one of a kind machine, and that was the first time we ran the production of program development along with these -all in assembly language- along with these typists typing real applications and uh (unclear) on unprotected machines.
MSM: You hadnít gone over to C yet.
Thompson: No. C was fairly late.
MSM: (unclear) DEC machines. Did DEC ever show any interest in what you were doing?
Thompson: No. At one point we put a notion to them, and said um...the way it started running internally, is the word just got around, and these random groups would come in and say you know....everyone, technically would look around for a machine and choose a DEC machine, and DEC had no software, they were real late in the delivery of their software, and when it came it was horrible, just...that was probably the early reason UNIX thrived is because it had no vendor competition at all, none. It was the only software around for the DEC machine, and so theyíd look around, theyíd find DEC machines, they were just without technical peer. And a project would buy a DEC machine and theyíd look around for software after that, and theyíd hear our names, theyíd come and talk to us. And theyíd decide that they were going to run UNIX for their software development, develop their application and then run their application standalone; it never happened, they never got rid of UNIX. You know their application would run on top of UNIX (unclear). The story was always the same: that they were going to use software development, develop their application, and then deliver it with the application. And these things kept proliferating, more and more and more of these things, it was all underground. No one, you know, we were the only ones doing the development, and theyíd want records or theyíd want this or theyíd want that, and weíd tell them no, we didnít like that, doesnít fit into our plans, so to hell with it. Uh and thatís how the UNIX development group got involved is to do the to be more responsive than we were to the needs of the people. (Unclear) were starting to put these machines in.(Unclear) telephone applications.
MSM: Would you say... is this the point at which UNI X became standardized in some sense, got settled?
Thompson: No, no um we had momentum, in our department, we were working...these guys were just fighting - the development group Ė were just fighting to learn what it was. By the time they learned, it was something else. I mean things were really moving fast in those days, and they would spend most of their time, coming up to speed with some version theyíd cloned in the past, and in the meantime there would be three more of their customers that would have cloned from us, because they needed some new thing that we had and, then theyíd end up retrofitting our current version back into you know their version, and most of their updates were in fact taking our version out. And this went on for several years, where they were just some sort of conduit buffer, and trying to learn what was going on. Um probably an unenviable position for them to be in. And it wasnít until there was a second group from (unclear) that cloned a version of our system and started doing...going off in their own direction, really doing their own work for it themselves. And they merged um (unclear) and, and a combination of our system and their system turned into the development system, which was the standard system, which, I think, turned into System 3. And from then on they were fairly separate, and just took ideas rather than massive amounts of code.
MSM: I was going to say, when did it wind down for you, or did it ever?
Thompson: Um at some point it got ponderous, and I decided to get out from under it. And I did, I took a sabbatical at Berkeley, just disappeared for a year. And you know, if youíre here, you are indispensable; if youíre not here, you know, (laughing).
MSM: Thatís why Iím going away next year...
Thompson: Yeah so I went away for a year and when I came back it was just...I was just not in it anymore, not in the mainstream and not needed for(unclear). It was actually planned.
MSM: So what did you go off to do?
Thompson: Uh, (unclear) I actually did a lot of system development for them, it was the start of their ascension. Most of the names you know were in fact students of mine.
MSM: I see...(unclear) the development of UCB.
Thompson: Yeah...when I went there they had one UNIX machine . It was (unclear) statistics, that theyíd run it three days a week and statistics would run it three and theyíd fight over it the last day, or run maintenance on it that day. When I left it was in almost every one of their courses. They had about three machines on order, they had three or four machines installed, real confident students.
MSM: Looking back (!phone!)...just one more, because we have been going for a long while. Um you look back on it...is there something fundamental youíd like to have done differently?
Thompson: Um, I had regrets about a couple things. Um probably the biggest thing is I had some little lash-up applications, uh with remote shells, and distributed machines where you had a shell where you could execute pipelines with different parts of different machines. Um and I never pursued that, and because of it thereís these, you know, the stuff that we laid out and distributed was hardly ever changed; our file system, the composed read/write, the pipes, and you know all these things. The stuff that we left fuzzy, got done either poorly or multiple ways and their different, different systems and their addressing strange (unclear). And I think that if we had worked harder, had the insight or done more in networking in those days then you know things would be different. But uh it was, itís, itís nothing that I could have predicted, it had to be in retrospect to think of that. I dabbled in that area, and just never, never installed it, never distributed it as part of the system.
MSM: Itís always easy to be wise after the fact, and hard to imagine how one thought otherwise about certain things, but, one thing about these timesharing systems of the late-60's...operating systems and the notion of making them time-sharing was the shared notion that computing was going to become generally available, it was going to be through a plug in the wall to a large central computer, when, in fact, thatís not how computing has become available to people at all. Were you still thinking in the late-60's early-70's when you were doing UNIX about using a central computer, or did you...
Thompson: Yeah, in fact, I still think that way. Um in those days I typically thought of using one central computer, um mainly because of the expense and stuff like that, you know, RAM, and lack of networking although Iíd done the interconnection of UNIXes to some extent. But now I am thinking of still central computing, but with multiple central computers, designing new operating systems that way, um I think that there... the comp centers maintain hardware better than people do in their offices. And um if you compute in your office, like a workstation mentality notion, that you are always stuck with a small machine, that you canít afford, on that machine, to put a hundred megabytes, but uh, on a central machine you can, and that for short periods of time you can use the 100megabytes... um for you applications. And that the sum of these, the distributions of how much memory you need and how much I need averages out much quicker on big machines than they do with one person with one machine on your desk. So I think that small amounts of sharing in central computing is better for everyone; itís cheaper and everybody gets better service and better cycles than they do on their machine on their desk. So I still feel that way.
MSM: But it would be a distributed system.
Thompson: Yeah, it would be several of these uh compute servers, call them whatever they are, but theyíre just there for these economies of scale, and you get these economies of scale with a few users.