Friday, March 21, 2008

Computing History Matters

In a few milliseconds my brain made the connection and I had one of those "so that is where it came from!" moments.

When I was in Silicon Valley last week, I had the privilege of taking a tour of the Computer History Museum. One of the stops was in front of these hanging grids:

I had no idea what they were. My friendly tour guide, Fred Ware, told me that I was looking at "core memory." It was an early form of memory technology that used small holes, or cores, through which wires were inserted. This setup allowed the machine to control the polarity of the magnetic field created and thereby store data inside of them.

When I first started programming on a Unix machine in college, I would occasionally do something stupid like try to peek at the contents at memory address zero which would do bad things, cause a crash, and give me a "core dump." I had always assumed that the "core" referred to in the error message was an adjective for "main," but when I saw real "core memory," I realized that I had misunderstood the term for years.

Our industry is full of terms that seem awkward unless you know their history. Even basic things like the "print" method in almost all languages doesn't make much sense unless you see that it goes back to the days when you connected to a computer via a TeleTYpe (TTY) machine that printed its result on rolled paper.

This isn't unique to computing. Human languages have quite a bit of history in their words. For example, the word "salary" comes from the bygone days when you were paid for your labor in salt. The difference with computing history is that most of it happened recently and most of the people that created it are still alive.

Great stories are just below the surface of things you use each day. While I knew that Linux is a Unix-like operating system that was started by Linus Torvalds when he was a student the University of Helsinki, I didn't realize until last week the interesting story of "Unix" itself. The name probably came about as a result of a silly pun. It was a "castrated" version of MULTICS. The MULTICS operating system was developed in the mid 1960's to be a better time-sharing operating system for the expensive computer time of the day. Perhaps frustrated with the complexity of MUTLICS, Ken Thompson wrote a rough "simplified" version of it in a month. Thus, the story goes, it was "UNIplexed" and simpler where MULTICS was "MULTiplexed" and complicated.

Unix's history gives color to the people that created it. I'm certainly no Ken, but I can relate to the feeling that some code seems overly complex and feel the itch to rewrite it to something simpler. It's neat to see that his diversion worked out so well. It's also a testament to the people behind MULTICS that most of the central ideas in our "modern" operating systems trace their origin to it.

Programming languages tend to have a story as well. Usually the most interesting is the philosophy that drove their creation. A good representative sample is the philosophical gap between Simula and C++.

Simula is regarded as the first object-oriented programming language. It too was developed in the golden era of Computer Science in the 1960's. As stated by its creators:

"From the very outset SIMULA was regarded as a system description language"

This naturally led to the bold design objective #6:

"It should be problem-oriented and not computer-oriented, even if this implies an appreciable increase in the amount of work which has to be done by the computer."

It was so bold, that they had to tone it down a bit. They still had 1960's hardware after all:

"Another reason for the [de-emphasizing the above goal] was that we realized that the success of SIMULA would, regardless of our insistence on the importance of problem orientation, to a large to a large extent depend upon its compile and run time efficiency as a programming language."

Regardless, it's telling of its philosophy. As David West writes:

"Both Parnas and the SIMULA team point to an important principle. Decomposition into subunits is necessary before we can understand, model, and build software components. If that decomposition is based on a 'natural' partitioning of the domain, the resultant models and software components will be significantly simpler to implement and will, almost as a side effect, promote other objectives such as operational efficiency and communication elegance. If instead, decomposition is based on 'artificial,' or computer-derived, abstractions such as memory structures, operations, or functions (as a package of operations), the opposite results will accrue."

As David continues, the philosophy of Simula was:

"... to make it easier to describe natural systems and simulate them in software, even if that meant the computer had to do more work."

Bjarne Stroustrup took a different approach with C++:

"SIMULA's class-based type system was a huge plus, but its run-time performance was hopeless:

The poor runtime characteristics were a function of the language and its implementation. The overhead problems were fundamental to SIMULA and could not be remedied. The cost arose from several language features and their interactions: run-time type checking, guaranteed initialization of variables, concurrency support, and garbage collection..." (Emphasis added)

"C with Classes [precursor to C++] was explicitly designed to allow better organization of programs; 'computation' was considered a problem solved by C. I was very concerned that improved program structure was not achieved at the expense of run-time overhead compared to C." (Emphasis added)

This simple historical account alone can probably guide you to answers to things that seem odd or annoying about C++. It also reveals a larger truth. Usually if someone way smarter than me, like Bjarne, creates or does something that I think is weird, there's probably an interesting historical reason behind it. In C++, the driving force was almost always performance. I find it amusing that a lot of the "new" ideas in languages and runtimes are just bringing back things from Simula that C++ took out.

Practicalities like performance often hinder the adoption of otherwise superior technology. Betamax lost because it only held an hour of video compared to VHS's three. Not being able to store a full length movie is a real problem. The QWERTY keyboard layout was designed in the 1860's when it was important to separate common letter pairs like "th" in order to prevent typebar jamming. When computer keyboards were designed, the designers simply copied the already popular QWERTY layout to ease adoption even though other, potentially more efficient, layouts were available. Sometimes a "good enough" solution wins because the better approach isn't worth the transition cost.

Popular history often misleads people into thinking that things came easily to the innovators. The real stories offer hope because they demonstrate that you too can make history if you're willing to persevere.

Google's founders Larry Page and Sergey Brin were turned down when they tried to sell their core PageRank algorithm to AltaVista and Yahoo. One of my favorite stories is that almost no one believed Bob Kahn, the inventor of TCP, when he advocated that congestion control would be important on a large network. It wasn't until the first 12 packets were sent out and congestion actually occurred did people finally agree that it could be a problem.

Probably the biggest misconception is how Isaac Newton came to explain gravity. Scott Berkun writes:

"It's disputed whether Newton ever observed an apple fall. He certainly was never struck by one, unless there's secret evidence of fraternity food fights while he was studying in Cambridge. Even if the apple incident took place, the telling of the story discounts Newton's 20 years of work to explain gravity, the feat that earned him the attention of the world" (Emphasis added)

Tim Berners-Lee, the primary man behind the web, tells a similar story:

"Journalists have always asked me what the crucial idea was or what the singular event was that allowed the Web to exist one day when it hadn't before. They are frustrated when I tell them there was no Eureka moment. It was not like the legendary apple falling on Newton's head to demonstrate the concept of gravity... it was a process of accretion (growth by gradual addition)."

What's the history lesson? Don't worry if you never have an apple falling on your head moment. Newton didn't have one either. Things take a lot of hard work and perseverance. True stories of innovator's persistence go on and on. While you're slogging away, don't worry about mistakes too much:

"Anyone who was never made a mistake has never tried anything new." - Einstein

Simply learn from them and move on.

Although doing something insanely great might take a long time, it demonstrates how important people are in our industry. People are the most important part of any industry because it is through them and their stories that history is created. People are especially important in computing. Unfortunately, some of their most interesting stories are practically unknown by most:

  • Alan Turing wrote the paper that effectively started computing as we know it.
  • William Shockley practically put the "silicon" in "Silicon Valley,"
  • Our modern Internet exists in large part because of the fascinating story of J.C.R. Licklider who used ARPA funding to work on his dream that everyone should have a personal computer connected to an "Intergalactic Computer Network."
  • Bob Taylor, the man that led the wildly successful Xerox Palo Alto Research Center (PARC) inspired creativity in the great people that worked with him by creating environment that encouraged exploration and laughter.
  • Dave Cutler was in large part the reason why we have the much more reliable NT kernel running on our computers and aren't stuck with "toy" Windows 9x one.

Their stories are full of little gems. You sometimes get to see what drove their curiosity. Bob Barton, arguably one of the greatest hardware designers ever, made the comment "I often thank IBM because they gave me so much motivation to do better." I'm sure that Google's founders could say the same thing about AltaVista of the 1990's.

I think that one of the reasons for the plummeting enrollment in Computer Science majors is that there is a lack of understand of people and their stories in our field. This leads to the perception that a career in computing will result in a "social death." I might be biased, but I think we have great stories. If you haven't visited it yet, check out the videos on the Computer History Museum section on YouTube. They're quite interesting.

I'll close with one lesson from computing history: don't be afraid to dream big. When Vint Cerf had to put an upper limit in 1977 on the number of addresses for what would become the Internet, he probably thought that 4 billion addresses would never be used, but that's the "address exhaustion" problem that we're facing now. Be careful when creating something, it just might exceed your wildest dreams.

Why do we as an industry largely ignore our history and rarely mine its rich stories? Do we think we're moving too fast for it to be relevant? I think that it's important to understand our history exactly because we're moving so fast. It seems that learning principles that are timeless or much longer lived is more valuable than keeping up with the deep intricacies of the latest technology of the day that won't matter in a year or two.

What's your favorite piece of computing history? Who are the people that you remember? What are their stories?

P.S. Special thanks to Alan Kay for sharing some of his stories with me last week.


mattl said...

Linus did not write an OS. He wrote a kernel for one, typically GNU.

John said...

At one time I worked at a company that used PDP-8e computers with core memory. It came in handy for those systems that wouldn't boot to tape. You would move the memory board to a working system then load the tape diags, from tape of course. After transferring the memory board to the other system you would just start up the diagnostic up from the front panel.

They would hold a program for weeks without power. Aside from their cost and size, Core memory worked quite nicely.


Jeff Moser said...

mattl: Good point. I didn't mean to imply that he wrote the whole OS. You're right, he did the kernel where the rest was GNU as mentioned in the Usenet post I linked it.

john: interesting! Thanks for sharing.

Anonymous said...

I'm currently studying os development, and I'm very interest in the history of computer and os, do you guys know of any good books on the subject? Thanks! (wliao)

Anonymous said...

The US Defense Department dared to dream big in the way they snatched all those IP adresses

Des Courtney said...

If you have a Mac-focused bent, there's an entire website dedicated to the creation process for that platform...

I left a post on reddit that have more general links as well...

Thomas Wahl said...

Just a nit, but...

TTYs did not print on tape.
They printed on rolled paper.
They punched paper tape which
corresponded to what was being printed, and the paper tape could
be read to replay the printout.

I used a TTY in high school for many hours, using a GE-235 timesharing sytem running Darmouth Basic
in 1967-1968.

T. Wahl

Christopher Bennage said...

Fantastic post. I really appreciate all the effort (research and cross-linking) that you put into it. Thanks!

Jeff Moser said...

Anonymous #1: I'm currently reading "The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal" by M.Mitchell Waldrop. It's sort of what you're asking for.

Anonymous #2: Even still, it was several orders of magnitude off. I can feel Vint's dilemma as most of the address bits would be zero back in the days of even smaller pipes. It'd be nice if moving to IPv6 were easier now.

Des Courtney: I didn't know about Thanks for that and the other links.

Thomas Wahl: Thanks for the clarification. I'll update my post. I also liked the TTY in high school comment. What types of programs did you use on that machine? What were the quirks? What did you like about it?

Christopher Bennage: I appreciate the comment. I sometimes worry I hyperlink too much, but I like that style in other people's posts. It lets you dig deeper if you want to.

ohxten said...

mattl: You know what he meant. I think we're all tired of Linux fanboys making that point clear over, and over, and over... unless you're talking to Richard Stallman, nobody really cares.

Yes, I use Linux.

Nice post anyway, Jeff.

Anonymous said...

I used an ASR-33 Teletype in high school also, using Coursewriter BASIC on an IBM 370. We also used IBM 029 keypunch machines to submit batch FORTRAN IV and COBOL programs. Now there's some physical evidence of history - look at the hole patterns on punch cards, and compare to the EBCDIC character set used on IBM and Burroughs systems. That explains why there are "gaps" in the sequences.

The TTYs were connected through dial phones with built-in modems; you'd dial the phone, set down the receiver, and then pull up one of the receiver buttons after you heard it connect. After hours, they would put locks on the dials. More than once I was caught connected with the phone locked - the teachers didn't seem to realize that you could dial a pulse line by tapping the pulses using the receiver buttons. Then I sent a letter to Coursewriter and got a copy of the BASIC system manual. It included the default superuser ID and password, which the school district of course hadn't changed...

Then I got a part-time job my senior year programming physics apps on an Apple ][ for the science department. More BASIC and some 6502 assembly.

Boy, it was a lot harder to play with computers back then!

All through school, I kept hearing about how I'd move on to "real" computers later. My first serious machine after that was a Univac 494 - core memory, drums, no disks, etc.

I'm forever grateful that I had the chance to work on such old systems, because it did give me a much richer understanding of why things are the way they are than I find in any of my coworkers nowadays.

Anonymous said...

Nice article. Just to throw it out there though, there's some disagreement on the QWERTY story.

"qwerty myth" on google gives a few others.

Jeff Moser said...

Anonymous: Great link! After looking at that link and doing some more researching, I've significantly revised the statement. Thanks for pointing me to the deeper look at the story behind the layout.

SimonTeW said...

One thing that I frequently marvel at is how memory has come down in size.

My Mum started as a computer operator in a data centre in 1999. Occasionally I would get to see around the centre. They had huge, standalone disk drives. Each one took a stack of about half-a-dozen 14 or 18 inch disks on a common axis, that could be plugged in and out of the drives.

These drives were about the size of two dishwashers, back to back, or perhaps a small dining table. Their storage capacity was 100 MB.

These days I can easily get 4 GB on a pen drive that is so small I can forget where I put it down.


Daniel Watkins said...


If you're using Linux, then I'm typing this message on a CPU.

Mike Petry said...

Sorry to pry but what was going on in SV?

Jeff Moser said...

Mike: Very cool things with lots of smart people -- most with PhD's. It's a project that could significantly change thinking about computing and what types of cool things get discovered and created. If you read this post, you can sort of think of it like finally going beyond the golden age of the 1960's I talk about here.

It'll definitely be topics of upcoming posts. It just takes time to get enough things in place and research to get to that point.

Sunrise Programmer said...

And an even neater thing about core memory: it was the hot new memory at the time. It was very cheap and very fast.

The alternatives? Well, low-end computers used their magnetic drum (like a disk, only with less data and a bit faster) as their main memory. Others used various types of delay lines (put a bit in, get it out a while later, and cycle it around). For a while people used what are essentially TV tubes: turn a dot on, and then you can measure if it's on or not.

All of those, in turn, were the hot new technology, totally beating the old technology: just use vacuum tubes at the cost of a couple of tubes per bit.

And along side of all of these: seemingly dozens of 'other' memory that people dreamed up. IBM even made the most gigantic dinosaur of a beast: writing involved pointing light at some undeveloped film. The machine would them automatically develop and store the film for later retrieval.

The moral of the story? When technology is new, people cast about for whatever they think they can make work. They often can make it work, and then it's just up to history to pick which ideas work cheaply and reliably (CMOS memory) and which don't.

Jeff Moser said...

sunrise programmer: Nice summary! History is often written by the winners and in hindsight things seem too obvious. It's good to know that the past, just like the present, is often certain and full of rabbit trails. However, as you mentioned, people can be creative with anything.

Anonymous said...

Great post. Enjoyed it immensely. A couple of "war stories" for you;

I remember chatting to my fellow programmers in the mid-1980's about a rumoured new PC hard disk that could hold 10 MB. Wow! One colleague said "You know, one day your watch will have that storage capacity..." Good call - wrong implementation.

We used to print out our programs on 132 column "music paper" (remember that?, I miss it). Anyway, it was printed by a chest-high dot matrix impact printer that was so loud that it had to be enclosed in a perspex case - I think it was an IBM 1403 or somesuch - It had a "feature" that - if it ran out of paper the perspex hood would automatically rise to allow a paper change. Well, One day we were all chewing the fat in the office when we heard the tell-tale noise of the hood being raised. No problem you might think. Problem. One of my colleagues had left his stack of punched cards (his source code) on top of the cabinet. He moved fast but not fast enough - he spent the next hour or so sorting his source code! I wonder if he used a bubble sort?

Finally, in answer to a previous anonymous - I found "The Soul of a New Machine" by Tracie Kidder far and away the best and most readable account of the creation of hardware and software. It describes the early days of DEC.

Thanks again for your post and for the thoughtful comments.

Jeff Moser said...

Anonymous: Loved the punched card story! I've heard that it was a common practice to take a marker to the side of the stack and create a sloped line. That way, if you dropped them, you could quickly put them back in order by looking to see if everything "lined up."

I'll have to take a peek at Kidder's book. Thanks for the recommendation. You might be interested in my Licklider post about "The Dream Machine."

John said...

I really found Kidder's book a great read. Then again, I'm "old school", started programming way back in the 70's in high school: )