How the Internet was born: the ARPANET Comes to Life

This essay is the third of a four-part series, which commemorates the anniversary of the first ever message sent across the ARPANET, the progenitor of the Internet on October 29, 1969 - Read: Part 1, Part 2 and Part 4.

After Charles Herzfeld, the Director of ARPA, gave his blessing to commence the first stage of the ARPANET project, Robert Taylor began circulating the plan to some ARPA’s contractors. Just like Baran, Taylor soon found that good ideas are not always easy to sell. The initial reaction to the ARPANET was one of suspicion: “Most of the people I talked to,” said Taylor, “were not initially enamoured with the idea”. Many feared that a network would be “an opportunity for someone else to come in and use their [computing] cycles”. However, the project finally started coming to life when Lawrence G. Roberts, a very talented researcher from the Lincoln Lab at MIT, was chosen as the ARPANET project manager.

Only 29 at the time, Roberts had already worked on another groundbreaking experiment in computer networks. In 1966, with his colleague Thomas Marrill, Roberts used the Western Union Telephone Line to link two super computers across the country (the Q–32 at the System Development Corporation in Santa Monica, California, and the TX–2 at the Lincoln Lab, in Lexington, Massachusetts) in a time-shared environment.

The experiment was a “testing environment” that aimed to verify whether it was possible to build a computer network on a continental scale “without enforcing standardisation”. Since the network was built “to overcome the problems of computer incompatibility”, it would have been ill-advised to enforce a standard protocol “as a prerequisite of membership in the network”. Instead, Roberts and Merrill argued, for a network to work efficiently, it required maximum flexibility.

If a protocol which is good enough to be put forward as a standard is designed, adherence to this standard should be encouraged but not required.

The idea of flexibility is an important building block of the Internet we use today. It allows the development of different networks, with different standards, all of which are able to connect with each other. This variety of networks and the lack of enforced standardisation, in time, have become a very important asset of the Internet. It has also made it a much more difficult environment to control as a whole.

From a purely technical perspective, Marrill and Roberts’ experiment proved that it was possible to connect different computers and share resources between them. However, both researchers faced the same problem Baran had foreseen for his distributed adaptive network: “dial communications based on the telephone network were too slow and unreliable to be operationally useful”.

As such, one of the important lessons learned from the experiment with the Q–32 and the TX–2 was that, in order to improve the speed and reliability the network, the programmers had to use packet–switching.

Larry Roberts was an advocate for “knowledge sharing”, believing that “sharing” everyone’s work was the only way to advance knowledge. Inspired by Licklider’s ideas of the Intergalactic Network, Roberts was fascinated by the untapped potential of a wide communication network linking symbiotically people, machines and resources. As Roberts explained:

At that point [in 1962], we had all of these people doing different things everywhere, and they were all not sharing their research very well. So you could not use anything anybody else did. Everything I did was useless to the rest of the world, because it was on the TX-2 and it was a unique machine. So unless the software was transportable, the only thing it was useful for was writing technical papers, which was a very slow process. So, what I concluded was that we had to do something about communications, and that really, the idea of the galactic network that Lick[lider] talked about, probably more than anybody, was something that we had to start seriously thinking about.

As soon as he was appointed the ARPANET Program Manager, Roberts began sketching out plans for the network. His starting point was the lesson he learnt while working with Marrill on linking the Q–32 and the TX–2 computers. He drew several sketches of the possible topology and, after discussing the network specifications with many fellow researchers – who included, among others, Licklider, Kleinrock, Donald Davies, Davies’ representative at the Gatlinburg Symposium, Roger Scantlebury, and Baran in Santa Monica – Roberts came up with two indispensable features for the network: a computer interface protocol that all 16 research groups participating in the project could accept; with the capacity to support the estimated 500, 000 packets of traffic per day between the 35 computers that were connected to the 16 hosts.

As originally envisioned by Baran, the ARPANET was set to be a fully distributed network that made use of routers (small computers called Interface Message Processors - IMPs) at every node to speed up communication between computers. Each router had four critical tasks to accomplish:

• to receive packets of data from both the computers connected to it,

• break the message blocks into 128 byte packets, or 1024 bits (In his study of packet-switching, Donald Davies theorised that “the length of a packet can be any multiple of 128 bits up to 1,024 bits”. The 128-bit unit length guaranteed some measure of “flexibility to the size of packets” without ever overloading the computer while handling them.

• add the destination and the sender address, and

• use a “dynamically updated routing table”, or an updated map of the routes available in the network (“considering both line availability and queue lengths”) to send the packet over whichever free line was currently the fastest route toward the destination.

As in Baran’s distributed network, at each node, Roberts wrote, the “minicomputer would acknowledge it and repeat the routing process independently”.

On July 29, 1968, ARPA issued a “request for quotation” (RFQ) to several companies in the computer sector to build the network switches (the IMPs).

Some major companies, including IBM and Control Data Corporation (CDC), declined the offer, on the ground that packet–switching would never work. Others responded with detailed proposals. At the end of the day, the two best contenders for the contract were Bolt, Beranek and Newman (BBN) and Raytheon. The former was a small company, the latter a major Defence contractor. Normally, Raytheon would have been the favourite to win this kind of contract. Yet, contrary to the Department of Defence logic, but in line with ARPA’s unconventional approach, in January 1969, BBN was awarded the $1 million contract to build four IMPs for a four-site network by the end of that year. The success of BBN’s bid was a clear sign of the anti-bureaucratic nature around which the Internet was originally built.

However small, BBN was, in the words of one of its most famous researchers Robert Khan, “the cognac of the research business, very distilled”. It was a sort of haven where people like Licklider worked, where dozens of graduate students and faculty members from either Harvard or MIT, free from any university duties but research, were encouraged “to do interesting things and move on to the next interesting thing”. They weren’t required to try “to capitalise on them once they had been developed.”

Moreover, unlike the other bidders, Frank Heart (Head of the Computer System Division at BBN) and his team submitted a 200-page detailed proposalwith flowcharts, calculations and tables explaining how the IMP network would work.

Certainly the well-crafted proposal was an important element in BBN’s winning bid, but it was not the only reason. In the decision taken by ARPA’s committee to award the contract to the team led by Frank Heart, two factors were decisive. First was Roberts’ personal acquaintance with many of the researchers at BBN. Some of them like Heart and Kahn had already informally participated in the early development of the ARPANET project. Licklider, who regularly collaborated with Roberts, also had strong ties with BBN. The second factor was Roberts’ dislike for bureaucracy: as per the style of major defence contractors, Raytheon’s proposal was very complex and presupposed an even more complex and multi-layered team in order to manage it.

In Roberts’ experience, Raytheon’s weighty bureaucratic structure would have not only made things more complicated, it would ultimately slowed down the whole project as well. Dealing with companies like Raytheon meant wasting half of your time trying to find the right person to talk to about any problems the project encountered. On the other hand, the BBN team was small and simple: Frank Heart was the head of the team and the whole communication process between ARPA and BBN only required a telephone call between Roberts and Heart.

The first four nodes of the ARPANET Network were the University of California Los Angeles (UCLA), the University of California Santa Barbabra (UCSB), the University of Utah, and the Stanford Research Institute (SRI).

The first computer was installed at UCLA September 1, 1969. UCLA was chosen because of Leonard Kleinrock and his ARPA’s funded Network Measurement Center. The centre focused on analysing and measuring the network traffic, as well as producing relevant statistics to be used in the implementation of the network. Stanford was selected because of Doug Engelbart’s Augmentation of Human Intellect project.

Engelbart was already an eminent figure in computer science (he is most renowned for the invention of the mouse). His work on developing a series of tools (a database, a text–preparation system, and a user–friendly interface messaging system) was vital for Roberts to make the network more user–friendly. The first connection between UCLA and SRI was the product of Kleinrock and Kline’s experiment on October 29, 1969. The first message ever sent over the ARPANET took place at 2230 hours. It was a message transmission between the UCLA SDS Sigma 7 Host computer and the SRI SDS 940 Host computer.