The Net that Shaped the World   Madhukar Shukla     The Internet is now becoming ubiquitous. But many would remember that its roots lie in a mid-60s US Defense project, ARPANET. Those were the Cold War years, and paranoia reigned. One of the major concerns for US Defense was to develop a “survivable communication” system, which would withstand any surprise attack by the USSR. The Rand Corporation - a government funded non-profit outfit for R&D on technology of strategic and military significance - was entrusted the task of developing such a system. The project was conceptualised by a young Rand engineer, Paul Baran, in a series reports written during 1960-62, and published under the title On Distributed Communications in 11 compendious volumes in 1964. However, the network, initially with just four switching nodes, became operational only in the fall of 1969. ARPANET was, of course, a significant strategic and technological breakthrough. But the concept and design of this system were far more than just that. ARPANET was a conceptual breakthrough, which defied and challenged the conventional wisdom. But for Baran’s rather proper background, one would be tempted to say that in the guise of technology, it was a political statement for the counterculture that prevailed in the US during those times: it was “a drastic change of basic architecture” of what a communication system was and could achieve. Its effectiveness relied on paradoxes: it aimed to help the army maintain “proper command and control” under wartime situation by offering a system which had no centralised control; it was one of the most robust and efficient system which relied on inefficient and distributed, but interconnected, parts; and, it created efficiencies in communication by building redundancies into the system. Central to ARPANET was the concept of, what Baran termed as the “distributed communication”. Conventional communication systems follow a centralised and hierarchical structure. A phone call made to another city first goes to a local exchange, from where it is then transferred to the regional/ state hub. The regional hub then switches it to the local exchange of the target city, which in turn connects it to the dialed number. In this system, each subscriber is linked to only one exchange, and each exchange - which switches the calls - services many users. Baran reasoned that in case of a surprise strike, it was the centralised hub, which made the system most vulnerable. Destruction of one central exchange could cut off thousands of users from the network. The obvious solution was to do away with the necessity of a hub to route the message. Instead, Baran proposed a system, which would have many switching nodes, and each node was connected to many other nodes - something akin to a fishing net. In this system, a message did not follow any pre-determined path; when a message reached a particular node, it could be directed to any other node, depending on the traffic density on that particular route. As Baran noted: “There is no central control; only a simple local routing policy is performed at each node, yet the over-all system adapts.” This was a major departure from the conventional logic of communication networks, which relied on the efficiency of routing a message through the shortest possible distance. Baran, however, saw a larger efficiency in this inefficiency. While using a longer than required route made the transmission of a single message inefficient, this method increased the overall efficiency of the network. In the conventional routing system, traffic density across different routes could fluctuate to the extremes at different times. At a particular time, certain routes may be more in use, and therefore, choked, while, at the same time, other routes may remain grossly underutilised. By routing the messages through a longer, but available, routes, the system optimised the load across the network. Baran’s proposal for a distributed communication system was actually not entirely original. AT&T was already running the AUTOVON network with ten switching nodes for the Defense Department since early 60s. There was, however, a significant difference in Baran’s proposal from the AT&T’s existing network. AUTOVON was a distributed network only in its design, but not in its operation: the ten switching nodes of the network were still controlled from a centralised operations centre. The monitoring and control of the system, and traffic level analysis was done manually by operators from this centre, who would take the re-routing decisions and then communicate them to the operators at the switching nodes. Baran’s proposal for the net, on the other hand, relied on completely autonomous nodes, which would automatically take the re-routing decisions without any need for human intervention at all. In his design, the “intelligence required to switch signals to surviving links is at the link nodes and not at one or a few centralised switching centers.” But how does one build this “intelligence” in the nodes? The nodes of the distributed communication systems at that time used analogue technology; intelligent switching nodes, on the other hand, required computers and digital transmission lines. Though AT&T, Baran (Gilder, 1997) described much later, ”did have digital transmission under examination, it was in the context of fitting directly into the plant by replacing existing units on a one-for-one basis. A digital repeater unit would replace an analog-loading coil. A digital multiplexer would replace an analog channel bank—always a one-for-one conceptual replacement, never a drastic change of basic architecture.” Baran’s design of the network relied on technologies, which were just gaining foothold. He envisioned the communication system as consisting of a "network of unmanned digital switches implementing a self-learning policy at each node... so that overall traffic is effectively routed in a changing environment." That is, the decision-making about the best path to any destination - and the updation of information as the network conditions changed - must be done at the switching nodes itself. Such computerised switches had never been designed before, and the digital computer technology was itself in its infancy - the first digital minicomputer, Digital Equipment’s PDP-8 was launched only in 1965. Moreover, Baran’s distributed network could work only if the message passed through a number of short links, instead of a few long ones. The conventional in-use analogue technology put limitations on the number of links through which a signal could pass, since the quality of signal deteriorated with each link. Baran proposed the use of digital transmission lines, which were supposed to transcend this limitation. But digital transmission was still a novelty: AT&T had started developing its T1 digital trunk lines in mid 50s (which became commercially available only around 1962), and the experts were still skeptical about Baran’s claim that an all-digital system would overcome the limitations on number of links per call. But perhaps, the most radical of Baran’s proposal was the paradox of using unreliable, inefficient and redundant - but interconnected - parts for creating an overall efficiennt, robust and reliable communication system. This paradox was built into both the hardware technology, which transmitted the message, as well in the way the message was transmitted. On the hardware front, Baran departed from the standard practice of the phone companies, which tried to increase the reliability of the system by making each component of the system as reliable as possible, with lowest error rate. Baran reasoned that such high-quality links - and so many in numbers - would make his system too expensive to be economically viable. Instead, he chose to build the system with low quality communication links, but provided redundancy (i.e., more than required number of links) in the system to compensate for failures and errors. He justified this by observing that, “reliability and raw error rates are secondary. The network must be built with the expectation of heavy damage anyway.” The other part of Baran’s inefficient-in-parts-but-efficient-as-a-whole proposed system was the manner in which the messages - whether digitised voice or computer data - were to be transmitted. In the conventtional telecommunication systems each call was assigned an exclusive bandwidth frequency and a link for its entire duration. Baran, on the other hand, proposed that instead of sending messages of varying sizes, they can be broken down and transmitted in uniform blocks of 1024 bits (a technique, which later came to be known as “packet switching”). If a message was large, it would be broken into multiple equal-sized packets, and the multiplexing station (which connects the user to the network) would add a header specifying the sender’s and receiver’s addresses to each packet. It would then transmit each packet through different routes. When these packets reached the sender, the local multiplexer station would reassemble them in proper sequence. Obviously, this method increased both the complexity and costs of the transmission process. Since a separate header was assigned to each outgoing packet (instead of one per message), it increased the amount of data that had to be transmitted through the network. Moreover, it created additional task for the system’s computers to assign headers to packets and to reassemble them. Since packets would travel through different routes, they would not arrive in the correct sequence, and the system had to provide for reassembling them in proper order. But for Baran, these redundancies and inefficiencies were a small price to pay for larger advantages. An obvious strategic benefit of splitting the message into parts and transmitting them through different routes was the security against the message being intercepted by enemy spies. But packet switching had two even greater technical and economic advantages. Firstly, the fixed-size small packets simplified the design of the switching nodes, and therefore, simple inexpensive computers could be used to man the system. Given the state of technology of the time, this was absolutely critical to the success of Baran’s proposal. Secondly - and this is what made ARPANET a secular communication medium, stimulating its exponential growth - packet switching enabled users with different devices (telephones, computers, microwave, etc.) to hook into the system. Each of these communication devices transmitted information at different and fixed data rates (i.e., number of bits of information per second which can be transmitted on a given link), and therefore, in conventional communication systems they were incompatible with each other. Packet switching, however, enabled rearrangement of data flow from different devices to the outgoing links, making it easier for devices transmitting at different data rates to share a common link to the network. As Baran noted, “...standardized data blocks permit many simultaneous users, each with widely different bandwidth requirements, to economically share a broad-band network made up of varied data rate links.”   Over a period of time, the Net expanded, first to connect universities research labs, and then… well, the rest is history.