Chapter 2

How Web Browsers and Servers Work Together

by Mark Brown

CONTENTS

• How the Web Works

  • The Client Computer

  • The Server Computer

  • Communication: Requesting Information via URLs

• Servers and HTTP

  • What Servers Do

  • HTTP

  • An Overview of Popular HTTPD Server Programs

• Browsers and HTML

  • What Browsers Do

  • HTML

  • An Overview of a Few Popular Web Browser Programs

They say that the camera never lies. A photograph is a record of a real scene as it was captured by the camera's lens.

On the other hand, two artists never paint the same scene the same way. Every element is subject to the artist's interpretation. One artist might paint a photo-realistic image that's almost sharper than reality; another might render an abstract smear that communicates mood more accurately than it portrays the scene. They might move, add, remove, or alter objects or change lighting to suit their depictions of the scene.

The World Wide Web works more like painting than it does like taking photographs. A Web server "serves up" a page over the Internet, and a Web browser program interprets it. The results may be almost as different as the two painting styles described above. This is the central, surprising truth about the World Wide Web: what you see on a Web page is described, not defined, by its HTML code. It is up to the user's browser program to render it in its final form.

Many new Web developers are shocked to discover that they have so little control over the final appearance of their pages. But, for better or worse, that's how the Web works. If you want to create Web pages that can be viewed accurately by a wide range of WWW users, you're going to have to be aware of the different server and browser programs that are out there and how they work together.

In this chapter, you learn about the following:

• The steps that occur when you view a Web page

• The difference between HTTP and HTML

• What a URL is

• How browser programs differ in the way they display Web pages

• Which server and browser programs are the most popular

How the Web Works

The World Wide Web has a client/server architecture. This means that a client program running on your computer (your Web browser) requests information from a server program running on another computer somewhere on the Internet. That server then sends the requested data back over the Net to your browser program, which interprets and displays the data on your screen. Figure 2.1 shows the whole process graphically. The following steps explain the process:

Figure 2.1 : This is what happens when you view a page on the World Wide Web.

1. You run a Web browser client program on your computer.

2. You connect to the Internet-at work or school via a direct T1 or T3 line; at home via a modem dial-up connection to an Internet service provider (ISP).

3. You request a page from a site on the Web. Your browser sends a message over the Internet that includes the following:

   • The transfer protocol (http://)

   • The address, or Uniform Resource Locator (URL), e.g., www.somesite.net

4. The server receives your request and retrieves the requested Web page, which is composed in HTML (HyperText Markup Language).

5. The server then transmits the requested page back across the Internet to your computer.

6. Your browser program receives the HTML text and displays its interpretation of the page you requested.

The Client Computer

To browse the Web, you need a client computer. There are two basic requirements for this machine: it must have a connection to the Internet and must be capable of running a Web browser program.

The Internet connection can be hard-wired, or it can be a dial-up phone connection via modem to an Internet service provider (ISP). You're most likely to have the former at work or school and the latter at home. The only difference you will notice between the two is speed; otherwise, they work identically.

There are Web browser programs for just about any computer you can name, from dumb text-only terminals running on mainframes to off-brand personal computers, such as the Amiga. (I'll list and discuss the most popular browser programs later in this chapter.)

The Server Computer

On the content-provider side of things, you need a server computer. This machine has requirements similar to those of the client computer: it must be connected to the Internet and must be able to run a Web server program.
 

However, a Web server needs a more robust Internet connection than a Web client does. A server should ideally be hooked up to the Internet via a fast dedicated T1 or T3 line that remains connected all the time. Otherwise, people trying to access your Web site will often find that it just isn't there.

It is possible (though excruciatingly slow) to run a Web site on a dial-up line, especially if you can find an Internet service provider who will let you stay dialed in 24 hours a day without disconnecting you. However, you must make sure that your ISP can assign a permanent IP address to your machine-not a new IP address each time you connect. Otherwise, people won't even be able to find your site.
 

HTTPD server software is available for a wide variety of computers (see fig. 2.2). Surprisingly, server computers don't have to be very powerful; serving up Web content is simply not that demanding. More of a concern is having a multithreaded, multitasking operating system so that the server can handle several tasks at once without bogging down. Storage is a concern, however, because Web sites are notorious for growing without limit.

Figure 2.2 : The Spy Web Servers for UNIX XWindows has a rich GUI for configuration and maintenance tasks.

Communication: Requesting Information via URLs

An HTTP connection is said to be stateless. That is, no permanent connection is maintained between the server and the client. A request is made, and the connection is broken. Then a response is sent back, and the connection is broken again. This process is repeated for every request and often even for parts of a request. Programmers refer to this as a query-response model of interaction. It's the reason your browser program always seems to be saying "Waiting for reply…" in its status display line. (See the message at the bottom of fig. 2.3.)

Figure 2.3 : Web browsers let you know where you are on the Web by displaying the page's URL. Netscape shows this near the top of the display window and labels it "Location".

A browser's request for information takes the form of a URL (Uniform Resource Locator), which is also referred to as a page's address or location. URLs almost always look something like this:

http://www.somesite.net/path/webpage.html
 

Web browsers generally display the URL of the Web page currently being viewed near the top of the window (see fig. 2.3).

Here's a real world example of a URL, taken from figure 2.3:

http://www.microsoft.com/msoffice/freestuf/msword/download/ia/default.htm

The "http://" portion of the URL indicates that the browser has requested a transfer via HTTP protocol; that is, it wants a Web page. "www.microsoft.com" is the domain name of the server being queried; in this case, it's the Web server at Microsoft. The "msoffice/freestuf/msword/download/ia/" portion of the URL is the path name on the server's hard drive for the file you want. (Fortunately, this example is of one of the longer path names you're likely to run into on the Web.) "default.htm" is the name of the actual HTML file on the server; it's what is being displayed on the screen in figure 2.3.
 

URLs can reference not only Web pages, but also just about any service on the Internet, including FTP, Gopher, WAIS, Usenet, and Telnet. You can even load in a file from your own computer! Table 2.1 shows the syntax for the various types of sites that can be accessed via a Web browser.
 

Table 2.1  URL Syntax for Addressing Various Types of Internet Sites

The domain name portion of a URL may include a colon followed by a port number, like this:

http://www.somesite.net:80/path/webpage.htm

This tells the server to access the site via a specific assigned port.
 

Web page file names usually end in .htm or .html to indicate that they are HTML content files. Many home pages don't have path names or file names at all. Their URLs are in a very abbreviated format:

http://www.somesite.net

These addresses access a page that is stored in the server's root directory; that is, they don't need a path name because they aren't stored in a subdirectory. Most servers also assume a default file name for the home page, such as hompage.html or index.htm. If no page file name is specified, the server will automatically serve up the default page.

URLs can point to other types of files than HTML Web pages, of course. For example, the URL http://www.somesite.net/logo.gif would display a GIF image file. http://www.somesite.net/path/program.zip would, depending on how your browser is configured, prompt you to save the specified ZIP file to disk or would decompress the file and store the resulting program files to disk.

Sometimes you may see a URL that looks something like this:

http://www.somesite.net/cgi-bin/findit&toad+frog

Servers can include gateways, which enable them to run application programs called CGI (Common Gateway Interface) scripts or programs. These are, in most ways, just like any other programs run by a computer. Depending on the machine and the gateway, CGI-bin (bin for binary) programs might be written in C, C++, BASIC, or the PERL scripting language. An URL like the one above instructs the server to run the CGI program called findit using the data toad and frog as inputs. In this example, it would be a fair guess that the server program, findit, was some sort of indexing program and that the user had instructed it to find references in its database to toads and frogs.

Some hyperlinks may reference a relative URL; that is, one in the same path name on the same server as the page currently being viewed. For example, if the current page is http://www.somesite.net/path/thispage.html, a relative link to thatpage.html would load in the page whose absolute address is http://www.somesite.net/path/thatpage.html. This technique not only saves page creation time and space on the server, but it also makes it very easy to move all of the files associated with a Web site to a new directory or even to a new machine. Only the references to the home page need to be changed; relative URLs remain the same.

Servers and HTTP

A Web server computer runs an HTTPD (HyperText Transfer Protocol Daemon) program (see fig. 2.4). The history behind the development of HTTP is told in chapter 1, "Introducing the World Wide Web." This section will concentrate on how Web servers use HTTP to communicate with browser clients.

Figure 2.4 : NetPublisher is an HTTPD server program for Windows NT. Unlike many bare-bones UNIX servers, NetPublisher sports an intuitive GUI.

What Servers Do

When you analyze what a server does, you begin to fully realize just how much of the look and feel of the World Wide Web is in the browser program.

The browser client strips an URL down to its component parts: protocol, address, path name, and file name. From the protocol portion, it determines how it is going to interact with the server that it's addressing and how to display the data it will receive. It then calls the address contained in the URL and waits for a response from the server.

Once the server realizes that a request is coming through, it likewise checks the URL for the connection protocol (e.g., http:// for a Web page). It takes the path name and file name that it has been given, finds them on its hard drive, and sends the data off to the browser using the correct protocol.

Then it's the browser's turn again. This time it gathers in, interprets, and properly displays the data it has received.

It probably seems as if there is a lot more for the browser to do in this process than there is for the server, and in the case of a simple transaction, such as viewing a Web page, that's probably true. But there's much more than that going on in the background.

For example, if there's an error somewhere along the way-such as a request for a page that doesn't exist-the server has to send back the proper error message. If the user has requested an action that requires running a CGI program, the server has to load and run the program. This process usually means creating a custom HTML page "on the fly" that contains the results of the program's action, then sending that back to the browser.

Then, too, every data file transmitted by the server has to be properly identified by type and tagged with the appropriate MIME (Multipurpose Internet Mail Extension) data type header so the browser will know what to do with it. Most Web pages include a mix of HTML formatted text, GIF and JPEG graphics, and maybe even audio and video clips; each must be properly tagged, or the browser won't know how to interpret the pieces when they arrive.

Today's Web servers also include all kinds of esoterica-for example, data encryption and client authentication. These take up a great deal of the server's time, too.

Factor in the fact that a server is often handling requests from hundreds of clients at any one time, and you'll see that there's more than enough going on to keep it busy.
 

HTTP

The HyperText Transfer Protocol (HTTP) was designed to be quick, simple, and nonintrusive. The connection between a server and a client program (or agent) is temporary and must be reestablished for every data transfer.

The HTTP specification incorporates a whole set of methods that are used to perform the tasks associated with servicing a Web site, including information retrieval, searching, front-end updating, and annotation. The specification is open-ended, so additional functionality can be added without making the whole Web obsolete.

As discussed previously, messages are passed in a format that is similar to Internet Mail and the Multipurpose Internet Mail Extensions (MIME); gateways enable browsers to request the execution of CGI applications on the server hardware; and communication is possible with other Internet protocols, such as SMTP, NNTP, FTP, Gopher, and WAIS.

HTTP is, like these earlier protocols, a TCP/IP protocol. However, it can be implemented on top of any other protocol implementation that can communicate over the Internet or on other networks, including LANs.
 

An Overview of Popular HTTPD Server Programs

Almost any computer platform that you can name (and some you probably can't) can act as a Web server. There are HTTPD server programs for systems as varied as multimillion-dollar mainframes and PCs costing under $1000.

Which one is best? That's a question that is highly subjective, and it's certainly beyond the scope of this book. A more important question might be, "Do I need an HTTPD server program at all?"

If you usually hook up to the Web through an Internet service provider (ISP) and are thinking about setting up a personal Web site, the answer is almost certainly no! What you need is an account on somebody else's Web server, probably that of your ISP. Most ISPs offer a place for your own home page for little or nothing if you're already using their dial-up services. For example, my local ISP charges me $30 a month for unlimited Internet access and includes a 10 megabyte UNIX account on which I can keep my own World Wide Web pages.

If you think about it, this is an incredible bargain. For $1 a day-which is less than I pay for cable TV-I maintain a Web site that can be seen by anyone in the world, 24 hours a day! And they take care of all the headaches of setting up and configuring the server, maintaining the site, and handling glitches and bugs. It's great. All I have to do is develop the content for my site, write it in HTML, and upload it to my provider.
 

Even small- to medium-sized companies might want to consider having an ISP host their sites rather than setting up their own Web server. It can mean a major investment of both time and money to set up a server computer, hook it up via a permanent T1 or T3 land line, install the HTTPD software, and maintain all of that hardware and software over time. If your needs are moderate, having an ISP take care of all that for you can be a major headache reliever.
 

However, if you are developing Web pages for a college or major corporation, you'll probably want to run your own site. This is the best way to go if your site:

• s already hard wired to the Internet

• Will contain a great number of pages

• Needs to run custom CGI applications

• Will require frequent updating

• Needs to be secure

If any of these criteria describe the site you want to set up, then pick a computer platform and HTTPD server software that match your requirements.

It is beyond the scope of this book to review all of the many HTTPD servers that are currently available; nonetheless, I can discuss the ones that are the most popular. Although popularity is not exclusively determined by quality-price and compatibility are certainly major issues as well-it can serve as a good measure of which server programs are already working for a large number of sites.
 

Accurate statistical data about Web server usage is hard to come by. Most is based on random samplings of sites or on volunteer surveys, which can be highly skewed by inaccurate survey samples. But a comparison of the best data currently available on the Web seems to lead to the following conclusions about which computer platforms are probably the most popular Web servers:

• A little less than one-third of all Web servers run on Sun workstations.

• About one-third of all Web sites are running on Macintoshes or on IBM-compatible personal computers with some version of the Windows operating system. These are about evenly split between the two platforms. Twice as many of the IBM-compatible sites run NT as run Windows.

• The remaining one-third of all Web sites run on UNIX systems.

• Only about 4 percent of Web sites run on OS/2 or other operating systems.

This breakdown is shown graphically in the pie chart in figure 2.5.

Figure 2.5 : Sun workstations, personal computers, and UNIX systems split the server market into nearly equal thirds.

A little over a year ago, more than 80 percent of the servers on the Web were UNIX-based. The shift shown in figure 2.5 certainly reflects two current trends: The rapid growth of the World Wide Web has moved it beyond the confines of academia and corporate America, where UNIX is most popular, and there has been, in the last year, a proliferation of HTTPD software for platforms other than UNIX.

Let's take a look at specific servers.

One of the major reasons for UNIX's predominance has been the number of freely distributable HTTPD server programs available for it. CERN HTTPD and NCSA HTTPD have historically been the two most popular server programs on the Web, and they remain at or near the top of the list.

However, a plethora of new server programs has been introduced in the past year, and some of them are gaining market share quickly. Although there are a few new freebies-Apache, for instance-many of the fastest-growing servers are commercial packages from such companies as Netscape, Quarterdeck, and Open Market.

There are many good reasons for the fast growth of commercial Web server programs. For the price, those buying commercial servers get the peace of mind that comes with knowing that the product is fully supported, not freeware with a nebulous cloud of hobbyist developers working on it in their spare time. Then, too, these products are finished goods; they don't need to be compiled to run and are packaged with complete documentation and a full set of additional Web development software tools.

Further, commercial servers usually offer additional functionality that just isn't found in freeware servers, such as encryption and security. And finally, most of the current growth of the Web can be directly attributed to the addition of many new commercial sites. Unlike the personal and academic sites that preceded them, commercial sites aren't scared off by the $400-$5,000 price tag that commercial server software carries.

Table 2.2 lists the most popular servers on the Web, as nearly as can be determined by current data.* Figure 2.6 shows the same data graphically.

Figure 2.6 : Many of the most popular Web server software in use today are freeware, but commercial servers are gaining ground.
 

Table 2.2  The Most Popular HTTPD Server Software

There are, of course, a great many more HTTPD server programs available than those shown in table 2.2. Table 2.3 lists HTTPD server programs for a variety of platforms. To find out more about any of these servers, check out Paul Hoffman's Server Comparison Chart on the Web. This site contains a wealth of data for anyone who is thinking about establishing a presence on the World Wide Web, including hypertext links to the publishers of most of the Web server programs listed in table 2.3. Its location is http://www.proper.com/www/servers-chart.html.
 

Table 2.3  HTTPD Server Programs by Platform

Browsers and HTML

To a large degree, the Web is what your Web browser makes it. Because Web pages are written in HTML and because HTML is subject to interpretation, your browser profoundly affects the appearance of Web pages and, consequently, your impression of the World Wide Web as a whole.

What Browsers Do

Figure 2.7 shows the same Web page as viewed by three different browser programs: the all-text UNIX Lynx browser (upper left), an older Windows browser called Cello (upper right), and Netscape Navigator 2.0 (lower right). The fourth image (lower left) is the same page displayed by Netscape 2.0 with customized color and font settings. You can see how differently various browsers can interpret the same page and how significant user settings are in the outcome.

Figure 2.7 : The same Web page can be displayed in very different ways by different browser programs.

That's why it's important to make sure you have a good browser program (see fig. 2.8) if you're going to develop HTML documents for the World Wide Web. You want to make sure that your viewers are seeing the same pages you are.

Figure 2.8 : Netscape Navigator, shown here in version 2.0 for Windows 95, is the most popular browser program in use on the Web today. In this screen shot, Netscape displays a 3-D VRML (Virtual Reality Modeling Language) virtual world using the WebFX plug-in.
 

HTML

HTML is not intended to be an all-encompassing, all-powerful page layout environment. HTML describes a page's look by using markup tags to indicate the relative position of elements on the page. For example, you can specify which lines of text are headings and their level of importance. You can show where in the text an in-line image should appear and whether certain blocks of text should appear with a particular type of formatting. You can even create tables and forms. With some of the most recent versions of HTML, you can create frames in which different parts of pages are displayed, put up graphic background wallpaper, and change the color of text on the fly.

HTML cannot, however, determine which font, font size, or color will be used to display text; what the screen background color will be; how the colors in graphics will be interpreted; or any of a wide variety of other variables that are at the mercy of browser programs or the users' settings of various options in their browser programs.

"Why doesn't HTML give a page creator more control?" you ask.

HTML's main appeal is that it is easy to learn and easy to use. Ease of implementation was, in fact, the major design criterion for HTML, and it worked. A recent survey of Web page creators found that over half of them learned the basics of HTML in under three hours, and another quarter took only six hours. Most said that a good book (like this one!) and the Web itself were the only tools they needed to begin creating Web pages.

HTML was also designed to be compatible across a wide range of machines, from text-only UNIX terminals to the flashiest Silicon Graphics workstation. To a large degree, that goal has been met, too. Although there will certainly be some differences in the same Web page viewed on different machines with different browsers, the results will likely be similar enough and acceptable enough to convey the information presented in the manner intended by the page's creator.

The responsibility for making sure this happens rests squarely on the shoulders of the Web developer-you! Properly applied, HTML can make your pages look good on a wide variety of platforms. The more aware you are of the differences in Web browsers, the better you'll be able to make sure your pages look good on all (or at least most) of them.
 

An Overview of a Few Popular Web Browser Programs

So which Web browser clients are the most popular? That's one with an easy answer. The most popular Web browser today is Netscape Navigator, which is used by over 80 percent of those cruising the Web. Other browsers don't account for over four percent of the market each.

Does this mean that you can, with impunity, develop only for Netscape and ignore the rest? In a word, no. First of all, there are several different versions of Netscape Navigator out there, running on UNIX, Windows, Windows 95, and Macintosh platforms. If you want to use some of the latest and greatest Netscape features-such as frames -you'll leave behind the three-quartersor more of your Netscape audience who are, as of this writing, still using Netscape 1.1. (See chapter 15, "Netscape-Specific Extensions to HTML.") And the 20 percent of Web users who don't use Netscape Navigator are a sizable chunk of your audience too. You don't want to leave them out in the cold, do you?

Table 2.4 should give you some idea of the wide variety of client programs that are out there browsing the Web.
 

Table 2.4  Web Browser Programs for Various Platforms

Figure 2.9 : NCSA Mosaic was the original Web browser program. It has kept up well and is still available for Macintosh, Windows 3.1, Windows NT, Windows 95 (shown here), and X Windows systems. Many other browsers are derived from the original NCSA Mosaic source code.

Figure 2.10: Microsoft's Internet Explorer (shown here in its v2.0 release for Windows 95) is one of the most recent and one of the most powerful Web browser programs currently available.

The information in this table came from the World Wide Web Frequently Asked Questions (FAQ) list, which is maintained by Thomas Boutell. You can download the latest version at http://www.boutell.com/faq/ or http://www.shu.edu/about/WWWFaq/.