After the user has submitted whatever information they need to submit into whatever GUI you have designed, it is time to get that information from the web browser to the web server.

To do this, the information has to pass through the Communication Layer.

As you might imagine, there are a host of technologies which all poke their way into the Communication Layer. You'll hear terms like TCP/IP, OSI/ISO, HTTP, HTTP-S, SSL, etc. You'll also hear about technologies that modify the data in transmission such as encryption.

Whichever the case, the main idea that you should get used to when thinking about the Communication Layer is that information needs to get from one place to another and that data needs to get there in one piece, in the right order, and with no parts of that data changed or corrupted. In addition, the information could/should be transferred securely. That is, it should go from one place to another without anyone being able to read it, change it, or fake authorship of it.

When we say that information must go from one place to another, we mean specifically, from one computer to another computer and more specifically, from one software program on one computer to another software program on another computer.

Since there are so many types of computers and so many software packages that must communicate, you can imagine that it all just works out to a mass of blabber.

Well, to prevent such chaos, and to make the web work, everyone (software and hardware) is required to follow very rigorous standards of communications called protocols. Protocols guarantee that each participant of the discussion can understand the other.

For the purposes of the web, there are three types of communications protocols we care about:

• Computer to computer communication - This is handled by the IP protocol
• Encryption of plain text over the wires - This is handled by a motley of technologies but most commonly involve some form of public key encryption.
• Software to software communication - Generally, TCP will move data from the hardware level to the software level. Once the data has been moved to the software level, the software packages themselves will provide their own protocols for communication. In the realm of web serves and browsers, this is usually handled by HTTP or SSL in the case of encrypted communication.

Consider the following diagram in which hardware hosts open a TCP/IP connection to exchange data between software programs. Once the TCP/IP connection is opened, the software communicates via HTTP or SSL. The actual data exchanged may or may not be encrypted.

You will be familiar with IP because in order to address web pages, you must use domain names like www.barclayscapital.com, or their numerical counterparts called IP addresses.

IP addresses are unique sets of four period delimited octets that represent individual hosts on specific networks. A TCP/IP octet is a number between 0 and 255. Thus, an IP Address might look something like:

 
          30.85.1.26 

which you might read as something like "computer named 26, on network 1, on host 85, of network 30."

Besides defining the address scheme, IP also handles the transmission of data from an originating computer to the computer specified by the IP address. It does so by breaking up large, unwieldly chunks of data into easily manageable IP packets that it can deliver across the network.

That is why we talk of IP being a connectionless protocol. In an effort to better manage network traffic, IP specifies the protocol for breaking single messages into a slew of portions. Each portion is responsible for finding its way across the network based on changing traffic congestion and the IP protocol.

Each time a message arrives at an IP router, the router decides where to send it next. There is no concept of a session with a preselected path for all traffic. Routers can send data along the path of least resistance regardless of local network traffic congestion.

Recall that since IP chops messages up into packets, we need to assure that the messages are assembled correctly at the other end of transport. And if any errors occur during transport, such as message degradation or the loss of some packets, the sender will know to resend.

For example, browsers and servers must understand how to exchange web documents (multimedia files such as images, text, audio, etc). To do so, browsers and servers must speak their own standard protocol to help them exchange these files as well as meta information about those files.

Specifically, browsers and servers speak the HTTP protocol Where TCP/IP drives the internet, HTTP drives the web.

HTTP is a stateless, "request/response" protocol that specifies the means of transport as well as the process for maintaining the integrity of those multimedia files. It does so by using the Multipurpose Internet Mail Extensions (MIME) specification that describes the transfer and format of multimedia files.

The web consists of millions of hyperlinked files. These millions of files in turn, each may contain webs of links pointing to other files in the network. However, because the web is used to deliver files of all types (audio, video, text, etc), web browsers and servers need some way of defining what it is that is being linked to and transported.

To do so, MIME headers are used by HTTP to specify the contents of any transported file. The header will specify a file's type.

MIME Types and subtypes examples include the following:

The list is hardly exhaustive of course, and in fact web servers and web browsers can easily extend the MIME specification to add new MIME types. All you need to do is make sure that the server is configured to output the new type and that browsers are configured to expect the new type.

Getting browsers to understand new types is usually a function of the options or preferences settings in that browser. Consider the Netscape preferences dialog.

MIME types are specified as part of the "HTTP header" which defines meta-information about the document and the session. MIME type is specified in the Content-type field.

Thus, if a web server were to send a document of MIME type...

          Content-type: application msword

the web browser would know that it should open Word and load the document.

Besides specifying the MIME type, the HTTP header also contains other important information including the following:

Accept: image/gif
Accept: image/jpeg

Accept: */*

HTTP provides two primary methods to request documents: GET or POST

The foundation of HTTP/0.9 (the first implementation of the HTTP protocol) was the definition of the GET method that was used by a web browser to request a specific document.

For example, the following HTTP request would return the document "index.html" that was located in the web server's root directory called "webdocs"

          GET /webdocs/index.html CRLF

Notice that the GET request began with the GET keyword, included a document to retrieve, and ended with a carriage return and line feed combination.

If you would like, you can try making a GET request by connecting to your favorite web server and sending the GET request yourself (as if you were a web browser).

Below is a GET session I cut and pasted from a telnet window. In this case, I used telnet to contact the web server "www.extropia.com" and asked for the file "irobot.html" in the "Scripts/Columns" directory (Don't forget the two carriage returns at the end). The server responded by sending me the contents of that file (the HTML code you see).

	selena: telnet www.extropia.com 80
	Trying 206.53.239.130...
	Connected to www.extropia.com.
	Escape character is '^]'.
	GET /Scripts/Columns/irobot.html
	<HTML>
	<HEAD>
	<TITLE>Hello there</TITLE>
	</HEAD>
	<BODY>
	Hello there.  My, you are awfully good-looking to be a web 
	browser!
	</BODY>
	</HTML>
	Connection closed by foreign host.
	selena: 

The beauty of web browsers of course, is that they take care of the HTTP protocol specifications so that the user only needs to enter the URL of the page they want to see. The web browser formulates the actual GET request, sends it to the web server, receives the HTML document back, and then displays the HTML document according to the HTML instructions.

Besides allowing web browsers (or you pretending to be one) to get documents from a web server, the GET method also implements a method for a web browser to send optional search parameters as well (it was used with ISINDEX HTML files originally).

Search parameters were encoded in a special way that the web server can deal with.

Encoding works like this:

The URL is differentiated from the search parameters by a question mark (?). In other words, a URL generically looks like the following:

http://www.domain.com/dir/file?search parameters

Since you may want to have multiple search parameters, the GET method specifies that parameters are differentiated by placing an ampersand sign (&) between them. Thus, the encoded URL above becomes something like the following:

http://www.domain.com/dir/file?search1&search2&search3

Next, search parameters themselves are specified as "name/value pairs" separated by an equal sign (=) such as in the following example that sets the variable "lname" equal to "Sol" and the variable "fname" equal to "Selena":

http://www.domain.com/dir/file?lname=Sol&fname=Selena

Further, any spaces in the encoding string are replaced by plus signs (+) as in the following example:

http://www.domain.com/dir/file?name=Selena+Sol&age=28

Finally, any non-alphanumeric characters are replaced with their hexadecimal equivalents that are escaped with the percent sign (%). For example, a single quote character (') is encoded as %27 and a line break (which is a carriage return plus a line feed) is encoded as %0D%0A. Thus, we might see the following example that specifies that the variable pageName is equal to "Selena Sol's Page":

http://www.domain.com/dir/file?pageName=Selena+Sol%27s+Page

Though the GET method was very useful, a couple of serious problem remained.

First, the GET method only allowed a limited amount of data (1024 characters) to be sent as URL encoded data.

If there were too many name/value pairs, some of them would be clipped and data would get lost.

Further, since the information was sent as part of the URL, the user could see all of that data. On the one hand, that made URL's look really could see all of that data. On the one hand, that made URL's look really ugly and scary. On the other hand, it meant that the user got to see all of the inner workings of your CGI input.

This all changed with the development of HTTP/1.0.

The HTTP/1.0 protocol was developed from 1992 to 1996 in order to satisfy the need to exchange more than simple text information.

The first major change from the HTTP/0.9 specification was the use of MIME-like headers in request and response messages.

The next HTTP change was the definition of new request methods: HEAD and POST.

Let's look at both of these changes in greater depth. Under HTTP/1.0 an HTTP transaction consisted of a header followed by an empty line and then some extra data.

We have already talked about the header. The POST method of input was the other important change brought about by the introduction of HTTP/1.0.

The POST method allowed web browsers to send an unlimited amount of data to a web server by allowing them to tag it on to an HTTP request after the request headers as the message body.

Typically, the message body would be our old familiar encoded URL string after the question mark (?).

Thus, it would not be strange for a web server to get a POST request that looked something like the following:

          POST /cgi-bin/phone_book.cgi HTTP/1.0
          Referer: http://www.somedomain.com/Direcory/file.html
          User-Agent: Mozilla/1.22 (Windows: I: 32bit)
          Accept */*
          Content-type: application/x-www-form-urlencoded
          Content-length: 29

          name=Selena+Sol&phone=7700404

Notice that the "Content-length" request header is equal to the number of characters in the body of the request. This is important because a CGI script could easily parse through the variables in the body using the length.

Of course, as with the GET method, the user never needs to deal with the protocol itself. Instead, the browser does all the work of preparing the POST request headers and body.

So the million-dollar question is how does the browser get the name/value pairs to put into the HTTP message body?

As we mentioned earlier, when a browser requests something from the server, it uses, for the most part, either the GET or POST methods of HTTP to do so.

The most common form of data protection is encryption. An encrypted transmission is a transmission that contains plain text data which has been mathematically altered so as to be unreadable, but which can be transformed back to the original by a reverse mathematical algorithm.

Consider the simple encryption algorithm used in Arthur C Clarke's 2001 of letter = letter + 1 in which HAL is turned into IBM. Notice that H + 1 = I, A + 1 = B and L + 1 = M

There are several problems with this approach. For one, how do you exchange keys safely. If you are worried about the security of your communication, you can't just send the private key to the receiver over the network. If someone gets a hold of the private key, all communication can be decoded.

Unfortunately, using private key encryption, you must exchange the key in person.

Further, what happens when you exchange your key with someone who is less than dependable and some third party steals it?

Modern day encryption gets around these problems through the use of public key encryption.

Public key encryption is based on a certain type of mathematical algorithm that provides one way encryption/decryption. That is, a public key algorithm allows you to encrypt a message with a special key that has some very special properties....

1. Encrypted messages can only be unencrypted with a single private key.
2. Having both a public key and a message encoded with a public key is not realistically enough to crack the secret key.

Thus, rather than distributing your private decryption and encryption keys to trusted parties and crossing your fingers, you distribute your public key to anyone who wants it with full confidence that messages sent to you that have been encrypted with your public key will only be readable by you.

Any sender can encrypt their message with your public key and that message will only be able to be unencrypted with your private key.

There are several popular of these magical algorithms for public key encryption including RSA, Diffie-Hellman, and Elliptic-Curve cryptography and several packages such as PGP to implement them within your web applications.

For our purposes, SSL provides these services between SSL-enabled browsers and SSL-enabled servers. Fortunately, since both Netscape and Microsoft have implemented SSL, we as web developers can use SSL without much fear.

For the most part, as a developer implementing SSL will be out of your hands. Your code remains the same. All that changes is the web server that you serve your application from. When served from an SSL enabled server and directory, the browser and server will do all the work of encryption.

The browser will even let the client know that they have moved into a secure transmission mode for you. Consider the Netscape lock symbol