Chapter 35

Adding Additional Protocols to HotJava

by David Baker

• Writing a Protocol Handler
  • Step One: Decide Upon a Package Name
  • Step Two: Create the Directories
  • Step Three: Set Your CLASSPATH
  • Step Four: Implement the Protocol
  • Step Five: Create the Handler Class
  • Step Six: Compile the Sources
• Using Protocol Handlers with HotJava
  • Step One: Update the properties File
  • Step Two: Run HotJava
• Using Protocol Handlers with Your Own Applications
  • The main() Method: Starting FetchWhois
  • The FetchWhois Constructor: Where the Work Gets Done
  • The whoisUSHFactory Class: Registering the Protocol Handler
  • Running FetchWhois
• More on URLStreamHandlerFactory

Anessential feature that Java attempts to impart to computing is extensibility. Over time, newer and better means of communicating will be created. Application protocols will evolve and programs must be able to adapt. Support for new protocols can be added to Java and the HotJava browser through protocol handlers.

Protocol handlers are a key tool for maintaining an adaptable environment with Java. They extend support to new URL schemes, which can access new application protocols, or present existing ones in new ways.

Writing a Protocol Handler

ORG.netspace.dwb.protocol.whois

Step Two: Create the Directories

The necessary code for the protocol handler must be placed into a directory named in correlation to the package name identified. This directory should reside within a directory into which your own Java code is placed, usually called classes within your home directory.

Listing 35.1 shows an example for creating these directories under Windows NT and Windows 95.

Lising 35.1  Creating the Directories Under Windows NT and Windows 95

%HOMEDRIVE%
cd %HOMEPATH%
mkdir classes
mkdir classes\ORG
mkdir classes\ORG\netspace
mkdir classes\ORG\netspace\dwb
mkdir classes\ORG\netspace\dwb\protocol
mkdir classes\ORG\netspace\dwb\protocol\whois

The process is similar under the UNIX operating system, as shown in Listing 35.2.

Listing 35.2  Creating the Directories Under UNIX

cd ~
mkdir classes
mkdir classes/ORG
mkdir classes/ORG/netspace
mkdir classes/ORG/netspace/dwb
mkdir classes/ORG/netspace/dwb/protocol
mkdir classes/ORG/netspace/dwb/protocol/whois

Step Three: Set Your CLASSPATH

The CLASSPATH environment variable tells the Java compiler and interpreter where to find Java classes, enabling the dynamic linking feature of the Java execution environment. When installing the JDK, HotJava, or a Java-aware browser, you may have set the CLASSPATH environment variable. If this is so, it is critical that you avoid overwriting that data. First, you must find out what your CLASSPATH current setting is. Under Windows NT/95, just type the following command:

SET

Look for the CLASSPATH value. Under UNIX systems, you can display the CLASSPATH value with this command:

ECHO $CLASSPATH

Now you need to reset your CLASSPATH, including the previous data, if any. Under Windows 95, if your CLASSPATH was .;C:\JAVA\LIB\CLASSES.ZIP, you can add the following line to your AUTOEXEC.BAT and reboot:

SET CLASSPATH=.;%HOMEDRIVE%%HOMEPATH%\CLASSES;C:\JAVA\LIB\CLASSES.ZIP

Under Windows NT, presuming that the CLASSPATH value was the same as under the Windows 95 example, use the System Control Panel to add a CLASSPATH environment variable with the value .; %HOMEDRIVE%%HOMEPATH%\CLASSES;C:\JAVA\LIB\CLASSES.ZIP.

Under UNIX, assume that your old CLASSPATH was .:/usr/java/lib. If you are using the C shell, place the following into your .cshrc file:

setenv CLASSPATH .:${HOME}:/usr/java/lib

If you are on a UNIX system using the Korn or a POSIX-compliant shell, add the following line to whatever file your ENV environment variable points. If ENV is unset, then you could add the line to your ~/.profile file:

CLASSPATH=.:${HOME}:/usr/java/lib
export CLASSPATH

Step Four: Implement the Protocol

Within the directory created in Step Two, create a class that extends URLConnection. This class should have a constructor that takes an URL object as an argument and calls its superclass with that object. In addition, most protocol handlers should extend the following methods:

public void connect() throws IOException;

This method should connect to the remote resource and perform the network transaction, as appropriate.

public String getContentType();

This method should indicate the MIME content type of the data returned by the object.

public synchronized InputStream getInputStream() 
   throws IOException;

This returns an InputStream containing the data from the remote system.

Listing 35.3 shows the class used to implement the WHOIS protocol. This class supports the URL formats shown in Table 35.1.

Listing 35.3   whoisURLConnection.java

// This is the package identified for this protocol handler.
package ORG.netspace.dwb.protocol.whois;

import java.io.*;    // Import the package names used.
import java.net.*;

/**
 * This class implements a connection to the new "whois"
 * URL scheme.
 * @author David W. Baker
 * @version 1.0
 */
class whoisURLConnection extends URLConnection {
   // Some defaults for the WHOIS protocol implementation:
   //    site defaults to rs.internic.net
   //    port defaults to 43
   //    query defaults to QUIT
   private static final String DEF_SITE = "rs.internic.net";
   private static final int DEF_PORT = 43;
   private static final String DEF_QUERY = "QUIT";
   private static final String CONT_TYPE = "text/html";
   static final int URL_BASE = 16;
   InputStream fromHandler;   // Input from the handler
   Socket whoisSocket;        // Socket for communication
   boolean gotQuery = false;  // Did we get the data?

   /**
    * Given a URL will instantiate a whoisURLConnection 
    * object.
    * @param getURL The URL to contact.
    */
   whoisURLConnection(URL getURL) {
      super(getURL); // Call superclass with the URL.
   }

   /**
    * Connect to the WHOIS server, obtain, and format the
    * data.
    * @exception java.io.IOException Indicates a problem 
    *    connecting to URL.
    */
   public void connect() throws IOException {
      String whoisSite;
      int whoisPort;
      String whoisQuery;
      PrintStream toApp;   // Send data to app using handler.
      DataInputStream fromWhois = null;
      PrintStream toWhois = null;
      String dataLine;

      // Set up piped streams for communication between
      // this handler and the application using it.
      PipedOutputStream pipe = new PipedOutputStream();
      toApp = new PrintStream(pipe);
      fromHandler = new PipedInputStream(pipe);
    
      // Get host from the URL, using default if omitted.
      if (url.getHost().length() == 0) {
         whoisSite = DEF_SITE;
      } else {
         whoisSite = url.getHost();
      }
      // Get port from the URL, using default if omitted.
      if (url.getPort() < 1) {
         whoisPort = DEF_PORT;
      } else {
         whoisPort = url.getPort();
      }
      // Get file from the URL, using default is "/"
      if (url.getFile().equals("/")) {
         whoisQuery = DEF_QUERY;
      } else {
         whoisQuery = url.getFile().substring(1);
      }
      // Decode the query from the URL.
      whoisQuery = decodeURL(whoisQuery);
      // Open a socket to the whois server.
      whoisSocket = new Socket(whoisSite,whoisPort);
      // Open streams to communicate with the whois server.
      fromWhois = 
         new DataInputStream(whoisSocket.getInputStream());
      toWhois = 
         new PrintStream(whoisSocket.getOutputStream());
      // Send the query to the server.
      toWhois.println(whoisQuery);
      // Print out some HTML.
      toApp.println("<!DOCTYPE HTML PUBLIC \"-//IETF//DTD "
                     + "HTML//EN\">");
      toApp.println("<HTML>");
      toApp.println("<HEAD>");
      toApp.println("<TITLE>Whois Query for: " + url 
                     + "</TITLE>");
      toApp.println("</HEAD>");
      toApp.println("<BODY>");
      toApp.println("<H1>Whois Query for : " + url 
                     + "</H1>");
      toApp.println("<PRE>");
      // Loop through the data from the whois server,
      // printing it all out within the preformatted
      // text section.
      while ((dataLine = fromWhois.readLine()) != null) {
         toApp.println(dataLine);
      }
      // Some last HTML.
      toApp.println("</PRE>");
      toApp.println("</BODY>");
      toApp.println("</HTML>");
      toApp.flush();       // Flush the pipe.
      toWhois.close();     // Close the streams.
      fromWhois.close();
      whoisSocket.close(); // Close the socket.
      toApp.close();       // Close one end of the pipe.
      gotQuery = true;     // Data has been obtained.
   }

   /**
    * Determine the content type of the data returned.
    * @return The content type.
    */
   public String getContentType() {
      return CONT_TYPE;
   }

   /**
    * Obtain a stream to get data from this protocol handler.
    * @return The stream to read data.
    */
   public synchronized InputStream getInputStream() 
      throws IOException {
      // If where has not been obtained, connect()
      if (!gotQuery) {
         connect();
      }
      // Return the stream.
      return fromHandler;
   }

   /**
    * This method decodes the URL encoded format.
    * i.e. %XX -> char and + to space
    * @param decode The String to decode.
    * @return The decoded String.
    */
   protected String decodeURL(String decode) {
      StringBuffer decoded = new StringBuffer();
      char nextChar;
      String encString;
      Integer encInteger;

      // Go through the String character by character.
      for(int index=0; index < decode.length(); index++) {
         // Get the next character in the String.
         nextChar = decode.charAt(index);
         // If the character is +, then convert it to
         // a space.
         if (nextChar == '+') {
            decoded.append(" ");
         }
         // If the character is a %, then the next two
         // characters store the value of the encoded
         // character.
         else if (nextChar == '%') {
            // Create an Integer object containing the
            // integer value of the next two characters
            // in the string, assuming a base 16 notation.
            encInteger = Integer.valueOf(
               decode.substring(index+1,index+3),URL_BASE);
            // Increment our counter by 2 - we just read
            // two characters.
            index += 2;
            // Return the int value within the Integer
            // and then cast that into a char type.
            nextChar = (char)encInteger.intValue();
            // Add the coded character.
            decoded.append(nextChar);
         }
         // Otherwise, just add the character.
         else {
            decoded.append(nextChar);
         }
      }
      // Return the decoded string.
      return decoded.toString();
   }
}

The whoisURLConnection constructor merely makes the appropriate call to the URLConnection superclass, passing it the URL object. The connect() method is where all of the work is done. First, it creates a number of streams for reading in data and passing it back to your Java application. Next, it uses various URL class methods to parse the URL, supporting the formats identified in Table 35.1. The default values of the host (for example, "rs.internic.net") and port (i.e., "35") are stored within static final variables. Then, the connect() method opens a TCP socket connection to the remote system, sends the query, and prepares to read the response. It reads in the query response, embedding it within HTML. Finally, connect() closes the streams and TCP socket.

The getContentType() method is used to indicate that the returned data is an HTML document. getInputStream() returns the PipedOutputStream, into which this protocol handler has pushed the formatted data. The Java application uses this stream to access the data obtained by the new protocol handler.

The whoisURLConnection has an additional protected method called decodeURL(). Certain characters, such as spaces and other special characters, cannot be represented literally within URLs, and these characters are encoded with a special format. For example, a space is encoded as %20. Since your WHOIS query may need to use such characters, the WHOIS protocol handler must have a method to decode this data. decodeURL() is called from the connect() method, and examines a string byte-by-byte. When it encounters a percent-sign, it uses the next two characters as the Unicode value to the encoded char-acter.

Step Five: Create the Handler Class

Within the same directory, you must create a file called Handler.java. The Handler class must extend the URLStreamHandler class and return an instance of the class created in Step Four. Listing 35.4 shows the Handler class for the WHOIS protocol.

Listing 35.4  Handler.java

// This is the package identified for this protocol handler.
package ORG.netspace.dwb.protocol.whois;

import java.net.*;   // Import the package names used.

/**
 * This class is a subclass of URLStreamHandler and provides
 * an implementation of the abstract openConnection() method
 * to support the "whois" scheme.
 * @author David W. Baker
 * @version 1.0
 */
public class Handler extends URLStreamHandler {
  /**
   * Given a URL return an appropriate URLConnection.
   * @param requestedURL The URL instance to contact.
   * @return The connection to the resource.
   */
   public synchronized URLConnection 
      openConnection(URL requestedURL) {
      return new whoisURLConnection(requestedURL);
   }
}

Step Six: Compile the Sources

Use javac to compile the sources created in Steps Four and Five, leaving the compiled class files in the originating directory.

Using Protocol Handlers with HotJava

Eventually, as part of HotJava's vision, protocol handlers will be dynamically downloaded and utilized. At the time of this writing, this feature is not yet supported by HotJava. Protocol handlers must be manually installed, as previously described, in order for HotJava to utilize them.

Once a protocol handler has been installed, the following additional steps are necessary to use it within your HotJava browser.

Step One: Update the properties File

You must instruct HotJava to load additional protocol handlers. This is done by updating the properties file, which is located within the .hotjava directory in your home directory.

Within this file, you must set the java.protocol.handler.pkgs property to include the new protocol package. The value to add should be everything from the protocol token leftward. Thus, you want to add the following line for your WHOIS protocol handler:

java.protocol.handler.pkgs=ORG.netspace.dwb.protocol

If you already have this property set within your HotJava properties file, append a pipe character (i.e., |) to the end of the line and then add ORG.netspace.dwb.protocol. This syntax enables you to install several new protocol handlers, like the following:

java.protocol.handler.pkgs=COM.company.protocol|ORG.netspace.dwb.protocol

Step Two: Run HotJava

Start up HotJava and test out the new protocol handler. Go under the File menu, select Open Page, and type the following text into the URL field:

whois:internic.net

Figure 35.1 shows what should be displayed within the HotJava window.

Figure 35.1 : HotJava using the WHOIS protocol handler.

Using Protocol Handlers with Your Own Applications

The usefulness of new protocol handlers extends beyond the HotJava browser. They can also be utilized within your own applications; the key step being a method to register the new protocol handler using a concept known as a factory.

As an example, a simple application will be developed to make use of the WHOIS protocol handler. FetchWhois will take its arguments and send them as a query to the WHOIS service at rs.internic.net. The source of this example is shown in Listing 35.5.

Listing 35.5  FetchWhois.java

import java.net.*;   // Import the package names used.
import java.io.*;

/**
 * This is an application which uses our new whois
 * protocol handler to obtain information.
 * @author David W. Baker
 * @version 1.1
 */
public class FetchWhois {

   /**
    * The method launches the application.
    * @param args Arguments which are the query string.
    */
   public static void main (String args[]) {
      if (args.length < 1) {
         System.err.println(
            "usage: java FetchWhois query string");
         System.exit(1);
      }
      FetchWhois app = new FetchWhois(args);
   }

   /**
    * This constructor does all of the work of obtaining
    * the data from the server.
    * @param args The tokens of the query string.
    */
   public FetchWhois(String args[]) {
      String encodedString;   // Hold the URL encoded query.
      String nextLine;        // Line from the handler.
      URL whoisURL;           // URL to whois resource
      URLConnection whoisAgent;  // Connection to whois.
      DataInputStream input;     // Stream from whois.

      // Create a buffer to place in all of the query
      // string tokens.
      StringBuffer buffer = new StringBuffer();
      // Append all of the tokens to the buffer.
      for(int index = 0; index < args.length; index++) {
         buffer.append(args[index]);
         if (index < args.length-1) {
            buffer.append(" ");
         }   
      }
      // URL encode the query buffer.
      encodedString = URLEncoder.encode(buffer.toString());
      // Set the factory to register the whois handler.
      URL.setURLStreamHandlerFactory(new whoisUSHFactory());
      try {
         // Create the whois URL object.
         whoisURL = new URL("whois:" + encodedString);
         // Open the connection.
         whoisAgent = whoisURL.openConnection();
         // Get an input stream from the whois server.
         input = 
            new DataInputStream(whoisAgent.getInputStream());
         // Print out the data line-by-line.
         while((nextLine = input.readLine()) != null) {
            System.out.println(nextLine);
         }
         input.close(); // Close the stream.
      } catch(MalformedURLException excpt) {
         System.err.println("Mailformed URL: " + excpt);
      } catch(IOException excpt) {
         System.err.println("Failed I/O: " + excpt);
      }
   }
}

/**
 * This class implements the URLStreamHandlerFactory
 * interface to register the whois protocol handler.
 * @see java.net.URLStreamHandlerFactory
 */
class whoisUSHFactory implements URLStreamHandlerFactory {
   /** 
    * This method returns the protocol handler to the
    * calling object.
    * @param scheme The URL scheme to be obtained.
    * @return The protocol handler.
    * @see java.net.URLStreamHandlerFactory#createURLStreamHandler
    */
   public URLStreamHandler 
      createURLStreamHandler(String scheme) {
      // Make sure that this is for a whois URL
      if (scheme.equalsIgnoreCase("whois")) {
         // If so, create the handler and return it.
         return 
            new ORG.netspace.dwb.protocol.whois.Handler();
      // Otherwise print an error message and return null.
      } else {
         System.err.println("Unknown protocol: " + scheme);
         return null;
      }
   }
}

The main() Method: Starting FetchWhois

The simple main() method checks to see that the program was invoked properly. It then creates a FetchWhois object, passing it the array of command line arguments.

The FetchWhois Constructor: Where the Work Gets Done

The constructor is where the connection to rs.internic.net is opened through the use of your new protocol handler. The key line is where it uses the setURLSreamHandlerFactory() static method from the URL class. This is where FetchWhois directs new URL instances to the new WHOIS protocol handler, using the whoisUSHFactory described later.

The constructor uses a StringBuffer to consolidate the array of command line arguments into a single String, and then uses the static encode() method of URLEncoder to properly format the data. Then, it creates an URL object with the new whois URL scheme. Finally, it opens a connection to the URL and receives the WHOIS data.

The whoisUSHFactory Class: Registering the Protocol Handler

The URL class uses a URLStreamHandlerFactory implementation to choose an appropriate protocol handler. In order to use the WHOIS protocol handler, you must implement the URLStreamHandlerFactory interface. The constructor should take a String that contains the scheme of the URL object being instantiated. whoisUSHFactory checks to ensure that the scheme is whois, and if so, returns an instance of the WHOIS Handler. Otherwise, it returns null.

Running FetchWhois

Compile FetchWhois with javac and then try running it by using the following command. The program should print an HTML document that contains the same data as when you used the whois URL within HotJava.

java FetchWhois internic.net

HotJava will soon support dynamically downloaded protocol handlers, enabling a much more flexible use of this feature. Manually installing these extensions will continue to be a useful alternative.

More on URLStreamHandlerFactory

The URLStreamHandlerFactory class is a means of passing out specific protocol handlers for each supported protocol handler. In Listing 35.5, you used a custom factory called whoisUSHFactory. This factory supported only one scheme, whois. However, factories can be much more general, and can support both custom protocol handlers and those provided within the JDK.

Listing 35.6 shows another application, ParseURL, which uses its own URLStreamHandlerFactory implementation. This application will take a series of URLs, including new URL schemes, as command line arguments and then print out how the URL class parses them. This demonstrates how the URL class deals with different forms of URLs.

Listing 35.6  ParseURL.java

import java.net.*;

/*
 * This simple application demonstrates how a custom 
 * URLStreamHandlerFactory can be used to provide custom-
 * written protocol handlers as well as those within the 
 * JDK. This program uses a dummy protocol handler so that
 * it can construct a URL object to unknown protocols and
 * parse out that URL.
 * @author David W. Baker
 * @version 1.0
 */
public class ParseURL {

  /**
   * This method allows ParseURL to be executed as a 
   * stand-alone application. It takes a series of URLs
   * as arguments and then prints out the various portions
   * of those URLs.
   * @param args The command-line arguments.
   */
  public static void main(String[] args) {
    URL urlToParse;   // We use this to parse each URL.
    
    // First, we must use the custom URLStreamHandlerFactory
    // which is included below.
    URL.setURLStreamHandlerFactory(new ParseURLFactory());
    // Go through each command-line argument.
    for(int index = 0; index < args.length; index++) {
      try {
        // Create the URL object.
        urlToParse = new URL(args[index]);
        // Print the URL to be parsed.
        System.out.println("Parsing URL: " + args[index]);
        // Print out the various portions of the URL.
        System.out.println("\tScheme:\t" + 
                           urlToParse.getProtocol());
        System.out.println("\tHost:\t" + 
                           urlToParse.getHost());
        System.out.println("\tPort:\t" + 
                           urlToParse.getPort());
        System.out.println("\tFile:\t" + 
                           urlToParse.getFile());
        System.out.println("\tRef:\t" + urlToParse.getRef());
        System.out.println("==============================");
      } catch(MalformedURLException excpt) {
        // We should catch this exception, but our custom
        // URLStreamHandlerFactory will ensure such is
        // never thrown.
        System.err.println("Malformed URL: " + args[index]);
        System.err.println(excpt);
      }
    }
  }
}
      
/**
 * This custom URLStreamHandlerFactory allows us to create
 * URL objects that will use the JDK protocol handlers for
 * known schemes. If the scheme is not one of those within
 * the JDK, we create an instance of our DummyHandler and 
 * return that. If this factory returned null instead, 
 * a MalformedURLException would be thrown for unknown
 * schemes.
 */
class ParseURLFactory implements URLStreamHandlerFactory {
  public URLStreamHandler createURLStreamHandler(String 
                                                 scheme) {
    // First, go through the JDK's protocol handlers for
    // known URL schemes. All of these protocol handlers
    // reside within sun.net.www.protocol.<scheme>.
    if (scheme.equalsIgnoreCase("http")) {
      return new sun.net.www.protocol.http.Handler();
    }
    if (scheme.equalsIgnoreCase("ftp")) {
      return new sun.net.www.protocol.ftp.Handler();
    }
    if (scheme.equalsIgnoreCase("gopher")) {
      return new sun.net.www.protocol.gopher.Handler();
    }
    if (scheme.equalsIgnoreCase("file")) {
      return new sun.net.www.protocol.file.Handler();
    }
    if (scheme.equalsIgnoreCase("mailto")) {
      return new sun.net.www.protocol.mailto.Handler();
    }
    // If it is none of the above, create an instance of our 
    // dummy URLStreamHandler.
    return new DummyHandler();
  }
}

/**
 * This is a dummy protocol handler. This handler will allow
 * us to create a URL object for this scheme, but we won't
 * be able to retrieve any data through it.
 */
class DummyHandler extends URLStreamHandler {
  /**
   * This method will only return null. Data cannot be 
   * obtained from this handler.
   * @param u The URL to obtain data from.
   * @return A null URLConnection.
   */
  public URLConnection openConnection(URL u) {
    return null;
  }
}

ParseURL also elucidates how the MalformedURLException is created for unknown schemes. If the factory being used with your application doesn't return a protocol handler for a scheme, any attempt to create an URL instance with such a scheme will throw a MalformedURLException. In ParseURL, however, you prevent this from occurring so that you can see how new URLs are parsed.

ParseURL has three classes, ParseURL, ParseURLFactory, and DummyHandler. The first is the only public class and is used to invoke the application. The main() method within the ParseURL class sets the URLStreamHandlerFactory to your custom one, ParseURLFactory. It then iterates through the command line arguments, creating URL instances for each and then printing out the portions of each URL.

The ParseURLFactory is your custom factory. It inspects the scheme of the URL instance to be created. If the scheme happens to be one of the standard schemes supported within the JDK, it returns the corresponding protocol handler. All of these protocol handlers within the JDK are contained within the package sun.net.www.protocol.scheme. If the scheme is not one of the standard ones, ParseURLFactory returns an instance of DummyHandler, rather than null. If null were returned, a MalformedURLException would be thrown if you attempted to create any URLs with non-standard schemes.

The DummyHandler class does very little, as its name suggests. This class allows you to create URL instances for non-standard schemes. You can then parse such URLs with methods like getHost() and getFile() without having to create a protocol handler for each new scheme. However, DummyHandler does not implement any connection mechanism. Thus, you will not be able to obtain any data through non-standard URL instances that use the DummyHandler.

To test your ParseURL application, first compile it with the Java compiler. Then execute it with the Java interpreter with one or more URLs as arguments. For instance, invoking it with:

java ParseURL http://www.yahoo.com/ newscheme:newstuff

returns the following information:

Parsing URL: http://www.yahoo.com/
        Scheme: http
        Host:   www.yahoo.com
        Port:   -1
        File:   /
        Ref:    null
==============================
Parsing URL: newscheme:newstuff
        Scheme: newscheme
        Host:
        Port:   -1
        File:   /newstuff
        Ref:    null
==============================