★ wanayoo — archive 1999 http://www.gnu.org/software/classpath/doc/hacking.htmlNouvelle recherche | Portail wanayoo

GNU Classpath Hacker's Guide

Aaron M. Renn
Paul N. Fisher
John Keiser


Table of Contents


Introduction

The Classpath Project is a dedicated to providing a 100% free, clean room implementation of the standard Java class libraries. Because there is currently no free implementation of the Java environment, no free operating system can ship with Java included. Parts of a free Java implementation have already been written, including free Java virtual machines (JVM's) such as Kaffe and Japhar, and Java compilers such as Guavac. However, there is currently no free replacement for Sun's proprietary libraries. This Classpath project aims to correct this problem by supplying a free class library implementation that will allow a 100% free Java platform to be distributed. Note that Kaffe now ships with a partial class library that is also free, so there is more than one group working towards a common goal.

Requirements

Although Classpath is following an open development model where input from developers is welcome, there are certain base requirements that need to be met by anyone who wants to contribute code to this project. They are mostly unfortunately dictated by legal requirements and are not arbitrary restrictions chosen by the Classpath team.

You will need to adhere to the following things if you want to donate code to the Classpath project:

Volunteering to Help

The Classpath project needs volunteers to help us out. People are needed to write unimplemented Java packages, to test Classpath on various platforms, and to port it to platforms that are currently unsupported.

While pretty much all contributions are welcome (but see see section Requirements) it is always preferable that volunteers do the whole job when volunteering for a task. So when you volunteer to write a Java package, please be willing to do the following:

Nobody likes to write documentation and test cases, but they are vital to a complete and robust product. Writing them as you go is much easier than going back at the end and adding them.

Project Goals

The goal of the Classpath project is to produce a free implementation of the standard class library for Java. However, there are other more specific goals as to which platforms should be supported.

Classpath is targeted to support the following operating systems:

  1. Free operating systems. This includes GNU/Linux, GNU/Hurd, and the free BSDs.
  2. Other Unix like operating systems.
  3. Platforms which currently have no Java support at all.
  4. Other platforms such as MS-Windows.

While free operating systems are the top priority, the other priorities can shift depending on whether or not there is a volunteer to port Classpath to those platforms and to test releases.

Eventually we hope the Classpath will support all JVM's that provide JNI support. However, the top priority is free JVM's. The JVM support priority list is:

  1. Japhar
  2. Kaffe
  3. Sun's JDK
  4. Other JNI Compliant JVM's.

As with OS platform support, this priority list could change if a volunteer comes forward to port, maintain, and test releases for a particular JVM. Kaffe is now developing its own class library, so the priority of supporting that platform is not as high as for Japhar.

The initial target version for Classpath is Java 1.1. Java 1.2 can be implemented if desired, but please do not create classes that depend on 1.2 features in other packages.

Programming Tools

If you want to hack on Classpath, you should download, install, and familiarize yourself with the following tools:

All of these tools are available from prep.ai.mit.edu via anonymous ftp. With the exception of perl, they are fully documented with texinfo manuals. Texinfo can be browsed with the Emacs editor, or with the text editor of your choice.

Here is a brief description of the purpose of those tools.

CVS
A version control system that maintains a centralized Internet repository of all code in the Classpath system. Access to the repository requires an account. Contact Paul Fisher (rao@gnu.org) for details.
automake
This tool automatically creates Makefile.in files from Makefile.am files. The Makefile.in is turned into a Makefile by autoconf. Why use this? Because it automatically generates every makefile target you would ever want (clean, install, dist, etc) in full compliance with the GNU coding standards. It also simplifies Makefile creation in a ton of different ways I can't describe here. Read the docs for more info.
autoconf
Automatically configures a package for the platform on which it is being built and generates the Makefile for that platform.
libtool
Handles all of the zillions of hairy platform specific options needed to build shared libraries.
m4
The free GNU replacement for the standard Unix macro processor. Proprietary m4 programs are broken and so GNU m4 is required for autoconf to work.
perl
Larry Wall's scripting language. It is used internally by automake.
MP
Required for java.lang.Float, java.lang.Double, java.math.BigInteger, and java.math.BigDecimal.

Programming Standards

For C code, follow the GNU Coding Standards. The standards also specify various things like the install directory structure. These should be followed if possible.

For Java code, please follow the GNU Coding Standards, with the exception of naming conventions. Please follow Sun's naming conventions.

For documentation comments, please follow How to Write Doc Comments for Javadoc.

Programming Goals

When you write code for Classpath, write with three things in mind, and in the following order: portability, robustness, and efficiency.

If efficiency breaks portability or robustness, then don't do it the efficient way. If robustness breaks portability, then bye-bye robust code. Of course, as a programmer you would probably like to find sneaky ways to get around the issue so that your code can be all three ... the following chapters will give some hints on how to do this.

Portability

The portability goal for Classpath is the following:

  1. native functions for each platform that work across all VMs on that platform
  2. a single classfile set that work across all VMs on all platforms that support the native functions.

For almost all of Classpath, this is a very feasible goal, using a combination of JNI and native interfaces. This is what you should shoot for. For those few places that require knowledge of the Virtual Machine beyond that provided by the Java standards, the VM Interface was designed. Read the Virtual Machine Integration Guide for more information.

Right now the only supported platform is Linux. This will change as that version stabilizes and we begin the effort to port to many other platforms.

Robustness

Native code is very easy to make non-robust. (That's one reason Java is so much better!) Here are a few hints to make your native code more robust.

Always check return values for standard functions. It's sometimes easy to forget to check that malloc() return for an error. Don't make that mistake. (In fact, use JCL_malloc() in the jcl library instead--it will check the return value and throw an exception if necessary.)

Always check the return values of JNI functions, or call ExceptionOccurred to check whether an error occurred. You must do this after every JNI call. JNI does not work well when an exception has been raised, and can have unpredictable behavior.

Throw exceptions using JCL_ThrowException. This guarantees that if something is seriously wrong, the exception text will at least get out somewhere (even if it is stderr).

Check for null values of jclasses before you send them to JNI functions. JNI does not behave nicely when you pass a null class to it: it terminates Java with a "JNI Panic."

In general, try to use functions in native/lib/jcl.h. They check exceptions and return values and throw appropriate exceptions.

Java Efficiency

For methods which explicitly throw a NullPointerException when an argument is passed which is null, per a Sun specification, do not write code like:

int 
strlen (String foo) throws NullPointerException
{
  if (foo == null)
    throw new NullPointerException ("foo is null");
  return foo.length ();
}

Instead, the code should be written as:

int
strlen (String foo) throws NullPointerException
{
  return foo.length ();
}

Explicitly comparing foo to null is unnecessary, as the virtual machine will throw a NullPointerException when length() is invoked. Classpath is designed to be as fast as possible -- every optimization, no matter how small, is important.

Native Efficiency

You might think that using native methods all over the place would give our implementation of Java speed, speed, blinding speed. You'd be thinking wrong. Would you believe me if I told you that an empty interpreted Java method is typically about three and a half times faster than the equivalent native method?

Bottom line: JNI is overhead incarnate. In Sun's implementation, even the JNI functions you use once you get into Java are slow.

A final problem is efficiency of native code when it comes to things like method calls, fields, finding classes, etc. Generally you should cache things like that in static C variables if you're going to use them over and over again. GetMethodID(), GetFieldID(), and FindClass() are *slow*.

Here are a few tips on writing native code efficiently:

Make as few native method calls as possible. Note that this is not the same thing as doing less in native method calls; it just means that, if given the choice between calling two native methods and writing a single native method that does the job of both, it will usually be better to write the single native method. You can even call the other two native methods directly from your native code and not incur the overhead of a method call from Java to C.

Cache methodIDs and fieldIDs wherever you can. String lookups are expensive. The best way to do this is to use the native/lib/jnilink.h library. It will ensure that jmethodIDs are always valid, even if the class is unloaded at some point. In 1.1, jnilink simply caches a NewGlobalRef() to the method's underlying class; however, when 1.2 comes along, it will use a weak reference to allow the class to be unloaded and then re-resolve the jmethodID the next time it is used.

Cache classes that you need to access often. jnilink will help with this as well. The issue here is the same as the methodID and fieldID issue--how to make certain the class reference remains valid.

If you need to associate native C data with your class, use Paul Fisher's native_state library (NSA). It will allow you to get and set state fairly efficiently. Japhar now supports this library, making native state get and set calls as fast as accessing a C variable directly.

Specification Sources

There are a number of specification sources to use when working on Classpath. In general, the only place you'll find your classes specified is in the JavaDoc documentation or possibly in the corresponding white paper. In the case of java.lang, java.io and java.util, you should look at the Java Language Specification.

Here, however, is a list of specs, in order of canonicality:

  1. Clarifications and Amendments to the JLS - 1.1
  2. JLS Updates - 1.1
  3. The 1.0 JLS
  4. JVM spec - 1.1
  5. JNI spec - 1.1
  6. Sun's javadoc - 1.1 (since Sun's is the reference implementation, the javadoc is documentation for the Java platform itself.)
  7. JVMDI spec - 1.2, JNI spec - 1.2 (sometimes gives clues about unspecified things in 1.1; if it was not specified accurately in 1.1, then use the spec for 1.2; also, we are using JVMDI in this project.)
  8. Sun's javadoc - 1.2 (sometimes gives clues about unspecified things in 1.1; if it was not specified accurately in 1.1, then use the spec for 1.2)
  9. The Bug Parade: I have obtained a ton of useful information about how things do work and how they *should* work from the Bug Parade just by searching for related bugs. The submitters are very careful about their use of the spec. And if something is unspecified, usually you can find a request for specification or a response indicating how Sun thinks it should be specified here.

You'll notice that in this document, white papers and specification papers are more canonical than the JavaDoc documentation. This is true in general.

Directory and File Naming Conventions

The Classpath directory structure is laid out in the following manner:

classpath
 |
 |---->java
 |       |
 |       |-->awt
 |       |-->io
 |       |-->lang
 |       |-->util
 |       |     |
 |       |     |--->zip
 |       |     |--->jar
 |       |-->net
 |       |-->etc
 |
 |---->gnu
 |       |
 |       |-->java
 |             |
 |             |-->awt
 |             |-->lang
 |             |-->util
 |             |     |
 |             |     |-->zip
 |             |-->etc
 |
 |---->native
 |       |
 |       |-->java.io
 |       |-->java.lang
 |       |-->java.net
 |       |-->java.util.jar
 |       |-->etc
 |
 |---->test
 |       |
 |       |-->java.io
 |       |-->java.lang
 |       |-->etc
 |
 |---->compat
         |
         |-->java.io
         |-->java.lang
         |-->etc

Here is a brief description of the toplevel directories and their contents.

java
Contains the source code to the Java packages that make up the core class library. Because this is the public interface to Java, it is important that the public classes, interfaces, methods, and variables are exactly the same as specified in Sun's documentation. The directory structure is laid out just like the java package names. For example, the class java.util.zip would be in the directory java/util/zip.
gnu/java
Internal classes (roughly analogous to Sun's sun.* classes) should go under the gnu/java directory. Classes related to a particular public Java package should go in a directory named like that package. For example, classes related to java.util.zip should go under a directory gnu/java/util/zip. Sub-packages under the main package name are allowed. For classes spanning multiple public Java packages, pick an appropriate name and see what everybody else thinks.
native
This directory holds native code needed by the public Java packages. Each package has its own subdirectory, which is the "flattened" name of the package. For example, native method implementations for java.util.zip should go in native/java.util.zip.
test
This directory contains test packages written for DejaGnu used to test releases of Classpath. The test scripts for a given package go in the subdirectory that is the same as the "flattened" name of the package. For example, test scripts for java.util.zip should go in test/java.util.zip
compat
This directory contains misc scripts designed not to test an implementation, but to determine various things about Sun's reference implementation that are needed in order to write a compatible package. Each package has its own directory which is the "flattened" package name. For example, compatibility scripts for java.util.zip go in compat/java.util.zip

Each person working on a package get's his or her own "directory space" underneath each of the toplevel directories. In addition to the general guidelines above, the following standards should be followed:

Character Conversions

Java uses the Unicode character encoding system internally. This is a sixteen bit (two byte) collection of characters encompassing most of the world's written languages. However, Java programs must often deal with outside interfaces that are byte (eight bit) oriented. For example, a Unix file, a stream of data from a network socket, etc. Beginning with Java 1.1, the Reader and Writer classes provide functionality for dealing with character oriented streams. The classes InputStreamReader and OutputStreamWriter bridge the gap between byte streams and character streams by converting bytes to Unicode characters and vice versa.

In Classpath, InputStreamReader and OutputStreamWriter rely on an internal class called gnu.java.io.EncodingManager to load translaters that perform the actual conversion. There are two types of converters, encoders and decoders. Encoders are subclasses of gnu.java.io.encoder.Encoder. This type of converter takes a Java (Unicode) character stream or buffer and converts it to bytes using a specified encoding scheme. Decoders are a subclass of gnu.java.io.decoder.Decoder. This type of converter takes a byte stream or buffer and converts it to Unicode characters. The Encoder and Decoder classes are subclasses of Writer and Reader respectively, and so can be used in contexts that require character streams, but the Classpath implementation currently does not make use of them in this fashion.

The EncodingManager class searches for requested encoders and decoders by name. Since encoders and decoders are separate in Classpath, it is possible to have a decoder without an encoder for a particular encoding scheme, or vice versa. EncodingManager searches the package path specified by the file.encoding.pkg property. The name of the encoder or decoder is appended to the search path to produce the required class name. Note that EncodingManager knows about the default system encoding scheme, which it retrieves from the system property file.encoding, and it will return the proper translator for the default encoding if no scheme is specified. Also, the Classpath standard translator library, which is the gnu.java.io package, is automatically appended to the end of the path.

For efficiency, EncodingManager maintains a cache of translators that it has loaded. This eliminates the need to search for a commonly used translator each time it is requested.

Finally, EncodingManager supports aliasing of encoding scheme names. For example, the ISO Latin-1 encoding scheme can be referred to as "8859_1" or "ISO-8859-1". EncodingManager searches for aliases by looking for the existence of a system property called gnu.java.io.encoding_scheme_alias.<encoding name>. If such a property exists. The value of that property is assumed to be the canonical name of the encoding scheme, and a translator with that name is looked up instead of one with the original name.

Here is an example of how EncodingManager works. A class requests a decoder for the "UTF-8" encoding scheme by calling EncodingManager.getDecoder("UTF-8"). First, an alias is searched for by looking for the system property gnu.java.io.encoding_scheme_alias.UTF-8. In our example, this property exists and has the value "UTF8". That is the actual decoder that will be searched for. Next, EncodingManager looks in its cache for this translator. Assuming it does not find it, it searches the translator path, which is this example consists only of the default gnu.java.io. The "decoder" package name is appended since we are looking for a decoder. ("encoder" would be used if we were looking for an encoder). Then name name of the translator is appended. So EncodingManager attempts to load a translator class called gnu.java.io.decoder.UTF8. If that class is found, an instance of it is returned. If it is not found, a UnsupportedEncodingException.

To write a new translator, it is only necessary to subclass Encoder and/or Decoder. Only a handful of abstract methods need to be implemented. In general, no methods need to be overridden. The needed methods calculate the number of bytes/chars that the translation will generate, convert buffers to/from bytes, and read/write a requested number of characters to/from a stream.

Many common encoding schemes use only eight bits to encode characters. Writing a translator for these encodings is very easy. There are abstract translator classes gnu.java.io.decode.DecoderEightBitLookup and gnu.java.io.encode.EncoderEightBitLookup. These classes implement all of the necessary methods. All that is necessary to create a lookup table array that maps bytes to Unicode characters and set the class variable lookup_table equal to it in a static initializer. Also, a single constructor that takes an appropriate stream as an argument must be supplied. These translators are exceptionally easy to create and there are several of them supplied in the Classpath distribution.

Writing multi-byte or variable-byte encodings is more difficult, but often not especially challenging. The Classpath distribution ships with translators for the UTF8 encoding scheme which uses from one to three bytes to encode Unicode characters. This can serve as an example of how to write such a translator.

Many more translators are needed. All major character encodings should eventually be supported.

Localization

There are many parts of the Java standard runtime library that must be customized to the particular locale the program is being run in. These include the parsing and display of dates, times, and numbers; sorting words alphabetically; breaking sentences into words, etc. In general, Classpath uses general classes for performing these tasks, and customizes their behavior with configuration data specific to a given locale.

In Classpath, all locale specific data is stored in a ListResourceBundle class in the package gnu/java/locale. The basename of the bundle is LocaleInformation. See the documentation for the java.util.ResourceBundle class for details on how the specific locale classes should be named.

ListResourceBundle's are used instead of PropertyResourceBundle's because data more complex than simple strings need to be provided to configure certain Classpath components. Because ListResourceBundle allows an arbitrary Java object to be associated with a given configuration option, it provides the needed flexibility to accomodate Classpath's needs.

Each Java library component that can be localized requires that certain configuration options be specified in the resource bundle for it. It is important that each and every option be supplied for a specific component or a critical runtime error will most likely result.

As a standard, each option should be assigned a name that is a string. If the value is stored in a class or instance variable, then the option should name should have the name name as the variable. Also, the value associated with each option should be a Java object with the same name as the option name (unless a simple scalar value is used). Here is an example:

A class loads a value for the format_string variable from the resource bundle in the specified locale. Here is the code in the library class:

  ListResourceBundle lrb = 
    ListResourceBundle.getBundle ("gnu/java/locale/LocaleInformation", locale);
  String format_string = lrb.getString ("format_string");

In the actual resource bundle class, here is how the configuration option gets defined:

/**
  * This is the format string used for displaying values
  */
private static final String format_string = "%s %d %i";

private static final Object[][] contents =
{
  { "format_string", format_string }
};

Note that each variable should be private, final, and static. Each variable should also have a description of what it does as a documentation comment. The getContents() method returns the contents array.

There are many functional areas of the standard class library that are configured using this mechanism. A given locale does not need to support each functional area. But if a functional area is supported, then all of the specified entries for that area must be supplied. In order to determine which functional areas are supported, there is a special key that is queried by the affected class or classes. If this key exists, and has a value that is a Boolean object wrappering the true value, then full support is assumed. Otherwise it is assumed that no support exists for this functional area. Every class using resources for configuration must use this scheme and define a special scheme that indicates the functional area is supported. Simply checking for the resource bundle's existence is not sufficient to ensure that a given functional area is supported.

The following sections define the functional areas that use resources for locale specific configuration in GNU Classpath. Please refer to the documentation for the classes mentioned for details on how these values are used. You may also wish to look at the source file for gnu/java/locale/LocaleInformation_en as an example.

String Collation

Collation involves the sorting of strings. The Java class library provides a public class called java.text.RuleBasedCollator that performs sorting based on a set of sorting rules.

Note that some languages might be too complex for RuleBasedCollator to handle. In this case an entirely new class might need to be written in lieu of defining this rule string.

Break Iteration

The class java.text.BreakIterator breaks text into words, sentences, and lines. It is configured with the following resource bundle entries:

Date Formatting and Parsing

Date formatting and parsing is handled by the java.text.SimpleDateFormat class in most locales. This class is configured by attaching an instance of the java.text.DateFormatSymbols class. That class simply reads properties from our locale specific resource bundle. The following items are requiered (refer to the documentation of the java.text.DateFormatSymbols class for details io what the actual values should be):

Note that it may not be possible to use this mechanism for all locales. In those cases a special purpose class may need to be written to handle date/time processing.

Decimal/Currency Formatting and Parsing

NumberFormat is an abstract class for formatting and parsing numbers. The class DecimalFormat provides a concrete subclass that handles this is in a locale independent manner. As with SimpleDateFormat, this class gets information on how to format numbers from a class that wrappers a collection of locale specific formatting values. In this case, the class is DecimalFormatSymbols. That class reads its default values for a locale from the resource bundle. The required entries are:

Note that several of these values are an individual character. These should be wrappered in a String at character position 0, not in a Character object.


This document was generated on 9 April 1999 using the texi2html translator version 1.54.


Return to GNU's home page.

Please send FSF & GNU inquiries & questions to gnu@gnu.org. There are also other ways to contact the FSF.

Please send comments on these web pages to webmasters@www.gnu.org, send other questions to gnu@gnu.org.

Copyright (C) 1999 Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA

Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.

Updated: 09 Apr 1999 unknown