As you start to use LDAP-based servers in your organization, you'll no doubt discover that you want to synchronize data between them. This will be the case if you've set up multiple servers across your organization to increase reliability by ensuring that if something happens to one of the machines, you won't lose access to all of your LDAP data. It will also apply if you've set up a series of caching servers in order to speed up LDAP lookups, or if you have several servers from different vendors (such as Sun's NIS and/or Microsoft Exchange Server) that speak LDAP. Finally, it will be true if you're using a centralized LDAP-based server as a metadirectory (more on this in a minute), to manage the data from legacy mainframe applications or existing relational database management systems (RDBMS).

In all these scenarios, you'll want the data in two (or more) separate systems to be maintained in a consistent manner. For example, your organization may have a Netscape Directory Server that's the authoritative source for the names, managers, and phone numbers of employees in the organization, and that also gives their email addresses. You may also have a Microsoft Exchange Server for internal email communications that's the authoritative database for employees' email addresses. If you change the surname of a recently married employee in your Directory Server, you'll want that change to be reflected in the Exchange Server. Or if in the Exchange Server you change the email address of an employee, you'll want that change reflected in the Directory Server. Synchronization is the process by which these updates occur.

In this column, I'll show you how to synchronize LDAP-based servers using the standard command-line tools of ldapsearch and ldapmodify. These tools have been with us since the first University of Michigan servers and come with the C Software Development Kit (SDK) and the Netscape Directory Server. They give us a quick and easy way to synchronize data. You could also write your own synchronization application using any of the SDKs, but in this article I'll stick with the command-line utilities and a favorite friend of mine, the Perl programming language.

At some point you may find yourself in a situation where you want to synchronize data between an LDAP-based server like Netscape Directory Server and a non-LDAP database, such as a PeopleSoft database, but this is a different situation and one I'll discuss in a future column.

SYNCHRONIZATION AND METADIRECTORIES

Before showing you the details of my synchronization method, I want to briefly mention metadirectories, because many people are confused by the concept. A metadirectory (which may also be called a registry) is a centralized LDAP-based server that brings together and synchronizes many different databases in a standard format. A metadirectory will be filled with data from a variety of sources, such as existing mainframe data, RDBMs data, existing proprietary directory service data (like data on a Sun NIS), and data from other LDAP-based servers. The metadirectory server feeds all of these other sources with updates made to the metadirectory. This is accomplished with synchronizing agents such as the program I'm about to show you or the PerLDAP for Peoplesoft.

Setting up a metadirectory server isn't an easy project and should be implemented in stages. For more ideas about metadirectories, check out the Stanford Registries Project.

THE SYNCHRONIZATION PROGRAM

In writing the synchronization program I'm going to show you, I chose to use Perl because of its "duct tape" qualities. We need to tape together into one application the search parameters for both LDAP-based servers, the attributes to modify, and error trapping. Perl's abilities to control external processes, in this case calling ldapsearch or ldapmodify, and to process text, make it my personal favorite choice.

Our program to synchronize two LDAP-based servers consists of four parts:

• ldapsearch - the ldapsearch command-line tool
• ldapmodify - the ldapmodify command-line tool
• ldiffer.cfg - a text file that contains all of the configuration information we need
• ldiffer.pl - the Perl "glue" script

The ldapsearch and ldapmodify command-line tools are standard utilities, and if you want to learn more about them you can consult the help files that come with Netscape Directory Server. Here we'll focus on the ldiffer.cfg and ldiffer.pl files.

THE CONFIGURATION FILE

The ldiffer.cfg file contains the configuration information we need to synchronize our LDAP-based servers. This is where we indicate which databases and attributes to synchronize and which database is the "root authority" for each attribute. For example, we may decide that our Human Resources database is the root authority for employees' names, but that their email addresses should come from our Exchange Server and their phone numbers should come from our PBX software. In this case we'll need to have a metadirectory for this information, and that will be our Netscape Directory Server.

Listing 1 is an example ldiffer.cfg file. It's just a simple text file. Comments (lines starting with "#") and blank lines are ignored. Each line consists of a field name followed by a colon and a value or values. This file uses the notion of a source server (fields starting with src_) and a destination server (fields starting with dst_), but we could just as easily have used the labels "X" and "Y" for the two servers.

The sample code included here is provided for your use on an "AS IS" basis, under the Netscape License Agreement - Terms of Use.

src_D:cn=Directory Manager
src_b:o=Airius.com
src_filter:uid=*
src_h:src_host
src_p:389
src_w:src_DN_passwd
src_key_attrs:sn,givenName
src_comp_attrs:mail,telephonenumber
src_masters:telephonenumber;telephonenumber
dst_D:cn=Directory Manager
dst_b:o=exchange.airius.com
dst_filter:MAPI-Recipient=TRUE
dst_h:dst_host
dst_p:389
dst_w:dst_DN_passwd
dst_key_attrs:sn,givenName
dst_comp_attrs:mail,telephonenumber
dst_masters:mail;mail

# The next three lines aren't used by the ldiffer.pl script. They
# indicate whether an entry should be added, modified, or deleted.
# The "1" after "mods" indicates that an entry should be modified.
# The modify value also works as "add" if the attribute doesn't exist
# yet.

adds:
mods:1
dels:
ldapmodify:./ldapmodify
ldapsearch:./ldapsearch
debug:1

Here's a description of each field:

src_D	Distinguished name of the source server's Directory Manager.
src_b	Source server's search base. In our example, it's the database for the organization ("o") named Airius.com.
src_filter	Search filter to use on the search base for the source server. In our example, it's the user ID ("uid").
src_h	Hostname for the source server.
src_p	Port for the source server.
src_w	Password for the Directory Manager of the source server.
src_key_attrs	Key attributes of a source server entry used to determine the matching entry on the destination server. In our example, we assume that using the attributes "sn" and "givenName" will be sufficient to identify matching entries; you may need to use different attributes.
src_comp_attrs	Attributes to compare with those on the destination server. In our sample, these are "mail" and "telephone number."
src_masters	Attribute pairs for which the source server is the authoritative (master) source. The attribute named on the left is found on the source server, while the one named on the right is on the destination server. In our example, these are "telephonenumber" and "telephonenumber." These attributes are always in pairs; if you have more than one pair, separate the pairs with a comma.
dst_D	Distinguished name of the destination server's Directory Manager.
dst_b	Destination server's search base. In the example, it's the Exchange Server with the hostname of exchange.airius.com.
dst_filter	Search filter to use on the search base for the destination server. Note that I'm assuming that our destination server contains at least one user entry with an attribute called MAPI-Recipient and that the value for this attribute can be TRUE or FALSE (case is insensitive). This filter will be combined with the source key attributes and their values from the search using the source server's filter.
dst_h	Hostname for the destination server.
dst_p	Port for the destination server.
dst_w	Password for the Directory Manager of the destination server.
dst_key_attrs	Key attributes of a destination server entry used to identify a matching entry on the source server. In our example, we assume that using the attributes "sn" and "givenName" will be sufficient to identify matching entries; you may need to use different attributes.
dst_comp_attrs	Attributes to compare with those on the source server. In our example, these are "mail" and "telephonenumber."
dst_masters	Attribute pairs for which the destination server is the authoritative (master) source. The attribute named on the left is found on the source server, while the one named on the right is on the destination server. In our example, these are "mail" and "mail." These attributes are always in pairs; if you have more than one pair, separate the pairs with a comma.
adds	Instruction to add a new entry. Not used in our example.
mods	Instruction to modify any entries. Not used in our example.
dels	Instruction to delete any entries. Not used in our example.
ldapmodify	Path to the ldapmodify command-line utility.
ldapsearch	Path to the ldapsearch command-line utility.
debug	Instruction to the ldiffer.pl application to print out some information.

In a nutshell, then, we want to synchronize the values of the "mail" and "telephonenumber" attributes for entries whose "sn" and "givenName" attributes match on the source and destination servers. The source server is the authoritative source for the "telephonenumber" attribute, while the destination server is the authoritative source for the "mail" attribute.

Now let's look at how the configuration file is used in our synchronization program.

THE PERL SCRIPT

The Perl script I've written, ldiffer.pl, uses the configuration file plus ldapsearch and ldapmodify to synchronize data between two different LDAP-based servers. It's especially useful in cases where you need to synchronize LDAP data between servers from two different vendors or wish to have more control over the replication of your data.

Listing 2 is the ldiffer.pl script. Note that this script doesn't check for existing values in entries and will totally replace any existing data, and if an attribute has multiple values, the script will replace all existing values, so you use it at your own risk.While I've tested it out (against a test server I set up on a Windows NT server using Directory Server 3 and Perl 5.0002, the non-ActiveState Perl port) to verify that it works, it's not supported or certified for use in a production environment. The script comments should explain what's going on, but here's a brief outline of what the script does:

1. Reads in the configuration file. Uses the hard-coded defaults in the script to fill in any blanks. You can change these hard-coded defaults in the AskCfg subroutine.
2. Searches the source server using the src_* parameters.
3. Cycles through each of the results from the previous search, if any. Proceeds if no error.
4. Prepares a search of the destination server using the attributes specified in the dst_key_attrs field and the values of the src_key_attrs field obtained from the entry in the source search.
5. Searches the destination server.
6. Compares the source server's comp_attrs values with the destination server's comp_attrs values. If they match, proceeds; otherwise, gets next source server's search entry.
7. Updates the destination server's attributes if the source server is the master server for any attribute(s).
8. Updates the source server's attributes if the destination server is the master server for any attribute(s).
9. Exits.

# Read in the configuration file.
if (defined($ARGV[0])) {
    $cfg_file = $ARGV[0];
} else {
    die "You must provide a config file name!\n";
}
print "Working.\n";

# Check if all $cfg vars are defined. If there are undefined ones,
# run &AskCfg and &WriteCfg.
if (&ReadCfg($cfg_file)) {
} else {
    if (!(&AskCfg)) {
      die "Error in AskCfg\n";
    }
    if (!(&WriteCfg)) {
      die "Error in WriteCfg\n";
    }
}

# Open the log.
&OpenLog;
$/ = "";

$addcnt = 0;
$modcnt = 0;
$delcnt = 0;
$srccnt = 0;
$dstcnt = 0;
$totcnt = 0;
$debug = $cfg{debug};
$src_search_string = " -b \"$cfg{src_b}\"";
$src_search_string .= " -h $cfg{src_h}" if ($cfg{src_h});
$src_search_string .= " -p $cfg{src_p}" if ($cfg{src_p});
$src_search_string .= " -D \"$cfg{src_D}\"" if ($cfg{src_D});
$src_search_string .= " -w \"$cfg{src_w}\"" if ($cfg{src_w});
$src_mod_string = "";
$src_mod_string .= " -h $cfg{src_h}" if ($cfg{src_h});
$src_mod_string .= " -p $cfg{src_p}" if ($cfg{src_p});
$src_mod_string .= " -D \"$cfg{src_D}\"" if ($cfg{src_D});
$src_mod_string .= " -w \"$cfg{src_w}\"" if ($cfg{src_w});

$dst_search_string = " -b \"$cfg{dst_b}\"";
$dst_search_string .= " -h $cfg{dst_h}" if ($cfg{dst_h});
$dst_search_string .= " -p $cfg{dst_p}" if ($cfg{dst_p});
$dst_search_string .= " -D \"$cfg{dst_D}\"" if ($cfg{dst_D});
$dst_search_string .= " -w \"$cfg{dst_w}\"" if ($cfg{dst_w});
$dst_mod_string = "";
$dst_mod_string .= " -h $cfg{dst_h}" if ($cfg{dst_h});
$dst_mod_string .= " -p $cfg{dst_p}" if ($cfg{dst_p});
$dst_mod_string .= " -D \"$cfg{dst_D}\"" if ($cfg{dst_D});
$dst_mod_string .= " -w \"$cfg{dst_w}\"" if ($cfg{dst_w});

# Next, convert our scalar data (variables that start with a $) into
# arrays. See www.Perl.com for
# more information about Perl variables.

@src_key_attrs = split(/,/,$cfg{src_key_attrs});
@src_comp_attrs = split(/,/,$cfg{src_comp_attrs});
@src_masters = split(/,/,$cfg{src_masters});
@dst_key_attrs = split(/,/,$cfg{dst_key_attrs});
@dst_comp_attrs = split(/,/,$cfg{dst_comp_attrs});
@dst_masters = split(/,/,$cfg{dst_masters});

# Read source sequentially. Note that we're reading data in from the
# command line ldapsearch utility. That's why the "|" is in the open
# statement. Check out www.Perl.com
# for more information about using
# command line commands like this.

print "$cfg{ldapsearch} $src_search_string
\"$cfg{src_filter}\"\n" if ($debug);
open (SRC, "$cfg{ldapsearch} $src_search_string $cfg{src_filter} |");
while (<SRC>) {
    $srccnt++;
    $src_curr_rec = $_;

# Here's where we need to handle the list of src_key_attrs.
    $src_key_match = 1;
    @keys = ();
    foreach $src_key_attr (@src_key_attrs) {
 if (/\n$src_key_attr: (.*)\n/i) {
     push @keys, $1;
 } else {
     $src_key_match = 0;
 }
    }
 if ($src_key_match) {
 print "srckeys:@keys:\n" if ($debug);
 $hash{@keys} = 1;

# Build up proper destination filter.
 $current_filter = "\"(&"; # hack hack hack
 for ($i = 0; $i <= $#keys; $i++) {
     $current_filter =
"$current_filter($dst_key_attrs[$i]=$keys[$i])";
 }
 if ($cfg{dst_filter}) {
     $current_filter = "$current_filter($cfg{dst_filter})";
        }
 $current_filter = "$current_filter)\""; # hack hack hack

# Read destination randomly.
print "$cfg{ldapsearch}
$dst_search_string $current_filter\n" if ($debug);
 open (DST, "$cfg{ldapsearch} $dst_search_string $current_filter |");
 while (<DST>) {
     $dstcnt++;
     $dst_curr_rec = $_;

# Here's where we need to handle the list of dst_key_attrs.
     $dst_key_match = 1;
     @keys = ();
     foreach $dst_key_attr (@dst_key_attrs) {
  if (/\n$dst_key_attr: (.*)\n/i) {
      push @keys, $1;
  } else {
      $dst_key_match = 0;
  }
     }
     if ($dst_key_match) {
          print
"dstkeys:@keys:\n" if ($debug);
  if (defined($hash{@keys})) {

# We found it in both source and destination; check if they match.
      $match = 1;
      for ($i = 0; $i <= $#src_comp_attrs; $i++) {
           $src_field = "";
           $dst_field = "";
   if ($src_curr_rec =~ /\n$src_comp_attrs[$i]: (.*)\n/i) {
       $src_field = $1;
   }
   if ($dst_curr_rec =~ /\n$dst_comp_attrs[$i]: (.*)\n/i) {
       $dst_field = $1;
   }
   if ($src_field ne $dst_field) {
       $match = 0;
   }
      }
      if (!($match)) {
   $modcnt++;
   print LOG "MOD: @keys\n";

# Grab the relevant dn's.
   $src_dn = "";
   if ($src_curr_rec =~ /^dn: (.*)\n/i) {
       $src_dn = $1;
   }
   $dst_dn = "";
   if ($dst_curr_rec =~ /^dn: (.*)\n/i) {
       $dst_dn = $1;
   }

# Iterate through all attrs mastered by source.
   for ($i = 0; $i <= $#src_masters; $i++) {
       ($src_attr,$dst_attr) =
split(/;/,$src_masters[$i]);
       print
"LOOK:src_attr:$src_attr,dst_attr:$dst_attr\n";

# Try to find the src_attr in the src_record.
       if ($src_curr_rec =~
/\n$src_attr:(.*)\n/i) {
    $src_value = $1;

# If you find the attr in the source, modify it in the destination!
    print "$cfg{ldapmodify} $dst_mod_string";
    open(DST_MOD, "| $cfg{ldapmodify} $dst_mod_string");
    print DST_MOD "dn: $dst_dn\n";
    print "dn: $dst_dn\n";

# Check if it's a modify of an existing attr or an add of a new attr.
    print DST_MOD "changetype: modify\n";
    print "changetype: modify\n";

# Check if it's an add of a multivalued attr or a replace of an
# existing attr.
    print DST_MOD "replace: $dst_attr\n";
    print "replace: $dst_attr\n";
    print DST_MOD "$dst_attr: $src_value\n";
    print "dst_attr: $src_value\n";
    close (DST_MOD);
       }
   }

# Iterate through all attrs mastered by destination.
   for ($i = 0; $i <= $#dst_masters; $i++) {
       ($src_attr,$dst_attr) =
split(/;/,$dst_masters[$i]);

# Try to find the dst_attr in the dst_record.
       if ($dst_curr_rec =~
/\n$dst_attr:(.*)\n/i) {
    $dst_value = $1;

# If you find the attr in the destination, modify it in the source!
    open(SRC_MOD, "| $cfg{ldapmodify} $src_mod_string");
    print SRC_MOD "dn: $src_dn\n";

# Check if it's a modify of an existing attr or an add of a new attr.
    print SRC_MOD "changetype: modify\n";

# Check if it's an add of a multivalued attr or a replace of an
# existing attr.
    print SRC_MOD "replace: $src_attr\n";
    print SRC_MOD "$src_attr: $dst_value\n";
    close (SRC_MOD);
       }
   }
      }
  } else {

# We didn't find it in the destination; should do an add.
      $addcnt++;
      print LOG "ADD: @keys\n";
  }

# Mark that it's been processed.
  $hash{@keys} = 0;
     } else {
  print "Can't find entry with key: @keys in destination\n";
     }
 }
 close (DST);
    } else {
 print "Can't find all source key attributes in: $src_curr_rec\n";
    }
}
close (SRC);

# Read through hash for unmatched records to build delete list.
foreach $key (keys %hash) {
    $totcnt++;
    if ($hash{$key}) {
 print LOG "DEL: $key\n";
 $delcnt++;
    }
}
print LOG "addcnt: $addcnt\n";
print LOG "modcnt: $modcnt\n";
print LOG "delcnt: $delcnt\n";
print LOG "srccnt: $srccnt\n";
print LOG "dstcnt: $dstcnt\n";
print LOG "totcnt: $totcnt\n";

&CloseLog;

# Exit.

################## Start subroutines ###################

sub ReadCfg {
    if (open (CFG, "<$cfg_file")) {
 while (<CFG>) {
     chomp;
     if ((/^#/) || (/^$/)) {

# It was a comment or a blank line; ignore it.
     } else {
         my ($lhs,$rhs);
  ($lhs,$rhs)=split(/:/,$_);
  $cfg{$lhs}=$rhs;
     }
 }
 close (CFG);
 return 1;
    } else {
 return 0;
    }
}
sub AskCfg {

# Set default values.
    $cfg{src_D}="cn=Directory Manager"
 unless ($cfg{src_D});
    $cfg{src_b}="o=Airius.com"
 unless ($cfg{src_b});
    $cfg{src_filter}="objectclass=*"
 unless ($cfg{src_filter});
    $cfg{src_h}="hostname"
 unless ($cfg{src_h});
    $cfg{src_p}="389"
 unless ($cfg{src_p});
    $cfg{src_w}="password"
 unless ($cfg{src_w});
    $cfg{dst_D}="cn=Directory Manager"
 unless ($cfg{dst_D});
    $cfg{dst_b}="o=Airius.com"
 unless ($cfg{dst_b});
    $cfg{dst_filter}="objectclass=*"
 unless ($cfg{dst_filter});
    $cfg{dst_h}="hostname"
 unless ($cfg{dst_h});
    $cfg{dst_p}="389"
 unless ($cfg{dst_p});
    $cfg{dst_w}="password"
 unless ($cfg{dst_w});
   
$cfg{ldapmodify}="/usr/netscape/suitespot/userdb/ldap/tools/ldapmodify"
 unless ($cfg{ldapmodify});
   
$cfg{ldapsearch}="/usr/netscape/suitespot/userdb/ldap/tools/ldapsearch"
 unless ($cfg{ldapsearch});
   
$cfg{ldif}="/usr/netscape/suitespot/slapd-hostname/ldif/Airius.ldif"
 unless ($cfg{ldif});
    foreach $cfg (sort (keys(%cfg))) {
 print "Enter the value for $cfg (default: $cfg{$cfg})\n";
 chomp($answer = <STDIN>);
 $cfg{$cfg} = $answer if ($answer);
    }
    return 1;
}
sub WriteCfg {

# This thing has a side effect of trashing all the comments.
    open (CFG, ">ldiffer.cfg");
    foreach $cfg (sort (keys(%cfg))) {
 print CFG "$cfg:$cfg{$cfg}\n";
    }
    return 1;
}
sub ReadLDAP {
    my($dir) = @_;
    open ($dir, "$cfg{ldapsearch} -b \"$cfg{b}\" -h $cfg{h}
-p $cfg{p} -D \"$cfg{D}\" -w \"$cfg{w}\" \"$cfg{filter}\" |");
    while (<$dir>) {
 print;
    }
    close ($dir);
    return 1;
}

# This routine ignored right now.
sub ReadLDIF {
    my($dir) = @_;

   
open($dir,"</export/home/ss35/slapd-bklynsun/ldif/Airius.ldif");
    while (<$dir>) {
 print;
    }
    close ($dir);
    return 1;
}

# This routine ignored right now.
sub WriteDir {
    return 1;
}
sub OpenLog {
    open (LOG, ">ldiffer.log");
}
sub CloseLog {
    close (LOG);
}

In order to run this script, Unix/LINUX users will need to make sure that the path to their Perl executable is set correctly on the first line and that they've set the executable permissions for the script. Then they'll need to type ldiffer.pl ldiffer.cfg at the Unix command prompt (%). Windows users (and Unix/LINUX users who don't want to use the Perl executable) should type Perl ldiffer.pl ldiffer.cfg at the command prompt (C:\ for Windows, % for Unix/LINUX). If you want to add attributes to the master list, you need to add them to both the master field (either src_masters or dst_masters) for the server that will be the master source and to the compare field (src_compare_attrs and dst_compare_attrs) for both servers.

Be forewarned that not all systems will be able to accept all of the data of the other, even if both can accept similar data. For example, in the case of passwords, the storage format will likely be different (Directory Server uses SHA-1 encryption and the NIS server uses Unix crypt, for instance). So you'll have to come up with a different way to synchronize that kind of data. One way you might do this is to create a secure web site where users fill in a form on a web server using Secure Socket Layers (SSL). The form sends its data to a CGI script, which then makes the necessary API calls to change your password on all affected systems.

After you get this script working, you can set it up to run regularly using Unix/LINUX cron or NT's Scheduler service. If the fact that this script doesn't check for multiple values in any updated attribute is a problem for you, let me know. If you figure out a solution, send it to me. I'll post it in a future column.

OTHER USES FOR LDAPSEARCH AND LDAPMODIFY

The command-line tools ldapsearch and ldapmodify provide a quick and easy way to manipulate LDAP data aside from synchronization.

For example, when I'm writing a new LDAP application and need to build a query, I often test the query out with the ldapsearch command. At a command prompt (the % on Unix or C:\ on Windows) I might type something like this:

ldapsearch -h "ldap.mycompany.com" -b
"o=airius.com" "(uid=scarter)"

I use the ldapmodify command to update the LDAP server. For example, the following typed at a command prompt would add a value of bigboss@airius.com to the mail attribute of an entry:

ldapmodify -h "ldap.mycompany.com" -D
"uid=some_user_that_can_update_records,o=airius.com" -w "password"
dn: uid=some_existing_user,ou=People,o=airius.com
changetype: modify
add:mail
mail: bigboss@airius.com

If the mail attribute didn't exist yet (and it was allowed by the schema of the entry's object class), the mail attribute would be added first and then the value. The "changetype" value can be add, modify, or delete. Modify also works as add if the attribute doesn't yet exist. If the attribute is multivalued (for example, you have more than one common name), all the existing values will be replaced with the single value you pass.

If you're new to LDAP or to the command-line utilities, you should notice that the command-line utilities all use the LDAP Data Interchange Format (LDIF), the human readable version of LDAP data. If you're familiar with SQL-based database applications, you can think of LDIF as the "query" language for LDAP. This is how we pass LDAP data between user and server.

AU REVOIR

The script I've presented here synchronizes two LDAP-based servers that have different roots and directory tree structures. Using scripts such as this one will give you better control over your synchronization processes with non-LDAP resources (such as relational databases) until more useful synchronization technologies appear. In future columns I'll show how to create a similar system with one of the SDKs and how to synch up with existing non-LDAP systems like databases.

Until next time.

FURTHER RESOURCES

• Directory Developer Central, Netscape DevEdge Online
• www.Perl.com
• Stanford Registries Project
• Tools for LDAP, mozilla.org

Thanks to David McNeely for helping to get this script out.

Mark Wilcox is the web administrator at the University of North Texas. He's been involved with LDAP since the alpha-release days of Netscape Directory Server 1.0. When not working with web technologies, he likes to spend time with his wife.

---

Related Readings:

• Frequently Asked Questions: Find the answers to your questions on Internet technologies.
• Documentation: A library of technical information.
• Developer Central: Collections of developer resources organized by Internet technology areas.

Any sample code included above is provided for your use on an "AS IS" basis, under the Netscape License Agreement - Terms of Use