★ wanayoo — archive 1999 http://www.volano.com/linuxnotes.htmlNouvelle recherche | Portail wanayoo

Volano LLC

 Support

 · Home
 · News
 · Awards
 · Demos
 · Products
 · Documentation
 · Browser Support
 · Server Support
    · FreeBSD
    · Linux
    · Mac OS
    · NetWare
    · OS/2 Warp
    · Solaris
    · Windows NT
 · Customers
 · Partners
 · Benchmarks
 · Company
 · Site Map

Note that the information on this page has been superseded by our new Linux Support page. The information on this page is provided for reference in rebuilding the Linux kernel and thread library, and for installing the TowerJ version of VolanoChat.

Linux

Linux 2.2 kernel file descriptors and tasks

When running on the Linux 2.2 kernel, each process is limited by default to 1,024 file descriptors. Since the VolanoChat server requires one file descriptor for each person chatting through a VolanoChat applet, you are limited by default to a maximum of 1,024 simultaneous users when using the Linux 2.2 kernel.

Some Java virtual machines, such as the IBM Developer Kit for Linux, Java Technology Edition, use native threads on Linux which require one task for each Java thread. Since VolanoChat requires two Java threads for each connection, you may need to increase the maximum number of tasks per user for such virtual machines. The default maximum number of tasks in a Linux 2.2 system is 512, with a maximum of 256 tasks per user. So by default, a VolanoChat server running under IBM JDK 1.1.6 for Linux can support a maximum of 128 people chatting at the same time.

You can increase the limits on the number of file descriptors and the number of tasks in the Linux 2.2 kernel by following the steps below. These steps were performed on a Red Hat Linux 6.0 system using Linux kernel 2.2.5-15. The instructions may be different for your Linux system.

To increase the maximum file descriptors per process to 4,096 (from 1,024) and the maximum tasks per user to 4,090 (from 256) on Linux 2.2, do the following steps.

  1. Change the following C header files.

    In /usr/include/linux/tasks.h, change: 

    NR_TASKS            512          -> 4090
MAX_TASKS_PER_USER  (NR_TASKS/2) -> NR_TASKS

    In /usr/include/linux/limits.h, change: 

    NR_OPEN             1024 -> 4096
OPEN_MAX             256 -> 4096

    In /usr/include/linux/posix_types.h, change: 

    __FD_SETSIZE        1024 -> 4096

    In /usr/include/bits/types.h, change: 

    __FD_SETSIZE        1024 -> 4096
  2. To allow users to increase their file descriptor limits, change the following configuration files:

    In /etc/security/limits.conf, add the lines: 

    *       soft    nofile  1024
*       hard    nofile  4096

    In /etc/pam.d/login, add: 

    session required /lib/security/pam_limits.so
  3. To increase the system-wide file descriptor limit, add the following three lines to the /etc/rc.d/rc.local startup script: 
    # Increase system-wide file descriptor limit.
echo  8192 > /proc/sys/fs/file-max
echo 24576 > /proc/sys/fs/inode-max
  4. Change to the directory /usr/src/linux and set up the kernel sources with: 
    make mrproper
  5. Make any other kernel configuration changes, such as setting the processor family type, by entering the command below. Save your changes and exit the configuration panel when you are done. 
    make xconfig
  6. Prepare for the kernel build by entering: 
    make dep
make clean
  7. Rebuild your kernel with the commands: 
    make bzImage
make modules
  8. Backup your old modules directory by entering (for example): 
    mv /lib/modules/2.2.5-15 /lib/modules/2.2.5-15.old
  9. Create a new set of kernel modules with the command: 
    make modules_install
  10. Backup your old kernel by entering (for example): 
    mv /boot/vmlinuz /boot/vmlinuz.old
  11. Copy over your new kernel image with: 
    cp /usr/src/linux/arch/i386/boot/bzImage /boot/vmlinuz
  12. Edit the Linux loader configuration in /etc/lilo.conf so that your new kernel is the default and your old kernel is still available from the boot prompt, as in the example shown below: 
    boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
image=/boot/vmlinuz
        label=linux
        root=/dev/hda1
        read-only
image=/boot/vmlinuz.old
        label=linux.old
        root=/dev/hda1
        read-only
  13. Generate the new Linux loader configuration by running: 
    /sbin/lilo
  14. Shutdown and reboot with your new kernel by entering: 
    shutdown -r now
  15. After rebooting, verify that the changes were effective as shown in the sections below.

LinuxThreads 0.7 limits

Even after you rebuild your Linux 2.2 kernel to increase the maximum number of threads per Linux user from 256 to 4,090, there are still definitions in the thread library itself which limit the number of threads per process to less than 1,000. The VolanoChat server requires two threads for each person chatting through a VolanoChat applet, so the LinuxThreads implementation creates a limit of 500 people to a single VolanoChat server.

You can increase this limit to 8,192 threads with the following steps. Even though the thread library will allow 8,192 threads per process after these changes, you'll still be limited by the Linux 2.2 kernel to a maximum of 4,090 threads for the Linux user account running the VolanoChat server after the changes in the previous section. The steps below are shown for a Red Hat Linux 6.0 system, so they may be different for your Linux distribution.

  1. Install the Glibc 2.1.1 source package from /mnt/cdrom/SRPMS/glibc-2.1.1-6.src.rpm on the Red Hat Linux CD-ROM #2.
  2. Unpack the source archive /usr/src/redhat/SOURCES/glibc-990416.tar.gz so that you can modify two of the files.
  3. Change the following C header files.

    In /usr/src/redhat/SOURCES/glibc/linuxthreads/internals.h, change the size of the thread stack reserve from 2 megabytes down to 256 kilobytes (with a page size of 4,096 bytes): 

    STACK_SIZE  (2 * 1024 * 1024) -> (64 * PAGE_SIZE)

    In sysdeps/unix/sysv/linux/bits/local_lim.h under /usr/src/redhat/SOURCES/glibc/linuxthreads, change the Posix thread implementation limit from 1,024 per process to 8,192 per process: 

    PTHREAD_THREADS_MAX  1024 -> 8192
  4. Rebuild the source archive, /usr/src/redhat/SOURCES/glibc-990416.tar.gz, to include your changes, replacing the original copy.
  5. Change to the RPM specification directory, /usr/src/redhat/SPECS, and enter the command: 
    rpm -ba glibc-2.1.spec
  6. After a few hours, the new LinuxThreads library will be found as libpthread.so under: 
    /usr/src/redhat/BUILD/glibc/build-i386-linux/linuxthread
  7. Copy the new thread library into your own lib subdirectory: 
    cd /usr/src/redhat/BUILD/glibc/build-i386-linux
cp linuxthread/libpthread.so ~/lib
cd ~/lib
ln -s libpthread.so libpthread.so.0
  8. Define the environment variable LD_LIBRARY_PATH to include the subdirectory containing the new thread library libpthread.so.
  9. Verify the new thread limits as instructed below.

Verifying the new file descriptor limit

Increase your new limits to their maximum by entering "ulimit -n 4096" for the Bash shell or "unlimit" for the C shell. You can then check whether your changes were effective by running the following C program: 

#include <sys/types.h>
#include <fcntl.h>
#include <stdio.h>

void main(void) {
  int k = 3;
  while (dup(0) != -1)
    k++;
  printf("Opens = %d Fdsize = %d\n", k, sizeof(fd_set) * 8);
}

Both numbers printed should give your new file descriptor limit.

Verifying the new thread limit

You can use the following program to verify your new thread limits in the modified Linux 2.2 kernel and in the modified LinuxThreads library. 

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>

#define MAX_THREADS 10000
int i;

void run(void) {
  char c;
  if (i < 10)
    printf("Address of c = %u KB\n", (unsigned int) &c / 1024);
  sleep(60 * 60);
}

int main(int argc, char *argv[]) {
  int rc = 0;
  pthread_t thread[MAX_THREADS];
  printf("Creating threads ...\n");
  for (i = 0; i < MAX_THREADS && rc == 0; i++) {
    rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL);
    if (rc == 0) {
      pthread_detach(thread[i]);
      if ((i + 1) % 1000 == 0)
        printf("%i threads so far ...\n", i + 1);
    }
    else
      printf("Failed with return code %i creating thread %i.\n",
              rc, i + 1);
  }
  exit(0);
}

Compile the program with: 

gcc -lpthread -o count count.c

Running with the original LinuxThreads library will print: 

blue:/home/john> count
Creating threads ...
Address of c = 3137535 KB
Address of c = 3135487 KB
Address of c = 3133439 KB
Address of c = 3131391 KB
Address of c = 3129343 KB
Address of c = 3127295 KB
Address of c = 3125247 KB
Address of c = 3123199 KB
Address of c = 3121151 KB
1000 threads so far ...
Failed with return code 11 creating thread 1023.

while running with the modified LinuxThreads library will print: 

blue:/home/john> count
Creating threads ...
Address of c = 3144703 KB
Address of c = 3144447 KB
Address of c = 3144191 KB
Address of c = 3143935 KB
Address of c = 3143679 KB
Address of c = 3143423 KB
Address of c = 3143167 KB
Address of c = 3142911 KB
Address of c = 3142655 KB
1000 threads so far ...
2000 threads so far ...
3000 threads so far ...
4000 threads so far ...

You may need to enter Ctrl-C to exit the program.

Notice that the thread stack addresses are spaced apart by 256 kilobytes instead of the earlier reserve of 2 megabytes per stack. Also notice that the program can create threads right up to the Linux 2.2 kernel limit of 4,090 for the system rather than the earlier LinuxThreads limit of 1,024 per process.

TowerJ for Linux

You can use the TowerJ High Performance Java Compiler from Tower Technology to run your VolanoChat server on Linux. TowerJ did very well in our latest benchmark tests of the VolanoChat server.

After installing VolanoChat and Blackdown JDK 1.1.7 Version 3, you'll need to download the following additional files in order to run the TowerJ-compiled version of the VolanoChat server: 

ftp://ftp.volano.com/Main               - 2,226,355 bytes
ftp://ftp.volano.com/tjinterp.tjlib     - 3,393,021 bytes
ftp://ftp.volano.com/libtjrtl_t.so.31.0 -   670,053 bytes

Place all three files in the VolanoChat installation directory, called vchat2.1.3 by default. Mark the Main file as executable by entering the command "chmod +x Main".

Define the following environment variables, shown below for the UNIX C Shell with an installation of VolanoChat and Blackdown JDK 1.1.7 Version 3 in your home directory: 

setenv TOWERJ_TJLIB_PATH $home/vchat2.1.3
setenv LD_LIBRARY_PATH $home/vchat2.1.3
setenv CLASSPATH .:$home/jdk117_v3/lib/classes.zip

You can then start the TowerJ-compiled VolanoChat server by entering the command Main from the VolanoChat installation directory, as shown below: 

blue:/home/john/vchat2.1.3> Main
VolanoChat(TM) Server Version 2.1.3
Copyright (C) 1996-1999 Volano LLC.  All rights reserved.
Loading server properties from /home/john/vchat2.1.3/properties.txt.
interp: Warning:
        Error opening message file "/home/john/vchat2.1.3/lib/base.msg".
interp: Warning:
        Error opening message file "/home/john/vchat2.1.3/lib/towerj.msg".
security properties not found. using defaults.
blue.volano.com (192.168.0.3) VolanoChatPro - 5 connection limit.

You can safely ignore the warning messages that are printed.

  Copyright © 1999 Volano LLC. All rights reserved.