|
|
|
|||||
|
|
|||||
|
BIG SPLASH |
|||||
|
Sidebar: Java 3DTM and the Virtual FishTank
Boston's Computer Museum is home to a group of Java technology-powered fish you can feed, rename, and experiment on -- you can even build your own.
|
|||||
|
by John Zukowski, with contributions from John Papageorge
Are we at the Monterey Aquarium? Noooo. You are on the top floor of The Computer Museum in Boston, Massachusetts, although you may think you've just entered an exhibit at Walt Disney's Epcot Center. On a 70-foot wall inside a 2,200-square-foot room are twelve high-resolution projection screens defining a nearly 10-million-pixel, cartoon-like virtual fish tank. Throw in the carefully designed environmental elements, like the wavy ceiling, the water-blue carpet, and the aquatic paint job, and even a mermaid would feel at home. The Virtual FishTankTM (VFT) exhibit presents an interactive underwater aquatic environment where visitors are immersed in a world of brightly-colored mechanical fish that might seem more likely to appear in a Disney cartoon than in a program created in JavaTM technology. Different areas of the exhibit allow visitors to interact with and control the environment as they gain a feeling for how complex systems interact. For instance, inside the tank is one of the developers in a diver's suit. Outside the tank is an air control lever. Give the diver too much air and he "explodes." Give him too little, and he bangs on his helmet a few times, grows faint, and keels over. Luckily, he is reborn for the next calamity.
Experiments That TeachVisitors can introduce up to ten new fish into the environment at the three "Build Your Own Fish" stations. They can control the fish school rules at the four "Schooling" stations. Using touch-screen computers, visitors create fish by setting up rules for traits like human interaction, hunger, and water-depth preference. Each of these rules has a corresponding visual representation, like a big mouth to represent hunger. The visitor then tags the fish with an initial, releases it into the tank through a virtual metal pipe, and watches it interact with the environment. The Schooling stations enable visitors to control the behavior of all the fish in a particular school. By changing the behavior of individual fish, the behavior of whole schools changes. For instance, instructing each fish to swim in the same direction as its neighbors causes all the fish to swim together, with no centralized control. Mitchel Resnick, research professor in education at the MIT Media Lab, and Oliver Strimpel, the Museum's executive director emeritus, together lead the FishTank project. Resnick explains the educational value of the exhibit: "The Virtual FishTank aims to provide museum visitors with a more 'decentralized' perspective on the world. Most people grow up with a 'centralized mindset,' assuming that organized patterns can be created only by centralized control. This mindset affects the ways that people design (and think about) all types of systems in the world (corporate organizations, computer architectures, school classrooms, national economies). The Virtual FishTank exhibit helps people move beyond the centralized mindset, giving them a chance to create and participate in an artificial ecosystem -- and learn how patterns can, in fact, arise from decentralized interactions." The VFT illustrates "emergent behavior" -- complex behavior that emerges from interactions among individuals following basic rules. In a school of fish, each fish maintains a specific set of rules, like swimming at a certain distance from other fish, avoiding obstacles and aggressors, and following the direction of the school. The result is a complex pattern of interactions. While aimed at a target audience from eight-year-olds up to adults, even toddlers seem fascinated by the exhibit. Without understanding the principles involved, visitors can create humorous fish for the tank, using visual clues for certain behaviors, like bulging eyes to signify fear. Visitors are also amazed when the fish follow their movements at the motion-sensitive camera. Some visitors thoroughly enjoy "playing God" by controlling the environment, for example, by setting the sharks' level of hunger at "starving" so that they devour other people's newly created fish.
Development That Paid OffAccording to Gail Jannes of The Computer Museum, the week after the exhibit opened attendance was up approximately 25% over the same week a year before. And, she reports, visitor feedback has The Virtual FishTank as the museum's favorite exhibit, bar none. Developed in collaboration with the MIT Media Lab and Nearlife, Inc., a Media Lab spin-off that designed and developed the VWT, the $1.2 million exhibit was primarily funded from a $600,000 National Science Foundation grant. Over nine months, eighteen people, of whom eight were core, brought the exhibit to life on 21 networked Sony 266 MHz Pentium II computers with Diamond Fire GL 4000 3D graphic accelerator cards running Microsoft Windows NT 4.0, Java Development Kit (JDKTM) 1.2 Beta3, and Java 3DTM Alpha3. The twelve projection screens used for the tank are 1024x768 resolution Mitsubishi LCD projectors. According to Brian Knep, head of Nearlife's core technologies, "The VFT is almost entirely written in the Java programming language. The only exception is a bit of code used to read images and detect motion from a digital camera. This was done not for speed but because there is no other way to interface to the camera." Knep envisions a future where people can log onto the web and observe the fish. "There's also potential for the Virtual FishTank to be a TV experience," he notes. The remaining 99% was developed with the Java programming language, the Java 2DTM API, the Java 3D API, the Swing component set, the Java Shared Data Toolkit, and the Java Communications API. Internally, the effort used the Collections Framework, Reflection, Object Serialization, and Java Native Interface (JNI) technologies of the JavaTM 2 platform (formerly code-named "JDK"). This includes Nearlife's Directable CharacterTM architecture, which allows real-time, interactive, networked, autonomous character behaviors. The Directable Character technology scales from small single-user applets to large scale location-based entertainment experiences such as the VFT. "The Directable Character architecture is a system we found we could use for all our projects, from the immediate response of shoot-em-up style games, to the emergent behavior of artificial life, to the reactive nature of agents or bots. We also needed a system that could run on a variety of platforms," Knep remarks. "Java technology dramatically increased our productivity. It decreased edit-compile-run cycle times, eased the burden of memory management, and isolated us from the vagaries of the platform. It enabled us to develop a solution that we can bring both to the desktop, and to large, immersive, multi-machine environments." Using custom spatial data structures to limit the necessary character interactions, the Directable Character technology represents the 3D visualization as well as each character's behavioral capabilities. One server controls the state of the entire system. Each window into the FishTank is the screen of a separate computer. And the server determines behavior and detects when a fish needs to swim from one window onto the next, ensuring proper flow of motion across multiple monitors. Minimal messages are sent between systems, enabling a 12-24 frames per second animation rate.
Far Beyond Applets
Beside all the Java 3D rendering for the windows into the enormous tank, the Java Communications API (Beta 1) was used to read the environment control devices, for both the diver air volume level and the hand-crank food wheel. If you designed a custom fish to be hungry, the virtual food tube would be a good place for predatory sharks to be found, as your fish will serve as its food, while your fish searches for its own. The three Build Your Own Fish stations and the four Schooling stations mix Java 2D within Swing components for gathering input, as found with the custom sliders that grow and shrink in width around an arc of a circle. Although the sliders are moved without a physical mouse, the MicroTouch touch screens act as a virtual mouse. With the Build Your Own Fish stations, the current state of the custom fish within a virtual fish bowl is shown using Java 3D. The Java 3D API is also used to transition the input modes from one option to the next, rotating the virtual bowl where the fish is configured to the next configuration scene. The Schooling stations primarily use the Java 2D API layering capabilities to help visitors visualize the effect of the changed school behaviors.
Other development issues required the engineers to understand the internal algorithms used by Java 3D. Since the API was still in an alpha state when the Virtual FishTank went live in June 1998, it was necessary to try to understand the internal workings of the then current Java 3D API release to better optimize performance. One capability that was abandoned was the use of the 3D sound capabilities, which conflicted with the beta JDK 1.2 [the JavaTM 2 platform] sound architecture available at the time. The audio in the exhibit is a custom piece of recorded music.
Five-Step Production ProcessFor those interested in a more in-depth look at the production process, Henry Kaufman described the various elements involved in creating the VFT. It started with a five-step process, just to get animated fish, outside the fish tank environment:
The first two phases only had printed output for design reviews. Once things moved into production, the 3D Studio Max tool was used to create the fish as a hierarchy of independently moving parts (for example, the fish, fins, mouth, and so on), defined by approximately 150 to 250 triangles per fish. Once created, the appropriate models and animation sequences were output into VRML (Virtual Reality Modeling Language). Because of the need to store the model, animations, and scenes, VRML served as a good graphics file format as 3D Studio Max generates fairly good VRML output.
See AlsoThe Virtual FishTank @ The Computer Museum:
Nearlife, Inc.: The MIT Media Lab:
Exploring Emergence:
About the Author
*As used on this web site, the terms "Java virtual machine" or "JVM" mean a virtual machine for the Java platform.
|
|||
A fish swims to the glass front of a
colossal aquarium. Pausing, it blows bubbles while curiously looking
you in the eye. Then, without warning, it swishes away into the blue
distance. Across the way, others watch as a panicked pint-sized fish
races across the tank, trying to outswim the gaping maw of a hungry
pursuer. Out of the corner of your eye, a man waves his hand and calls
a fish over to him.
To engage the audience's attention, the project team came up
with appropriate names for the different fish schools. There are
two predatory fish schools, the Sharky's and Flash Gortons, as well
as two non-predatory schools, the Angels and Phil A's.
Knep observes: "From comments we have gotten about the FishTank, it
seems like many people are amazed that we did it in Java technology at all.
Most folks are used to seeing little applets on the web,
while our project is large, immersive, complex, and entertaining."
Henry Kaufman, Senior Software Developer and User Interface
Designer with Nearlife (and the diver who frequently perishes in
the FishTank), discussed several non-intuitive problems while
developing the project. For instance, with regards to the Java
virtual machine's* (JVM)
automatic garbage collection, he
states, "Even for a real-time system, it was better to let the
system just garbage collect when it needed to instead of trying to
predict when you thought it was a good time. The system just
responded better that way."
In sum, the exhibit is an amazing piece of work, given its size and relatively
short development time. The fact that it was built with early releases of Java
technologies is even more amazing -- an impressive testament to the stability of
the code from the start. So stop in next time you're in Boston, check it out --
the virtual fun starts here.
