The Sound of Java

Thomas Dolby Robertson demonstrates the future of interactive sound

By Dave Orchard, JavaWorld

San Francisco (April 7, 1997) -- JavaOne attendees got all they could want and more at this year's JavaOne conference. Enterprise frameworks, sound, video, 3-D, and Beans were everywhere. But the loudest addition to Java stole the Pavillion and Scott McNealy's keynote. With rakish headgear and an entertainer's flair, Thomas Dolby Robertson demonstrated the future of interactive sound. While the rest of the Net has thought that push media and serialized sound was sufficient, Headspace (www.headspace.com) and JavaSoft have combined the digital revolution with sound and Java interactivity.

I had the pleasure of speaking with Chris Muir, VP of technology at Headspace, about the technology which had JavaOne abuzz. Muir explained that Sun has licensed three different pieces for inclusion in the Java Virtual Machine and class libraries -- the synthesizer, Rich Music Format (RMF), and a great content store with thousands of songs and audio clips.

The Digital Sound Brain
The 32-channel synthesizer provides the heart of the interactive experience. The interactive aspect was shown brilliantly during a realistic demonstration. While a user is looking at a sound-enabled Web page, a pager message is received. The pager message is delivered to the user by decreasing the volume of the Web page and mixing in the pager message, all in a seamless, real-time environment.

A compelling feature of the Headspace synth is the widespread system availability -- Solaris, Mac, Be, Windows, MagicCap, and a host of others.

"The content creators at Headspace are also consumers of technology, which provides a very tight feedback loop," Muir said. Clearly, insanely great developers and content authors provide a virtuous circle.

The New Virtual Machine: A Library
That's no surprise given Dolby's musician background. "Headspace is driven by artistic sense," notes Muir. "Almost everyone at Headspace is a musician."

This is particularly noticeable in the choice of the thousands of audio clips for the Java VM. "A certain attitude is evident in the sound bank and samples," explains Muir.

Future Web pages will be able to employ a megabyte of sound bytes that are included in the Java VM and thus effectively incorporate approximately one megabyte of sounds without significant download time; only an instruction file (detailing how to use the sound in the Java VM) need be downloaded. (Interestingly, it seems the Java VM is becoming not only a behavior repository but a data repository as well.) The audio clips will be in a variety of formats, including the new Headspace Rich Music Format (RMF).

Your Sounds Will Be Assimilated
The RMF format is a method of collecting and compressing certain types of audio content into its base components and through RMF and Java Sound the components can be reused over and over again. The components can be raw audio or MIDI clips. RMF is a container for all the common digital sound standards -- .au, .midi, .wav, .mod, .aiff. "The RMF content portion has musical content and audio content," Muir said. "The musical content is in the MIDI and e-MIDI standards. The audio content, such as .au, is just a stream of sound. The audio content playback can be controlled by MIDI sequencing. Also supported is the .mod standard, a combination of audio and musical content."

A variety of compression techniques are used. LZSS is used on MIDI, resulting in up to 70 percent compression. A proprietary algorithm provides two-and-a-half-times compression of audio. As we'd expect, the compression techniques are not very lossy. However, MIDI files are generally very small as they manipulate sounds in the sound box. Re-using sounds and simply sending the sound deltas provides the highest theoretical compression.

There are a couple of sections of metadata in addition to the content portion. Here as well, the author's desire for copyright protection shows through. An encrypted composer field contains copyright information about the author, publisher, and licensees. This field is intended for traceability as there is no enforcement logic in the synthesizer. By employing 40-bit encryption, Headspace retains its ability to legally export its product.

Muir also knows that categorization of sounds, indeed any stored content, is crucial to enabling content delivery. As MIT software agent researcher Pattie Maes says, "Context is King." The keyword fields enable authors to create descriptive textual information about the sounds. These, as well as the composer fields, can be searched and/or used as database entries. RMF will undergo an extension in mid-1997.

"We are very excited at the prospect of incorporating Headspace technology into our platform," said, David Rivas, staff engineer in the Java Media Engineering group at JavaSoft and architect of Java Sound and the Java Media Framework. "With the Headspace Audio Engine, the Java Platform uniquely gives content providers cross-platform, high-quality sound synthesis capabilities with the Java platform. Musicians and other content producers can now rely on audio content that always sounds as it was composed on all Java platforms."

Give Me the Sound
The Headspace environment currently is available as a plug-in for most operating systems. We can expect the sound engine and samples in JDK 1.2, followed by extensive APIs.

On the tools side, Headspace has created a "Beatnik" editor to create the RMF files. The content can then be incorporated into web pages with the help of wizards.

The Sweet Sound of Calitas
Many of the faithful were considerably impressed with Headspace's sound "bite." Sun's policy of partnering with best-of-breed is evident in the partnership between Sun and Headspace. As much as the convergence theme has been bandied about, the convergence of content, computing, and communication blares through in Java Sound. Java is poised to be the content consumer's platform of choice by the end of the millenium.

Dolby's Story

Thomas Dolby Robertson: from rock musician to Headspace founder

San Francisco (April 7, 1997) -- Thomas Dolby Robertson's career was, until the early 1990s, purely musical. His 1982 hit "She Blinded Me with Science" -- from his album titled The Golden Age of Wireless -- was one of the tunes that gained popularity primarily from the airtime the video received on MTV.

Like many rockers who have found later fame and fortune, Dolby started out as "a guitar player in Paris -- in the Metro." He moved to the States in the early 1980s because "it was the most exciting place to be, musically." Although he had always tinkered with off-the-shelf music-generating software, such as MIDI for the Mac, it was not until he assisted on a virtual-reality display for the Guggenheim Museum in the early 1990s that he began to think in terms of actually improving upon what was available.

Said Dolby: "I was leaning over a C programmer's shoulder while he was programming the music and I realized that although there was great stuff out there for making records, there was nothing for run-time interactivity." With that idea in mind, Dolby founded Headspace in 1992, hiring moonlighting software engineers to generate necessary code. Headspace incorporated in 1996.

April 3, 1997	Complete Contents