[Editor's Note: This article was prepared by I-Kinetics for inclusion in the Third-Party Features section of View Source; although it has been reviewed for accuracy, appropriateness, originality, and editorial quality by Netscape Communications, View Source and Netscape make no guarantees about the contents of the article or the third-party products it mentions.]

The past two years have given us the explosive growth and acceptance of the Internet, the near instantaneous adoption of Java, and the rapid shift of the software industry from client-server to distributed object architectures. The adoption of object technology is motivated by a demand for fast response to requirements changes; objects facilitate complexity management. Internet-based systems provide a mechanism to rapidly and cost-effectively deliver IT to the enterprise's new frontiers.

The fact that the IT infrastructure evolution is driven by significant capability gains for the organization makes it no less uncomfortable. After the initial value is realized from the Internet's low-cost management and distribution of documents, the problem remains of how to integrate into the new infrastructure the enterprise's IT assets that do the organizational heavy lifting. Billions of lines of code, representing thousands of staff-years of domain expertise, are already in production. These mission-critical applications are strategic assets that cannot be discarded, yet redesigning and redeveloping them for the Internet's distributed object infrastructure is blocked by tremendous cost and risk.

The I-Kinetics "Component-First" approach described in this article has been distilled from experience gained from a number of large integration and reengineering projects and three architecture generations of the ComponentWare product line using CORBA distributed object technology. The shared goal of these projects was to maximize the integration of the enterprise's legacy assets while minimizing disruption of the enterprise's in-production systems and processes. Legacy applications are noninvasively transformed into CORBA-based application components. The enterprise gains new distributed object-based applications from different assemblies of the repackaged legacy application components.

GETTING STARTED: AUDITING YOUR LEGACY SYSTEM

It's worth noting that software is not designed this way. Normally these applications begin life as well-designed implementations. They're useful and valuable to the organization, so over time they're "extended" in a number of ways; organizational requirements change and new logic is added. They're modified to communicate with other applications, a user interface is added, state is made persistent, log files are added, and so on. Most of these changes are infrastructure changes: they lie outside the domain of the business application and exist only to support the implementation of it. These enhancements are also typically incremental; the new functionality is added not all at once as part of a redesign, but gradually and more or less independently of other enhancements.

This evolution results in a system that's difficult to maintain and extend. New enhancements often introduce as many bugs as they do features. This is a familiar refrain, but a key distinction is that the homegrown infrastructure is the chief culprit. Generally, such infrastructures evolve to address a narrowly defined problem, while a comprehensive infrastructure solution is too broad to lie within the expertise of any but system vendors. In the pathological case, the state of the system actually drives business decisions instead of supporting them ("We can't do that because our system won't be able to support it.").

Unfortunately, the application logic is still useful; in fact, by this time it's often perceived as "mission-critical." Resources continue to be devoted to its maintenance. Alternatively, maintenance cost and downtime risks become so great that the application can no longer be extended -- it's "frozen."

How can such applications be transitioned to a distributed object architecture? There are two basic approaches: complete reimplementation, or incremental integration and reengineering of the legacy system. The prospect of wholesale reengineering is fraught with well-known risk -- all of us have witnessed some spectacular failures in the last ten years. Incremental reengineering using encapsulation may offer a way out of this thicket. We'll look at how encapsulation naturally transforms a legacy system into a set of usable and reusable enterprise components.

LEGACY INTEGRATION AND REENGINEERING USING ENCAPSULATION

In referring to legacy application encapsulation, we'll use the terms encapsulation and wrapping interchangeably. Also, we'll use the terms encapsulation wrapper and wrapper interchangeably. Wrapping is not an innovative concept. A casual web search for encapsulation or wrapping will easily yield hundreds of examples where this technique has been used. Accomplishing integration with encapsulation wrappers is a standard technique. It has been used excessively for prototypes or a quick fix where functionality has been traded in for lower project costs. However, because of standards-based distributed object technology, such as Netscape ONE, and when used with a systematic discipline, such as the Component-First approach, wrapping lowers the cost and risk of integration as well as improves the overall architecture and increases the capabilities of the legacy system.

We've found that there are three major encapsulation strategies: Ad-Hoc, Legacy-First, and Component-First. First we'll examine the Ad-Hoc and Legacy-First approaches, showing why these common strategies are not suitable for transitioning legacy applications to distributed object architectures. We'll then consider the requirements a legacy application encapsulation wrapper must meet if it is to satisfactorily serve as an integration mechanism; this is the basis of the Component-First approach, which we'll then look at in detail.

THE AD-HOC AND LEGACY-FIRST APPROACHES

The Legacy-First approach is appropriate when the goal of the wrapping is to export the application interfaces into a new environment. Examples of this are "shell" wrapping like the MS-DOS prompt in MS Windows, or rehosting the terminal screen output of a mainframe application onto a Windows platform using screen scrapping. We don't want to change the user model or add new functionality; we just want it to be available in a different platform environment.

The starting point of the Legacy-First encapsulation approach is the legacy application specification. The question answered is "How can I export the interfaces to this application into the new environment?" This is a slightly different question from "How can I integrate this into a new environment?" or "How can I transition this into a new environment?" Considering these questions leads us to view encapsulation in a different light.

Driving the encapsulation from the functional specification of a legacy application is not always appropriate. The Legacy-First approach is particularly inappropriate when the application is part of a transition to a distributed object system. Encapsulating an application merely to use it in a new environment suggests different design goals than encapsulating it as a step in a transition process. Yet this distinction is rarely made.

The chief consequence of the Legacy-First approach is the exposure of legacy infrastructure constraints into the new system. For distributed components, such exposure can have disastrous consequences. We typically intend these components to interact; indeed we want to construct new applications with the components we've made by wrapping the legacy applications. Yet the applications were never designed to be building blocks or to work in concert with other components. The interfaces they export do not fit into a framework or normally even adhere to a single strategy. Consequently this technique is not effective as a transition method to distributed object systems.

Our experience, distilled from migrating many large production legacy systems to CORBA, has shown that encapsulation is a complex and difficult problem to address because of the number of roles a legacy application wrapper must fulfill. Encapsulations of different systems seldom look the same. Should they? We feel the answer to that question is "Yes." The basis of transforming legacy systems into application components is that there are sets of needs that are common across the enterprise. These needs translate into patterns of encapsulation -- the basis of the systematic Component-First encapsulation approach.

REQUIREMENTS OF A COMPLETE ENCAPSULATION WRAPPER

• Provide connection protocol management on different levels.
• Accomplish data translation and information processing on potentially different levels.
• Implement some sort of error detection/recovery mechanism.
• Manage the application environment.
• Implement new object interfaces that conform to widely supported distributed object specifications as much as possible.

Connection Protocol Management

Second, the wrapper must often implement some application-level protocol (for example, a login or a dialog). It must at least know how to format the different commands that drive the legacy application and how to make sense of its output. The application-level dialog can easily become complex when the legacy application maintains state.

Data Translation and Information Processing

• Architectural-level transformations may need to be performed. Examples are binary buffers of data being passed among machines with different internal representations, or data being inserted or extracted from a bit stream with a proprietary format.
• Higher-level transformations may be required if the wrapper interface needs to return data derived from application results. For example, if a legacy application returns a time interval and a distance traveled, and the wrapper interface is required to supply velocity, the remaining computation must be performed in the wrapper.

Error Detection and Recovery

Environment Management

Consider an application that performs a simple data transformation. It reads an input file (in a certain format) and writes an output file (in possibly a different format). Simply to run this program, a wrapper needs to take the following steps: ensure that there are no file naming conflicts, create and write the input file in a specific format, invoke the application, monitor and check completion status, read and parse the output file, remove the input and output files, and return the result. Even this trivial example requires seven or eight actions. It's easy to project the amount of complexity involved in robustly wrapping a moderately complex application.

THE COMPONENT-FIRST APPROACH

• To maximally leverage legacy assets (that is, legacy domain logic).
• To minimize the exposure of any legacy architectural constraints into our new system. This is where our use of encapsulation diverges from the Legacy-First approach, which naturally exposes legacy partitioning into the new environment. The partitioning of the distributed component architecture must be driven by a well-defined object model that is based on the enterprise domain, not the existing legacy application partitioning.

Rather than exporting legacy application interfaces and environments into a new domain, the Component-First approach mandates constructing our new domain first and then determining the minimal set of legacy application functions we need to populate it. We may want only a small fraction of what some large legacy application can do. In fact, in general we won't export any interfaces of legacy applications -- they will all be hidden behind a new object model.

The Component-First encapsulation strategy enables us to reengineer functionality piece by piece, at a pace driven by the new business needs, and to control the transition. This control allows us to do incremental reengineering, which significantly reduces the risk involved in undertaking this sort of infrastructure surgery. Application component specifications driven by the new organizational needs, rather than the legacy needs that are compiled into the legacy interface specification, is the founding principle of the Component-First encapsulation strategy.

Steps in the Component-First Process

1. Perform domain analysis and generate the object model.
2. Identify public interfaces of the object model.
3. Encapsulate legacy applications to populate specific functionality in the object model.

Legacy Transition Case Study: Telemetry Management System

The valuable domain logic part of the Telemetry Manager is the telemetry data "decommutation," where bit streams are unpacked and data values for dozens of separate fields are extracted. The Display & Control client "controls" the Telemetry Manager and Telemetry Analysis processes by sending commands through the file IPC mechanism which register the Display & Control client's interest in specific fields. Either the Telemetry Manager process or the Telemetry Analysis process sends via the file-based IPC the indicated field values to the Display & Control process.

Several unpleasant constraints are apparent:

• The system is strictly single-threaded, and performance cannot be scaled up by upgrading to a multiprocessor platform or distributing across multiple hosts. Because of the way the files are specified, only one set of processes can run per machine.
• New processes cannot be added. Only one pair of clients can connect to the Telemetry Manager. The Telemetry Manager cannot maintain different sets of fields for different clients.

However, the sponsor needed the next generation of the telemetry management system to have scalability, fault-tolerance based on redundancy, and integration with other legacy systems, such as high-performance analytics hosted at remote installations. The sponsor agreed to use the Component-First approach, trading up for the critical capabilities that were needed.

Following the three-step process of the Component-First strategy, we did the following:

1. Designed an object model from a telemetry management domain analysis.
2. Used the object model to specify telemetry management component interfaces with CORBA IDL.
3. Wrapped the telemetry management system to populate the specific functionality of each component.

Listing 1 shows the IDL specification for some of the component interfaces of the telemetry management system. Exception handling has been omitted for brevity. The legacy telemetry management system is encapsulated with the DataSource interface. New applications and client components interact through the FieldManager interface, which in turn is the only component to interact with the DataSource interface. These well-defined telemetry management component interfaces completely hide the existence of the legacy telemetry management system.

module Telemetry 
{
   // Client subscription list for fields
   typedef sequence<string> FieldNameList;
  
   // Base interface for all telemetry fields
   interface Field {
      readonly attribute string name;
      readonly attribute long   dev_id;
      attribute          long   time;
      attribute          any    value;
      attribute          short  color;
   };

   // Manager for all Fields; interacts with DataSource interface
   interface FieldManager {
      boolean       setup (in long dev_id, in string pass, in Output outRef);
      FieldNameList subscribe (in FieldNameList list);
      boolean       advise (in string fname, in double displayValue, in long seconds, 
                            in short colorValue);
      boolean       updateSpecial (in string name, in string value);
      oneway void   startReplay ();
      boolean       stopReplay ();
   };

   // Legacy system wrapper interface
   interface DataSource {
      boolean       setup (in long dev_id, in string pass, in FieldManager dsRef);
      oneway void   replayBegin ();
      boolean       replayEnd ();
      void          subscribe (in FieldNameList slist);
   };
};  // end module Telemetry

The DataSource component itself is subject to certain constraints, such as running on the same machine as the legacy telemetry management system. However, because the FieldManager is a logically centralized source of all telemetry Field objects, the full benefits of a CORBA-based telemetry management system can be realized. We're freed from the constraints of the legacy system, as they're isolated and contained by the DataSource component encapsulation wrapper.

The FieldManager can receive (and discard) redundant copies of uniquely identified Field objects from two or more DataSource components. We can achieve fault-tolerance by replicating the legacy telemetry management system with a "hot-swap" redundant spare. There can be multiple Display & Control components, enabling access to the telemetry management system from multiple sites. The system can scale to meet the demands of a growing population of users by copying and broadcasting the Field objects and running multiple FieldManagers on multiple machines. Finally, we can answer to the large and constant need to add new Telemetry Analysis processing. Using the same DataSource component encapsulation pattern, specialized "hothouse" analytics can be incrementally added that consume the raw telemetry Field objects and write new derived data Field objects back into the system.

The component wrappers protect new applications from changes in the legacy system. We can proceed with reengineering the legacy implementations with no effect on client applications. A client requesting notification upon change in state of one of the Field objects has no way of knowing (nor does it care) whether the notification is originally generated by the legacy system or some new mechanism. If we had chosen the Legacy-First approach and created a wrapper that mirrored the command interface of the legacy telemetry management system, we would have been able to achieve the full benefit of the distributed object infrastructure. Clients using the legacy server would be manipulating it directly through interfaces customized for the legacy application; any reengineering of the original application would be far more likely to impact those interfaces (and thus any new application using them).

THE RIGHT PERSPECTIVE

Legacy encapsulation is inherently complex, due to the many functions a robust legacy encapsulation wrapper must perform. Almost every legacy application seems unique when viewed in isolation; from this view the prospect of automating the wrapping process seems dim. However, decomposing wrapper functionality gives insight into this complexity. Viewing the functions separately allows us to apply design patterns to the problem of automating encapsulation.

The systematic Component-First legacy encapsulation strategy enables us to better predict the cost of and identify risk points earlier in the process of transitioning legacy systems to a distributed object infrastructure, such as CORBA. The Component-First approach is continually being advanced and matured into a formal methodology, distilling our experience in transitioning enterprise systems to Internet distributed object architectures using CORBA.

FURTHER RESOURCES

• The Essential Distributed Objects Survival Guide, Robert Orfali, Dan Harkey, and Jeri Edwards, John Wiley and Sons, Inc., 1996.
• Client/Server Programming with Java and CORBA, Robert Orfali and Dan Harkey, John Wiley and Sons, Inc., 1997.
• CORBA Design Patterns, Thomas J. Mowbray and Raphael C. Malveau, John Wiley and Sons, Inc., 1997.

Ted Morin is Vice President, Professional Services of I-Kinetics. Previously, Mr. Morin was a lead distributed object system technologist at MIT Lincoln Laboratory, where for over ten years he designed and built very large distributed object systems for radar imagery analysis and air traffic management. At I-Kinetics, Mr. Morin and his staff are shepherding very large CORBA legacy integration projects with the Component-First methodology in such areas as bioinformatics, Fortune 500 IT, JPL's mission operations, image analysis, enterprise resource planning, logistics, satellite operation management, and multi-enterprise collaborative engineering.

I-Kinetics is a leading provider of CORBA-based enterprise component software, tools, and professional services. Among the tools it offers is the I-Kinetics ComponentFactory, which partially automates the Component-First encapsulation process.

ComponentWare is a registered trademark and ComponentFactory, Component-First, DataBroker, and I-Kinetics are trademarks of I-Kinetics, Incorporated. 

Any sample code included above is provided for your use on an "AS IS" basis, under the Netscape License Agreement - Terms of Use