Integration Design Approaches

Integration Design Approaches

The J2EE 1.4 platform provides several technologies for integration, and you can combine them to fit your integration requirements. These technologies, together with the other platform capabilities, give you a rich platform for designing an integration solution. IT architects face the challenge of combining the technologies in the most effective and flexible manner possible to create an integration architecture that can adapt to changing business needs and strategies.

Often, enterprises attempt integration as a "one-off" solution—that is, they treat each integration problem as a unique instance with its own special solution. With this type of approach, an enterprise integrates each system in an individual, custom manner—not the most efficient or effective integration approach. It is far better to have an integration architecture that grows with the needs of the enterprise. The J2EE platform, with its extensive support for Web services, makes such an architecture possible. A J2EE application server acts as the universal connector between the EIS systems and new EAI applications. (This chapter uses the term EAI applications to refer to applications specially designed to solve the problems of integrating enterprise applications.) In a sense, the J2EE application server is the integration hub. The data model in the application server becomes the canonical agreed-upon data model to which all others—EISs and new EAI applications—adapt. Figure shows how this might look from a high level.

A good integration architecture consists of a set of "integration layers" each of which provide certain quality of services. By integration layer we mean the interface or endpoint at which distinct systems intersect. Enterprise architects and software developers must decide how to implement these various integration layers in the most suitable manner. Generally, the requirements of each situation drive the choice of integration layer. For example, the adventure builder enterprise uses Web services as the integration layer for its supply chain. As a result, all interactions with its suppliers must happen through the Web service interfaces.

The integration layer defines the interface between the EIS systems and new applications, thus exposing an EIS system to an application.

An integration layer needs to be the stable point in your system—the fixed standard against which new applications are programmed. While both the new application and the existing EIS may change, the integration layer should remain the same. Otherwise, changes in one place will require changes to numerous dependent systems.

Integration approaches vary, and often the situation dictates the best approach. In some situations, the integration approach is fairly obvious. For example, when a client requires a tightly coupled interaction with a relational database, clearly the JDBC API is the design strategy to use. Similarly, another design strategy uses the J2EE Connector architecture for data integration with non-relational databases. These two design strategies are particularly appropriate when the client requires an API for connecting, querying, and updating a database with transactional semantics. Other situations require you to combine technologies to meet the integration requirements. You might use connectors on top of a non-relational database, but then, based on the client requirements, you might add a Web service layer on top of the connector.

It is helpful to view the process of formulating an integration approach as three separate tasks (each requiring a design and implementation phase):

• Decide on the integration layer, including where it should be located and what form it should take. With the J2EE platform, various integration layers are possible, including connectors, enterprise beans, JMS, and Web services. Remember that you want the integration layer to be such that other applications or systems can evolve easily.

• Decide how to adapt each EIS to the integration layer.

• Decide how to write new applications against these integration layers.

Keep in mind that an important reason for using an integration layer is the services it provides for distribution. Usually, EISs are located on different physical machines, or they may be run as separate processes. To integrate these EISs, you need to use a distributed technology. J2EE application servers are an excellent choice for integration because they can handle multiple protocols for distributed computing: Internet Inter-ORB Protocol (IIOP), messaging, HTTP and Web services. In addition, J2EE application servers can handle native EIS protocols through connectors. These native protocols include proprietary protocols used by ERP and database vendors, among others.

Figure shows the integration design approaches that are possible with the J2EE technologies.

Note: This figure uses dashed lines to represent asynchronous calls, such as sending a JMS message. The solid lines show synchronous calls, which are made either through EJB or Web service APIs. The three applications, which are shown on the left side of the diagram, are new EAI applications that you might be writing. The right side of the diagram shows some of the existing EIS systems that need to be integrated with the EAI applications.

Since it requires very tight coupling with EIS A, EAI application 1 bypasses the J2EE application server but reuses the connector layer. Application 2 needs to use some business logic present in the application server. As a result, it does not want to access the EIS directly. Application 2 instead uses a combination of synchronous and asynchronous calls to the EJB tier. These calls (to enterprise beans and message-driven beans) access EIS A and the relational database through JDBC, EJB container-managed persistence, or connectors. EIS B also generates asynchronous messages for implementing the business logic, and these messages must be delivered to a message-driven bean. Application 3 uses the least tightly coupled interface, Web services, and it accesses available functionality through a Web service interface. The Web service internally uses the business logic stored in the J2EE application server. (In the next sections we take a closer look at these different EAI design approaches and then provide some guidelines for implementing these approaches.)

Let's see how you might apply this strategy to the adventure builder enterprise. The adventure builder enterprise decides to use several layers for integration:

1 Web Services Approach

One approach for EAI is to use a Web service as the integration layer. With this approach, an enterprise's EIS systems expose their functionality by implementing Web services. They make their Web service interfaces available to other applications by providing WSDL descriptions of them. In addition, the integration layer may also include XML schemas for the documents used as parameters and return values. Essentially, the WSDL description of the service interface and document schemas becomes the integration layer, or the point of stability.

Developers who use the Web service approach for integration can leverage the advantages of Web services. Developers can write new enterprise applications with any technology that supports Web services, and the applications may be run on a variety of hardware and software platforms. However, this approach falls short when the new applications have certain additional requirements, such as transactional semantics or stringent security requirements. We discuss how to handle these issues in subsequent sections.

The adventure builder application is a good example of the Web services approach. As noted, the adventure builder enterprise uses Web services for integrating its supply chain. (Chapter 8 discuss the exact structure of the application, but here we highlight those details that pertain to integration.) The adventure builder architects decide, in consultation with the suppliers, on the schemas for the documents that they intend to exchange. (See "Designing Domain-Specific XML Schemas" on page 131.) Since their business depends on this exchange of documents—the adventure builder application sends purchase orders to various suppliers who fulfill adventure requests, and, in turn, the suppliers invoice adventure builder—the adventure builder enterprise and the suppliers need to agree on schemas that describe the content of these documents. For example, a lodging supplier might use an invoice document schema. Code Figure shows an example document corresponding to that schema. Similar schemas are defined for other kinds of suppliers, such as activity and airline suppliers.

1. Lodging Supplier Invoice

<?xml version="1.0" encoding="UTF-8"?>

<Invoice xmlns="http://java.sun.com/blueprints/ns/invoice"

   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

   xsi:schemaLocation="http://java.sun.com/blueprints/ns/invoice  http://java.sun.com

/blueprints/schemas/invoice-lodging.xsd">

   <ID>1234</ID>

   <OPCPoId>AB-j2ee-1069278832687</OPCPoId>

   <SupplierId>LODGING_INVOICE</SupplierId>

   <status>COMPLETED</status>

   <HotelId>LODG-6</HotelId>

   <HotelAddress>1234 Main Street, Sometown 12345, USA

   </HotelAddress>

<CancelPolicy>No Cancelations 24 hours prior</CancelPolicy>

</Invoice>

Similarly, the architects standardize on the WSDL to use for invoices when fulfilling an order. Suppliers, such as airlines, hotels, and activity providers, use these schemas and WSDL descriptions when interacting with the adventure builder enterprise. Code Figure shows the WSDL for the Web service endpoint to which the suppliers send their invoices.

The WSDL shown in Code Figure provides a single operation, submitDocument, which has a single xsd:anyType parameter representing the invoice XML document and which returns a single string value indicating a confirmation. The use of xsd:anyType enables an endpoint to receive multiple types of invoice documents with one method. See "Exchanging XML Documents" on page 107. A WSDL document normally begins with the type definitions that it uses, such as the exception type InvalidDocumentException. (For brevity, the code sample omits these type definitions.) The WSDL code then describes the SOAP messages used in the SOAP request and reply calls that correspond to the submitDocument method. It also describes a WSDL port corresponding to the JAX-RPC interface for receiving invoices, plus it binds the port to the HTTP protocol and indicates the RPC-literal binding. Finally, the WSDL code defines the name of the service and the URL at which the service is available. (For a more detailed explanation of a WSDL document, see "Web Services Description Language" on page 36.)

2. WSDL for Receiving the Invoices

<?xml version="1.0" encoding="UTF-8"?>

<definitions name="WebServiceBroker" targetNamespace= ...>

   ...

   <message name="BrokerServiceIntf_submitDocument">

      <part name="Invoice" type="xsd:anyType"/></message>

   <message name="BrokerServiceIntf_submitDocumentResponse">

      <part name="result" type="xsd:string"/></message>

   <message name="InvalidDocumentException"><part name=

          "InvalidDocumentException" element=

          "tns:InvalidDocumentException"/>

   </message>

   <portType name="BrokerServiceIntf">

      <operation name="submitDocument" parameterOrder="Invoice">

         <input message="tns:BrokerServiceIntf_submitDocument"/>

         <output message

            ="tns:BrokerServiceIntf_submitDocumentResponse"/>

         <fault name="InvalidDocumentException" message=

                       "tns:InvalidDocumentException"/>

      </operation>

   </portType>

   <binding name="BrokerServiceIntfBinding" type=

                            "tns:BrokerServiceIntf">

      ...

   </binding>

   <service name="WebServiceBroker">

      ...

   </service>

</definitions>

Although you can modify the EIS systems that participate in Web service interactions to export the Web service interfaces, a better approach uses a J2EE connector to plug the EIS system into the J2EE application server. The connector by its nature does not intrude on the EIS. This latter strategy takes advantage of the capabilities of the J2EE server. Among other advantages, using connectors allows the transactional and security contexts of a J2EE application server to carry forward to the EIS application. Plus, the EIS can use the thread pools of the application server. In conjunction with connectors, you use JAX-RPC from the J2EE application server to expose a Web service interface. See Figure.

To expose a Web service interface for an EIS, use JAX-RPC in front of a connector.

For example, the adventure builder enterprise wants to use Web services to integrate its existing CRM system, which provides services to manage customer relations, to process orders. The department that owns the CRM module not only wants to maintain control of the software, it wants to use a generic interface that can handle user requests from multiple sources (Web site, telephone, OEM channels, and so forth). Web services are the best way to create such a generic interface, for these reasons:

• Web services establish clear, defined boundaries. Since Web services can provide an interface with clear, defined boundaries of responsibilities, the CRM department has the responsibility to only maintain the endpoint and publish a WSDL describing the endpoint.

• Web services provide controlled access. Outside requests to the CRM must come in through the service interface, and the CRM department can apply its access control parameters to limit access, plus it can log each access.

• Web services support multiple platforms. Because it does not control the hardware and software platforms other departments use, the CRM department can accommodate any platform by using Web services.

• The current generation of Web services is best suited for applications with a limited need for transactions and security. The main purpose of the CRM system is to allow status queries on existing orders. As such, it has little need for transactions. It also has limited need for security, since all access to the module happens within the corporate firewall.

Developers have a choice of endpoint types for implementing the Web service representing the EIS: either an EJB service endpoint or a Web-based JAX-RPC service endpoint. Solely from an integration point of view, either type of endpoint works well since Web and EJB components can both directly use JDBC and J2EE connectors. For example, the adventure builder enterprise receives invoices using an EJB service endpoint, as well as JAX-RPC service endpoints for other functions.

Returning to the CRM problem, the adventure builder enterprise could use a small J2EE application that uses a J2EE connector to connect to the CRM system. Such an application needs to have a JAX-RPC endpoint to expose the required Web service functionality.

Today, since many EIS vendors are providing built-in Web services support, developers can use connectors and leverage this support directly to avoid writing a JAX-RPC wrapper. If you choose to follow this path, ensure that the EIS vendor provides a WS-I Basic Profile-compatible Web service. If the vendor does not, consider writing an adapter layer yourself to handle any differences. This approach may require upgrading your EIS system to a new software version, a risk you may not want to take. In such cases, consider using the recommended connectors and JAX-RPC approach.

Consider using the built-in Web services support provided by the EIS vendors to avoid writing additional interfaces.

2 Enterprise Beans and JMS Approach

You can use enterprise beans and JMS layers, separately or combined, to develop an integration layer. With this approach, you use a J2EE application server to provide an enterprise bean layer for synchronous interactions. For asynchronous interactions, you place a message bus in the enterprise and require that applications use it.

The adventure builder enterprise uses this strategy for integrating applications within one department. For example, a single department owns the order processing module. Within that department, different groups handle various aspects of order processing, such as credit card payments, supply chain interactions, customer relations, and so on. In this workflow arrangement, interactions among these departmental groups are handled in a loosely coupled, asynchronous manner using JMS. When it needs to provide synchronous access, a group may use a remote enterprise bean interface. (See Figure on page 345.)

Using enterprise beans and JMS does not preclude the use of XML, which is an additional benefit. For example, the adventure builder enterprise uses the same invoice document listed in Code Figure when sending a JMS message within its order processing center. The message-driven bean that receives the JMS message applies XML validation and translation logic just like any Web service endpoint.

3 The Connector Approach

When using connectors, the new EAI application is programmed directly against an interface provided by the EIS. For new applications that are also J2EE applications, you use a connector to access the EIS. You can either buy an off-the-shelf connector or write your own.

If you want to provide a simple isolation layer, you should consider writing a data access object, or DAO class, to hide the internal details of the EIS. A data access object encapsulates access to persistent data, such as that stored in a data management system. It decouples the user of the object from the programming mechanism for accessing the underlying data. It can also expose an easier-to-use API. Typically, you develop a data access class for a specific business function or set of functions, then client applications use these classes. Another common use is to layer data access objects on top of connectors, or resource adapters, thus isolating applications from the details of the connector interactions. Often, each resource adapter interaction has a corresponding data access object, whose methods pass input records to the adapter as parameters and return output records to the client application as return values.

Using a connector reduces the complexity of the integration layer. Note that when using the connector approach, the EAI application is tightly coupled with the EIS since the application directly uses the data model and functionality of the EIS. Since there is minimal layering, it also increases performance. However, it is an approach that works best when the integration problem is small in scope. Since it does not put into place a true integration architecture, this approach may limit the ability to scale the integration layer as the enterprise grows. Given these advantages and disadvantages, consider using this approach as a basic building block for other strategies.

4 Combining Approaches into an Integration Architecture

In many cases, architects combine these various integration layers into a single integration architecture. The end result—the mix of integration layer types—is in large measure driven by the requirements of each enterprise's situation. Architects need to consider the realities of the current state of the technologies and weigh that against future promises.

For example, Web services, as they exist today, have some shortcomings: They do not deliver the heavy-duty process integration, data transformation, and security capabilities required by many EAI scenarios. Similarly, services for transactional integrity and reliable messaging are not yet in place. Since security and transactional context propagation are critical business requirements, these are important factors to consider when using a Web services approach.

Data binding is another issue to consider. Web services are a good solution when data binding requirements are straightforward, such as mapping simple data types to Java data types. However, when it is necessary to manipulate complex relational or binary from an EIS, you may want to consider other solutions, such as using the J2EE Connector architecture, which provides a metadata facility to dynamically discover the data format in the EIS.

Figure compares these approaches for different integration problems and indicates the approaches best suited for these integration problems. The adventure builder enterprise uses Web services for partner and inter-department interactions, and the enterprise beans with JMS (EJB/JMS) approach for intra-department interactions.