This, however, could finally give a shot in the arm to the long stagnated SOA projects. What makes me say that? Well, as I’ve been advocating all along, SOA represents an inexpensive way of adding flexibility and achieving new goals. SOA does not have to require significant new investments. Done smartly, it could be a great way of implementing new functionality without ripping the existing systems apart, which is undoubtedly won’t fly in light of highly constrained budgets.

To prove this point, in spite of lackluster economy, the demand for SOA architects has been booming.

There is one caveat to this view – organizations need to stop approaching SOA from the infrastructure standpoint (i.e., what ESB product do I need?) and start using it as a solution to real business needs. I’m not entirely sure that it’s going to happen, but bad economy could be just the motivation for that.

What many people don't realize is just how many SOA-related features are already provided by application servers. Open-source Glassfish server supports JAXR Web services repositories, XSLT mediations, JBI integration and provides extensive Web services monitoring capabilities. It can be augmented by several open-source JBI-based ESBs, such as OpenESB. Other application servers provide similar set of features, although using different technologies (e.g., SCA instead of JBI in IBM products). With an application server being so powerful, there should be a very good reason and business rationale for upgrading to commercial SOA/integration products.

In my mind the right strategy for any SOA is to first implement some services that provide immediate value to the business stakeholders. These services should be implemented using the products already owned and used by an organization (or by using open source ones). After the initial success of this implementation, the organization can evaluate commercial products to see how these products will help implement next set of business requirements/goals more efficiently.

• SOAP tight coupling with WSDL. Until recently, SOAP was the only supported WSDL binding. WSDL, with all of its issues (such as the convoluted structure), remains the only widely accepted vendor-neutral way of defining services.
• In Java world, SOAP was promoted by adding its support to J2EE/Java EE via JAX-RPC and JAX-WS specifications. There is a way to create RESTful services using JAX-WS, however this approach has not gained wide acceptance due to a number of reasons (lack of standardized WSDL binding being one of them).
• Solid support by software vendors. SOAP is supported in all application servers; there are plenty of development tools (including Eclipse and NetBeans) that can help with creating SOAP-based services.
• Availability of a number of WS-* specifications that only work with SOAP. WS-* specs, such as WS-Security and WS-ReliableMessaging heavily rely on SOAP headers for passing message metadata.

SOAP, however, is far from perfect. It adds complexity and overhead and makes implementation of message intermediaries (e.g., gateway products) more difficult (XML message has to be parsed in order to obtain any information about the message, such as an operation name). On the other hand, the uptake of WS-* standards has been very slow, perhaps with the exception of WS-Security, so the benefit of WS-* integration with SOAP failed to materialize. Another supposed SOAP benefit is the protocol neutrality, i.e., a SOAP message looks the same over HTTP, JMS, RMI or any other supported protocol. However, most services are implemented using HTTP, or in some cases, JMS without any headers (unless WS-* is used), so this benefit has limited value.

So it is fair to say that in many instances, SOAP itself does not add any value to Web services implementation. Nevertheless, it's continued to be used because of the reasons listed above. But this situation might be changing in the future.

WSDL 2.0 fully supports HTTP binding. WSDL 2.0 stirred a lot of controversy (I personally think that it's a step forward from WSDL 1), however, it was finally accepted as a W3C recommendation in June, so we can fully expect that vendors support will follow.

Additionally, JAX-RS specification was created and included into Java EE 6. This will also promote development of RESTful services behind the firewall.

Currently, there a preconceived notion that REST is only good for Internet/WEB 2.0 environment, whereas enterprise services should all be using SOAP. The emergence of the two new standards will help change this notion.

Ideally, developers should be able to use the simplest binding that gets the job done. HTTP/REST can be a good choice for many "query-style" services. Services that accept more complex data can use "Plain Old XML" over HTTP post. Finally, services that have to rely on WS-* specifications will utilize SOAP. In all these cases, developers should be able to use same set of APIs for invoking the service and same set of annotations (obviously, with different parameters) for creating the service.

Perhaps this is how Web service development will be done in the future, but we are certainly not there yet.

How fast are XML appliances? I’m not aware of published benchmarks; however, I did have a chance to conduct an informal performance testing of DataPower XI50 for a client and the results were quite impressive; for example we saw little to no overhead validating medium-size XML schemas.

Clearly, offloading as much XML processing as possible to an appliance could be a performance booster. So what kind of processing could be done on the device? Here are my recommendations based on the capabilities of the DataPower appliance; the situation could be slightly different with devices from other vendors.

• Validation. Turning schema validation on is a no-brainer, especially for complex schemas that can choke software parsers. In addition to schema validation, XSLT can be used to validate rules that are not grammar-based. You can use Schematron to generate XSLT that validates these rules. Combination of Schema and Schematron provides very powerful validation mechanism. The beauty of this approach is that failed XML documents never reach the application thus reducing resource consumption of the application server.
• Splitting. It is possible to define XPath-rules that in combination with XSLT will split complex documents into multiple fragments that can then be sent to different back-end services for processing.
• Filtering. Sometimes an application only needs a subset of elements provided in a message. Eliminating unnecessary tags will reduce the application’s XML processing overhead.
• Flattening. Complex XML documents with deep nesting can be “flattened” using XSLT to eliminate some of the nesting. This could make application XML parsing quicker.
• "Canonicanization". In many cases, XML schema may allow for different representation of the same piece of data to accommodate needs of different clients. It could be allowing for SSN with and without dashes or supporting different name formats (e.g., "firstName, lastName" vs. "fullName"). XSLT logic on the device could implement the rules to clean up incoming messages and transform them into one strict canonical representation so that the application only needs to implement one way of parsing data.

"Shredding"” could also be very effective for high-throughput XML processing. Shredding is the process of breaking up large documents into smaller fragments and sending them off to the application for concurrent processing. Shredding helps dealing with the documents consisting of a large number of repetitive groups. Each group/fragment can be processed independently without waiting for the completion of the processing of the entire document. Note that shredding is different from "splitting" which allows for processing of different logical parts (different types) of the document independently (as opposed to repetitive groups). Shredding is the most useful for processing large bathes of data of the same type.

Unfortunately, shredding does not seem to be supported by DataPower, although the appliance does support streaming XML processing which would’ve worked very nicely in conjunction with shredding logic. Perhaps devices from other vendors do provide this support.

The bottom line is that just plugging the appliance in and using it in a gateway mode is not necessarily going to improve XML processing performance by much. Web services (and any other application components that interface via XML such as XML-based batch file processing components) have to be designed with the appliance in mind. In the course of the design effort, developers need to identify pre-processing or post-processing logic that can be implemented in XSLT as opposed to making it all application’s responsibility. Anything that uses XSLT can then go to the appliance to improve the processing speed and throughput.

There are a number of commercial registry/repository products but all of them are quite pricey. Also, smaller organizations or organizations just starting with SOA may not need the full power of a commercial registry/repository product; most of it functionality will remain unutilized. On the other hand, existing registries do not do a particularly good job of managing dependencies between services as well as between service consumers and service providers. Dependency management must be a key consideration in SOA since, unlike in the case of monolithic applications, an SOA consists of a large number of “moving parts”, that is, services. In a well developed SOA most services depend on other services and these services in turn may rely on some other services. Being able to manage and analyze services dependencies is required in order to be able to implement robust and maintainable SOA.

Open source Maven project has become something close to a de-facto standard in the area of dependency management for building Java-based projects and components. Also, Maven dependency mechanism can be used with any Ant build by utilizing Maven Ant dependency tasks (i.e., there is no need to migrate existing Ant build processes to Maven). Maven also has a pretty sophisticated repository support allowing implementing custom central repositories as well as local repository snapshots. Projects like artifactory further enhance Maven repository and add nice UI.

So why not use a Maven repository for storing Web services artifacts, such as WSDL, schemas and policy files? Each service or a group of related services from the same service provider can have its own POM. Using Maven (i.e., by executing “deploy” goal) service providers can publish WSDL and other artifacts (but, obviously, no binaries) along with their POMs in a Maven repository. Service consumers could then download these artifacts based on the dependencies defined in their POMs. If the version stays the same, service consumers don’t need to do anything. In case of a change, service consumers may need to make changes to their client’s implementation, including re-generating client’s classes (this is, of course, a manual process that requires some analysis of the changes). In other words, the process is very similar to how we deal with changes in APIs in third-party libraries.

The entire process can be supported by out-of –the-box Maven build process. The only required change is a custom artifact handler to support new “service” extension and packaging type.

An existing change management process can be adapted for Web services so that service consumers are notified about new versions and their backward compatibility. Again, there is nothing new here; most organizations already have a change management process in place to handle changes in shared libraries.

Using Maven for managing Web services releases and dependencies has multiple benefits:

• Ability to use full range of Maven dependency management capabilities including version range dependency, snapshot and central repositories and so on.
• Ability to use Maven dependency reports for dependency analysis.
• Web services dependency management is consistent with the dependency management process used for other artifacts, such as jars, wars and so on. Commercial repository products often provide a proprietary mechanism that can only be used for Web services - related artifacts.
• Tight integration with any Maven or Ant-based build process, no need for using UDDI or proprietary APIs for publishing services.
• Service consumers that do not use Ant or Maven could still participate in the process by downloading Web services artifacts manually using UI provided by Artifactory or a similar product (although using automated approach should be the preferred option).

Of course, the Maven-based approach does not do anything for managing and enforcing (including run-time enforcement) of Web services policies which is something that commercial registry/repository products can do quite well. So organizations need to evaluate the need for a commercial registry/repository based on types and complexity of the policies they would like to enforce. In many if not most cases the policies are quite straightforward in which case Maven-based solution becomes a viable option. ]]> http://myarch.com/using-maven-repository-as-web-services-registry/feed 6 http://myarch.com/xml-appliances-begin-supporting-java http://myarch.com/xml-appliances-begin-supporting-java#comments Fri, 22 Jun 2007 03:14:54 +0000 Alexander Ananiev http://myarch.com/xml-appliances-begin-supporting-java Layer7 announced that its appliances will support Java-based custom assertions. Details are sketchy at this point but apparently Layer7 will provide a proprietary SDK for developing assertions. This could be much more powerful mechanism than XSLT-only facilities in IBM DataPower.

So how long before appliances begin supporting JBI and SCA components? The trick here of course is to make sure that an appliance is still a robust low-maintenance device that can't be brought down by a rogue application thread. But if Layer7 was able to figure it out for Java in general (I wonder if they use a general Sun JDK or some kind of a special JDK build), there is no reason why an appliance can't start supporting more advanced specifications such as JBI/SCA.

Formation of SOA suites is having a tremendous impacts on how SOA products are selected as many organizations are being tempted to settle for "one stop shop" approach as opposed to doing proper product evaluation within each SOA product category. (Interesting discussion about SOA suites is available at ZDNet).

So what is a SOA suite and how offerings from different vendors support different aspects of SOA? The table below attempts to answer this question.

One thing to keep in mind is that SOA products within the same category can differ substantially in terms of their feature sets. Definitions of ESB, registry and other SOA product categories are not standardized and so vendors are free to categorize their products as they wish. Detailed analysis and evaluation is still a requirement when selecting SOA products.

IBM, BEA and Oracle are emerging as leaders in terms of completeness of their SOA suites. Microsoft's products are not as comprehensive, nevertheless Microsoft's marketshare makes it a signifcant player - Micorsoft's shops tend to be very loyal to the vendor even if Microsoft's SOA story is not as compelling.

Other vendors, including Software AG/webMethods and HP have interesting offerings too, however, they still have some gaps in their capabilities and so they are not covered here (perhaps I'll add more vendors in the future).

I certainly agree with his assessment that SCA (can't really speak about JBI) is more of a generic component-centric programming model than a SOA implementation framework. I blogged about it before. However, I do think that SCA is going to provide a lot of value in the area of application architecture and just general component reuse. A technology or a framework does not have to be about SOA to be useful. ]]> http://myarch.com/sca-and-jbi-mean-nothing-for-soa/feed 3 http://myarch.com/will-jax-ws-become-the-primary-mechanism-for-invoking-restful-services http://myarch.com/will-jax-ws-become-the-primary-mechanism-for-invoking-restful-services#comments Sun, 03 Jun 2007 01:15:40 +0000 Alexander Ananiev http://myarch.com/will-jax-ws-become-the-primary-mechanism-for-invoking-restful-services Developers working with REST and XML/HTTP services have traditionally used light-weight APIs, such as java.net classes or Apache HttpClient. Web services APIs provided by JAX-RPC were SOAP and "enterprise" only and they required J2EE libraries. The situation changed with the release of JAX-WS and its inclusion into Java SE 6. JAX-WS supports RESTful services; also, its Dispatch interface allows clients to work with XML directly as opposed to having to use object mapping or completely unwieldy SAAJ API. A nice example of using JAX-WS to implement a client for a RESTful service is available in Mark Hadley's blog.

However, I don't think that JAX-WS is going to become the primary way of implementing RESTful clients just yet. In many situations, HttpClient provides more flexibility and control over HTTP calls.

For example, it is very straightforward to turn on basic authentication for a client using HttpClient APIs:

 HttpState httpState =httpClient.getState();
 httpState.setCredentials( AuthScope.ANY, 
     new UsernamePasswordCredentials(username, password));
 // Set the authentication header on the first request, don't wait for 401
 httpClient.getParams().setAuthenticationPreemptive(true);
        
 postMethod.setDoAuthentication(true);

Implementing the same requirement in JAX-WS is not a big deal either:

Map requestContext = dispatcher.getRequestContext();
requestContext.put(BindingProvider.USERNAME_PROPERTY, username); 
requestContext.put(BindingProvider.PASSWORD_PROPERTY, password);

However, it is not clear how to tell the client to set basic authentication header on the very request in case of the service does not issue 401 basic authentication challenge (which is pretty common). It is also not clear if there is a way to change authentication scheme (say, to digest) or authentication scope.

There are also other examples that demonstrate that in attempt to abstract the details of the transport protocol JAX-WS limits capabilities of the clients.

However, using JAX-WS REST capabilities still makes sense for applications that use both SOAP and RESTfull services. This allows developers to use consistent API for both types of services. Also, in some instances, a service provider may initially implement a service using XML/HTTP and later decide to switch to SOAP, say, to be able to use some of the WS-* specifications (this is actually a fairly common situation within an enterprise). In this case, using JAX-WS will certainly make the transition easier for the clients. ]]> http://myarch.com/will-jax-ws-become-the-primary-mechanism-for-invoking-restful-services/feed 3 http://myarch.com/is-esb-mediation-infrastructure-of-web-services-platform http://myarch.com/is-esb-mediation-infrastructure-of-web-services-platform#comments Tue, 29 May 2007 00:48:07 +0000 Alexander Ananiev http://myarch.com/is-esb-mediation-infrastructure-of-web-services-platform Lori MacVitte from F5 had a few comments about my post on using XML appliances in ESB capacity. While I don't completely agree with some of her specific points about what ESB capabilities are missing from XML appliances (for example, parallel processing is employed by most appliance vendors, transactionality is also supported for relevant protocols (although no XML appliance can act as an XA transaction coordinator)), the overall observation is correct – there is (and there always will be) a big difference between software-based ESBs and hardware-based XML appliances.

Software ESBs are implemented on top of application servers or messaging products. As such, they are infinitely flexible. Even if certain feature (e.g., some fancy transformation) is not supported by the ESB product out of the box, it can always be custom-developed using Java or other language supported by the product. This custom code can use all the facilities of the underlying application server, such as transaction management, connection pooling and so forth.

Extensibility of hardware appliances is typically very limited. This is the cost of having closed-down low-maintenance device. Allowing running arbitrary Java code on it would simply defeat the purpose of going the appliance route in the first place. There is obviously no way to run business logic on the appliance.

In many cases, organizations implement ESBs to serve very simple needs, such as authentication/authorization, management/monitoring, schema validation and, perhaps some relatively simple transformations dealing with cleansing data formats and reconciling version incompatibilities. The actual business logic involving business process management, service orchestration and other aspects is very often pushed down to the application tier. In other words, ESB in this case is treated as a low-level integration and service management infrastructure as opposed to a container for Web services. Appliance-based ESB would fit very well to this environment.

On the other hand, if there is a desire to use ESB as a platform and container for Web services, appliances are certainly not appropriate. Such a Web services platform acts as a next generation application server. It allows developers to implement Web services using technologies supported by the ESB (such as mediation flows and BPEL) and/or using technologies supported by the underlying application server (Java EE). Product such as WebSphere Proces Server, and, to some degree BEA AquaLogic Bus (it does not support BPEL) fit into this category.

As a side note, I believe that ESB capabilities will be moving down the stack to the base application server products. It becomes more and more difficult for vendors to charge for application server licenses as the competition in the open source space intensifies (just look at the buzz generated by the latest version of Glass Fish). At the same time, more and more enterprise applications are being built using Web services and more and more applications become Web service consumers. This means that soon most applications will require using ESB-like integration technologies that would allow them to orchestrate and mediate services easily. So software ESB products will be used more and more as platforms for service-based applications whereas XML appliances will be playing a key role in the Web services integration/management space. ]]> http://myarch.com/is-esb-mediation-infrastructure-of-web-services-platform/feed 1