XML Appliances Begin Supporting Java

Layer7 announced that its appliances will support Java-based custom assertions. Details are sketchy at this point but apparently Layer7 will provide a proprietary SDK for developing assertions. This could be much more powerful mechanism than XSLT-only facilities in IBM DataPower.

So how long before appliances begin supporting JBI and SCA components? The trick here of course is to make sure that an appliance is still a robust low-maintenance device that can’t be brought down by a rogue application thread. But if Layer7 was able to figure it out for Java in general (I wonder if they use a general Sun JDK or some kind of a special JDK build), there is no reason why an appliance can’t start supporting more advanced specifications such as JBI/SCA.

Comparison of SOA Suites

Several SOA vendors are trying to put together comprehensive suites of SOA products that in theory should be capable of
all aspects of SOA, including governance, integration, business process management and others.

Formation of SOA suites is having a tremendous impacts on how SOA products are selected
as many organizations
are being tempted to settle for “one stop shop” approach as opposed to doing proper product evaluation
within each SOA product category. (Interesting discussion about SOA suites is available at

So what is a SOA suite and how offerings from different vendors support different aspects of SOA?
The table below attempts to answer this question.

Read the rest of this post »

SCA and JBI Mean Nothing for SOA?

There is an interesting and somewhat controversial interview with ZapThink’s Jason Bloomberg where he claims that SCA and JBI are just going to “muddy the waters” as opposed to provide real help to SOA architects.

I certainly agree with his assessment that SCA (can’t really speak about JBI) is more of a generic component-centric programming model than a SOA implementation framework. I blogged about it before. However, I do think that SCA is going to provide a lot of value in the area of application architecture and just general component reuse. A technology or a framework does not have to be about SOA to be useful.

Will JAX-WS Become the Primary Mechanism for Invoking RESTful Services?

Developers working with REST and XML/HTTP services have traditionally used light-weight APIs, such as java.net classes or Apache HttpClient. Web services APIs provided by JAX-RPC were SOAP and “enterprise” only and they required J2EE libraries. The situation changed with the release of JAX-WS and its inclusion into Java SE 6. JAX-WS supports RESTful services; also, its Dispatch interface allows clients to work with XML directly as opposed to having to use object mapping or completely unwieldy SAAJ API. A nice example of using JAX-WS to implement a client for a RESTful service is available in Mark Hadley’s blog.

However, I don’t think that JAX-WS is going to become the primary way of implementing RESTful clients just yet. In many situations, HttpClient provides more flexibility and control over HTTP calls.

For example, it is very straightforward to turn on basic authentication for a client using HttpClient APIs:

Read the rest of this post »

Is ESB The Mediation Infrastructure of Web Services Platform?

Lori MacVitte from F5 had a few comments about my post on using XML appliances in ESB capacity. While I don’t completely agree with some of her specific points about what ESB capabilities are missing from XML appliances (for example, parallel processing is employed by most appliance vendors, transactionality is also supported for relevant protocols (although no XML appliance can act as an XA transaction coordinator)), the overall observation is correct – there is (and there always will be) a big difference between software-based ESBs and hardware-based XML appliances.

Software ESBs are implemented on top of application servers or messaging products. As such, they are infinitely flexible. Even if certain feature (e.g., some fancy transformation) is not supported by the ESB product out of the box, it can always be custom-developed using Java or other language supported by the product. This custom code can use all the facilities of the underlying application server, such as transaction management, connection pooling and so forth.

Read the rest of this post »

Is XML Appliance the Ultimate ESB?

IBM recently announced that DataPower XI50 appliance now supports transformations developed using WebSphere Transformation Extender design studio. This is the same technology used in the Message Broker product. Additionally, support for WebSphere Registry and Repository has been added. From the announcement:

New offerings being announced today include the WebSphere DataPower Integration Appliance XI50, which now supports direct database connectivity including IBM’s DB2 data server extending the XI50’s enterprise service bus capabilities. It also adds support for WebSphere Transformation Extender design studio providing common data transformation tooling across IBM’s ESB portfolio. The XI50, along with a new version of the WebSphere DataPower XML Security Appliance XS40, now integrates with the IBM WebSphere Service Registry and Repository (WSRR) to offer enhanced governance capabilities and improve service interoperability, reuse and connectivity.

So why is it significant?

Read the rest of this post »

Finally, Somebody is Thinking about Batch Processing

Batch processing (a.k.a. “bulk processing”) is dull and boring compared to the new world of SOA, Software as a Service and Web 2.0. It’s hardly ever mentioned these days, so one can get an impression that batch processing all but disappeared from an enterprise and got replaced by “enterprise mashups”, or, at the very least, Web services.

Of course, nothing can be further from the truth. While Web services indeed begin playing a key role in many organizations, batch files still remain one of the most widespread form of system integration due to a large number of “legacy” systems that rely on it. At the same time, batch processing presents certain challenges to developers; these challenges include high volume/high throughput requirements since batch processing is inherently “spiky”, ability to deal with failures, which usually requires checkpoint/restart capabilities and many others.

The way batch files are processed varies greatly from system to system; in most cases developers end up creating their own custom frameworks to handle batch processing requirements.

Read the rest of this post »

Person/Organization Schema – Why Reinvent The Wheel?

Almost any enterprise SOA effort begins with an attempt to define XML schemas for key entities used by organization’s business processes. These entities inevitably include “person”, “organization” and associated information such as contact data with addresses, phone numbers, etc. While not every organization needs to support (and model) all CRM-related functions, person/organization (sometimes a term “party” is used to refer to these entity types) is almost always present in some aspects of a business (usually to represent customers and partners).

Of course, the need for these common data definitions goes far beyond just SOA; any integration/EDI effort has to rely on a language that can be understood by all participants of this effort. This is why many industries and government agencies try to define standard schemas that can be used for information exchange between multiple organizations (and not all of it is implemented using Web services). Examples in US include ACORD for insurance, MISMO for mortgage lenders and the Department of Justice JXDM XML schemas. All such schemas have to be able to represent customer/organization data.

I find it surprising that with all the effort that has gone into modeling shared schemas there is still no universally accepted standard that would once and for all define common XML data types.

Read the rest of this post »

Is SCA New Java EE?

Service Component Architecture (SCA) so far has largely gone unnoticed by developers of enterprise applications, even though there are already several SCA implementations on the market. Additionally, open-source SCA implementation from Apache called Tuscany is gaining momentum.

I suspect that most developers think of SCA as yet another “enterprisey” SOA-related standard along the lines of the mostly obscure WS-* specifications. The fact that SCA is being developed by a vendor consortium as opposed to a grass-root open source effort does not add to its popularity either.

However, SOA and Web services are not the main focus of SCA. SCA is a generic component/application framework that is totally separate from Web services. SCA allows defining component interfaces and implementations in Java (although WSDL interfaces are supported as well). I would actually speculate that Java interfaces are going to be used in the majority of cases (at least, for Java applications) and WSDL will be left to special circumstances.

SCA also supports remoting by allowing to expose component interfaces using Web services or other technologies, such as JMS (using so-called “bindings”). But this capability is secondary to the SCA assembly model that explains how components are defined and composed.

The bottom line is that it is perfectly valid to use SCA without ever creating a single Web service.

Moreover, SCA directly competes with Spring and other application and IOC frameworks. SCA’s component is very similar to Spring’s bean. SCA components can have properties and references. References can be “wired” to other components. Similarly to Spring, references and properties translate into setters or constructors that are populated via injection mechanism. SCA 1.0 even supports component auto-wiring that matches auto-wiring Spring feature.

SCA does support Spring via Spring Component Implementation specification, which allows to define SCA components using beans defined in a Spring’s context file. However, for brand new SCA applications, there is really no need to use any other IOC container in addition to SCA itself (in fact, I would argue that using multiple component/IOC frameworks in the same application may make the application architecture really confusing). Of course, Spring does much more than just providing IOC functions (e.g., AOP, integration with persistence frameworks, MVC support) and so these other Spring capabilities could provide sufficient rationale for deciding to use Spring in conjunction with SCA. But given that SCA will be integrated with many commercial products (IBM has already started down that path with adding SCA to the WebSphere application server) including development tools support, I can see how SCA could become a very substantial alternative to Spring.

SCA also directly competes with Java EE. This can be illustrated by the following diagram which is part of the documentation provided with WebSphere SOA Feature Pack:

Indeed, SCA bindings insulate developers from specific communication protocols and mechanism so they don’t have to deal directly with Web services and JMS. On the Web services side, it is not entirely clear to me at this point how SCA is going to be able to support all the capabilities currently provided by JAX-WS, such as “bare” and “wrapped” styles, JAXB serialization and others but I guess it could be possible in theory.

SCA policy framework allows to declaratively specify various policies for SCA components. Currently the policies mostly focus on reliable messaging and security. In theory, they could also include defining transaction boundaries similar to EJB’s transaction attributes.

Another common misconception about SCA is that it must to be used together with Service Data Objects (SDO). This is not the case. While SCA recommends using SDOs, regular Java classes and simple types are supported as well.

SCA also has a number of unique benefits, the most prominent one being its support for different languages. There are SCA implementations for C++ and PHP; Apache Tuscany supports Ruby, Groovy and JSR 223 component implementations (what’s interesting, component interfaces still have to be defined in Java or WSDL).

Binging flexibility is another interesting SCA feature. Today we have to use completely different APIs and libraries for different ways of inter-system communication, e.g., JAX-WS has very little in common with XML over JMS. SCA supports a uniformed declarative binding abstraction so that developers don’t have to be concerned with the intricacies of the specific communication type. New protocols and data formats can be added. For example Tuscany is going to support JSON as one of the bindings. One can imagine that REST or even COBOL Copybook format can be added in the future.

I’m somewhat concerned that SCA binding is actually a combination of a data format and a transport protocol. So, for example, JMS binding translates into XML transported using JMS text message with certain headers. So it is not clear to me if JMS binding can support other data formats (e.g., positional string) over JMS. Perhaps this is something that can be clarified in the future versions of the spec.

So is SCA going to win over hearts and minds of developers and become the de-facto standard for enterprise applications (and not just the ones with heavy Web services aspect)? That remains to be seen. I’ll be watching closely how it evolves, especially the Tuscany project which at this point is still in the incubation stage.

soapUI vs. JUnit

Open source soapUI tool has a number of benefits over developing test cases in Java using JUnit framework. Some of these benefits include:

  • Use of UI for creating/configuring tests. This makes it possible for non-programmers to develop tests, which means that testing of Web services can be performed by a QA/system test group.
  • Auto-generation of test messages based on WSDL.
  • Ability to externalize test data using property files.
  • Built-in assertions, such as checking for SOAP faults and validity of the schema.
  • Built-in stress testing capabilities.
  • Management of endpoint URLs.

Overall, soapUI allows developers and testers to create tests quicker and also makes tests easier to manage and update.

However, one needs to understand shortcomings of this approach. soapUI uses its own Web services client. The way it creates test messages is different from how regular (JAX-WS-based and other) Web services clients work. soapUI completely skips the step of generating Java classes from WSDL. It does not have to deal with XML marshalling/unmarshalling from/to Java objects. As a result, you are not invoking your service the way it will be invoked by real Web service consumers.

Technically, it may not be an issue in a majority of cases. The fact that soapUI is able to parse your WSDL/schema and generate test message means that you have a valid WSDL that should also be consumable by other technologies. However, it is possible that your WSDL/ schema may lead to sub-optimal (albeit perfectly valid) generated classes. For example, the schema that defines <addresses><address>…</address>…</addresses> XML will result in JAXB compiler generating Addresses class that returns the list of type Address. This may or may not be how you want Java classes to look like for the clients. Perhaps you may want to consider to get rid of <addresses> so that no extra class is generated.

In other words, using soapUI (or similar UI-based Web services testing tools, such as SOAPScope) does not allow you to understand all the details of how your Web service will be consumed in the real world.

Another advantage of JUnit test cases is that they can be used as examples illustrating the use of your service. The test classes can be published along with other Web service documentation or handed over to consumers to help them jump-start their client implementations. Unfortunately, soapUI test cases can’t be used for this purpose.

The shortcomings of visual testing tools such as soapUI can be addressed by complementing soapUI tests with JUnit tests. I usually start with soapUI since it is so easy to get up and running with it. Once soapUI test is implemented, I go back to Java, generate Web services client classes from WSDL and implement a JUnit test class. The logic that goes into a JUnit test class obviously depends upon a Web service. But at the very least, there should be one JUnit test per service that invokes the service using generated classes and libraries provided by your Web service product of choice. This way, we can see exactly how our services will be consumed by our clients.

Create JAX-WS Service in 5 Minutes (Tutorial)

This is a brief tutorial describing how to create a Web service using WSDL and annotations.
The approach presented here allows you to design and implement your WSDL/Schema and Java classes
independently without having to generate anything. You can then use annotations
to map Java classes to appropriate WSDL and Schema elements.

Since JAXB and JAX-WS have sensible defaults, the number of annotations can be kept
to the minimum.

In my opinion, it is always best to develop WSDL and schemas by hand to ensure that the
service contract is appropriately defined and also that the schema
can be re-used (by other services) and extended if necessary. I do not recommend using annotations
for automatically producing WSDL and schemas at runtime as this leads to
simplistic schemas and WSDLs.

Generating classes from WSDL/schema sometimes makes sense, say in situations
when you have to use a pre-defined schema. However, as with any code generation,
it creates a maitainance problem especially if there is a need to put behavior
into generated classes. So it is desirable to be able to evolve object model and
service implementation classes independently
from WSDL/schemas. You can do it quite easily by following the steps below.

Even though I use the word “create” when describing some of the steps, the same approach
will also work for updates or refactoring caused by changes in
service requirements.

The steps are the following:

  • Design your service interface. In our case the service has a single “add” operation that accepts
    a “person” document, adds it to a database and returns back the status:

    public String add( Person person )
  • Design your data/object model. In our case Person class has three fields
    (ssn, firstName, lastName) and the list of addresses.
    Real world examples will certainly be more complex, but this is good enough for our purposes.
  • Define XML schema based on your object model.
    Use strict datatypes where possible. For example, we use
    xsd:token instead of xsd:string since “token” does not allow
    carriage return, tabs, and leading spaces:
  •     <xsd:element name="person">
            <xsd:complexType >
                    <xsd:element name="ssn" type="xsd:token" 
                            minOccurs="1" maxOccurs="1"/>
                    <xsd:element name="firstName" type="xsd:token" 
                            minOccurs="1" maxOccurs="1"/>
                    <xsd:element name="lastName" type="xsd:token" 
                            minOccurs="1" maxOccurs="1" />
                    <xsd:element name="address" type="Address" 
                            minOccurs="1" maxOccurs="unbounded"/> 

    Complete schema file

  • Create an XML file for your schema so you can test the schema.
    You can generate an XML instance document
    automatically from Eclipse (assuming you’re using
    Eclipse Web Tool Platform
    by right-clicking on the schema in navigator and selecting “generate”.
    Test the schema by changing data and making sure that you’re getting correct
    validation errors.
  • Sample XML file

  • Create WSDL. Our WSDL imports the schema that we just created.
    Note that the WSDL target namespace is different from the schema namespace since the
    same schema can be reused by different WSDLs:
  •     <wsdl:definitions 
                <xsd:import namespace="http://person" 
                        schemaLocation="person.xsd" />
                <!-- Return value -->
                <xsd:element name="status" type="xsd:string" />

    Complete WSDL file

  • Create your object model classes and annotate them with appropriate JAXB annotations:
  •     // By default JAXB uses getters/setters for marshaling/unmarshaling
        // Using fields access allows us not to define getters/setters for 
        // all mapped instance variables.
        // We need to provide the namespace, otherwise it defaults to
        // the package name
        public class Person {
            // we only need annotations for variables that don't match 
            // element names
            private String ssn;
            private String firstName;
            private String lastName;
            @XmlElement(name = "address")
            protected List<Address> addresses;

    Person class

    Address class

  • Create ObjectFactory class. This class is required by JAXB and it
    should contain factory methods for all JAXB-mapped classes. ObjectFactory
    must resides in the same package with object model classes.
  •     @XmlRegistry
        public class ObjectFactory {
            public Person createPerson() {
                return new Person();
            public Address createAddress() {
                return new Address();

    ObjectFactory class

  • Create Web Service implementation class. Note that with JAX-WS
    Service Endpoint Interface (SEI) is optional, so all you need is an implementation class:
  •       @WebService(
                  // all values must match corresponding attributes 
                  // and elements of the WSDL file
                  // Name of the port type in WSDL
                  // Target namespace of WSDL, could be 
                  // different from the schema namespace
                  // the file must be available to Web container
           * We'are using "bare" style since "add" operation 
           * takes only one argument, person document. Therefore, there 
           * is no need to "wrap" it by nesting it inside operation element.
           * Note that "wrapped" style is the default.
          public class PersonService {
              @WebResult(name = "status")
              public String add( Person person ) {
                  System.out.println("Adding person :"+person );
                  return "Added "+person.getFullName();

    PersonService class

  • Deploy your Web service to your application server or Web services container.
    Refer to your application server documentation for details.
  • Test the service.
    The easiest way to test the service is to use open source soapUI tool.
    You can install it as Eclipse plugin (there are also plugins for
    NetBeans and IDEA) following these instructions.
    In Eclipse, open SoapUI view, right click and create a project from your WSDL file. SoapUI will automatically
    create a sample SOAP request which you can update with your data. You can run it right away.
    Later, you can create a repeatable test suite with parametirized data using SoapUI properties and

SoapUI Project (you can import it into SoapUI).

You can also download all files in a zip file.

WS-I Basic Security Profile Has Been Released

The final version of Basic Security Profile 1.0 (BSP) was recently been released by WS-I. This is certainly a welcome event; WS-Security is broad and complex and it has been plagued by interoperability issues for quite some time (although the situation has improved in the last year or so). The BSP document seems to have pretty good level of details with plenty of good examples that are easy to follow.

I looked at a few messages generated by Axis used in conjunction with Apache WS-Security implementation called Rampart and at the first glance they seem to be compliant with BSP, although all my examples use WS-Security username token profile, which is the simplest form of passing credentials with WS-Security.

As an example, Rampart generates the correct form of the password tag, which includes “type” attribute (this attribute is not mandated by WS-Security, however, it is required by BSB):


BSP is going to make the use of WS-Security more widespread, although this is not going to happen overnight. Currently, none of the public Web services (Yahoo, Ebay, Amazon) use WS-Security, which is not surprising since the standard is not fully supported by such languages as PHP, Python and Ruby (although some aspects of WS-Security are supported in PHP). Even platforms for commercial services, such as StrikeIron, don’t use it (StrikeIron does support WS-Security for communication between its broker and service providers, but not for client calls).

In any event, I’m pretty confident that we can count on rising WS-Security adoption in an enterprise.

WSDL Naming Conventions

Web service providers communicate with their customers (consumers) by the means of publishing WSLD of the service. In most cases, developers create the client code for the service by generating classes from the published WSDL file. While JAX-WS makes it possible to avoid code generation entirely (as I described in
this post) and hand-code all client-side classes, in all but the most trivial cases using code generation is the easiest way to start developing a Web service client.

Service providers need to keep in mind that WSDL is the only “bridge” between providers and consumers. Service providers may have great clean code with nicely defined interfaces and object model; however, this code won’t be shared with consumers (unless, perhaps, we are talking about an internal service used for integrating components of the same application, but even in this case developers should beware of created tight coupling). Therefore we need to make sure that WSDL that we publish will help service consumers generate good client-side code.

JAX-WS mandates the following mappings between WSDL and the generated client code:

  • wsdl:portType translates to the Java interface (business interface) of a Web service (where each operation maps to a method of the interface).
  • wsdl:service translates to, essentially, a factory class that extends javax.xml.ws.Service. This class has “get” methods for each port defined as part of the service element. “getPortName” methods return dynamic proxy that implements the service business interface.
  • Some tools (e.g., WebSphere Web Services Feature Pack) generate additional proxy classes (not to be confused with the dynamic proxy created by the Service subclass) for each port. The proxy class encapsulates logic for accessing javax.xml.ws.Service to invoke the right “get” method. Client code then only needs to instantiate the proxy class and call a business method. The name of the proxy class is NameOfThePortProxy.

Keeping these rules in mind, I use the following naming conventions to ensure that the generated code will contain classes with meaningful names:

  • I use the name of the service business interface as the name of the port type, e.g., POProcessor or PersonService. This ensures that service provider and service consumer will use the same name to refer to the business interface (remember that in general service providers and service consumers do not share any Java classes or interfaces). I never add “PortType” to the name, as this obscures the name of the interface generated for the client. Additionally, it makes names long and POProcessorPortType.process(po) just does not look too good in the code.
  • I add “Ports” to the service name, e.g., POProcessorPorts or PersonServicePorts. I don’t like adding “Service” since most Web service business interfaces already have the name “Service” in them, and “PersonServiceService” looks rather ugly.
  • For ports, I also use the name of the service business interface (i.e. the port type name). More often then not, SOAP/HTTP is the only service binding and so port type name can be used without any changes. If there are other bindings, I add the binding type to the port type name, e.g., PersonServiceJMS.

The following example illustrates the approach:

SOA Need Not Be Costly

I’ve come across a nice chart in the baseline magazine that shows that cost is the biggest obstacle to SOA adoption (unfortunately, this chart is not available in the online version of the magazine). I think that this is very telling, it really shows how often SOA is either misunderstood or applied incorrectly (or both).

SOA is about reducing the cost of implementing business functions across the enterprise by leveraging existing systems instead of rewriting them. Any alternative architecture will most likely be more expensive than SOA (unless we’re dealing with some special/unusual requirements, such as real-time device management). Doing nothing, i.e., just maintaining the existing IT architecture without attempting to improve it, is also more expensive (unless the existing architecture is so great that it does not have any issues) than adopting SOA since the cost of maintaining duplicate functionality in multiple siloed systems is usually quite high.

SOA does not have to be costly. The cost of implementing a service on top of several existing systems usually is not that great. The cost of consuming a Web service in an application built using modern technologies should not be high (and if it is, then there is something wrong with the design or the technologies used by this application).

However, it is very easy to make SOA implementation costly. Following are some of the leading causes of that:

  • Re-writing existing systems in order to implement services. In most instances, full rewrite is not needed; services can be implemented as wrappers.
  • Trying to implement “big bang” SOA that involves deploying massive number of services at once. SOA is most effective when implemented in small increments.
  • Over-architecting implementation of technical aspects of SOA, such as transformation, security, management, etc. Don’t get me wrong, security and management are very important; however, their design should depend on the types of business services that are being created as part of SOA. Attempting to implement SOA infrastructure “in general” will certainly lead to high price solution. For example, SSL/TLS-based approach with basic authentication works fine in many situations (not all, but many), however countless hours are spent on trying to put together security infrastructure using SAML, WS-Trust, WS-Security or whatever the latest and greatest set of buzzwords might be.

  • Procuring and implementing expensive SOA infrastructure products (ESB, registries, Web service management, etc.) before a single service is put in place. My opinion (perhaps, somewhat extreme) is that at the initial stages of SOA many of these products are unnecessary.

To control the cost, SOA should be implemented using short iterations where each iteration results in implementing several services dealing with a specific problem or functional area. An iteration should also include implementation of service consuming applications for the newly developed services, so that the services don’t remain unused. Ideally, benefits/ROI of each iteration should be quantifiable.

In other words, “good” SOA is not costly; the high cost is an indication of mistakes made during SOA planning, design and implementation.

Yahoo Pipes – A Great Way to Create Composite Applications

Yahoo Pipes web site was launched last week and almost immediately drew the attention of a large crowd – I think the site actually went down for a few hours after the launch.

Yahoo Pipes makes it extremely easy to “mash” different Web sources together – without any programming, using drag-and-drop AJAX UI. The UI is actually very slick, it loads fast and provides very intuitive environment. Basically, the UI allows users to create a “message flow” or a pipe using predefined customizable operations, which is a paradigm familiar to any enterprise developer.

I literally took me twenty minutes to put together a simple “pipe” aggregating different SOA-related feeds and allowing user to filter feeds (title or body) using keywords – you can check it out here (you don’t even need a Yahoo account to run it). Learning time was close to zero (perhaps five minutes).

This experience got me thinking. Why is it so easy to create composite applications on the Web (it was easy enough before and with Yahoo pipes it’s just gotten easier) and why is it so hard to do it in an enterprise SOA environment? Why ESB tools don’t have the same level of ease of use and the same degree of “zero administration”? Visual message flow and workflow editors for ESBs are nothing new, but they still come with steep learning curve, tricky configuration requirements and hefty price tags. Of course it’s not fair to compare a simple RSS aggregation/filtering function with typical enterprise tasks (e.g., try to reconcile three different customer XML schemas with different field semantics), but I still think that we have plenty of room for improvement in the enterprise SOA space.