Category Archives: Web services

Is The End of SOAP Dominance Nearing?

SOAP-based services currently dominate the enterprise landscape. Main reasons this are:

  • SOAP tight coupling with WSDL. Until recently, SOAP was the only supported WSDL binding. WSDL, with all of its issues (such as the convoluted structure), remains the only widely accepted vendor-neutral way of defining services.
  • In Java world, SOAP was promoted by adding its support to J2EE/Java EE via JAX-RPC and JAX-WS specifications. There is a way to create RESTful services using JAX-WS, however this approach has not gained wide acceptance due to a number of reasons (lack of standardized WSDL binding being one of them).
  • Solid support by software vendors. SOAP is supported in all application servers; there are plenty of development tools (including Eclipse and NetBeans) that can help with creating SOAP-based services.
  • Availability of a number of WS-* specifications that only work with SOAP. WS-* specs, such as WS-Security and WS-ReliableMessaging heavily rely on SOAP headers for passing message metadata.

SOAP, however, is far from perfect. It adds complexity and overhead and makes implementation of message intermediaries (e.g., gateway products) more difficult (XML message has to be parsed in order to obtain any information about the message, such as an operation name). On the other hand, the uptake of WS-* standards has been very slow, perhaps with the exception of WS-Security, so the benefit of WS-* integration with SOAP failed to materialize. Another supposed SOAP benefit is the protocol neutrality, i.e., a SOAP message looks the same over HTTP, JMS, RMI or any other supported protocol. However, most services are implemented using HTTP, or in some cases, JMS without any headers (unless WS-* is used), so this benefit has limited value.

So it is fair to say that in many instances, SOAP itself does not add any value to Web services implementation. Nevertheless, it’s continued to be used because of the reasons listed above. But this situation might be changing in the future.

WSDL 2.0 fully supports HTTP binding. WSDL 2.0 stirred a lot of controversy (I personally think that it’s a step forward from WSDL 1), however, it was finally accepted as a W3C recommendation in June, so we can fully expect that vendors support will follow.

Additionally, JAX-RS specification was created and included into Java EE 6. This will also promote development of RESTful services behind the firewall.

Currently, there a preconceived notion that REST is only good for Internet/WEB 2.0 environment, whereas enterprise services should all be using SOAP. The emergence of the two new standards will help change this notion.

Ideally, developers should be able to use the simplest binding that gets the job done. HTTP/REST can be a good choice for many “query-style” services. Services that accept more complex data can use “Plain Old XML” over HTTP post. Finally, services that have to rely on WS-* specifications will utilize SOAP. In all these cases, developers should be able to use same set of APIs for invoking the service and same set of annotations (obviously, with different parameters) for creating the service.

Perhaps this is how Web service development will be done in the future, but we are certainly not there yet.

Using Maven Repository as Web Services Registry

A Web services registry is arguably one of the most important components of SOA. The registry provides a single source of information about all services in an enterprise.

There are a number of commercial registry/repository products but all of them are quite pricey. Also, smaller organizations or organizations just starting with SOA may not need the full power of a commercial registry/repository product; most of it functionality will remain unutilized. On the other hand, existing registries do not do a particularly good job of managing dependencies between services as well as between service consumers and service providers. Dependency management must be a key consideration in SOA since, unlike in the case of monolithic applications, an SOA consists of a large number of “moving parts”, that is, services. In a well developed SOA most services depend on other services and these services in turn may rely on some other services. Being able to manage and analyze services dependencies is required in order to be able to implement robust and maintainable SOA.

Open source Maven project has become something close to a de-facto standard in the area of dependency management for building Java-based projects and components.

Read the rest of this post »

Comparison of SOA Suites

Several SOA vendors are trying to put together comprehensive suites of SOA products that in theory should be capable of
addressing
all aspects of SOA, including governance, integration, business process management and others.

Formation of SOA suites is having a tremendous impacts on how SOA products are selected
as many organizations
are being tempted to settle for “one stop shop” approach as opposed to doing proper product evaluation
within each SOA product category. (Interesting discussion about SOA suites is available at
ZDNet).

So what is a SOA suite and how offerings from different vendors support different aspects of SOA?
The table below attempts to answer this question.

Read the rest of this post »

Will JAX-WS Become the Primary Mechanism for Invoking RESTful Services?

Developers working with REST and XML/HTTP services have traditionally used light-weight APIs, such as java.net classes or Apache HttpClient. Web services APIs provided by JAX-RPC were SOAP and “enterprise” only and they required J2EE libraries. The situation changed with the release of JAX-WS and its inclusion into Java SE 6. JAX-WS supports RESTful services; also, its Dispatch interface allows clients to work with XML directly as opposed to having to use object mapping or completely unwieldy SAAJ API. A nice example of using JAX-WS to implement a client for a RESTful service is available in Mark Hadley’s blog.

However, I don’t think that JAX-WS is going to become the primary way of implementing RESTful clients just yet. In many situations, HttpClient provides more flexibility and control over HTTP calls.

For example, it is very straightforward to turn on basic authentication for a client using HttpClient APIs:

Read the rest of this post »

soapUI vs. JUnit

Open source soapUI tool has a number of benefits over developing test cases in Java using JUnit framework. Some of these benefits include:

  • Use of UI for creating/configuring tests. This makes it possible for non-programmers to develop tests, which means that testing of Web services can be performed by a QA/system test group.
  • Auto-generation of test messages based on WSDL.
  • Ability to externalize test data using property files.
  • Built-in assertions, such as checking for SOAP faults and validity of the schema.
  • Built-in stress testing capabilities.
  • Management of endpoint URLs.

Overall, soapUI allows developers and testers to create tests quicker and also makes tests easier to manage and update.

However, one needs to understand shortcomings of this approach. soapUI uses its own Web services client. The way it creates test messages is different from how regular (JAX-WS-based and other) Web services clients work. soapUI completely skips the step of generating Java classes from WSDL. It does not have to deal with XML marshalling/unmarshalling from/to Java objects. As a result, you are not invoking your service the way it will be invoked by real Web service consumers.

Technically, it may not be an issue in a majority of cases. The fact that soapUI is able to parse your WSDL/schema and generate test message means that you have a valid WSDL that should also be consumable by other technologies. However, it is possible that your WSDL/ schema may lead to sub-optimal (albeit perfectly valid) generated classes. For example, the schema that defines <addresses><address>…</address>…</addresses> XML will result in JAXB compiler generating Addresses class that returns the list of type Address. This may or may not be how you want Java classes to look like for the clients. Perhaps you may want to consider to get rid of <addresses> so that no extra class is generated.

In other words, using soapUI (or similar UI-based Web services testing tools, such as SOAPScope) does not allow you to understand all the details of how your Web service will be consumed in the real world.

Another advantage of JUnit test cases is that they can be used as examples illustrating the use of your service. The test classes can be published along with other Web service documentation or handed over to consumers to help them jump-start their client implementations. Unfortunately, soapUI test cases can’t be used for this purpose.

The shortcomings of visual testing tools such as soapUI can be addressed by complementing soapUI tests with JUnit tests. I usually start with soapUI since it is so easy to get up and running with it. Once soapUI test is implemented, I go back to Java, generate Web services client classes from WSDL and implement a JUnit test class. The logic that goes into a JUnit test class obviously depends upon a Web service. But at the very least, there should be one JUnit test per service that invokes the service using generated classes and libraries provided by your Web service product of choice. This way, we can see exactly how our services will be consumed by our clients.

Create JAX-WS Service in 5 Minutes (Tutorial)

This is a brief tutorial describing how to create a Web service using WSDL and annotations.
The approach presented here allows you to design and implement your WSDL/Schema and Java classes
independently without having to generate anything. You can then use annotations
to map Java classes to appropriate WSDL and Schema elements.

Since JAXB and JAX-WS have sensible defaults, the number of annotations can be kept
to the minimum.

In my opinion, it is always best to develop WSDL and schemas by hand to ensure that the
service contract is appropriately defined and also that the schema
can be re-used (by other services) and extended if necessary. I do not recommend using annotations
for automatically producing WSDL and schemas at runtime as this leads to
simplistic schemas and WSDLs.

Generating classes from WSDL/schema sometimes makes sense, say in situations
when you have to use a pre-defined schema. However, as with any code generation,
it creates a maitainance problem especially if there is a need to put behavior
into generated classes. So it is desirable to be able to evolve object model and
service implementation classes independently
from WSDL/schemas. You can do it quite easily by following the steps below.

Even though I use the word “create” when describing some of the steps, the same approach
will also work for updates or refactoring caused by changes in
service requirements.

The steps are the following:

  • Design your service interface. In our case the service has a single “add” operation that accepts
    a “person” document, adds it to a database and returns back the status:

    public String add( Person person )
  • Design your data/object model. In our case Person class has three fields
    (ssn, firstName, lastName) and the list of addresses.
    Real world examples will certainly be more complex, but this is good enough for our purposes.
  • Define XML schema based on your object model.
    Use strict datatypes where possible. For example, we use
    xsd:token instead of xsd:string since “token” does not allow
    carriage return, tabs, and leading spaces:
  •     <xsd:element name="person">
            <xsd:complexType >
                <xsd:sequence>
                    <xsd:element name="ssn" type="xsd:token" 
                            minOccurs="1" maxOccurs="1"/>
                    <xsd:element name="firstName" type="xsd:token" 
                            minOccurs="1" maxOccurs="1"/>
                    <xsd:element name="lastName" type="xsd:token" 
                            minOccurs="1" maxOccurs="1" />
                    <xsd:element name="address" type="Address" 
                            minOccurs="1" maxOccurs="unbounded"/> 
                </xsd:sequence>
            </xsd:complexType>
        </xsd:element>
      

    Complete schema file

  • Create an XML file for your schema so you can test the schema.
    You can generate an XML instance document
    automatically from Eclipse (assuming you’re using
    Eclipse Web Tool Platform
    )
    by right-clicking on the schema in navigator and selecting “generate”.
    Test the schema by changing data and making sure that you’re getting correct
    validation errors.
  • Sample XML file

  • Create WSDL. Our WSDL imports the schema that we just created.
    Note that the WSDL target namespace is different from the schema namespace since the
    same schema can be reused by different WSDLs:
  •     <wsdl:definitions 
              targetNamespace="http://personservice" 
              xmlns="http://personservice" 
              xmlns:pers="http://person"
              xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" 
              xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" 
              xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" 
              xmlns:xsd="http://www.w3.org/2001/XMLSchema">
      
          <wsdl:types>
                <xsd:import namespace="http://person" 
                        schemaLocation="person.xsd" />
                <!-- Return value -->
                <xsd:element name="status" type="xsd:string" />
          </wsdl:types>
          ...
      

    Complete WSDL file

  • Create your object model classes and annotate them with appropriate JAXB annotations:
  •     // By default JAXB uses getters/setters for marshaling/unmarshaling
        // Using fields access allows us not to define getters/setters for 
        // all mapped instance variables.
        @XmlAccessorType(XmlAccessType.FIELD)
        // We need to provide the namespace, otherwise it defaults to
        // the package name
        @XmlRootElement(namespace="http://person")
        public class Person {
        
            // we only need annotations for variables that don't match 
            // element names
            private String ssn;
            private String firstName;
            private String lastName;
            
            @XmlElement(name = "address")
            protected List<Address> addresses;
          ...
      

    Person class

    Address class

  • Create ObjectFactory class. This class is required by JAXB and it
    should contain factory methods for all JAXB-mapped classes. ObjectFactory
    must resides in the same package with object model classes.
  •     @XmlRegistry
        public class ObjectFactory {
        
            public Person createPerson() {
                return new Person();
            }
            
            public Address createAddress() {
                return new Address();
            }
        }
      

    ObjectFactory class

  • Create Web Service implementation class. Note that with JAX-WS
    Service Endpoint Interface (SEI) is optional, so all you need is an implementation class:
  •       @WebService(
                  // all values must match corresponding attributes 
                  // and elements of the WSDL file
              
                  // Name of the port type in WSDL
                  name="PersonService",
                  // Target namespace of WSDL, could be 
                  // different from the schema namespace
                  targetNamespace="http://personservice",
                  serviceName="PersonServicePorts",
                  portName="PersonService",
                  // the file must be available to Web container
                  wsdlLocation="WEB-INF/wsdl/PersonService.wsdl"
          )
          
          /*
           * We'are using "bare" style since "add" operation 
           * takes only one argument, person document. Therefore, there 
           * is no need to "wrap" it by nesting it inside operation element.
           * Note that "wrapped" style is the default.
           */
          @SOAPBinding(parameterStyle=SOAPBinding.ParameterStyle.BARE)
          
          public class PersonService {
              
              @WebResult(name = "status")
              public String add( Person person ) {
                  
                  System.out.println("Adding person :"+person );
                  return "Added "+person.getFullName();
              }
          }
      

    PersonService class

  • Deploy your Web service to your application server or Web services container.
    Refer to your application server documentation for details.
  • Test the service.
    The easiest way to test the service is to use open source soapUI tool.
    You can install it as Eclipse plugin (there are also plugins for
    NetBeans and IDEA) following these instructions.
    In Eclipse, open SoapUI view, right click and create a project from your WSDL file. SoapUI will automatically
    create a sample SOAP request which you can update with your data. You can run it right away.
    Later, you can create a repeatable test suite with parametirized data using SoapUI properties and
    assertions:

SoapUI Project (you can import it into SoapUI).

You can also download all files in a zip file.

WS-I Basic Security Profile Has Been Released

The final version of Basic Security Profile 1.0 (BSP) was recently been released by WS-I. This is certainly a welcome event; WS-Security is broad and complex and it has been plagued by interoperability issues for quite some time (although the situation has improved in the last year or so). The BSP document seems to have pretty good level of details with plenty of good examples that are easy to follow.

I looked at a few messages generated by Axis used in conjunction with Apache WS-Security implementation called Rampart and at the first glance they seem to be compliant with BSP, although all my examples use WS-Security username token profile, which is the simplest form of passing credentials with WS-Security.

As an example, Rampart generates the correct form of the password tag, which includes “type” attribute (this attribute is not mandated by WS-Security, however, it is required by BSB):

<wsse:Password
      Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">
      B5twk47KwSrjeg==
   </wsse:Password>

BSP is going to make the use of WS-Security more widespread, although this is not going to happen overnight. Currently, none of the public Web services (Yahoo, Ebay, Amazon) use WS-Security, which is not surprising since the standard is not fully supported by such languages as PHP, Python and Ruby (although some aspects of WS-Security are supported in PHP). Even platforms for commercial services, such as StrikeIron, don’t use it (StrikeIron does support WS-Security for communication between its broker and service providers, but not for client calls).

In any event, I’m pretty confident that we can count on rising WS-Security adoption in an enterprise.

WSDL Naming Conventions

Web service providers communicate with their customers (consumers) by the means of publishing WSLD of the service. In most cases, developers create the client code for the service by generating classes from the published WSDL file. While JAX-WS makes it possible to avoid code generation entirely (as I described in
this post) and hand-code all client-side classes, in all but the most trivial cases using code generation is the easiest way to start developing a Web service client.

Service providers need to keep in mind that WSDL is the only “bridge” between providers and consumers. Service providers may have great clean code with nicely defined interfaces and object model; however, this code won’t be shared with consumers (unless, perhaps, we are talking about an internal service used for integrating components of the same application, but even in this case developers should beware of created tight coupling). Therefore we need to make sure that WSDL that we publish will help service consumers generate good client-side code.

JAX-WS mandates the following mappings between WSDL and the generated client code:

  • wsdl:portType translates to the Java interface (business interface) of a Web service (where each operation maps to a method of the interface).
  • wsdl:service translates to, essentially, a factory class that extends javax.xml.ws.Service. This class has “get” methods for each port defined as part of the service element. “getPortName” methods return dynamic proxy that implements the service business interface.
  • Some tools (e.g., WebSphere Web Services Feature Pack) generate additional proxy classes (not to be confused with the dynamic proxy created by the Service subclass) for each port. The proxy class encapsulates logic for accessing javax.xml.ws.Service to invoke the right “get” method. Client code then only needs to instantiate the proxy class and call a business method. The name of the proxy class is NameOfThePortProxy.

Keeping these rules in mind, I use the following naming conventions to ensure that the generated code will contain classes with meaningful names:

  • I use the name of the service business interface as the name of the port type, e.g., POProcessor or PersonService. This ensures that service provider and service consumer will use the same name to refer to the business interface (remember that in general service providers and service consumers do not share any Java classes or interfaces). I never add “PortType” to the name, as this obscures the name of the interface generated for the client. Additionally, it makes names long and POProcessorPortType.process(po) just does not look too good in the code.
  • I add “Ports” to the service name, e.g., POProcessorPorts or PersonServicePorts. I don’t like adding “Service” since most Web service business interfaces already have the name “Service” in them, and “PersonServiceService” looks rather ugly.
  • For ports, I also use the name of the service business interface (i.e. the port type name). More often then not, SOAP/HTTP is the only service binding and so port type name can be used without any changes. If there are other bindings, I add the binding type to the port type name, e.g., PersonServiceJMS.

The following example illustrates the approach:

JSON Pros and Cons

JSON is a simple object serialization approach based on
the JavaScript object initializers syntax. The code for initializer (object
literal) is put into a string and then interpreted using JavaScript eval()
function or JSON parser (which is very lightweight):

serializedObj='{firstName:"john", lastName:"doe"}';
...
// This is just an example, JSON parser should be used instead
// to avoid security vulnerabilities of "eval"
var obj = eval("(" + serializedObj + ")");
document.getElementById("firstName").innerHTML=person.firstName;

JSON is used extensively in various AJAX frameworks and toolkits to provide easy
object serialization for remote calls. It is is supported by both GWT
and DOJO. There is a growing realization that
perhaps JSON should be considered as an option when implementing SOA, for
example Dion Hinchcliffe recently published a blog
entry
suggesting that JSON (and other Web 2.0 technologies) must be
seriously considered by SOA architects.

So what are the benefits of using JSON relative to XML (for this comparison I
want to focus just on XML and stay away from SOAP)? Here is my brief take on
that, there are also numerous other articles
and posts
on the subjects.

JSON XML
JSON strengths
Fully automated way of de-serializing/serializing JavaScript objects,
minimum to no coding
Developers have to write JavaScript code to serialize/de-serialize to/from
XML
Very good support by all browsers While XML parsers are built-in into all modern browsers, cross-browser XML
parsing can be tricky, e.g., see this
article
Concise format thanks to name/value pair -based approach Verbose format because of tags and namespaces
Fast object de-serialization in JavaScript (based on anecdotal evidence ) Slower de-serialization in JavaScript (based on anecdotal evidence )
Supported by many AJAX toolkits and JavaScript libraries Not very well supported by AJAX toolkits
Simple API (available for JavaScript and many other languages) More complex APIs
JSON weaknesses
No support for formal grammar definition, hence interface contracts are hard to
communicate and enforce
XML Schema or DTD can be used to define grammars
No namespace support, hence poor extensibility Good namespace support, many different extensibility options in Schema
Limited development tools support Supported by wide array of development and other (e.g., transformation)
tools
Narrow focus, used for RPC only, primarily with JavaScript clients (although
one can argue that it’s one of the strengths)
Broad focus – can be used for RPC, EDI, metadata, you name it
No support in Web services -related products (application servers, ESBs, etc),
at least not yet
Supported by all Web services products

So the bottom line is that JSON and XML are, of course, two very different
technologies; XML is much broader in scope so I’m not even sure if comparing
them side by side is fair.

As an object serialization technology for AJAX (or should it now be called AJAJ
since we’ve replaced XML with JSON?) JSON looks very appealing. Anybody who ever
tried parsing SOAP directly in a browser (while having to support multiple
browsers) can attest that this is not a straightforward task. JSON simplifies
this task dramatically. So I think that ESB vendors should definitely start
thinking about adding JSON to the list of formats they support.

One of the keys to SOA success is that it should be easy to consume a service,
i.e., the entry barrier for service consumers must be low to support “grass root” SOA adoption.
While a top-down SOA effort may succeed, it will certainly take
longer than bottom-up (“grass-root”) approach when developers are able to
consume services as they see fit. AJAX/JSON fits this bill perfectly – it is
easily understood by developers and it does not require any Web services -specific
tools or infrastructure.

So overall I’m pretty enthusiastic about JSON.

Who Needs Web Services Repository?

Web services repositories are almost always mentioned in conjunction with SOA governance as its key enabler and there is a widespread notion that a repository is a key component of an SOA.

To me there are two types of SOA governance. There is strategic governance which is part of the overall IT governance that deals primarily with funding and other “big” decisions (“big G”). There is also more “tactical” SOA governance which deals mostly with configuration management (CM) (e.g., release management) of individual Web services (“small g”).

There is also Web service management, including SLA monitoring and enforcement, and, perhaps, security, but to me that’s just an operational aspect of SOA governance and it’s distinctly separate from big G (funding) and small g (configuration and change management).

So, having been in a developer’s shoes for the most of my career, I’m primarily interested in “small g”, I’m sure management types can figure out “Big G” much better than I am.

So how this “small g”, or SOA CM, is different from CM that we’ve been doing routinely over the last, well, at least 20 years? Why all of a sudden we’re told that we need new tools for that, such as Web services registries and repositories?

I thought that we already have a repository that we know very well, and this is our version control repository – CVS, SVN, ClearCase, Perforce, whatever. So all our code and code-related artifacts, including, of course, WSDL and schema files (and whatever WS-* files, such as WS-Policy, we’ll need in the future) are already checked into our version control repository of choice. We can do a lot of good things with version control repositories, including all kinds of analysis, diffing, etc. We can develop ant or rake script to integrate build and deploy process with version control. With tools like Maven we can do pretty complicated dependency management. There are also continuous integration tools, build servers, change management tools, reporting tools and all kinds of other software helping us to deal with the code we’re developing. In other words, we know how to “govern” our code and so “governing” a few extra XML files (WSDL, Schema) should not be that difficult or special.

So I just don’t see how a Web services repository is going to make any of these tasks easier.

What we do need is better WSDL and Schema presentation and visualization tools, so that Web service consumers don’t have to always deal with raw XML. But I doubt this task warrants an expensive SOA repository, and there are some tools out there that can do it on the cheap.

SOA repositories also provide some run-time APIs so that service consumers or intermediaries can dynamically discover the most suitable service. Quite frankly, I think this scenario is a little far-fetched, especially given the lack of support for dynamic discovery in existing WS tools and products. Then there is also support for dynamic endpoints a la UDDI (or directly using UDDI standard), but, again, dynamic endpoints can be supported much more easily using configuration files as opposed to heavy-weight run-time APIs. Extremely low acceptance of UDDI is the proof of that.

So perhaps SOA registries and repositories are useful for “Big G” governance tasks (although I have my doubts – e.g., how relevant are WSDL files for funding decisions?), but the “small g”, that is, CM tasks, can certainly be more efficiently handled by existing CM tools. SOA repository vendors should think about extending existing CM tools instead of trying to create specialized environments just for Web services.

Schema Compliance is the Key to Interoperability

Good Web services interoperability
is an absolute must for a successful SOA implementation, but why
interoperability has been so difficult to achieve?

I think that inability to comply with a published Web services contract
expressed via its WSDL/Schema could be one of the leading causes of
interoperability problems (I use the term “interoperability” pretty broadly here).

For example:

  • Many “wrapper” service implementations use positional parameter binding, as I
    described in
    my previous post
    . This allows a service provider to accept messages
    will never validate against the schema. Then, certain changes in
    implementation (a different binding mechanism, switching from “wrapper” to “non-wrapper”),
    could all of a sudden start causing issues for clients that have never had any
    problems before.

  • Some Web services clients always generate messages using qualified local
    elements thus ignoring what is in the schema (the default is actually “unqualified”).
    This may work for some binding frameworks but not for others.

  • Different JAX-RPC implementation handle “xsd:anyType” differently. Some generate “Object”
    bindings; some use SOAPElement and some actually allow using DOM’s “Document”
    or “Element” (as an extension to JAX-RPC).
    This would be Ok if serialization/de-serialization
    process did not change depending on the binding,
    but that unfortunately is not always the case.

  • Handling of “nillable” (xsi:nil) and required elements. If an element is
    declared as required but nillable, it must be provided as part of a message.
    Instead, some Web service clients omit an element if it is “null” in Java.

My biggest pet peeve is that most JAX-RPC implementations today don’t even
support schema validation out of the box (hopefully this will change in JAX-WS).
I understand that full validation against a schema could be expensive, but can
we at least handle required parameters/fields as they defined in the schema?
Types mismatch will certainly cause an error, why not a missing mandatory field?

Note that I’m not talking about WS-I profiles, .NET/Java interop, and
differences in WS-* implementations. I’ve seen all of the above problems while working
with different Web services implementations in Java (mostly JAX-RPC).

And it is not just about interoperability – reliance on subtle conventions as
opposed to a published contract makes architectures more fragile and more
tightly coupled.

So my advice is simple – write a handler (at lest for development/test
environments) and enable schema validation on the server and, under some
circumstances (i.e., you don’t control the service), on the client. This will
save you from many interoperability problems, guaranteed.

“Wrapper”/”Non-wrapper” Web Service Styles – Things You Need to Know

“Wrapper”/”non-wrapper” Web services styles are mandated by JAX-RPC and JAX-WS
specifications
and are explained in details in documentation or in other sources, for
example in
this article
. However, from my experience, there is still a lot of
confusion among developers about differences between these two styles and also
how a particular style affects design and implementation of a Web service.

First of all, we need to understand that “wrapper”/”non-wrapper” style is a
characteristic of a Web services implementation framework (such as JAX-WS), as
opposed to a style defined in WSDL. In fact “wrapper”/”non-wrapper”
can only be used in conjunction with the document/literal style defined in WSDL.
“wrapper”/”non-wrapper” setting defines how Web service request/reply messages
are interpreted by a Web service provider/consumer. Quite simply, “wrapper”
style tells the Web service provider that the root element of the message (also
called “wrapper element”) represents the name of the operation and it is not
part of the payload. This also means that children of the root element must map
directly to parameters of the operation’s signature. The “non-wrapper” style (also
sometimes called “bare”), does not make this assumption; in this case the entire
message will be passed to the service operation. The reply message is handled in
a similar way.

“Wrapper” style can be used for the following method:


public String produceFullName( 
        // note we have to provide names in annotations
        // to comply with the schema.
        // otherwise generic "in0", "in1" names are used.
        @WebParam(name = "firstName")String firstName, 
        @WebParam(name = "lastName") String lastName );

The request SOAP messages for this method will look like this:


<soap:Envelope 
    xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soap:Body>
    <produceFullName xmlns="http://personservice">
      <firstName>John</firstName>
      <lastName>Doe</lastName>
    </produceFullName>
  </soap:Body>
</soap:Envelope>

However, suppose we implemented the same function differently:


public String produceFullName(
        @WebParam(name = "person", 
                targetNamespace="http://person") 
        Person person );

This method can be used with “non-wrapper” style resulting in the following
request message:


<soap:Envelope 
    xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soap:Body>
    <ns2:personExt xmlns:ns2="http://person">
      <firstName>John</firstName>
      <lastName>Doe</lastName>
    </ns2:personExt>
  </soap:Body>
</soap:Envelope>

In JAX-WS, “SOAPBinding” annotation can be used to specify the style, e.g.,
@SOAPBinding(parameterStyle=SOAPBinding.ParameterStyle.BARE).
Note that “wrapper” is the default. In JAX-RPC the “wrapper” style is
specified in the Web service mapping xml file by adding “wrapped-element”
element to the service endpoint mapping.

If you generate your Java artifacts from WSDL file, the “wsdl2java” tool
will automatically assume “wrapper” if operation name matches the wrapper
element name and if some other requirements are met (some “wsdl2java”
implementations also allow to control “wrapper”/”non-wrapper” using a command-line option).

So once again, “wrapper”/”non-wrapper” is a data binding style used in the
context of a particular Web service provider/consumer. It is not part of the
contract of a Web service, since it is not mentioned anywhere in the WSDL file.
In other words, if you are a Web service provider, your consumers don’t have to
know or care whether your implementation style is “wrapper” or “non-wrapper”.
They may very well choose the “non-wrapper” style for their client’s
implementation and this client should interoperate with your “wrapper” service
without a hitch (at least in theory).

In fact, the “wrapper” style is supported widely but not universally.
From what I understand, the support for the “wrapper”
style first appeared in .NET and later on Java-based Web services frameworks
began supporting it as well (in Axis and others). I think it is also supported
by Perl SOAP library. However, this style may not be supported by other languages,
for example, I believe that PHP 5 SOAP engine does not have a notion of the “wrapper”
style.

“Wrapper” style is really an RPC-style binding over “document/literal” WSDL
style. The basic premise of the “wrapper” style is that our service is a remote
method that takes some parameters (as opposed to pure message-based paradigm of
a “non-wrapper” Web service, where method name is not explicitly provided in the message).
So how “wrapper” with “document/literal is better
than “rpc/literal” WSDL style? Here are some of its advantages:

  • “Wrapper” service can only have one message part in WSDL, which guarantees that
    the message (consisting of the method element that contains parameters) will be
    represented by one XML document with a single schema.

  • RPC style message definitions in WSDL have to use schema types, whereas with the
    document style we can refer directly to element names. This makes XML message
    representation more “precise”, it is also easier to validate.

  • RPC/literal, while WS-I compliant, fell out of favor and being
    frowned upon, not
    in small part due to poor Microsoft support.

“Wrapper” style has one interesting drawback, which may not be immediately
obvious. Most Web services engines (at least the ones that I had a chance to
work with) use positional parameter binding with “wrapper” style. This means
that if a service signature changed (e.g., a new parameter was added), clients
will have to change as well. In other words, with “wrapper” style, all
parameters have to be present in the XML message (although they can be defined
as null using “xsd:nil” attribute). This is despite the fact that the element
corresponding to the parameter can be defined as optional in the schema. “non-wrapper”
style does not have this problem; adding a new optional element does not
affect clients, since binding is always name-based. This creates somewhat
tighter coupling between “wrapper” style consumers and providers. This may also
violate the contract of the Web service defined by its schema. To get around
the last problem, you may want to define all child elements of the wrapper root
element as required (minOccurs=1); optional elements must be made nillable. For
some reason JAX-WS spec does not explicitly state this.

JAX-WS, however, does impose some additional restrictions on the “wrapper” style,
the most important one being that the wrapper element’s content type must
be “sequence”. This makes sense, since we’re trying to model a method’s
signature, which is always represented as an ordered list of arguments (at least
in Java/C#).

As far as the “non-wrapper” style goes, probably the most obscure thing about it
deals with how a SOAP engine decides which operation to invoke based on the
message type (assuming a Web service has multiple operations), since operation
name is not explicitly provided by the message (unless “SOAPAction” header is
used, however, this header is HTTP-only and also optional in JAX-WS). With “non-wrapper”,
each operation must accept a message that corresponds to a unique XML element
name. Note that it is not enough to define different WSDL messages using “wsdl:message”
element, each message must be represented by a different element in the schema.
Web service engines use unique element names to determine Java method names (that
correspond to WSDL operations). This means that with “non-wrapper” you can’t
have different operation names processing the same message, e.g., process(Customer)
and processDifferently(Customer) can only be implemented by creating
different customer types.

So we have an interesting dilemma – “wrapper” does not support overloaded methods (in
fact, I don’t think WS-I allows them either), whereas “non-wrapper” does not support
methods with the same signature (at least under certain conditions).

So how should we decide when to use the “wrapper” style? My approach is always
the following:

  • Design good “tight” schemas for your request and reply messages. Do not rely on Java-to-WSDL/Schema
    generation. Hopefully, you have good requirements as the basis for your design.

  • You can now see if the request schema matches the Web service operation’s
    signature. If that is the case, “wrapper” might be your choice. Consider pros
    and cons of each style described above. Basically, “wrapper” provides more “natural”
    RPC implementation, whereas “non-wrapper” gives more control and somewhat
    looser coupling (from a client’s standpoint). If you decide to go with the “wrapper”,
    do not assume that all of your clients will be using the “wrapper” (unless you
    know for sure that this will be the case). So make sure that your WSDL can be
    easily consumed by both “wrapper” and “non-wrapper” clients. For example, an
    element with empty sequence maps to “void” return type if the “wrapper” style is
    used; however, “wsdl2java” (or a similar tool used by other languages) will
    generate an empty class as the return type for “non-wrapper”. This last option
    may not be very intuitive or desirable for your clients.

  • When publishing your service, provide examples of its client implementation. You
    may not be able to address all the different languages and platforms; however
    covering the most widely used ones (most often, Java in C# in an enterprise environment) is a
    good idea. This will help your clients consume your Web service and also clarify
    the “wrapper” versus “non-wrapper” question.

Reliability of SOAP over HTTP Web Services

HTTP or HTTPS can be viewed as the current de-facto standard transport binding
for Web services. It is frequently said, however, that HTTP is inherently
unreliable and not appropriate in situations where guaranteed delivery and other
Quality of Service (QoS) characteristics are required. To deal with this issue,
many Web services products, commercial and open-source, offer a non-standard (meaning
that it is not part of a WS-I profile) transport binding using JMS or other
messaging APIs. Alternatively,
WS-ReliableMessaging (WS-RM)
specification
defines a reliable messaging protocol for Web
services independently of the transport binding. Currently, WS-RM is supported
by several Web services products.

It is tempting to standardize on one of these alternatives for an enterprise SOA
implementation in order to meet QoS requirements. However, we need to realize that both
of these options add complexity to Web services implementation and greatly
impair interoperability. JMS binding requires that all Web service consumers
must support a particular messaging provider. Additionally, appropriate
messaging resources (queues, connection factories) must be configured
on the server and, sometimes, on the client’s side (e.g., MQ channels). A Web service
provider has to be able to communicate with a messaging provider e.g., to
consume a message from a queue (which calls for a Message Driven Bean in J2EE environment).
So while these issues are not insurmountable, they certainly make things
more complicated and, among other things, require full J2EE stack for Web
services implementation.

WS-RM is currently being standardized by
WS-I
, there is also
project “Tango”

that specifically addresses interoperability between Sun and Microsoft
implementations. Unfortunately, WS-RM support is still far from being uniformed among
Web services vendors. Also, from what I understand, WS-RM by itself is not
sufficient since it does not define how (or if) a WS-RM conversation
participates in a transaction. So then WS-Transaction or some proprietary vendor
extension is needed, and that in turn calls for the ability to support XA. I
also suspect that the use of a persistent storage on the client is the only way
to support all WS-RM requirements (e.g., what if the client goes down in the middle
of a message exchange?), so this makes clients “fatter” and more complex.

The bottom line is that “good old” SOAP over HTTP is the easiest and the most
interoperable way to implement Web services today. So how much can we trust
SOAP/HTTP Web services and should HTTP even be considered in
an enterprise setting where QoS is almost always a requirement?

First, we need to remember that HTTP as well as virtually any other
communication protocol today (including IIOP and protocols used by messaging
providers) is based on TCP. The whole purpose of TCP is to be able to transfer
data reliable and so it employs acknowledgements and sliding window mechanism to
guarantee the delivery of data. What does it mean in terms of Web services? Say,
we have a one-way service. If we invoked a service and received a successful
reply (no SOAP faults or HTTP errors), we can be confident that our SOAP message
reached its destination. But what if the connection went down in the middle of
the call and we received a timeout error? Our message exchange is now in an
ambiguous state. If the message has not been received, we need to invoke the
service again, but, on the other hand, if it has been received, the duplicate
message may lead to an inconsistent state if the service provider is not able to
handle duplicates correctly.

QoS provided by messaging systems or WS-RM helps us in this situation by ensuring
that the message will only be delivered once; messaging systems can also handle
re-delivery of messages and “store and forward” (there are other QoS policies
besides “exactly once”, but “exactly once” is the most stringent one).
Messaging systems also provide another important benefit by allowing a Web
services call to participate in an atomic transaction. This allows service
consumers to keep in synch multiple resources (including the ones provided by Web
services) thus improving data integrity and reliability.

So where does it leave SOAP/HTTP services? Based on the explanation above, SOAP/HTTP
is not the best fit when:

  • Service providers can’t handle message duplicates (in
    other words, an operation performed by the service is not idempotent).

  • Different data resources owned by service provider and service consumer must
    be in synch at all times.

However, we still might be able to use SOAP/HTTP if we make our service
consumers a little smarter. Specifically, service consumers must be able to meet
the following requirements:

  • Consumers must be able to retry a Web service call in case of a failure due to
    a connectivity error. In the simplest case, a consumer’s application may
    choose to show the error message to an end user and let the user press “submit”
    button again. However, most consumers will probably choose to automate the retry
    logic (e.g., have X number of attempts) and at least log unsuccessful attempts
    and potentially alert an application administrator (note that some Web services
    clients have built-in retry capabilities).

  • Consumers must be able to uniquely identify the data that they are sending to
    providers. Suppose we have “addEmployee” service that takes an employee XML
    message as an input. The employee message must have unique ID set by the
    consumer of the service, so that when the invocation is retried, the service
    will be able to detect that the employee was already added as part of the
    previous call. This means that popular techniques using database sequences or
    autoincrement fields for generating unique IDs do not work with SOAP/HTTP Web
    services. Service consumers must implement their own way of creating unique IDs
    (perhaps relying on some kind of a natural key or using UUID).

  • Consumers must be able to handle certain application-specific errors (SOAP
    faults). For example, “addEmployee” service may return “Employee with this ID
    already exists” error after “addEmployee” call was retried in response to a
    connectivity failure. The consumer’s application will have to stop retrying
    after catching this error. This situation may also require an end user (or an administrator) involvement to
    verify that the employee was indeed added.

As an example, let’s take a look at “Create/Read/Update/Delete” (CRUD)
operations. While real-life services oftentimes do much more than just “CRUD” (e.g.,
calculations), it is a valid simplification for our purposes.

Operation Is Indempotent? Required Service Consumer Behavior
Create No Must be able to retry the call

Must be able to handle “record already exists” error.
Read Yes None
Update Yes, unless update involves calculating new values based on the
existing values, e.g., incrementing a counter
Must be able to retry the call
Delete No Must be able to retry the call

Must be able to handle “record
does not exist” error.

So what is the takeaway? SOAP/HTTP can be used in many (if not most)
situations, however, implications of this decision must be fully
understood. All service consumers must be made fully aware of these implications.
Most importantly, service consumers must implement proper logic for handling
connectivity failures and application errors. In some cases, users of service
consuming application may need to be instructed about how to react
to certain application errors.

Using Schema Validation with JAXB and XFire

Schema validation framework that I covered earlier is actually fully supported by
JAXB. If you’re using XFire, you will have to switch to JAXB binding in order to
utilize it.

As described in XFire documentation, you can specify “schema” element as part
of your service definition and point it to your schema. From what I understand,
however, this does not enable validation, it just tells XFire to use
your schema as part of the generated WSDL (in response to “?wsdl” requests).
So I actually had to develop a simple handler to enable it:


public class JaxbValidationEnablingHandler extends AbstractHandler {
   
    public JaxbValidationEnablingHandler() throws SAXException{
        super();
        setPhase(Phase.PARSE);
        before(ReadHeadersHandler.class.getName());
    }
   
    public void invoke(MessageContext ctx) {
        ctx.setProperty(JaxbType.ENABLE_VALIDATION, "true");
    }
}

Another problem that I encountered was that validation exceptions don’t
reach the client, instead the client receives meaningless “Can’t unmarshal” exception.
A simple change in XFire’s JaxbType class fixes that though.

So how does this approach compare to the validation handler approach that I presented
in the previous post? On one hand,
relying on JAXB allows us to easily handle both “wrapped” and “bare” Web service styles.
It also requires minimal effort to implement. On the other hand, very often with Web services
there is a need to be able to process XML without reading it into objects, so, chances are,
JAXB won’t be used in all cases. Also, non-JAXB approach provides for more sophisticated
error handling using
custom error handlers that can be used in conjunction with the validation framework.

Another requirement that have to be considered regardless of the approach is
integration with various XML and Web service registries that are becoming
wide spread in large organizations. This means the ability to read schema from a URL or even
over UDDI or proprietary API.

Clearly, there is still some work to be done in order to make schema validation the norm
for Web services.

Web Services without Code Generation

JAX-RPC supports two major ways of developing Web services:
“bottom-up”, which allows to generate WSDL from a Java interface and “top-down”,
which generates Java classes from WSDL. Both of these approaches suffer from
one major drawback.
WSDL/Schema/mapping files or Java classes are fully re-created every time there is a need
to change Web service interface and so it becomes a developer’s responsibility to
merge existing WSDL files or Java code with the generated files. The need to manually
change WSDL/Schema stems from the fact that the generated files are usually very “basic”,
they do not take advantage of advanced schema capabilities, they
don’t use WSDL/Schema modularization and so on. Generated Java classes are even more problematic.
You really don’t want to have simple “value” classes without any behavior with fully exposed
properties. And adding behavior requires manual intervention.

In reality, there is a way to evolve two sets of artifacts (WSLD
and Java classes) independently without any generation by manually updating
Java-WSDL
mapping file
. The format of this file, however, is the antithesis of the
“convention over configuration” idea. The format is extremely verbose; each and every
field has to be mapped explicitly and so manual modification of this file poses a
real challenge for any more or less complex data structure.

The latest JEE Web service specification, JAX-WS, fixes this problem by
heavily leveraging annotations.
For data mapping, annotation support
is provided by JAXB specification. JAX-WS and JAXB finally free developers from
having to rely on the brittle code generation approach.

In fact, JAXB still supports code generation and so developers can use
it for generating Java or WSDL files. The generated artifacts could provide a good starting point for a
Web service interface. After that, however, it is pretty easy to keep both sets of
artifacts in synch by updating annotations and/or WSDL/Schema files.

How does it work in practice? Let’s say we have this very simple schema:


<element name="person">
  <complexContent>
    <sequence>
      <element name="firstName" type="xsd:string" />
      <element name="lastName" type="xsd:string"/>
    </sequence>
  </complexContent>
</element>

The following class will match the schema:


// By default JAXB uses getters/setters to drive marshaling/unmarshaling
// Using fields is easier, since they can be defined more compactly
// in one place and also moved around to manipulate order.
// Beware though - getters/setters are ignored in this case
@XmlAccessorType(XmlAccessType.FIELD)
// We need to provide the namespace, otherwise it defaults to
// the package name
@XmlRootElement(namespace="http://myarch.com/personservice")
public class Person {
    // note that JAXB marshals instance variables in the order they
    // declared
    protected String firstName;
    protected String lastName;
   ...

Note that JAXB provide several
additional attributes and annotations
which I’m not using here – I’m trying to rely as much as
I can on “convention over configuration”. Element names, for example, match my variable names,
so there is no need to provide annotations for individual fields (however, in this case,
you can’t use qualified locals
in your schema since JAXB won’t prefix nested elements.)

Many JAXB annotations, such as “XmlType” are only used for generating schema from Java classes.
Same is also true for most attributes of XmlElement.
Oftentimes developers specify “required” and “nillable” attributes of XmlElement annotation
thinking that JAXB will automatically enforce these constraints, e.g. :


@XmlElement( required = true, nillable=false)
private String lastName;

However, these parameters are not used at
run-time. JAXB 2.0 actually relies on the
schema validation framework
and so
these parameters can be omitted assuming that the schema contains the constraints.

In other words, we only have to use those annotations that help
us to correctly marshal/unmarshal our objects.

The only other step that we need to take is to create ObjectFactory class in same
package with the mapped class. In this class we need to define a factory method for the root
element of our content tree (you don’t need to worry about nested datatypes even
if they map to other classes). From what I understand, JAXB does not actually invoke any
factory methods, however, it does check for their presence in ObjectFactory.
The factory method looks simple enough:


public Person createPerson() {
    return new Person();
}

Now we can make changes to our Java classes and the corresponding WSDL/Schema files without
ever having to resort to code generation. In my mind, this is a preferred way.
We can see from Object-Relation mapping that “pure” “top-down” and
“bottom-up” don’t really exist. How often do we generate DDL from Java classes or vice versa?
Different data representations supported by different technologies have to be able to evolve
independently without having to play by each other rules. JAXB helps us to achieve it pretty easily.