• SOAP tight coupling with WSDL. Until recently, SOAP was the only supported WSDL binding. WSDL, with all of its issues (such as the convoluted structure), remains the only widely accepted vendor-neutral way of defining services.
• In Java world, SOAP was promoted by adding its support to J2EE/Java EE via JAX-RPC and JAX-WS specifications. There is a way to create RESTful services using JAX-WS, however this approach has not gained wide acceptance due to a number of reasons (lack of standardized WSDL binding being one of them).
• Solid support by software vendors. SOAP is supported in all application servers; there are plenty of development tools (including Eclipse and NetBeans) that can help with creating SOAP-based services.
• Availability of a number of WS-* specifications that only work with SOAP. WS-* specs, such as WS-Security and WS-ReliableMessaging heavily rely on SOAP headers for passing message metadata.

SOAP, however, is far from perfect. It adds complexity and overhead and makes implementation of message intermediaries (e.g., gateway products) more difficult (XML message has to be parsed in order to obtain any information about the message, such as an operation name). On the other hand, the uptake of WS-* standards has been very slow, perhaps with the exception of WS-Security, so the benefit of WS-* integration with SOAP failed to materialize. Another supposed SOAP benefit is the protocol neutrality, i.e., a SOAP message looks the same over HTTP, JMS, RMI or any other supported protocol. However, most services are implemented using HTTP, or in some cases, JMS without any headers (unless WS-* is used), so this benefit has limited value.

So it is fair to say that in many instances, SOAP itself does not add any value to Web services implementation. Nevertheless, it's continued to be used because of the reasons listed above. But this situation might be changing in the future.

WSDL 2.0 fully supports HTTP binding. WSDL 2.0 stirred a lot of controversy (I personally think that it's a step forward from WSDL 1), however, it was finally accepted as a W3C recommendation in June, so we can fully expect that vendors support will follow.

Additionally, JAX-RS specification was created and included into Java EE 6. This will also promote development of RESTful services behind the firewall.

Currently, there a preconceived notion that REST is only good for Internet/WEB 2.0 environment, whereas enterprise services should all be using SOAP. The emergence of the two new standards will help change this notion.

Ideally, developers should be able to use the simplest binding that gets the job done. HTTP/REST can be a good choice for many "query-style" services. Services that accept more complex data can use "Plain Old XML" over HTTP post. Finally, services that have to rely on WS-* specifications will utilize SOAP. In all these cases, developers should be able to use same set of APIs for invoking the service and same set of annotations (obviously, with different parameters) for creating the service.

Perhaps this is how Web service development will be done in the future, but we are certainly not there yet.

There are a number of commercial registry/repository products but all of them are quite pricey. Also, smaller organizations or organizations just starting with SOA may not need the full power of a commercial registry/repository product; most of it functionality will remain unutilized. On the other hand, existing registries do not do a particularly good job of managing dependencies between services as well as between service consumers and service providers. Dependency management must be a key consideration in SOA since, unlike in the case of monolithic applications, an SOA consists of a large number of “moving parts”, that is, services. In a well developed SOA most services depend on other services and these services in turn may rely on some other services. Being able to manage and analyze services dependencies is required in order to be able to implement robust and maintainable SOA.

Open source Maven project has become something close to a de-facto standard in the area of dependency management for building Java-based projects and components. Also, Maven dependency mechanism can be used with any Ant build by utilizing Maven Ant dependency tasks (i.e., there is no need to migrate existing Ant build processes to Maven). Maven also has a pretty sophisticated repository support allowing implementing custom central repositories as well as local repository snapshots. Projects like artifactory further enhance Maven repository and add nice UI.

So why not use a Maven repository for storing Web services artifacts, such as WSDL, schemas and policy files? Each service or a group of related services from the same service provider can have its own POM. Using Maven (i.e., by executing “deploy” goal) service providers can publish WSDL and other artifacts (but, obviously, no binaries) along with their POMs in a Maven repository. Service consumers could then download these artifacts based on the dependencies defined in their POMs. If the version stays the same, service consumers don’t need to do anything. In case of a change, service consumers may need to make changes to their client’s implementation, including re-generating client’s classes (this is, of course, a manual process that requires some analysis of the changes). In other words, the process is very similar to how we deal with changes in APIs in third-party libraries.

The entire process can be supported by out-of –the-box Maven build process. The only required change is a custom artifact handler to support new “service” extension and packaging type.

An existing change management process can be adapted for Web services so that service consumers are notified about new versions and their backward compatibility. Again, there is nothing new here; most organizations already have a change management process in place to handle changes in shared libraries.

Using Maven for managing Web services releases and dependencies has multiple benefits:

• Ability to use full range of Maven dependency management capabilities including version range dependency, snapshot and central repositories and so on.
• Ability to use Maven dependency reports for dependency analysis.
• Web services dependency management is consistent with the dependency management process used for other artifacts, such as jars, wars and so on. Commercial repository products often provide a proprietary mechanism that can only be used for Web services - related artifacts.
• Tight integration with any Maven or Ant-based build process, no need for using UDDI or proprietary APIs for publishing services.
• Service consumers that do not use Ant or Maven could still participate in the process by downloading Web services artifacts manually using UI provided by Artifactory or a similar product (although using automated approach should be the preferred option).

Of course, the Maven-based approach does not do anything for managing and enforcing (including run-time enforcement) of Web services policies which is something that commercial registry/repository products can do quite well. So organizations need to evaluate the need for a commercial registry/repository based on types and complexity of the policies they would like to enforce. In many if not most cases the policies are quite straightforward in which case Maven-based solution becomes a viable option. ]]> http://myarch.com/using-maven-repository-as-web-services-registry/feed 6 http://myarch.com/comparison-of-soa-suites http://myarch.com/comparison-of-soa-suites#comments Sun, 17 Jun 2007 22:53:39 +0000 Alexander Ananiev http://myarch.com/comparison-of-soa-suites Several SOA vendors are trying to put together comprehensive suites of SOA products that in theory should be capable of addressing all aspects of SOA, including governance, integration, business process management and others.

Formation of SOA suites is having a tremendous impacts on how SOA products are selected as many organizations are being tempted to settle for "one stop shop" approach as opposed to doing proper product evaluation within each SOA product category. (Interesting discussion about SOA suites is available at ZDNet).

So what is a SOA suite and how offerings from different vendors support different aspects of SOA? The table below attempts to answer this question.

One thing to keep in mind is that SOA products within the same category can differ substantially in terms of their feature sets. Definitions of ESB, registry and other SOA product categories are not standardized and so vendors are free to categorize their products as they wish. Detailed analysis and evaluation is still a requirement when selecting SOA products.

IBM, BEA and Oracle are emerging as leaders in terms of completeness of their SOA suites. Microsoft's products are not as comprehensive, nevertheless Microsoft's marketshare makes it a signifcant player - Micorsoft's shops tend to be very loyal to the vendor even if Microsoft's SOA story is not as compelling.

Other vendors, including Software AG/webMethods and HP have interesting offerings too, however, they still have some gaps in their capabilities and so they are not covered here (perhaps I'll add more vendors in the future).

However, I don't think that JAX-WS is going to become the primary way of implementing RESTful clients just yet. In many situations, HttpClient provides more flexibility and control over HTTP calls.

For example, it is very straightforward to turn on basic authentication for a client using HttpClient APIs:

 HttpState httpState =httpClient.getState();
 httpState.setCredentials( AuthScope.ANY, 
     new UsernamePasswordCredentials(username, password));
 // Set the authentication header on the first request, don't wait for 401
 httpClient.getParams().setAuthenticationPreemptive(true);
        
 postMethod.setDoAuthentication(true);

Implementing the same requirement in JAX-WS is not a big deal either:

Map requestContext = dispatcher.getRequestContext();
requestContext.put(BindingProvider.USERNAME_PROPERTY, username); 
requestContext.put(BindingProvider.PASSWORD_PROPERTY, password);

However, it is not clear how to tell the client to set basic authentication header on the very request in case of the service does not issue 401 basic authentication challenge (which is pretty common). It is also not clear if there is a way to change authentication scheme (say, to digest) or authentication scope.

There are also other examples that demonstrate that in attempt to abstract the details of the transport protocol JAX-WS limits capabilities of the clients.

However, using JAX-WS REST capabilities still makes sense for applications that use both SOAP and RESTfull services. This allows developers to use consistent API for both types of services. Also, in some instances, a service provider may initially implement a service using XML/HTTP and later decide to switch to SOAP, say, to be able to use some of the WS-* specifications (this is actually a fairly common situation within an enterprise). In this case, using JAX-WS will certainly make the transition easier for the clients. ]]> http://myarch.com/will-jax-ws-become-the-primary-mechanism-for-invoking-restful-services/feed 3 http://myarch.com/soapui-vs-junit http://myarch.com/soapui-vs-junit#comments Sun, 29 Apr 2007 03:15:10 +0000 Alexander Ananiev http://myarch.com/soapui-vs-junit Open source soapUI tool has a number of benefits over developing test cases in Java using JUnit framework. Some of these benefits include:

• Use of UI for creating/configuring tests. This makes it possible for non-programmers to develop tests, which means that testing of Web services can be performed by a QA/system test group.
• Auto-generation of test messages based on WSDL.
• Ability to externalize test data using property files.
• Built-in assertions, such as checking for SOAP faults and validity of the schema.
• Built-in stress testing capabilities.
• Management of endpoint URLs.

Overall, soapUI allows developers and testers to create tests quicker and also makes tests easier to manage and update.

However, one needs to understand shortcomings of this approach. soapUI uses its own Web services client. The way it creates test messages is different from how regular (JAX-WS-based and other) Web services clients work. soapUI completely skips the step of generating Java classes from WSDL. It does not have to deal with XML marshalling/unmarshalling from/to Java objects. As a result, you are not invoking your service the way it will be invoked by real Web service consumers.

Technically, it may not be an issue in a majority of cases. The fact that soapUI is able to parse your WSDL/schema and generate test message means that you have a valid WSDL that should also be consumable by other technologies. However, it is possible that your WSDL/ schema may lead to sub-optimal (albeit perfectly valid) generated classes. For example, the schema that defines <addresses><address>...</address>...</addresses> XML will result in JAXB compiler generating Addresses class that returns the list of type Address. This may or may not be how you want Java classes to look like for the clients. Perhaps you may want to consider to get rid of <addresses> so that no extra class is generated.

In other words, using soapUI (or similar UI-based Web services testing tools, such as SOAPScope) does not allow you to understand all the details of how your Web service will be consumed in the real world.

Another advantage of JUnit test cases is that they can be used as examples illustrating the use of your service. The test classes can be published along with other Web service documentation or handed over to consumers to help them jump-start their client implementations. Unfortunately, soapUI test cases can't be used for this purpose.

The shortcomings of visual testing tools such as soapUI can be addressed by complementing soapUI tests with JUnit tests. I usually start with soapUI since it is so easy to get up and running with it. Once soapUI test is implemented, I go back to Java, generate Web services client classes from WSDL and implement a JUnit test class. The logic that goes into a JUnit test class obviously depends upon a Web service. But at the very least, there should be one JUnit test per service that invokes the service using generated classes and libraries provided by your Web service product of choice. This way, we can see exactly how our services will be consumed by our clients.

In my opinion, it is always best to develop WSDL and schemas by hand to ensure that the service contract is appropriately defined and also that the schema can be re-used (by other services) and extended if necessary. I do not recommend using annotations for automatically producing WSDL and schemas at runtime as this leads to simplistic schemas and WSDLs.

Generating classes from WSDL/schema sometimes makes sense, say in situations when you have to use a pre-defined schema. However, as with any code generation, it creates a maitainance problem especially if there is a need to put behavior into generated classes. So it is desirable to be able to evolve object model and service implementation classes independently from WSDL/schemas. You can do it quite easily by following the steps below.

Even though I use the word "create" when describing some of the steps, the same approach will also work for updates or refactoring caused by changes in service requirements.

The steps are the following:

• Design your service interface. In our case the service has a single "add" operation that accepts a "person" document, adds it to a database and returns back the status:
  public String add( Person person )
• Design your data/object model. In our case Person class has three fields (ssn, firstName, lastName) and the list of addresses. Real world examples will certainly be more complex, but this is good enough for our purposes.
• Define XML schema based on your object model. Use strict datatypes where possible. For example, we use xsd:token instead of xsd:string since "token" does not allow carriage return, tabs, and leading spaces:

    <xsd:element name="person">
        <xsd:complexType >
            <xsd:sequence>
                <xsd:element name="ssn" type="xsd:token" 
                        minOccurs="1" maxOccurs="1"/>
                <xsd:element name="firstName" type="xsd:token" 
                        minOccurs="1" maxOccurs="1"/>
                <xsd:element name="lastName" type="xsd:token" 
                        minOccurs="1" maxOccurs="1" />
                <xsd:element name="address" type="Address" 
                        minOccurs="1" maxOccurs="unbounded"/> 
            </xsd:sequence>
        </xsd:complexType>
    </xsd:element>
  

Complete schema file

• Create an XML file for your schema so you can test the schema. You can generate an XML instance document automatically from Eclipse (assuming you're using Eclipse Web Tool Platform) by right-clicking on the schema in navigator and selecting "generate". Test the schema by changing data and making sure that you're getting correct validation errors.

Sample XML file

• Create WSDL. Our WSDL imports the schema that we just created. Note that the WSDL target namespace is different from the schema namespace since the same schema can be reused by different WSDLs:

    <wsdl:definitions 
          targetNamespace="http://personservice" 
          xmlns="http://personservice" 
          xmlns:pers="http://person"
          xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" 
          xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" 
          xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" 
          xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  
      <wsdl:types>
            <xsd:import namespace="http://person" 
                    schemaLocation="person.xsd" />
            <!-- Return value -->
            <xsd:element name="status" type="xsd:string" />
      </wsdl:types>
      ...
  

Complete WSDL file

• Create your object model classes and annotate them with appropriate JAXB annotations:

    // By default JAXB uses getters/setters for marshaling/unmarshaling
    // Using fields access allows us not to define getters/setters for 
    // all mapped instance variables.
    @XmlAccessorType(XmlAccessType.FIELD)
    // We need to provide the namespace, otherwise it defaults to
    // the package name
    @XmlRootElement(namespace="http://person")
    public class Person {
    
        // we only need annotations for variables that don't match 
        // element names
        private String ssn;
        private String firstName;
        private String lastName;
        
        @XmlElement(name = "address")
        protected List<Address> addresses;
      ...
  

Person class
Address class

• Create ObjectFactory class. This class is required by JAXB and it should contain factory methods for all JAXB-mapped classes. ObjectFactory must resides in the same package with object model classes.

    @XmlRegistry
    public class ObjectFactory {
    
        public Person createPerson() {
            return new Person();
        }
        
        public Address createAddress() {
            return new Address();
        }
    }
  

ObjectFactory class

• Create Web Service implementation class. Note that with JAX-WS Service Endpoint Interface (SEI) is optional, so all you need is an implementation class:

      @WebService(
              // all values must match corresponding attributes 
              // and elements of the WSDL file
          
              // Name of the port type in WSDL
              name="PersonService",
              // Target namespace of WSDL, could be 
              // different from the schema namespace
              targetNamespace="http://personservice",
              serviceName="PersonServicePorts",
              portName="PersonService",
              // the file must be available to Web container
              wsdlLocation="WEB-INF/wsdl/PersonService.wsdl"
      )
      
      /*
       * We'are using "bare" style since "add" operation 
       * takes only one argument, person document. Therefore, there 
       * is no need to "wrap" it by nesting it inside operation element.
       * Note that "wrapped" style is the default.
       */
      @SOAPBinding(parameterStyle=SOAPBinding.ParameterStyle.BARE)
      
      public class PersonService {
          
          @WebResult(name = "status")
          public String add( Person person ) {
              
              System.out.println("Adding person :"+person );
              return "Added "+person.getFullName();
          }
      }
  

PersonService class

• Deploy your Web service to your application server or Web services container. Refer to your application server documentation for details.
• Test the service. The easiest way to test the service is to use open source soapUI tool. You can install it as Eclipse plugin (there are also plugins for NetBeans and IDEA) following these instructions. In Eclipse, open SoapUI view, right click and create a project from your WSDL file. SoapUI will automatically create a sample SOAP request which you can update with your data. You can run it right away. Later, you can create a repeatable test suite with parametirized data using SoapUI properties and assertions:

SoapUI Project (you can import it into SoapUI).

You can also download all files in a zip file.

I looked at a few messages generated by Axis used in conjunction with Apache WS-Security implementation called Rampart and at the first glance they seem to be compliant with BSP, although all my examples use WS-Security username token profile, which is the simplest form of passing credentials with WS-Security.

As an example, Rampart generates the correct form of the password tag, which includes "type" attribute (this attribute is not mandated by WS-Security, however, it is required by BSB):

<wsse:Password
      Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">
      B5twk47KwSrjeg==
   </wsse:Password>

BSP is going to make the use of WS-Security more widespread, although this is not going to happen overnight. Currently, none of the public Web services (Yahoo, Ebay, Amazon) use WS-Security, which is not surprising since the standard is not fully supported by such languages as PHP, Python and Ruby (although some aspects of WS-Security are supported in PHP). Even platforms for commercial services, such as StrikeIron, don't use it (StrikeIron does support WS-Security for communication between its broker and service providers, but not for client calls).

In any event, I'm pretty confident that we can count on rising WS-Security adoption in an enterprise.

Service providers need to keep in mind that WSDL is the only "bridge" between providers and consumers. Service providers may have great clean code with nicely defined interfaces and object model; however, this code won't be shared with consumers (unless, perhaps, we are talking about an internal service used for integrating components of the same application, but even in this case developers should beware of created tight coupling). Therefore we need to make sure that WSDL that we publish will help service consumers generate good client-side code.

JAX-WS mandates the following mappings between WSDL and the generated client code:

• wsdl:portType translates to the Java interface (business interface) of a Web service (where each operation maps to a method of the interface).
• wsdl:service translates to, essentially, a factory class that extends javax.xml.ws.Service. This class has "get" methods for each port defined as part of the service element. "getPortName" methods return dynamic proxy that implements the service business interface.
• Some tools (e.g., WebSphere Web Services Feature Pack) generate additional proxy classes (not to be confused with the dynamic proxy created by the Service subclass) for each port. The proxy class encapsulates logic for accessing javax.xml.ws.Service to invoke the right "get" method. Client code then only needs to instantiate the proxy class and call a business method. The name of the proxy class is NameOfThePortProxy.

Keeping these rules in mind, I use the following naming conventions to ensure that the generated code will contain classes with meaningful names:

• I use the name of the service business interface as the name of the port type, e.g., POProcessor or PersonService. This ensures that service provider and service consumer will use the same name to refer to the business interface (remember that in general service providers and service consumers do not share any Java classes or interfaces). I never add "PortType" to the name, as this obscures the name of the interface generated for the client. Additionally, it makes names long and POProcessorPortType.process(po) just does not look too good in the code.
• I add "Ports" to the service name, e.g., POProcessorPorts or PersonServicePorts. I don't like adding "Service" since most Web service business interfaces already have the name "Service" in them, and "PersonServiceService" looks rather ugly.
• For ports, I also use the name of the service business interface (i.e. the port type name). More often then not, SOAP/HTTP is the only service binding and so port type name can be used without any changes. If there are other bindings, I add the binding type to the port type name, e.g., PersonServiceJMS.

The following example illustrates the approach:

serializedObj='{firstName:"john", lastName:"doe"}';
...
// This is just an example, JSON parser should be used instead
// to avoid security vulnerabilities of "eval"
var obj = eval("(" + serializedObj + ")");
document.getElementById("firstName").innerHTML=person.firstName;

JSON is used extensively in various AJAX frameworks and toolkits to provide easy object serialization for remote calls. It is is supported by both GWT and DOJO. There is a growing realization that perhaps JSON should be considered as an option when implementing SOA, for example Dion Hinchcliffe recently published a blog entry suggesting that JSON (and other Web 2.0 technologies) must be seriously considered by SOA architects.

So what are the benefits of using JSON relative to XML (for this comparison I want to focus just on XML and stay away from SOAP)? Here is my brief take on that, there are also numerous other articles and posts on the subjects.

So the bottom line is that JSON and XML are, of course, two very different technologies; XML is much broader in scope so I'm not even sure if comparing them side by side is fair.

As an object serialization technology for AJAX (or should it now be called AJAJ since we've replaced XML with JSON?) JSON looks very appealing. Anybody who ever tried parsing SOAP directly in a browser (while having to support multiple browsers) can attest that this is not a straightforward task. JSON simplifies this task dramatically. So I think that ESB vendors should definitely start thinking about adding JSON to the list of formats they support.

One of the keys to SOA success is that it should be easy to consume a service, i.e., the entry barrier for service consumers must be low to support "grass root" SOA adoption. While a top-down SOA effort may succeed, it will certainly take longer than bottom-up ("grass-root") approach when developers are able to consume services as they see fit. AJAX/JSON fits this bill perfectly - it is easily understood by developers and it does not require any Web services -specific tools or infrastructure.

So overall I'm pretty enthusiastic about JSON.

To me there are two types of SOA governance. There is strategic governance which is part of the overall IT governance that deals primarily with funding and other "big" decisions ("big G"). There is also more "tactical" SOA governance which deals mostly with configuration management (CM) (e.g., release management) of individual Web services ("small g").

There is also Web service management, including SLA monitoring and enforcement, and, perhaps, security, but to me that's just an operational aspect of SOA governance and it's distinctly separate from big G (funding) and small g (configuration and change management).

So, having been in a developer's shoes for the most of my career, I'm primarily interested in "small g", I'm sure management types can figure out "Big G" much better than I am.

So how this "small g", or SOA CM, is different from CM that we've been doing routinely over the last, well, at least 20 years? Why all of a sudden we're told that we need new tools for that, such as Web services registries and repositories?

I thought that we already have a repository that we know very well, and this is our version control repository - CVS, SVN, ClearCase, Perforce, whatever. So all our code and code-related artifacts, including, of course, WSDL and schema files (and whatever WS-* files, such as WS-Policy, we'll need in the future) are already checked into our version control repository of choice. We can do a lot of good things with version control repositories, including all kinds of analysis, diffing, etc. We can develop ant or rake script to integrate build and deploy process with version control. With tools like Maven we can do pretty complicated dependency management. There are also continuous integration tools, build servers, change management tools, reporting tools and all kinds of other software helping us to deal with the code we're developing. In other words, we know how to "govern" our code and so "governing" a few extra XML files (WSDL, Schema) should not be that difficult or special.

So I just don't see how a Web services repository is going to make any of these tasks easier.

What we do need is better WSDL and Schema presentation and visualization tools, so that Web service consumers don't have to always deal with raw XML. But I doubt this task warrants an expensive SOA repository, and there are some tools out there that can do it on the cheap.

SOA repositories also provide some run-time APIs so that service consumers or intermediaries can dynamically discover the most suitable service. Quite frankly, I think this scenario is a little far-fetched, especially given the lack of support for dynamic discovery in existing WS tools and products. Then there is also support for dynamic endpoints a la UDDI (or directly using UDDI standard), but, again, dynamic endpoints can be supported much more easily using configuration files as opposed to heavy-weight run-time APIs. Extremely low acceptance of UDDI is the proof of that.

So perhaps SOA registries and repositories are useful for "Big G" governance tasks (although I have my doubts - e.g., how relevant are WSDL files for funding decisions?), but the "small g", that is, CM tasks, can certainly be more efficiently handled by existing CM tools. SOA repository vendors should think about extending existing CM tools instead of trying to create specialized environments just for Web services.