First Impressions from Service Component Architecture

I'm currently playing with Service Component Architecture (SCA), currently supported in IBM products, such as WebSphere Process Server and WebSphere Integration Developer (WID).

IBM implementation "wraps" all SCA standard classes and namespaces using IBM proprietary classes and namespaces. As a result, XML tags and class names don't match what's in the spec. I suspect that the reason for that was that the products were developed when the specification was still in flux so hopefully it will change in future versions.

Looking at SCA, one immediately gets a deja vu feeling since SCA resembles a lot an approach used in Spring and other IOC containers. Similar to Spring, an SCA component definition file can contain references to other components and configuration properties. However, SCA lacks some sophistication of Spring, for example, there is no "auto-wiring" of references and so each reference has to be wired explicitly in the component definition file. On the other hand, SCA is supposed to be a cross-language framework (with even some talks of PHP being supported in the future).

On the bright side, SCA is simple and easy to understand. Editing component definition files by hands is a snap and these file formats are pretty intuitive (which is quite an achievement for an SOA-related spec). This is certainly an advantage over similar-in-purpose JBI which comes across as a much heavier framework with lifecycle methods and prescribed container interactions a la EJB 2.0. In SCA you only deal with components and modules and so the small number of key concepts certainly makes it easy to grasp.

Even though SCA is related to service integration and SOA, it does not force WSDL down developers throats; regular Java interfaces are also supported. Unfortunately, Java interfaces look like second-class citizens in WID since the the IDE generates WSDL by default (although I was able to create Java interfaces manually). Also, Java inteface-based components can't be referenced firectly from BPEL-based components (I guess, because references have to match partner links in BPEL and those partner links are WSDL-based). My personal preference would be to use Java intefaces whenever possible since they are easier to deal with and easy to enforce using regular language constructs.

SCA components could be implemented using POJOs. There are no magic methods and a component implementation class does not need to inherit from anything or implement any interface (except for its own Java interface, if it has one). As per the spec, annotations are also supported, although it does not look like this support is in IBM products at this point (they are still on JDK 1.4).

I was hoping that reliance on POJOs will provide for a nice test-driven development experience. However, at this point, I'm still trying to figure out a way for binding and invoking a component from a Junit test outside of a container. I can test a component's implementation directly (I construct SDO objects in my test classes and pass them to implementation classes) but I would like to be able to use proper SCA APIs and look up the component dynamically so I can test against its interface. Testing framework (which generates Junit test classes) that comes with WID for some reason was giving me some problems and, in any case, I prefer to write my test classes by hand.

So to sum up, other than implementation-related glitches, SCA looks like a nice generic component framework.

Is ESB the Starting Point for SOA?

This blog entry discusses Forrester Wave report on ESB market. The report endorses ESB products and suggests that ESB is “the most straightforward way to get started with service-oriented integration today”.

And I’ve always thought that the most straightforward way is to start implementing services instead of infrastructure products (however useful these products might be). As I blogged before, ESB is not a magic wand that will make SOA happen with a flick of a switch. SOA is about implementing services that provide some value to their consumers. Whether these services are mediated by an ESB is completely secondary.

In my opinion, an ESB should come into play later, after certain critical mass of services is designed ( and potentially implemented) and a need for mediations and transformation provided by an ESB becomes more obvious. In certain environments with a relatively homogeneous application set (e.g., an organization which is 100% J2EE shop), ESB may not be required at all, assuming that all applications can speak Web services and canonical data model is well defined.

Even in heterogeneous environments, there are options. For example, ESBs can be used to provide mainframe integration capabilities (conversion to copybook format, etc.), but a different approach could be to use CICS 3.1 which support SOAP/XML natively.

The bottom line is that the need for ESBs must be driven by specific business and technical requirements, not by some kind of perceived goodness of ESB products in general. That’s why I recommend analysing requirements, designing services (and, perhaps, implementing certain set of services) before implementing ESB infrastructure.

The Most Important Characteristic of an ESB

ESB and other SOA middleware can provide many important capabilities but at the end of the day what really counts is whether these products make developers and other stakeholders more productive.

There is nothing “magic” about ESB products. They are all based on well-known application servers and/or messaging products and so they really don’t change the underlying basic characteristics of these platforms. Some ESBs may provide a leg up if advanced Web services specifications, such as WS-Security and WS-ReliableMessaging, must be used; however, their use is still relatively rare even for internal integration, let alone for integrating with external system outside of an enterprise. As far as basic Web services go, ESBs don’t really have any advantage, since any application server and many open-source products can provide equally good Web service run-time platforms.

So the true value-add of ESBs is transformation and mediation capabilities that I described in one of my previous posts. However, there is nothing “magic” about these capabilities either. Any mediation flow or a BPEL flow can be implemented in a regular language, such as Java. There are many open-source and relatively light-weight workflow implementations (some of them even support BPEL) that can be used in place of a full-blown commercial ESB. There are also many adapters and converters for a variety of different applications and formats that can be purchased separately (and many ESB vendors do not package adapters with their products).

So it all boils down to whether an ESB product’s tools help create and modify mediation flows and BPEL flows quicker and easier than competitors. This is why all ESB vendors put great emphasis on visual modeling and development tools. And this is where an evaluation of ESB products must focus on. Ideally, users should be able to measure a productivity gain when creating a mediation flow/process and when making changes, such as adding an activity or re-wiring a flow. This gain could even be measured in absolute time it takes from the start of implementation to when changes are fully deployed in a target environment. Of course, any evaluation should include multiple products, potentially using different approaches (e.g., visual tool versus API-based workflow library).

My suspicion is that ESBs demonstrate productivity gains primarily in a case of simple to medium complexity flows. As I blogged before, BPEL and many proprietary mediation flow languages could be inadequate when dealing with complex flows with a very high number of activities and complex business logic. So an ideal product should also provide an easy-to-use APIs to allow developers to switch from “visual” mode to “textual” development environment where all features of Java or, perhaps, a scripting language, can be used (so that the appropriate reusable components can be created).

SOA and Agile Development

Today SOA is often viewed as a complex, “enterprisey” kind of architecture mostly targeting large enterprise-wide development and integration effort. This view could be justified as SOA technologies and products, could, indeed, be complex.

This, however, does not mean that development of a SOA-based system can’t be agile. In fact, in my view, agile, highly iterative development approach is in the “genes” of SOA. SOA can be a great agility enabler for a software development organization.

Many development efforts involve replacing existing “legacy” applications. In fact, cases of creating applications that provide brand new functionality are relatively rare in enterprise world; usually some (or all) functions provided by the new application already exist in a legacy application or applications. A legacy application’s users may desire new features or they may want to otherwise alter the behavior of the legacy application, hence creating the rationale for its complete or partial re-write.

The truth is, most key business processes and principles of running a business do not change very often. Certain details may change, of course; for example, a loan approval process could be completely automated based on the available credit history and other personal details. This, however, does not change how interest on a loan is accrued – I suspect that the compound interest formula is a few hundred years old (at least). Or take a double-entry bookkeeping which was invented in fifteenth century. There have been some refinements but key ideas have not changed much in accounting.

The need for a complete re-write of a legacy application forces developers to re-invent the wheel, over and over gain. It also kills the agility because of the requirement to re-implement all of the functionality of the legacy application before it can be shut down and replaced by a new one. This makes the initial release cycle a long one; and if it is sufficiently long, the “latest and greatest” technology selected for the new effort will becoming obsolete even before the development is complete thus perpetuating even more changes and churn. This is not to mention the fact that project stakeholders don’t see any return on investment for years to come.

SOA promotes a completely different approach to software development by allowing reuse of existing legacy code. Almost any code can be wrapped in a Web service; this includes COBOL/CICS, COBOL/IMS, PowerBuilder, VB, you name it. This obviates the need for a complete re-write of large chunks of functionality. Instead, new features can be added incrementally, on the top of the existing code. This means that the time required to release new features can be reduced drastically. The first version of a new application could simply be the old code wrapped in services stringed together using some new logic (potentially, with a new UI on the top). Then, new features (expressed as services) can be injected into this architecture as quickly as necessary. With every release the system contains complete functionality including all the functionality of the legacy code. This is much more agile than big bang approach with monolithic builds.

This all sounds fine in theory, reality, however, is much more complicated. The most obvious complication is that the old code could be extremely messy and poorly organized and so developers of a new application could have a natural desire to improve it by potentially re-writing it. This urge should be resisted. If the code does what it’s supposed to do, it should be left alone. All “improvements” should go into the Web service wrapper, and the old code should remain intact. While this approach creates a maintenance issue and potentially promotes bad code, it allows the development team to stay agile. Later on, after the new system has proved itself to its customers, legacy code could be replaced with the new technology. This, however, should be a gradual process with legacy modules switched off over several releases. This approach allows developers to focus on delivering value to their customers instead of being bogged down in endless technology improvements cycles.

Limits of Visual Service Orchestration

Visual tools for designing message or process flows are part of many ESB products. BPM products supporting BPEL or BPMN also heavily rely on visual tools. Flowchart-like visual notation used by these tools represents a specialized language that includes all elements of structural programming ("if-then-else", "while" and so on).

Unfortunately, these languages have very limited support for reuse. Not all such languages support sub-processes (i.e., subroutines). There is no inheritance or extension, in other words, I can't take an existing process flow or message flow and change some of its logic without duplicating the logic of an existing flow. Lack of these facilities makes it difficult to implement complex and reusable process and message flows.

From my experience any real-life process flow is going to be pretty complex. Three-step loan approval process widely used as an example in many different BPEL products does not exist in reality. Real-life business processes include many different activities and involve many different roles. They also have to take into account various exceptions, not just a "happy path" scenario. For example, if a loan application is rejected because of a credit history, a lender can try to offer a loan under different terms.

Also note that there is a difference between using visual tools for modeling and for coding. A model does not have to include each and every detail, its main goal is to convey key ideas; details can be taken care of during implementation. Not so if we want to visually create executable code; in this case all details have to be accounted for.

I am also not convinced that structural programming approach that BPEL and other "flow" languages support is best suited for visual tools. Structural programming constructs replace explicit "flow" represented by "go to". In a visual flow-oriented language, however, "go to" (i.e., explicitly connecting nodes) might be a better alternative.

I also doubt that visual approach works well for code with high cyclomatic complexity (and without subroutines high complexity is almost guaranteed), especially with high degree of nesting. Text-based representation of nested code blocks using indentation could be very compact. This is not the case when each branch has to be presented by its own "flow" of connected nodes.

So I think that text-based representation using a high-level specialized language could be a much better alternative for expressing process and message flows in SOA. This language could be a domain specific language (DSL) or a general purpose dynamically typed language with built-in XML and Web services support and other constructs required for supporting process flow. Syntax of this language should be simple enough and at the same time it needs to have good support for reuse (this may or may not require full OO support).

Visual representation of a program written in such language could still be supported; however I see it as a "report" that could be used to facilitate discussion with business users. Developers should not be required to use visual environment for programming; I don't think it makes anybody more productive relative to text-based approach (unless a flow is really simple). And I'm not talking about XML-based serialization of a visual flow, such as in BPEL where XML is so unwieldy that it is almost impossible to hand-edit it. We need a true powerful "service orchestration" language that will make developers (and business users along with them) more productive.

Web Services and Publishing Schema

Good Web service design starts with a schema. Binding, port type and all these other parameters of a WSDL file usually are not interesting at all – 99.9% of all services have trivial SOAP bindings, no headers and no declared faults. Also, majority of Web services today are document-style with one “part” per message. So schemas of input and output messages are really the key to understanding what service is about. In other words, schema truly defines the contract of a service.

There is no substitution for the schema – you can’t generate a good schema from a source code or a UML class diagram since they do not convey optionality, restrictions, patterns, element model (sequence or choice) and other aspect. XML Schema is verbose and complex (and that’s a topic for another post), but it is pretty expressive as far the data typing goes.

So I usually start a service design with a well-documented schema. The question is, however, how to then publish it so other people can review it (you know, people who represent service consuming applications – these folks are kind of important). Reviewing a schema in a “raw” format is out of question, especially if business people are involved. So a nicely formatted HTML documentation which could present the schema in a readable format (ideally, translating schema lingo into plain English, such as “minOccur=0 ” could translate into “optional”) is the way to go.

I suspect that SOA registries, such as Infravio, can take care of this problem. But oftentimes there is a need to prototype a solution quickly before SOA infrastructure is put into place. Also, at this day and age, open source/free/inexpensive tools might be a requirement for many projects.

The best known Schema documentation generator is xnsdoc. It is not free, but 49 EUR seems like a reasonable price. xnsdoc has an open source XSLT-based counterpart with fewer capabilities. xnsdoc does produce nice documentation along the lines of JavaDoc format (although I’m not convinced that this is the best choice – remember, the documentation has to be suitable for non-developers), however, in my view it does not do much in terms of explaining the schema in plain English. In other words, the assumption is that users are familiar with Schema concepts. I also found that the support for groups and multi-files schemas had some problems (but keep in mind that I’d done my testing almost a year back, so please check the latest version)

A better tool that I found is Bluetetra XSDDoc (99 USD). In my view, it provides more user-friendly schema documentations. You can see some examples here. Unfortunately, it is not clear if XSDDoc is still actively supported – its website provides only minimal information and the product has not been updated in a while.

I still think that there is a need for more interactive Wiki-style schema publishing tool that would allow users reviews schemas expressed in layman terms and comment on elements and types.

What is an ESB?

ESB products are touted by vendors as key infrastructure component of an SOA. ESB product selection is a great challenge because ESB as a product category is still very new. Unlike in JEE application server space, there are no standards that define ESB capabilities. So vendors are free to use ESB moniker for, essentially, any integration middleware that has some Web services support.

I’ve looked at several ESB products trying to understand exactly what these products are about and what the key features that characterize an ESB are. Here is the list of these features:

• Data transformation. Supports bi-directional transformation for XML and non-XML formats, such as CSV, EDI, COBOL copybooks. XML-to-XML transformation is done with the help of XSLT or XQuery; non-XML transformation is implemented using various proprietary mechanisms. Multiple transformations can be “chained” together to implement a complex transformation logic.
• Message routing and service orchestration flows. This facility allows ESB to create a flow that supports transformations, branching (“if” logic), and services invocation. Some vendors use BPEL for routing and orchestration (CapeClear, IBM ProcessServer), some use proprietary mechanisms (BEA, IBM Message Broker) and some use both (Fiorano).
• Support for multiple protocols. Almost all vendors support HTTP, JMS, FTP and email.
• GUI tools (typically, Eclipse-based) for developing data transformation and service orchestration flows. All vendors promote ESBs as a way to improve “agility ” of IT, in other words, allowing IT respond to business requirements quicker. Easy-to-use tools are the key piece of this vision; in theory they should allow for designing flows and transformations in a visual manner with minimal or no programming (I have to admit that I’m somewhat skeptical of this view).

This pretty much describes “core” ESB capabilities. Of course, many (if not all) ESB products go above and beyond this basic feature set and implement many other features, such as service management, security, additional WS-* specs (such as WS-ReliableMessaging), services registry, connectors to enterprise applications (SAP, etc.) and others. But I would argue that all these extra amenities are not really what ESBs are about. They can be supported either with other product categories (such as SOA management tools) or by application servers, especially since many ESB products run on top of existing application servers (e.g., BEA, CapeClear).

So when selecting an ESB for your project, focus on the core capabilities and move non-core stuff down the list. By the way, ESB roundup conducted by Network computing magazine provide a good starting point for ESB selection (although their review is a bit too superficial in my opinion).

REST Support in JAX-WS

I was intrigued by the REST support in JAX-WS. REST style could be very useful in certain situations, not to mention the fact that there are already many REST-style services out there.

While I don’t want to get into SOAP vs. REST debate here I still want to quickly mention one very important REST advantage, which is security.

Since each service call operation in REST is an HTTP GET URL, it makes it easy to use traditional Web authorization schemes, including Apache mod_authnz or J2EE declarative security defined in web.xml. Contrast that with SOAP, which requires looking inside the message to figure out what security policy should be applied. Another side of it is auditing and monitoring. With REST I can easily see all the details of the requests to my web service operations just by looking at the Web server’s access log and/or using one of the numerous log analysis tools. SOAP does not give me this information automatically; different application servers and ESB products may provide these capabilities, but there is no guarantee (since it is not part of any specification).

In any event, REST has its place and so it’s nice to see that JAX-WS authors recognized it.

Unfortunately, upon closer examination, it turned out that REST support in JAX-WS is pretty shallow. REST is really a second class citizen compared to SOAP. Here’s why I came to this conclusion:

• Automatic WSDL generation (when using annotations in service implementation class) is not supported. I was expecting to see the support of HTTP GET binding defined in WSDL 1.1, but it is not there. In general, the spec does not mandate any WSDL to Java or Java to WSDL binding support for REST.
• Object serialization/deserialization is not supported either. This is not unexpected since mapping of a complex object graph to name/value pair GET string may be non-trivial. But I don’t see a problem of doing it for simple flat objects that don’t have nesting.
• Consequently, in order to use REST, one has to rely on dispatcher API on the client and so the query string has to be created manually. Same is happening on the server where developers have to use WebServiceContext and parse the query string manually.

In short, using REST requires some low-level programming; its support is clearly limited compared to SOAP.

What I would like to see is the ability to implement a RESTful service by simply adding @BindingType(value=HTTPBinding.HTTP_BINDING) annotation without having to make any other changes relative to a regular SOAP service (or client). Clearly, we’re not there yet.

Tags: Web services, SOAP, REST

JAX-WS and Annotations: Looking Good

Lately I’ve been playing with JAX-WS reference implementation (version EA3). JAX-WS is the next version of JAX-RPC (i.e., JAX-WS is renamed JAX-RPC 2.0) and it comes with many interesting features, including annotations, HTTP binding, MTOM, asynchronous client operation, direct access to XML stream (finally!) and others.

In this entry I will focus on annotations. Annotations allow developers to configure binding, handler chains, set names of portType, service and other WSDL parameters. This makes it possible to use “Java to WSDL” approach as the primary way for developing Web services (for the server side). Previously with JAX-RPC I was leery of this approach since it was not possible to “fine tune” generated WSDL file.

Annotations greatly simplify Web services development process. For the server side, JAX-WS only generates a wrapper class for the parameters of the service operation. If “bare” (“unwrapped”) mode is used, then no code is generated and so this step can be skipped completely (in other words, all annotations will be handled at run time). This makes it easy to do all development from IDE, for example in my case I deploy to Tomcat and I use Eclipse with Sysdeo plugin, and so I was able to change the code in my Web service class without having to restart the server.

Changing annotations themselves does require bouncing the server, but still there is no code generation involved and so there is no build file to run.

Note that code generation is still required for the client side unless dynamic API is used.

Finally, here is an example of a simple Web service that takes Person object and returns concatenated full name.

import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;

@WebService(
    // Name maps to PortType in WSDL. Default is the name 
    // of the class which is not very intuitive
    name = "PersonService",
    // by default, the namespace is derived from the Java package 
    // name which is not always desirable
    targetNamespace = "http://myarch.com/personservice",
    /* I use the "Endpoint" suffix as it is better convey the 
     * meaning of this parameter in WSDL and also since this 
     * will be the name of the "factory" class generated for the 
     * client side. The default is the name of the class + "Service"
     */
    serviceName="PersonServiceEndpoint"
)

/*        
 * Note that document/literal/wrapped is the default, but using  
 * "bare" as opposed to "wrapped" allows to avoid code generation.          
 */
@SOAPBinding(
    parameterStyle=SOAPBinding.ParameterStyle.BARE)
        
public class PersonServiceImpl {
    
    public java.lang.String produceFullName( Person person) {

        return person.getFirstName()+" "+person.getLastName();
    }
}

Tags: Web services, JAX-WS, JAX-RPC

Client Library as WSDL Alternative

I think that it‘s time for us to start learning from the successes of public Web services, such as Google maps, Ebay and Amazon. Let‘s face it, today enterprise SOA implementations are still in infancy. Most organizations are just testing waters, developing prototypes and implementing pilot projects.

At the same time, Google maps, Ebay and Amazon Web services have been tremendously successful. I know that Google maps example has already been overused by the “Web 2.0” hype, but it is indeed pretty amazing. I subscribe to Google maps mania blog and there are several new “mashup” applications being announced every day. So how do these service providers manage to keep their diverse array of clients happy?

I think that the most important distinction of a successful Web service provider is that it goes above and beyond just publishing WSDL and schema files. These service providers take time to develop detailed samples, documentation and clients libraries. For example, Ebay provides client libraries for Windows, Java, Perl and so on. Google maps is obviously JavaScript. Google search is exposed using WSLD/SOAP but there are also client libraries for Java and .NET that hide complexities of WSDL. Amazon provides entire Associate Engine application written in Perl to interface with its Web services.

This goes against the common view that WSDL /Schema of a service provides complete service definition. This view also suggests that all that a service provider needs to do is to announce its WSDL to the world (or to a community of customers/partners) and the world will immediately adopt the service. This approach shifts the responsibility of implementing the code to access a service (client library or SDK ) to the clients. And for a complex WSDL (or multiple WSDLs), this may mean a lot of effort (especially if we take testing into account). We all know how tricky it is to develop an elegant API . Code generators typically don‘t do a very good job, or they are too inflexible. Just look at code generators supplied with today‘s JAX -RPC implementations that all insist on generating Java objects from schema. Besides, letting the clients develop the code does not allow service provider to implement validations on the client side, as I pointed out in my previous entry.

I think that successful SOA implementations will rely on the clients libraries developed by service providers. These libraries will utilize features of the target language and incorporate some client-side logic. They will also ensure interoperability. These libraries could also take into account capabilities of the client‘s underlying application server or platform. For example, a client library can utilize WS-ReliableMessaging if the client’s application server supports it.

As a result, the role of WSDL will diminish greatly. WSDL has many problems, including its hard-to-comprehend name-based links, verbosity and incompleteness (for example, it does not explain if any of WS-* specs, such as WS-Security, must be used in order to be able to access a service). Client libraries can fix these problems and shield users of a service from having to deal with low-level details of WSDL and WS-* specs.

Tags: WSDL, Web services

Design by Contract for Web Services

Contracts are important for Web services. Design by contract (DBC) is nothing new, but, unfortunately, formal precondition and postcondition specifications have never become mainstream, at least not in the way they were presented by Bertrand Mayer in Object Oriented Software Construction.

However, with the advent of Web services, I think it‘s time to re-introduce DBC to the mainstream. Here‘s why:

• From a client‘s prospective, testing and debugging a Web services call is much costlier than a local method call. For example, service providers may restrict the number of calls per day in test environment, so trial and error may not be an option. So it is important for a client to be able to understand exactly what is required from the clients (precondition) and what is returned by the service (postcondition).
• Web services can be exposed to a very wide range of clients that use different technologies. So it is important for service providers to be able to unambiguously define the contract with the clients to simplify error and exception processing by potentially separating (externalizing) validation logic from the service‘s business logic.

Preconditions and postconditions for Web services must support the following capabilities:

• Validation of input and output parameters. This is partly addressed by the Schema, but that are many types of validations that can‘t be expressed using Schema, such as cross-field or cross-parameter validations (e.g., “endDate > beginDate“). Postconditions should also be able to validate the outputs relative to the inputs, e.g., “resultFlightDate>=requestedDate“.
• Preconditions and postconditions should be able to validate the state of the service. Classical Meyer‘s example is a stack class with “remove“ operation where the precondition states the stack must not be empty. In Web services world, oftentimes there is a need to specify dependencies (so they would be clear to the clients) between calls to different operations, e.g., operation “b“ can only be called after operation “a“ was called. BPEL can do some of it but only on a very limited scale. And BPEL is not really a validation tool.
• Precondition and postcondition checks can run on the server or on the client. In test environment it might be beneficial to run them on the client for faster development.

So what is the solution? I‘m hoping that at some point DBC will find its way into one of the numerous WS specifications. For example, there are some “traces” of DBC in the newly announced WS-Semantics. Although at this point WS-* specifications proliferate at such speed that I doubt that anyone can keep up with them (except, of course, vendors and standard organizations who keep churning them out) so it may do more harm than good.

So the easiest solution for the time being could be for service providers to actually develop client libraries that implement preconditions (and may be even postconditions). This will allow the clients to deal with APIs (and perhaps the source code) expressed in the language that they understand instead of having to deal with bare-bone WSDL which has to be translated into the target language anyway. These libraries can even include “stubbed” versions of the services that can run entirely on the client for testing purposes. Yes, it‘s more work for service providers, but, realistically, how many different languages/platforms are we talking here? In enterprise it probably boils down to various flavors of J2EE and .NET. There are also Python, PHP , Ruby and JavaScript for AJAX clients. For a commercial service provider, the cost of developing these fairly simple client libraries will be returned many times over by the added convenience for the clients. For an example, just look at pygoogle and other Python wrappers for popular Web services. Granted, this approach does not fully address my second requirement for state validation (although it can be done using a smart client library), but I think it‘s a good start nevertheless.

Tags: Web services, Design by contract, Web services design

BPEL, SCA and Refactoring

With the advent of SOA and SOA -related technologies and standards, such as WSDL , BPEL, and, more recently, SCA and SDO, more and more application metadata (and just plain data) is externalized into XML . XML is used for:

• Flow definitions (BPEL).
• Interface definitions (WSDL).
• Component and assembly definitions (SCA).
• Data (SDO).

Business logic, however, largely remains written in Java. For many people, key strength of Java and Java IDEs has been very strong support for refactoring (other languages support refactoring as well, but in this blog I focus mostly on Java). In fact, in my view, refactoring made agile development possible; it finally combined design and coding activities into one continuous process.

Unfortunately, with more and more information going into XML files, benefits of pure Java refactoring are diminishing.

For example, with SOA technologies, an interface name could be used in several different places:

• WSDL file (probably in multiple places if we want to use naming conventions, e.g., in “portType“ and “service“ elements).
• BPEL (multiple places, e.g., in “import“ and “portType“).
• SCA component definition file.

So what happens if I want to change the name of an interface or rename an operation? Personally I am not aware of XML refactoring support in BPEL products (obviously, I have not tested all of them, so I might be wrong here) and I don‘t expect the situation to improve when SCA /SDO are added to the mix. For example, is SDO property renaming going to be supported? How will it work if I use XPath expressions against XML -backed SDO in BPEL ?

Now, dynamic APIs are nothing new, we‘ve been dealing with XML for quite some time now. However, the approach in Java has always been to map XML to statically defined classes and so the ripple effect from, for instance, renaming was somewhat contained (I know that I‘m simplifying here). With BPEL and SCA , the problem becomes more widespread.

Also, tools for working with these technologies are supposed to rely mostly on visual modeling and so their users are not necessarily J2EE developers. In fact, the idea of BPEL is to be able to change business process-related logic quickly and easily. The idea of SCA (among other things) is to be able to easily wire and assemble components together. I think that without refactoring, the ability to accomplish these goals could be impaired.

I just hope that tool vendors realize this risk; otherwise we‘re in for another round of “XML hell”.

To SOAP or Not To SOAP : That is The Question

SOAP gets mentioned every time people talk about Web services. SOAP is almost always considered an integral part of any “serious” Web services implementation, especially if there is also a desire to use WSDL.

But it does not have to be. WSDL 1.1 has HTTP binding support and this support will be beefed up in WSDL 2.0. Granted, the support for WSLD HTTP binding is not very good right now, for example JAX-RPC does not mandate it at all (SOAP binging and SOAP with Attachment are the only two requirements). Assuming the support for HTTP binding will improve or if using WSDL is not even a requirement (after all, a service can be described using “plain English”, as long as developers can understand this description), how should one decide whether to use SOAP?

To me, it all comes down to metadata. The main purpose of SOAP (in case of document-style services) is to provide a place where to put message metadata. This place is SOAP header. WS-Security digital signature, WS-Addressing headers or application-specific metadata all belong there. Today, many (if not most) Web services implementations don’t use SOAP headers and so for these implementations SOAP is just overhead and added complexity (just try get direct access to the InputStream of the message payload in any JAX-RPC-compliant implementaiton). Without anything in the header, SOAP does not have any advantage over good old XML over HTTP “post” (assuming that there is a schema for this XML).

One can argue that SOAP engines also provide XML mapping to the host language primitives, i.e., mapping to/from objects. I personally think that XML mapping capabilities must be de-coupled from SOAP processing. A good XML mapping framework should work with any XML source (in fact, this is exactly where JAX-WS, which is the next version of JAX-RPC is going). A developer must have a choice of receiving “raw” XML from SOAP engine or converting this XML into objects using a framework of his/her choice (e.g., what if I want to use Castor instead of JAXB), I really don’t understand why SOAP engine should make this choice for me.

Just to confirm my point that SOAP is not a requirement for a successful Web service implementation, here is the list of Internet heavyweights and their use of SOAP versus XML. Note that none of them (as far as I know) use WSDL HTTP binding, WSDL is only used when SOAP is used.

	SOAP	XML
Ebay	X	X
Yahoo(maps, search, Flickr, Upcoming, etc.)		X
Google maps		X (JavaScript)
Google search	X
Blogger		X
PayPal	X
del.icio.us		X
MSN Search	X

Of course, SOAP has some other benefits, such as solid tool support. SOAP also provides SOAP encoding (for RPC-style services), although the document style is quickly taking over. But is SOAP absolutely required to implement a “proper” Web service? I don’t think so.

Implementing Document-style Web Service with Sun’s JWSDP

I‘m currently using JWSPD to develop some prototypes and I thought it might be useful to document a few things that are not immediately clear from the documentation (at least, they were not to me).

Why use Sun’s JWSDP (JAX-RPC Standard Implementation)?

• From my experience, it has better WSDL and document-style web services support than Axis. For example, you can read this article or my blog entry about the problems that Axis has with document/literal services (alhough these problems might be fixed in Axis2).
• It‘s a reference implementation which should have better JAX -RPC compliance.

On the downside, JWSDP is more difficult to use than Axis, you have to use “wscomplile“ and “wsdeploy“ tools and it takes some time to set it up.

First of all, you probably want to run JWSDP under the latest version of the Jakarta/ Tomcat. JWSDP documentation says that it requires modified Tomcat 5.0 which you can download from Sun‘s website (or Sun‘s application server), but you don‘t need it. From my experience, it works with jakarta-tomcat-5.5.9 just fine. You just need to pull all JWSDP jars under “WEB-INF/lib”. Here is the list (I may have some extras there, I did not try to figure out which ones are not needed):

FastInfoset.jar
activation.jar
commons-beanutils.jar
commons-collections.jar
commons-digester.jar
commons-logging.jar
dom.jar
jaas.jar
jax-qname.jar
jaxb-api.jar
jaxb-impl.jar
jaxb-libs.jar
jaxb-xjc.jar
jaxp-api.jar
jaxrpc-api.jar
jaxrpc-impl.jar
jaxrpc-spi.jar
jsr173_api.jar
mail.jar
namespace.jar
saaj-api.jar
saaj-impl.jar
sjsxp.jar
xmlsec.jar
xsdlib.jar

You need to read this article first to understand the difference between wrapped and unwrapped mode.

I‘m assuming that you‘ll be generating Java classes from WSDL , not the other way around.

• If you‘re implementing document/literal service with wrapper (JAX-RPC style):
  This is the default in JWSDP . From my experience, JWSDP uses this style even if operation name does not match the outer element name in the schema as prescribed by JAX -RPC.
• If you‘re implementing document/literal non-wrapper (WS-I style):
  You need to use “-f:wsi“ to switch to this mode. Note that in this case you must use model file (“-model“). Model file is used to pass the information to “wsdeploy“ utility. “wsdeploy“ calls “wscompile“ under the covers, but you can‘t directly set “wscompile“ options on “wsdeploy“ command line. Model file is what‘s used to pass these options to “wsdeploy“. Also, don‘t forget to use “-f:wsi“ for the client‘s stub generation (“-gen:client”) option.

Finally, a sample build file is available here.

Tags: Web services, JAX-RPC, JWSDP, Sun

BPEL as a Modeling Tool

BPEL can be used to implement workflow and business processes. In theory, BPEL is not tied up with Web services or SOA since all J2EE BPEL vendors support callouts to EJB , JMS or even POJOs in their BPEL implementations (BPELWS, BPELJ ). BPEL has some interesting capabilities, such as the support for asynchronous and long-running processes, transaction compensation, and integration with UI for human activities. But the main point of BPEL is really its visual nature, the fact that the BPEL flow can be modeled in a visual designer tool. If not for that, it would’ve been more efficient to provide the aforementioned capabilities in the form of yet another Java API . The fact that BPEL is “visual” should make it easier to gather requirements and communicate changes to business stakeholders. Some vendors even go as far as proclaiming that now a business analyst can change and optimize a business process without requiring the involvement of a software developer.

However, for any real-life system some logic would still remain in the Java code. At the very least, the Java code would have some kind of persistence support for data transferred by the BPEL flow. More likely, complex business rules, the ones that would be too unwieldy to implement in BPEL will also be coded in Java. Also, since BPEL does not have the reuse capabilities of an OO language, anything reusable will probably be coded in Java (although, a piece of reusable logic could also be implemented as a reusable BPEL subprocess).

So on one hand, BPEL should be pretty high level and the business process‘ activities (and, therefore services APIs) should be coarse-grained. But this will externalize only the most rudimentary logic of the system. Granted, this high level flow could still be useful if the goal is to integrate different (probably heterogeneous) systems, but if we want to use BPEL for its process modeling capabilities, this may prove insufficient. Devil is always in details and so most likely in real life business process changes or optimization will require changes to these complex business rules implemented in Java. In other words, no matter how we try, business people still need to work together with the developers, make sure that the use case are right, changes are thoroughly tested and so on.

On the other hand, implementing very complex detailed business processes in BPEL is probably difficult at best. Visual modeling approach has its natural limits and so at some point it will become much easier to fire up a Java IDE . I guess it is also possible to hand-edit BPEL XML code, although this probably won‘t be pretty either, using XML as a procedural programming language has its limits too.

So what is the right usage of BPEL ? Is just a Web services orchestration language or a general purpose business modeling environment?