All posts by Alexander Ananiev

The Most Important Characteristic of an ESB

ESB and other SOA middleware can provide many important capabilities but at the
end of the day what really counts is whether these products make developers and
other stakeholders more productive.

There is nothing “magic” about ESB products. They are all based on well-known
application servers and/or messaging products and so they really don’t change
the underlying basic characteristics of these platforms. Some ESBs may provide a
leg up if advanced Web services specifications, such as WS-Security and WS-ReliableMessaging,
must be used; however, their use is still relatively rare even for internal
integration, let alone for integrating with external system outside of an
enterprise. As far as basic Web services go, ESBs don’t really have any
advantage, since any application server and many open-source products can
provide equally good Web service run-time platforms.

So the true value-add of ESBs is transformation and mediation capabilities that
I described in one of my previous posts. However, there is nothing “magic”
about these capabilities either. Any mediation flow or a BPEL flow can be
implemented in a regular language, such as Java. There are many open-source and
relatively light-weight

workflow implementations
(some of them even support
BPEL) that can be used in place of a full-blown commercial ESB.
There are also many adapters and converters for a variety of different applications and formats that can be purchased separately (and many ESB vendors do not package adapters with their products).

So it all boils down to whether an ESB product’s tools help create and modify
mediation flows and BPEL flows quicker and easier than competitors. This is why
all ESB vendors put great emphasis on visual modeling and development tools. And
this is where an evaluation of ESB products must focus on. Ideally, users should
be able to measure a productivity gain when creating a mediation flow/process
and when making changes, such as adding an activity or re-wiring a flow. This
gain could even be measured in absolute time it takes from the start of
implementation to when changes are fully deployed in a target environment. Of
course, any evaluation should include multiple products, potentially using
different approaches (e.g., visual tool versus API-based workflow library).

My suspicion is that ESBs demonstrate productivity gains primarily in a case of
simple to medium complexity flows. As I blogged

before
, BPEL and many proprietary
mediation flow languages could be inadequate when dealing with complex flows
with a very high number of activities and complex business logic. So an ideal
product should also provide an easy-to-use APIs to allow developers to switch
from “visual” mode to “textual” development environment where all features of
Java or, perhaps, a scripting language, can be used (so that the appropriate
reusable components can be created).

SOA and Agile Development

Today SOA is often viewed as a complex, “enterprisey” kind of architecture mostly targeting large enterprise-wide development and integration effort. This view could be justified as SOA technologies and products, could, indeed, be complex.

This, however, does not mean that development of a SOA-based system can’t be agile. In fact, in my view, agile, highly iterative development approach is in the “genes” of SOA. SOA can be a great agility enabler for a software development organization.

Many development efforts involve replacing existing “legacy” applications. In fact, cases of creating applications that provide brand new functionality are relatively rare in enterprise world; usually some (or all) functions provided by the new application already exist in a legacy application or applications. A legacy application’s users may desire new features or they may want to otherwise alter the behavior of the legacy application, hence creating the rationale for its complete or partial re-write.

The truth is, most key business processes and principles of running a business do not change very often. Certain details may change, of course; for example, a loan approval process could be completely automated based on the available credit history and other personal details. This, however, does not change how interest on a loan is accrued – I suspect that the compound interest formula is a few hundred years old (at least). Or take a double-entry bookkeeping which was invented in fifteenth century. There have been some refinements but key ideas have not changed much in accounting.

The need for a complete re-write of a legacy application forces developers to re-invent the wheel, over and over gain. It also kills the agility because of the requirement to re-implement all of the functionality of the legacy application before it can be shut down and replaced by a new one. This makes the initial release cycle a long one; and if it is sufficiently long, the “latest and greatest” technology selected for the new effort will becoming obsolete even before the development is complete thus perpetuating even more changes and churn. This is not to mention the fact that project stakeholders don’t see any return on investment for years to come.

SOA promotes a completely different approach to software development by allowing reuse of existing legacy code. Almost any code can be wrapped in a Web service; this includes COBOL/CICS, COBOL/IMS, PowerBuilder, VB, you name it. This obviates the need for a complete re-write of large chunks of functionality. Instead, new features can be added incrementally, on the top of the existing code. This means that the time required to release new features can be reduced drastically. The first version of a new application could simply be the old code wrapped in services stringed together using some new logic (potentially, with a new UI on the top). Then, new features (expressed as services) can be injected into this architecture as quickly as necessary. With every release the system contains complete functionality including all the functionality of the legacy code. This is much more agile than big bang approach with monolithic builds.

This all sounds fine in theory, reality, however, is much more complicated. The most obvious complication is that the old code could be extremely messy and poorly organized and so developers of a new application could have a natural desire to improve it by potentially re-writing it. This urge should be resisted. If the code does what it’s supposed to do, it should be left alone. All “improvements” should go into the Web service wrapper, and the old code should remain intact. While this approach creates a maintenance issue and potentially promotes bad code, it allows the development team to stay agile. Later on, after the new system has proved itself to its customers, legacy code could be replaced with the new technology. This, however, should be a gradual process with legacy modules switched off over several releases. This approach allows developers to focus on delivering value to their customers instead of being bogged down in endless technology improvements cycles.

Limits of Visual Service Orchestration

Visual tools for designing message or process flows are part of many ESB
products. BPM products supporting BPEL or BPMN also heavily rely on visual tools.
Flowchart-like visual notation used by these tools represents a specialized
language that includes all elements of structural programming (“if-then-else”, “while” and
so on).

Unfortunately, these languages have very limited support for reuse. Not all such
languages support sub-processes (i.e., subroutines). There is no inheritance or
extension, in other words, I can’t take an existing process flow or message flow
and change some of its logic without duplicating the logic of an existing flow.
Lack of these facilities makes it difficult to implement complex and reusable
process and message flows.

From my experience any real-life process flow is going to be pretty complex.
Three-step loan approval process widely used as an example in many different
BPEL products does not exist in reality. Real-life business processes include
many different activities and involve many different roles. They also have to
take into account various exceptions, not just a “happy path” scenario. For
example, if a loan application is rejected because of a credit history, a lender
can try to offer a loan under different terms.

Also note that there is a difference between using visual tools for modeling and
for coding. A model does not have to include each and every detail, its main
goal is to convey key ideas; details can be taken care of during implementation.
Not so if we want to visually create executable code; in this case all details
have to be accounted for.

I am also not convinced that structural programming approach that BPEL and other
“flow” languages support is best suited for visual tools. Structural
programming constructs replace explicit “flow” represented by “go to”. In a
visual flow-oriented language, however, “go to” (i.e., explicitly connecting
nodes) might be a better alternative.

I also doubt that visual approach works well for code with high cyclomatic
complexity (and without subroutines high complexity is almost guaranteed),
especially with high degree of nesting. Text-based representation of nested code
blocks using indentation could be very compact. This is not the case when each
branch has to be presented by its own “flow” of connected nodes.

So I think that text-based representation using a high-level specialized
language could be a much better alternative for expressing process and message
flows in SOA. This language could be a domain specific language (DSL) or a
general purpose dynamically typed language with built-in XML and Web services
support and other constructs required for supporting process flow. Syntax of
this language should be simple enough and at the same time it needs to have good
support for reuse (this may or may not require full OO support).

Visual representation of a program written in such language could still be
supported; however I see it as a “report” that could be used to facilitate
discussion with business users. Developers should not be required to use visual
environment for programming; I don’t think it makes anybody more productive
relative to text-based approach (unless a flow is really simple). And I’m not
talking about XML-based serialization of a visual flow, such as in BPEL where
XML is so unwieldy that it is almost impossible to hand-edit it. We need a true
powerful “service orchestration” language that will make developers (and
business users along with them) more productive.

Web Services and Publishing Schema

Good Web service design starts with a schema. Binding, port type and all these
other parameters of a WSDL file usually are not interesting at all – 99.9% of
all services have trivial SOAP bindings, no headers and no declared faults. Also,
majority of Web services today are document-style with one “part” per message.
So schemas of input and output messages are really the key to understanding what
service is about. In other words, schema truly defines the contract of a service.

There is no substitution for the schema – you can’t generate a good schema from
a source code or a UML class diagram since they do not convey optionality,
restrictions, patterns, element model (sequence or choice) and other aspect. XML
Schema is verbose and complex (and that’s a topic for another post), but it is
pretty expressive as far the data typing goes.

So I usually start a service design with a well-documented schema. The question
is, however, how to then publish it so other people can review it (you know,
people who represent service consuming applications – these folks are kind of
important). Reviewing a schema in a “raw” format is out of question, especially
if business people are involved. So a nicely formatted HTML documentation which
could present the schema in a readable format (ideally, translating schema lingo
into plain English, such as “minOccur=0 ” could translate into “optional”) is
the way to go.

I suspect that SOA registries, such as Infravio, can take care of this problem.
But oftentimes there is a need to prototype a solution quickly before SOA
infrastructure is put into place. Also, at this day and age, open source/free/inexpensive
tools might be a requirement for many projects.

The best known Schema documentation generator is xnsdoc.
It is not free, but 49 EUR seems like a reasonable price. xnsdoc has an open
source XSLT-based counterpart with fewer capabilities. xnsdoc does produce nice
documentation along the lines of JavaDoc format (although I’m not convinced that
this is the best choice – remember, the documentation has to be suitable for non-developers),
however, in my view it does not do much in terms of explaining the schema in
plain English. In other words, the assumption is that users are familiar with
Schema concepts. I also found that the support for groups and multi-files
schemas had some problems (but keep in mind that I’d done my testing almost a
year back, so please check the latest version)

A better tool that I found is Bluetetra
XSDDoc
(99 USD). In my view, it provides more user-friendly schema
documentations. You can see some examples here.
Unfortunately, it is not clear if XSDDoc is still actively supported – its
website provides only minimal information and the product has not been updated
in a while.

I still think that there is a need for more interactive Wiki-style schema
publishing tool that would allow users reviews schemas expressed in layman terms
and comment on elements and types.

What is an ESB?

ESB products are touted by vendors as key infrastructure component of an SOA.
ESB product selection is a great challenge because ESB as a product category is
still very new. Unlike in JEE application server space, there are no standards
that define ESB capabilities. So vendors are free to use ESB moniker for,
essentially, any integration middleware that has some Web services support.

I’ve looked at several ESB products trying to understand exactly what these
products are about and what the key features that characterize an ESB are. Here
is the list of these features:

  • Data transformation. Supports bi-directional transformation for XML and non-XML
    formats, such as CSV, EDI, COBOL copybooks. XML-to-XML transformation is done
    with the help of XSLT or XQuery; non-XML transformation is implemented using
    various proprietary mechanisms. Multiple transformations can be “chained”
    together to implement a complex transformation logic.
  • Message routing and service orchestration flows. This facility allows ESB to
    create a flow that supports transformations, branching (“if” logic), and
    services invocation. Some vendors use BPEL for routing and orchestration (CapeClear,
    IBM ProcessServer), some use proprietary mechanisms (BEA, IBM Message Broker)
    and some use both (Fiorano).
  • Support for multiple protocols. Almost all vendors support HTTP, JMS, FTP and
    email.
  • GUI tools (typically, Eclipse-based) for developing data transformation and
    service orchestration flows. All vendors promote ESBs as a way to improve “agility
    ” of IT, in other words, allowing IT respond to business requirements quicker.
    Easy-to-use tools are the key piece of this vision; in theory they should allow
    for designing flows and transformations in a visual manner with minimal or no
    programming (I have to admit that I’m somewhat skeptical of this view).

This pretty much describes “core” ESB capabilities. Of course, many (if not all)
ESB products go above and beyond this basic feature set and implement many other
features, such as service management, security, additional WS-* specs (such as
WS-ReliableMessaging), services registry, connectors to enterprise applications
(SAP, etc.) and others. But I would argue that all these extra amenities are not
really what ESBs are about. They can be supported either with other product
categories (such as SOA management tools) or by application servers, especially
since many ESB products run on top of existing application servers (e.g., BEA,
CapeClear).

So when selecting an ESB for your project, focus on the core capabilities and
move non-core stuff down the list. By the way, ESB
roundup
conducted by Network computing magazine provide a good starting
point for ESB selection (although their review is a bit too superficial in my
opinion).

REST Support in JAX-WS

I was intrigued by the REST support in JAX-WS. REST style could be very useful in certain situations,
not to mention the fact that there are already many REST-style services out there.

While I don’t want to get into SOAP vs. REST debate here I still want to quickly mention one
very important REST advantage, which is security.

Since each service call operation in REST is an HTTP GET URL, it makes it easy to use
traditional Web authorization schemes, including Apache mod_authnz or J2EE declarative security defined in web.xml.
Contrast that with SOAP, which requires looking inside the message to figure out what security policy should be applied.
Another side of it is auditing and monitoring. With REST I can easily see all the details of the requests to my web service
operations just by looking at the Web server’s access log and/or using one of the numerous log analysis tools.
SOAP does not give me this information automatically; different application servers and ESB products may provide these capabilities, but there is no guarantee (since it is not part of any specification).

In any event, REST has its place and so it’s nice to see that JAX-WS authors recognized it.

Unfortunately, upon closer examination, it turned out that REST support in JAX-WS is pretty shallow.
REST is really a second class citizen compared to SOAP. Here’s why I came to this conclusion:

  • Automatic WSDL generation (when using annotations in service implementation class) is not supported.
    I was expecting to see the support of HTTP GET binding defined in WSDL 1.1, but it is not there.
    In general, the spec does not mandate any WSDL to Java or Java to WSDL binding support for REST.
  • Object serialization/deserialization is not supported either. This is not unexpected since mapping of a complex object graph to name/value pair GET string may be non-trivial. But I don’t see a problem of doing it for simple flat objects that don’t have nesting.

  • Consequently, in order to use REST, one has to rely on dispatcher API on the client and so the query string has to be created manually. Same is happening on the server where developers have to use WebServiceContext and parse the query string manually.

In short, using REST requires some low-level programming; its support is clearly limited compared to SOAP.

What I would like to see is the ability to implement a RESTful service by simply adding
@BindingType(value=HTTPBinding.HTTP_BINDING)
annotation without having to make any other changes relative to a regular SOAP service (or client).
Clearly, we’re not there yet.

Tags: , ,

JAX-WS and Annotations: Looking Good

Lately I’ve been playing with
JAX-WS reference implementation
(version EA3).
JAX-WS is the next version of JAX-RPC (i.e., JAX-WS is renamed JAX-RPC 2.0) and it
comes with many interesting features, including annotations, HTTP binding, MTOM,
asynchronous client operation, direct access to XML stream (finally!) and others.

In this entry I will focus on
annotations
. Annotations allow developers to configure binding,
handler chains, set names of portType, service and other WSDL parameters.
This makes it possible to use “Java to WSDL” approach as the primary way for
developing Web services (for the server side).
Previously with JAX-RPC I was leery of this approach since it was
not possible to “fine tune” generated WSDL file.

Annotations greatly simplify Web services development process. For the server side,
JAX-WS only generates a wrapper class for the parameters of the service operation.
If “bare” (“unwrapped”) mode is used, then no code is generated and so this step can be
skipped completely (in other words, all annotations will be handled at run time).
This makes it easy to do all development from IDE, for example in my case I
deploy to Tomcat and I use Eclipse with Sysdeo plugin,
and so I was able to
change the code in my Web service class without
having to restart the server.

Changing annotations themselves does require bouncing the server, but still there is
no code generation involved and so there is no build file to run.

Note that code generation is still required for the client side unless dynamic API is used.

Finally, here is an example of a simple Web service that takes Person object
and returns concatenated full name.


import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;

@WebService(
    // Name maps to PortType in WSDL. Default is the name 
    // of the class which is not very intuitive
    name = "PersonService",
    // by default, the namespace is derived from the Java package 
    // name which is not always desirable
    targetNamespace = "http://myarch.com/personservice",
    /* I use the "Endpoint" suffix as it is better convey the 
     * meaning of this parameter in WSDL and also since this 
     * will be the name of the "factory" class generated for the 
     * client side. The default is the name of the class + "Service"
     */
    serviceName="PersonServiceEndpoint"
)

/*        
 * Note that document/literal/wrapped is the default, but using  
 * "bare" as opposed to "wrapped" allows to avoid code generation.          
 */
@SOAPBinding(
    parameterStyle=SOAPBinding.ParameterStyle.BARE)
        
public class PersonServiceImpl {
    
    public java.lang.String produceFullName( Person person) {

        return person.getFirstName()+" "+person.getLastName();
    }
}

Tags: , ,

Client Library as WSDL Alternative

I think that it‘s time for us to start learning from the successes of public Web
services, such as Google maps, Ebay and Amazon. Let‘s face it, today enterprise
SOA
implementations are still in infancy. Most organizations are just testing waters,
developing prototypes and implementing pilot projects.

At the same time, Google maps, Ebay and Amazon Web services have been
tremendously successful. I know that Google maps example has already been
overused by the “Web 2.0” hype, but it is indeed pretty amazing. I subscribe to Google
maps mania blog
and there are several new “mashup” applications being
announced every day. So how do these service providers manage to keep their
diverse array of clients happy?

I think that the most important distinction of a successful Web service provider
is that it goes above and beyond just publishing
WSDL
and schema files. These service providers take time to develop detailed samples,
documentation and clients libraries. For example, Ebay provides client
libraries
for Windows, Java, Perl and so on. Google maps is obviously
JavaScript. Google search is exposed using WSLD/SOAP but there are also client
libraries
for Java and .NET that hide complexities of WSDL. Amazon provides
entire Associate
Engine
application written in Perl to interface with its Web services.

This goes against the common view that
WSDL
/Schema of a service provides complete service definition. This view also
suggests that all that a service provider needs to do is to announce its
WSDL
to the world (or to a community of customers/partners) and the world will
immediately adopt the service. This approach shifts the responsibility of
implementing the code to access a service (client library or
SDK
) to the clients. And for a complex
WSDL
(or multiple
WSDLs), this may mean a lot of effort (especially if we take testing into account).
We all know how tricky it is to develop an elegant
API
. Code generators typically don‘t do a very good job, or they are too inflexible.
Just look at code generators supplied with today‘s
JAX
-RPC implementations that all insist on generating Java objects from schema.
Besides, letting the clients develop the code does not allow service provider to
implement validations on the client side, as I pointed out in my previous
entry
.

I think that successful
SOA
implementations will rely on the clients libraries developed by service
providers. These libraries will utilize features of the target language and
incorporate some client-side logic. They will also ensure interoperability.
These libraries could also take into account capabilities of the client‘s
underlying application server or platform. For example, a client library can
utilize WS-ReliableMessaging if the client’s application server supports it.

As a result, the role of
WSDL
will diminish greatly.
WSDL
has many problems, including its hard-to-comprehend name-based links, verbosity
and incompleteness (for example, it does not explain if any of WS-* specs, such
as WS-Security, must be used in order to be able to access a service). Client
libraries can fix these problems and shield users of a service from having to
deal with low-level details of
WSDL
and WS-* specs.

Tags: ,

Design by Contract for Web Services

Contracts are important for Web services. Design by contract (DBC) is nothing
new, but, unfortunately, formal precondition and postcondition specifications
have never become mainstream, at least not in the way they were presented by
Bertrand Mayer in Object
Oriented Software Construction
.

However, with the advent of Web services, I think it‘s time to re-introduce
DBC
to the mainstream. Here‘s why:

  • From a client‘s prospective, testing and debugging a Web services call is much
    costlier than a local method call. For example, service providers may restrict
    the number of calls per day in test environment, so trial and error may not be
    an option. So it is important for a client to be able to understand exactly
    what is required from the clients (precondition) and what is returned by the
    service (postcondition).
  • Web services can be exposed to a very wide range of clients that use different
    technologies. So it is important for service providers to be able to
    unambiguously define the contract with the clients to simplify error and
    exception processing by potentially separating (externalizing) validation logic
    from the service‘s business logic.

Preconditions and postconditions for Web services must support the following
capabilities:

  • Validation of input and output parameters. This is partly addressed by the
    Schema, but that are many types of validations that can‘t be expressed using
    Schema, such as cross-field or cross-parameter validations (e.g., “endDate >
    beginDate“). Postconditions should also be able to validate the outputs
    relative to the inputs, e.g., “resultFlightDate>=requestedDate“.
  • Preconditions and postconditions should be able to validate the state of the
    service. Classical Meyer‘s example is a stack class with “remove“ operation
    where the precondition states the stack must not be empty. In Web services world,
    oftentimes there is a need to specify dependencies (so they would be clear to
    the clients) between calls to different operations, e.g., operation “b“ can only
    be called after operation “a“ was called.
    BPEL
    can do some of it but only on a very limited scale. And
    BPEL
    is not really a validation tool.
  • Precondition and postcondition checks can run on the server or on the client. In
    test environment it might be beneficial to run them on the client for faster
    development.

So what is the solution? I‘m hoping that at some point
DBC
will find its way into one of the numerous WS specifications. For example, there
are some “traces” of
DBC
in the newly announced WS-Semantics.
Although at this point WS-* specifications proliferate at such speed that I
doubt that anyone can keep up with them (except, of course, vendors and standard
organizations who keep churning them out) so it may do more harm than good.

So the easiest solution for the time being could be for service providers to
actually develop client libraries that implement preconditions (and may be even
postconditions). This will allow the clients to deal with APIs (and perhaps the
source code) expressed in the language that they understand instead of having to
deal with bare-bone
WSDL
which has to be translated into the target language anyway. These libraries can
even include “stubbed” versions of the services that can run entirely on the
client for testing purposes. Yes, it‘s more work for service providers, but,
realistically, how many different languages/platforms are we talking here? In
enterprise it probably boils down to various flavors of
J2EE
and .NET. There are also Python,
PHP
, Ruby and JavaScript for
AJAX
clients. For a commercial service provider, the cost of developing these fairly
simple client libraries will be returned many times over by the added
convenience for the clients. For an example, just look at pygoogle
and other Python wrappers for popular Web services. Granted, this approach does
not fully address my second requirement for state validation (although it can be
done using a smart client library), but I think it‘s a good start nevertheless.

Tags: , ,

BPEL, SCA and Refactoring

With the advent of SOA and SOA -related technologies and standards, such as WSDL ,
BPEL, and, more recently, SCA
and SDO
, more and more application metadata (and just plain data) is
externalized into XML . XML is used for:

  • Flow definitions (BPEL).
  • Interface definitions (WSDL).
  • Component and assembly definitions (SCA).
  • Data (SDO).

Business logic, however, largely remains written in Java. For many people, key
strength of Java and Java IDEs has been very strong support for refactoring (other
languages support refactoring as well, but in this blog I focus mostly on Java).
In fact, in my view, refactoring made agile development possible; it finally
combined design and coding activities into one continuous process.

Unfortunately, with more and more information going into XML files, benefits of
pure Java refactoring are diminishing.

For example, with SOA technologies, an interface name could be used in several
different places:

  • WSDL file (probably in multiple places if we want to use naming conventions, e.g.,
    in “portType“ and “service“ elements).
  • BPEL (multiple places, e.g., in “import“ and “portType“).
  • SCA component definition file.

So what happens if I want to change the name of an interface or rename an
operation? Personally I am not aware of XML refactoring support in BPEL products
(obviously, I have not tested all of them, so I might be wrong here) and I don‘t
expect the situation to improve when SCA /SDO are added to the mix. For example,
is SDO property renaming going to be supported? How will it work if I use XPath
expressions against XML -backed SDO in BPEL ?

Now, dynamic APIs are nothing new, we‘ve been dealing with XML for quite some
time now. However, the approach in Java has always been to map XML to statically
defined classes and so the ripple effect from, for instance, renaming was
somewhat contained (I know that I‘m simplifying here). With BPEL and SCA , the
problem becomes more widespread.

Also, tools for working with these technologies are supposed to rely mostly on
visual modeling and so their users are not necessarily J2EE developers. In fact,
the idea of BPEL is to be able to change business process-related logic quickly
and easily. The idea of SCA (among other things) is to be able to easily wire
and assemble components together. I think that without refactoring, the ability
to accomplish these goals could be impaired.

I just hope that tool vendors realize this risk; otherwise we‘re in for another
round of “XML hell”.

To SOAP or Not To SOAP : That is The Question

SOAP gets mentioned every time people talk about Web services. SOAP is almost always considered an integral part of any “serious” Web services implementation, especially if there is also a desire to use WSDL.

But it does not have to be. WSDL 1.1 has HTTP binding support and this support will be beefed up in
WSDL 2.0
. Granted, the support for WSLD HTTP binding is not very good right now, for example JAX-RPC does not mandate it at all (SOAP binging and SOAP with Attachment are the only two requirements). Assuming the support for HTTP binding will improve or if using WSDL is not even a requirement (after all, a service can be described using “plain English”, as long as developers can understand this description), how should one decide whether to use SOAP?

To me, it all comes down to metadata. The main purpose of SOAP (in case of document-style services) is to provide a place where to put message metadata. This place is SOAP header. WS-Security digital signature, WS-Addressing headers or application-specific metadata all belong there. Today, many (if not most) Web services implementations don’t use SOAP headers and so for these implementations SOAP is just overhead and added complexity (just try get direct access to the InputStream of the message payload in any JAX-RPC-compliant
implementaiton). Without anything in the header, SOAP does not have any advantage over good old XML over HTTP “post” (assuming that there is a schema for this XML).

One can argue that SOAP engines also provide XML mapping to the host language primitives, i.e., mapping to/from objects. I personally think that XML mapping capabilities must be de-coupled from SOAP processing. A good XML mapping framework should work with any XML source (in fact, this is exactly where JAX-WS, which is the next version of JAX-RPC is going). A developer must have a choice of receiving “raw” XML from SOAP engine or converting this XML into objects using a framework of his/her choice (e.g., what if I want to use Castor instead of JAXB), I really don’t understand why SOAP engine should make this choice for me.

Just to confirm my point that SOAP is not a requirement for a successful Web service implementation, here is the list of Internet heavyweights and their use of SOAP versus XML. Note that none of them (as far as I know) use WSDL HTTP binding, WSDL is only used when SOAP is used.

  SOAP XML
Ebay X X
Yahoo(maps, search, Flickr, Upcoming, etc.)   X
Google maps   X (JavaScript)
Google search X  
Blogger   X
PayPal X  
del.icio.us   X
MSN Search X  

Of course, SOAP has some other benefits, such as solid tool support. SOAP also provides SOAP encoding (for RPC-style services), although the document style is quickly taking over. But is SOAP absolutely required to implement a “proper” Web service? I don’t think so.

Implementing Document-style Web Service with Sun’s JWSDP

I‘m currently using JWSPD to develop some prototypes and I thought it might be
useful to document a few things that are not immediately clear from the
documentation (at least, they were not to me).

Why use Sun’s
JWSDP
(JAX-RPC Standard Implementation)?

  • From my experience, it has better WSDL and document-style web services support
    than Axis. For example, you can read this
    article
    or my
    blog entry
    about the problems that Axis has with document/literal services (alhough
    these problems might be fixed in Axis2).
  • It‘s a reference implementation which should have better JAX -RPC compliance.

On the downside, JWSDP is more difficult to use than Axis, you have to use “wscomplile“
and “wsdeploy“ tools and it takes some time to set it up.

First of all, you probably want to run JWSDP under the latest version of the
Jakarta/ Tomcat. JWSDP documentation says that it requires modified Tomcat 5.0
which you can download from Sun‘s website (or Sun‘s application server), but you
don‘t need it. From my experience, it works with jakarta-tomcat-5.5.9 just fine.
You just need to pull all JWSDP jars under “WEB-INF/lib”. Here is the list (I
may have some extras there, I did not try to figure out which ones are not
needed):


FastInfoset.jar
activation.jar
commons-beanutils.jar
commons-collections.jar
commons-digester.jar
commons-logging.jar
dom.jar
jaas.jar
jax-qname.jar
jaxb-api.jar
jaxb-impl.jar
jaxb-libs.jar
jaxb-xjc.jar
jaxp-api.jar
jaxrpc-api.jar
jaxrpc-impl.jar
jaxrpc-spi.jar
jsr173_api.jar
mail.jar
namespace.jar
saaj-api.jar
saaj-impl.jar
sjsxp.jar
xmlsec.jar
xsdlib.jar

You need to read this
article
first to understand the difference between wrapped and unwrapped
mode.

I‘m assuming that you‘ll be generating Java classes from WSDL , not the other
way around.

  • If you‘re implementing document/literal service with wrapper (JAX-RPC style):
    This is the default in JWSDP . From my experience, JWSDP uses this style even if
    operation name does not match the outer element name in the schema as prescribed
    by JAX -RPC.
  • If you‘re implementing document/literal non-wrapper (WS-I style):

    You need to use “-f:wsi“ to switch to this mode. Note that in this case you must
    use model file (“-model“). Model file is used to pass the information to “wsdeploy“
    utility. “wsdeploy“ calls “wscompile“ under the covers, but you can‘t directly
    set “wscompile“ options on “wsdeploy“ command line. Model file is what‘s used to
    pass these options to “wsdeploy“. Also, don‘t forget to use “-f:wsi“ for the
    client‘s stub generation (“-gen:client”) option.

Finally, a sample build file is available here.

Tags: , , ,

What’s Missing from XML Schema

Over the last several weeks I‘ve been working on developing XML schemas for a
client to support information exchanges between several different organizations,
so it was important to make the schemas very explicit and “tight“ so that each
party can validate XML before or after sending it. The XML documents could be
used in conjunction with Web services or as part of “old-fashion“ file based
exchange. In short, this was pretty typical system integration task.

The client had already decided to standardize on XML Schema, so using Relax NG
or Schematron was not an option.

XML Schema provides a lot of different capabilities but based on my experience I
think that it could benefit from some improvements. Here are my random thoughts
on this. Now, I don‘t claim to be the ultimate XML Schema expert, so take it for
what it‘s worth.

  • Schema‘s verbosity and unwieldy syntax makes it a poor candidate for
    communicating and publishing XML structure and rules to the wide audience of
    technical people from different organizations that may or may not know XML
    Schema. For example, “minOccur=0“ means “optional field” which is probably not
    very intuitive to anyone unfamiliar with the Schema specification. Even after
    formatting the schema for publishing (e.g., by using xsddoc)
    schemas are still hard to understand. Of course, one can use the annotations and
    try to explain each type in plain English, but then the documentation always
    tends to get out of synch.

    The obvious counter-argument here is that XML Schema is designed to be the data
    modeling/validation tool and as such it is not suitable for capturing business
    requirements but I just think that it would be nice if it could really be used
    for both, essentially becoming the “system of records“ for integrating different
    systems and organization.

  • Error messages thrown by XML parsers are far from being the most intuitive (this
    obviously depends on the parser and I have not done any comparative analysis).
    For example, missing required element results in “Element ‘element name‘ is not
    valid for content model“ where ‘element name‘ is the name of the element
    following the missing required missing element. Why can‘t the parser simply say
    “Required element is missing“? Again, this problem is exacerbated when you‘re
    dealing with people with only cursory XML Schema knowledge. I‘m not aware of a
    standard way to customize error messages, so in my case developers will have to
    do error translation in the code.
  • XML Schema users are forced to use regular expressions for defining any more or
    less complex template for simple types (phone number, SSN , etc). This poses a
    problem in an environment where you can‘t expect all users to be familiar with
    regexp syntax. When you get a message “Value does not match regular expression
    facet ‘\+?[0–9\-\(\)\s]{1,25}“, it could very easily befuddle an uninitiated. I
    wish there was a simplified templating mechanism, may be something similar to
    java.text.MessageFormat “##.##“.
  • Reuse capabilities in XML Schema are not perfect. “extend“ only allows to append
    an element to the end of the sequence. “Restrict“ requires repeating the content
    model of the parent type. This creates very brittle schemas and violates DRY
    principle. There is no way to parameterize an XML type. Let‘s say there is “name“
    type with “first“ and “last“ elements. When a new person is added, I want “last“
    element to be mandatory. In “update“ situation all fields could be optional. I
    wish I could make “minOccur“ a parameter here.
  • XML Schema may seem very OO-like at the first glance, but in fact it is missing
    some important OO-like capabilities. For instance, there is no element-level
    polymorphism. In the example above, I wanted to change the “behavior“ of “last”
    (some aspect of this type definition) in a subtype and I can‘t do that.
    Inheritance by restriction for complex types (I don‘t have a problem with using
    it for simple types) is IMO counter-intuitive. So now I can have a child which
    does not have all properties of its parent, and so there is no way to enforce
    optional elements for all children.
  • Element and type scoping could‘ve been more flexible. All declarations are
    either parent element-scoped (local) or global. This does not allow me to define
    a reusable type or a group scoped to certain parent element or to a file (any
    more or less complex schema would have to broken down into multiple files for
    manageability sake). So say I have a name type for person‘s name (first, middle,
    last) and business‘ name type with a different structure. If I want use to
    person‘s name type for different elements within Person type, I will have to
    define as global and name it PersonNameType, essentially embedding parent‘s name
    into the child‘s name. I wish I could simply define NameType and specify what
    parent type or element it is scoped to.
  • XML Schema is a declarative language and so it lacks programming constructs,
    which is fine. But there is still a need for Schematron-like facility or the
    scripting language for expressing complex rules (such as cross-field validation).
    Schematron works fine when embedded inside annotations, but it requires a
    separate validation step and Schematron XSLT . So it would be great if this
    capability was supported by the Schema standard and natively understood by XML
    parsers. This would make schemas truly self-contained.

So my wish list is actually quite simple:

  • Make XML schemas easier to understand for non-technical users or people without
    schema knowledge perhaps via some intelligent translation mechanism.
  • Make the Schema more powerful by allowing programming constructs, variables and
    more flexible scoping.

Tags: ,

BPEL as a Modeling Tool

BPEL can be used to implement workflow and business processes. In theory, BPEL
is not tied up with Web services or SOA since all J2EE BPEL vendors support
callouts to EJB , JMS or even POJOs in their BPEL implementations (BPELWS, BPELJ
). BPEL has some interesting capabilities, such as the support for asynchronous
and long-running processes, transaction compensation, and integration with UI
for human activities. But the main point of BPEL is really its visual nature,
the fact that the BPEL flow can be modeled in a visual designer tool. If not for
that, it would’ve been more efficient to provide the aforementioned capabilities
in the form of yet another Java API . The fact that BPEL is “visual” should make
it easier to gather requirements and communicate changes to business
stakeholders. Some vendors even go as far as proclaiming that now a business
analyst can change and optimize a business process without requiring the
involvement of a software developer.

However, for any real-life system some logic would still remain in the Java code.
At the very least, the Java code would have some kind of persistence support for
data transferred by the BPEL flow. More likely, complex business rules, the ones
that would be too unwieldy to implement in BPEL will also be coded in Java. Also,
since BPEL does not have the reuse capabilities of an OO language, anything
reusable will probably be coded in Java (although, a piece of reusable logic
could also be implemented as a reusable BPEL subprocess).

So on one hand, BPEL should be pretty high level and the business process‘
activities (and, therefore services APIs) should be coarse-grained. But this
will externalize only the most rudimentary logic of the system. Granted, this
high level flow could still be useful if the goal is to integrate different (probably
heterogeneous) systems, but if we want to use BPEL for its process modeling
capabilities, this may prove insufficient. Devil is always in details and so
most likely in real life business process changes or optimization will require
changes to these complex business rules implemented in Java. In other words, no
matter how we try, business people still need to work together with the
developers, make sure that the use case are right, changes are thoroughly
tested and so on.

On the other hand, implementing very complex detailed business processes in BPEL
is probably difficult at best. Visual modeling approach has its natural limits
and so at some point it will become much easier to fire up a Java IDE . I guess
it is also possible to hand-edit BPEL XML code, although this probably won‘t be
pretty either, using XML as a procedural programming language has its limits too.

So what is the right usage of BPEL ? Is just a Web services orchestration
language or a general purpose business modeling environment?

How Loosely Coupled is Your SOA?

SOA is the all about loosely coupled services, right? The Web service behind
WSDL is an opaque black box, i.e., it might be implemented by an EJB today and
by some COTS package tomorrow. XML must to be used for passing data between
layers; we can‘t assume that service provider and service consumer understand
the same binary protocol.

What does it mean to a typical J2EE application? For the sake of the discussion,
let‘s assume that our application has rich domain object model (say, POJOs with
Hibernate persistence), EJB session fasade classes and Struts/JSP on the front-end.
Domain classes are directly used in the UI tier; there are no DTOs.

Now, some business functions of our application turned out to be pretty useful
so we now want to build Web services to allow the entire organization (or may be
even business partners) to benefit from these functions.

“Pure“ SOA would require us to change the UI so that it would rely on Web
services API and XML instead of POJOs to communicate with the back-end.
Potentially, we‘d also have some kind of ESB /messaging solution in the middle.
With that, the service becomes fully de-coupled from the UI and we‘re well on
our way to SOA nirvana. Now the implementation of the Web service can be changed
without affecting anything on the client‘s side.

But if you look at this approach closely, it immediately raises some nagging
questions:

  • Both UI and back-end still depend on the same object model. So whenever object
    model classes change, we want to re-deploy both the service and the UI component.
  • If data model changes, object model, UI and may be APIs will be affected anyway.
  • We loose the ability to call the service from other EJBs (still co-deployed
    with the UI) as part of the same transaction unless some kind of proprietary
    SOAP binding is implemented (and then we need two different WSDL files, one for
    external consumers, one for internal).
  • There are also some “minor“ issues, like concurrency. Let‘s say each updatable
    POJO has a version number and the UI keeps the object in session for each form.
    This is not exactly the best model for Web services that are supposed to be
    stateless; perhaps we should not assume that all service consumers will be able
    to comply with that.

The list can go on, but I‘ll stop here. Doesn‘t look like we‘ve accomplished
much by “SOA-izing” our application, does it? Perhaps, what we should‘ve done is
to leave the application alone and build the service just for the external
consumers. Oh, and it would be nice to know what their requirements are before
we do that, may be we should not assume that one size will fit all of them.