Monthly Archives: May 2006

Limits of Visual Service Orchestration

Posted on: May 29, 2006 | Posted in: BPEL, ESB, SOA

Visual tools for designing message or process flows are part of many ESB
products. BPM products supporting BPEL or BPMN also heavily rely on visual tools.
Flowchart-like visual notation used by these tools represents a specialized
language that includes all elements of structural programming (“if-then-else”, “while” and
so on).

Unfortunately, these languages have very limited support for reuse. Not all such
languages support sub-processes (i.e., subroutines). There is no inheritance or
extension, in other words, I can’t take an existing process flow or message flow
and change some of its logic without duplicating the logic of an existing flow.
Lack of these facilities makes it difficult to implement complex and reusable
process and message flows.

From my experience any real-life process flow is going to be pretty complex.
Three-step loan approval process widely used as an example in many different
BPEL products does not exist in reality. Real-life business processes include
many different activities and involve many different roles. They also have to
take into account various exceptions, not just a “happy path” scenario. For
example, if a loan application is rejected because of a credit history, a lender
can try to offer a loan under different terms.

Also note that there is a difference between using visual tools for modeling and
for coding. A model does not have to include each and every detail, its main
goal is to convey key ideas; details can be taken care of during implementation.
Not so if we want to visually create executable code; in this case all details
have to be accounted for.

I am also not convinced that structural programming approach that BPEL and other
“flow” languages support is best suited for visual tools. Structural
programming constructs replace explicit “flow” represented by “go to”. In a
visual flow-oriented language, however, “go to” (i.e., explicitly connecting
nodes) might be a better alternative.

I also doubt that visual approach works well for code with high cyclomatic
complexity (and without subroutines high complexity is almost guaranteed),
especially with high degree of nesting. Text-based representation of nested code
blocks using indentation could be very compact. This is not the case when each
branch has to be presented by its own “flow” of connected nodes.

So I think that text-based representation using a high-level specialized
language could be a much better alternative for expressing process and message
flows in SOA. This language could be a domain specific language (DSL) or a
general purpose dynamically typed language with built-in XML and Web services
support and other constructs required for supporting process flow. Syntax of
this language should be simple enough and at the same time it needs to have good
support for reuse (this may or may not require full OO support).

Visual representation of a program written in such language could still be
supported; however I see it as a “report” that could be used to facilitate
discussion with business users. Developers should not be required to use visual
environment for programming; I don’t think it makes anybody more productive
relative to text-based approach (unless a flow is really simple). And I’m not
talking about XML-based serialization of a visual flow, such as in BPEL where
XML is so unwieldy that it is almost impossible to hand-edit it. We need a true
powerful “service orchestration” language that will make developers (and
business users along with them) more productive.

Web Services and Publishing Schema

Posted on: May 20, 2006 | Posted in: Web services, XML

Good Web service design starts with a schema. Binding, port type and all these
other parameters of a WSDL file usually are not interesting at all – 99.9% of
all services have trivial SOAP bindings, no headers and no declared faults. Also,
majority of Web services today are document-style with one “part” per message.
So schemas of input and output messages are really the key to understanding what
service is about. In other words, schema truly defines the contract of a service.

There is no substitution for the schema – you can’t generate a good schema from
a source code or a UML class diagram since they do not convey optionality,
restrictions, patterns, element model (sequence or choice) and other aspect. XML
Schema is verbose and complex (and that’s a topic for another post), but it is
pretty expressive as far the data typing goes.

So I usually start a service design with a well-documented schema. The question
is, however, how to then publish it so other people can review it (you know,
people who represent service consuming applications – these folks are kind of
important). Reviewing a schema in a “raw” format is out of question, especially
if business people are involved. So a nicely formatted HTML documentation which
could present the schema in a readable format (ideally, translating schema lingo
into plain English, such as “minOccur=0 ” could translate into “optional”) is
the way to go.

I suspect that SOA registries, such as Infravio, can take care of this problem.
But oftentimes there is a need to prototype a solution quickly before SOA
infrastructure is put into place. Also, at this day and age, open source/free/inexpensive
tools might be a requirement for many projects.

The best known Schema documentation generator is xnsdoc.
It is not free, but 49 EUR seems like a reasonable price. xnsdoc has an open
source XSLT-based counterpart with fewer capabilities. xnsdoc does produce nice
documentation along the lines of JavaDoc format (although I’m not convinced that
this is the best choice – remember, the documentation has to be suitable for non-developers),
however, in my view it does not do much in terms of explaining the schema in
plain English. In other words, the assumption is that users are familiar with
Schema concepts. I also found that the support for groups and multi-files
schemas had some problems (but keep in mind that I’d done my testing almost a
year back, so please check the latest version)

A better tool that I found is Bluetetra
XSDDoc (99 USD). In my view, it provides more user-friendly schema
documentations. You can see some examples here.
Unfortunately, it is not clear if XSDDoc is still actively supported – its
website provides only minimal information and the product has not been updated
in a while.

I still think that there is a need for more interactive Wiki-style schema
publishing tool that would allow users reviews schemas expressed in layman terms
and comment on elements and types.

What is an ESB?

Posted on: May 12, 2006 | Posted in: ESB, SOA

ESB products are touted by vendors as key infrastructure component of an SOA.
ESB product selection is a great challenge because ESB as a product category is
still very new. Unlike in JEE application server space, there are no standards
that define ESB capabilities. So vendors are free to use ESB moniker for,
essentially, any integration middleware that has some Web services support.

I’ve looked at several ESB products trying to understand exactly what these
products are about and what the key features that characterize an ESB are. Here
is the list of these features:

  • Data transformation. Supports bi-directional transformation for XML and non-XML
    formats, such as CSV, EDI, COBOL copybooks. XML-to-XML transformation is done
    with the help of XSLT or XQuery; non-XML transformation is implemented using
    various proprietary mechanisms. Multiple transformations can be “chained”
    together to implement a complex transformation logic.
  • Message routing and service orchestration flows. This facility allows ESB to
    create a flow that supports transformations, branching (“if” logic), and
    services invocation. Some vendors use BPEL for routing and orchestration (CapeClear,
    IBM ProcessServer), some use proprietary mechanisms (BEA, IBM Message Broker)
    and some use both (Fiorano).
  • Support for multiple protocols. Almost all vendors support HTTP, JMS, FTP and
    email.
  • GUI tools (typically, Eclipse-based) for developing data transformation and
    service orchestration flows. All vendors promote ESBs as a way to improve “agility
    ” of IT, in other words, allowing IT respond to business requirements quicker.
    Easy-to-use tools are the key piece of this vision; in theory they should allow
    for designing flows and transformations in a visual manner with minimal or no
    programming (I have to admit that I’m somewhat skeptical of this view).

This pretty much describes “core” ESB capabilities. Of course, many (if not all)
ESB products go above and beyond this basic feature set and implement many other
features, such as service management, security, additional WS-* specs (such as
WS-ReliableMessaging), services registry, connectors to enterprise applications
(SAP, etc.) and others. But I would argue that all these extra amenities are not
really what ESBs are about. They can be supported either with other product
categories (such as SOA management tools) or by application servers, especially
since many ESB products run on top of existing application servers (e.g., BEA,
CapeClear).

So when selecting an ESB for your project, focus on the core capabilities and
move non-core stuff down the list. By the way, ESB
roundup conducted by Network computing magazine provide a good starting
point for ESB selection (although their review is a bit too superficial in my
opinion).