Improve Your Application Performance with XML Appliance

XML appliances are capable of extremely fast XML parsing and transformation (sometimes the term “wire-speed” is used). The speed is achieved by using hardware acceleration, specially written XML parsers and XSLT engines (you won’t find Xerces on these devices) and optimized operating system (usually, a trimmed-down version of Linux or BSD).

How fast are XML appliances? I’m not aware of published benchmarks; however, I did have a chance to conduct an informal performance testing of DataPower XI50 for a client and the results were quite impressive; for example we saw little to no overhead validating medium-size XML schemas.

Clearly, offloading as much XML processing as possible to an appliance could be a performance booster. So what kind of processing could be done on the device? Here are my recommendations based on the capabilities of the DataPower appliance; the situation could be slightly different with devices from other vendors.

• Validation. Turning schema validation on is a no-brainer, especially for complex schemas that can choke software parsers. In addition to schema validation, XSLT can be used to validate rules that are not grammar-based. You can use Schematron to generate XSLT that validates these rules. Combination of Schema and Schematron provides very powerful validation mechanism. The beauty of this approach is that failed XML documents never reach the application thus reducing resource consumption of the application server.
• Splitting. It is possible to define XPath-rules that in combination with XSLT will split complex documents into multiple fragments that can then be sent to different back-end services for processing.
• Filtering. Sometimes an application only needs a subset of elements provided in a message. Eliminating unnecessary tags will reduce the application’s XML processing overhead.
• Flattening. Complex XML documents with deep nesting can be “flattened” using XSLT to eliminate some of the nesting. This could make application XML parsing quicker.
• “Canonicanization”. In many cases, XML schema may allow for different representation of the same piece of data to accommodate needs of different clients. It could be allowing for SSN with and without dashes or supporting different name formats (e.g., “firstName, lastName” vs. “fullName”). XSLT logic on the device could implement the rules to clean up incoming messages and transform them into one strict canonical representation so that the application only needs to implement one way of parsing data.

“Shredding”” could also be very effective for high-throughput XML processing. Shredding is the process of breaking up large documents into smaller fragments and sending them off to the application for concurrent processing. Shredding helps dealing with the documents consisting of a large number of repetitive groups. Each group/fragment can be processed independently without waiting for the completion of the processing of the entire document. Note that shredding is different from “splitting” which allows for processing of different logical parts (different types) of the document independently (as opposed to repetitive groups). Shredding is the most useful for processing large bathes of data of the same type.

Unfortunately, shredding does not seem to be supported by DataPower, although the appliance does support streaming XML processing which would’ve worked very nicely in conjunction with shredding logic. Perhaps devices from other vendors do provide this support.

The bottom line is that just plugging the appliance in and using it in a gateway mode is not necessarily going to improve XML processing performance by much. Web services (and any other application components that interface via XML such as XML-based batch file processing components) have to be designed with the appliance in mind. In the course of the design effort, developers need to identify pre-processing or post-processing logic that can be implemented in XSLT as opposed to making it all application’s responsibility. Anything that uses XSLT can then go to the appliance to improve the processing speed and throughput.