Yahoo Pipes – A Great Way to Create Composite Applications

Yahoo Pipes web site was launched last week and almost immediately drew the attention of a large crowd – I think the site actually went down for a few hours after the launch.

Yahoo Pipes makes it extremely easy to “mash” different Web sources together – without any programming, using drag-and-drop AJAX UI. The UI is actually very slick, it loads fast and provides very intuitive environment. Basically, the UI allows users to create a “message flow” or a pipe using predefined customizable operations, which is a paradigm familiar to any enterprise developer.

I literally took me twenty minutes to put together a simple “pipe” aggregating different SOA-related feeds and allowing user to filter feeds (title or body) using keywords – you can check it out here (you don’t even need a Yahoo account to run it). Learning time was close to zero (perhaps five minutes).

This experience got me thinking. Why is it so easy to create composite applications on the Web (it was easy enough before and with Yahoo pipes it’s just gotten easier) and why is it so hard to do it in an enterprise SOA environment? Why ESB tools don’t have the same level of ease of use and the same degree of “zero administration”? Visual message flow and workflow editors for ESBs are nothing new, but they still come with steep learning curve, tricky configuration requirements and hefty price tags. Of course it’s not fair to compare a simple RSS aggregation/filtering function with typical enterprise tasks (e.g., try to reconcile three different customer XML schemas with different field semantics), but I still think that we have plenty of room for improvement in the enterprise SOA space.