Saturday, April 23, 2005

The future of the ESB market

There's been some interesting discussion of late regarding the value of ESBs.

Richard Turner, a Program Manager at Microsoft, recently posted To ESB or not to ESB, which asks some questions regarding the usefulness of an ESB once Indigo is fully baked, and the WS-* stack becomes fully mature. In particular, Richard asks:
  1. Does anyone out there want to build their Enterprise infrastructure on a technology from a single vendor, supporting a single platform?
  2. Does anyone out there see benefit from building their Enterprise upon a technology which is closed and proprietary?
These questions pertain to the fact that most of the ESB products on the market today are based on Java and proprietary MOM protocols. (Note that IONA Artix is based on C++, it provides native bindings for C++, Java, COBOL, PL/I, and .NET, and it doesn't use a proprietary MOM -- so not all ESBs are Java- and MOM-centric.)

Richard concludes his missive with this question:
  • What value, therefore, is an ESB? To my mind, an ESB is smart-plumbing to which to attach dumb nodes. To my mind, the ESB approach is less about providing customers with value and more about extending attempts to lock customers into a given platform. How, for example, would I add mainframe nodes, COM+ nodes, Web Service nodes etc. running on different platforms and technologies on top of JBI/Sonic/ without the use of adaptors and translators? And more importantly, if I chose to change vendors, how would I replace my ESB with that from someone else?
In response, Dave Chappell, renowned ESB evangelist for Sonic Software , posted ESB and the Road to Indigo. Dave makes a few good points about the value of an ESB:
  • an ESB is based on a philosophy that a SOA should be configured rather than coded
  • an ESB can mediate between different protocols, API conventions, and invocation styles
  • an ESB supports incremental adoption of a SOA
My take:
I think an ESB provides an important value-add today, but I view it as a short term solution, and people should use it with caution. Many ESBs promote the deployment of proprietary protocols, and in general, I think that's something you want to avoid. You don't want to lock yourself into a proprietary solution now that open. non-proprietary options are available. I also agree with Richard that once the core platforms (.NET, WebSphere, WebLogic, CICS, etc.) and application systems (e.g., SAP, Oracle, etc.) have inherent support for secure, reliable, transacted web services, the value of an ESB will be greatly diminished.

The ESB market is heading for a severe squeeze play; expect some significant market consolidation.

The first comment posted to Richard's post asks this question:
  • With WS, WSE and Indigo, aren't there still some missing "pieces", such as Monitoring, Auditing, Exception Management, Measurement, Definition and monitoring of SLAs, etc.? Isn't this the space that an ESB should cover?
I think this is a really interesting question. Today, none of the ESB products provide this type of functionality. Instead, this is the territory of the web services management (WSM) vendors (e.g., Actional, AmberPoint, Blue Titan, Infravio, SOA Software, and most recently Oracle [who acquired Oblix/Confluent]). Now what I find interesting is that some of these players -- particularly Blue Titan and SOA Software -- are starting to deliver ESB capabilities, such as reliable messaging and format/protocol/invocation mediation. And Systinet, Cape Clear, and IONA (traditionally web services platform [WSP] players) are also supplying ESB functionality. The major superplatform players also have plans to deliver ESB functionality (Microsoft "Indigo", IBM "Pyxis", BEA "QuickSilver", Oracle "Fusion", etc.) I just don't see the stand-alone ESB market surviving the combined assault from WSM and the superplatforms..

When it comes right down to it, a web services infrastructure consists of three types of entities:
  • endpoints
  • intermediaries
  • registries
Endpoints are typically implemented using a WSP, although you can also implement an endpoint using an ESB. Essentially, an ESB provides a WSP for legacy applications (via adapters), and an ESB can expose a composite application as a web service (most ESBs provide an orchestration engine). Some ESBs (including Sonic) allow you to create a new POJO service, host it in the ESB container, and expose it as a web service.

Intermediaries intercept web services traffic and enforce policies regarding security, reliability, routing, transformation, mediation, event processing, orchestration, transactions, monitoring, auditing, etc. Intermediaries can run either as proxies along the message path or as interceptors within an endpoint. Both WSM and ESB products provide intermediaries. WSM intermediaries typically support security, routing, transformation, mediation, monitoring, and auditing. ESB intermediaries typically support reliability, routing, transformation, mediation, orchestration, and event processing. (Obviously there's a lot of overlap between WSM and ESB.) You can also build custom intermediaries using a WSP. Most WSPs provide a mechanism to implement interceptors (handlers) that run inside the endpoint. And you can also develop your own proxy.

Web service registries (WSR) maintain information about endpoints and intermediaries. Registries can be used at development time to discover services and metadata; and they can be used at runtime for dynamic location resolution or runtime service selection. Many WSM products also use a registry for provisioning and runtime management -- Amberpoint, Blue Titan, Infravio, and SOA Software all use registries for management.

The future of the ESB market
I expect the pure-play ESB market to go away within the next 3-5 years. Today there are about a dozen vendors promoting their wares as ESB. I just don't see the market supporting this many vendors for very much longer.

Consider what happened to the WSP market. Four years ago, there were about a dozon independent WSP startups. The only survivors today are Systinet and Cape Clear. And, in fact, neither company really markets itself as a WSP player anymore. Systinet is the leading independent registry vendor. Cape Clear is struggling to make it as an ESB vendor now.

Also consider what has been happening in the WSM market. Three years ago, there were about a dozen WSM startups. But since then, CA acquired Adjoin, HP acquired Talking Blocks, Oracle acquired Oblix which acquired Confluent, SOA Software acquired Flamenco, and Actional acquired Westbridge. At this point, the survivors are Actional, Amberpoint, Blue Titan, Infravio, and SOA Software. I anticipate further consolidation.

The ESB market includes Cape Clear, Fiorano, IONA, Polar Lake, SeeBeyond, Sonic, TIBCO, webMethods, and I'm sure a few others. In addition, superplatform players IBM, Microsoft, BEA, and Oracle are heading into this space. And Blue Titan, SOA Software, and Systinet all provide ESB-lite solutions. Definitely this market will experience some serious consolidation. More to the point, I expect that the ESB, WSM, and WSR markets will merge.

In the long run, maybe 4-5 players will survive. My bets are on Sonic, Systinet, TIBCO, and webMethods.


Anne Thomas Manes said...

Barry Briggs responded here

Rich said...

Hey Anne. Great post - commented here:

Bob Evans said...


Great blog. I also listened to a podcast of one of your presentations from that I found insightful and useful.

Rich, some is funny about your comment. I had to view source to get the text.

Anyway, thanks alot for the extremely worthwhile reads and listens.

Matt Gunter said...

I am curious what your latest thoughts are about the relative value of .NET vs. J2EE "superplatforms"

Can they be fairly evaluated due to their radically different degrees of openness, standards support, and ability to leverage the value of open source-style collaboration?

The same trends that are propelling Google seem to be working against Microsoft. Do these same trends not affect the longer-term prospects for "superplatforms"?

Manuel Aldana said...

First of all thnx anne, for you thoughts.

you said, that ESB vendors will lock you in by using propriertary protocols and suggested that it is better to look out for esbs, which support webservices entirely for integration, so it is easier to switch from vendor.

I think changing ESB vendor-products implementing integration-technology by "webservices only" is still problematic:

1)Vendors still will have their "functional" ESB-USPs, otherwise there is no point to buy it (apart from pricing issues...). So by switching vendors you lose "functionality" of your esb and still get to the problem to integrate your IT again.
2)ROI focused management will anyway not agree to another migration even if webservices technology would make it easier.

So i think, either you develop your own ESB, if your integration requirements are simple(!) and your enterprise has got the IT-expertise.

Or you buy a webservices-based ESB, which has add-ons (e.g. a very special orchestration-tool), which suit your integration requirements.

Deepu said...

Hi Anne,
I have a wsdl provided by my client and it looks like "document literal unwrapped" as in the xsd the wsdl is importing doesn't have the operation name as the first element.
now i generated java client and server implementation and deployed using IBM Websphere Application server(also generated java aritifcats using IBM webservice) and client is able to invoke the server. I also checked the soap request and i can see that the reqiest is of type "document literal unwrapped" as the "operation" name is not the first element.
Now using the same webservice client which is running in the IBM websphere application server, i am invoking the server implementation which is now deployed in Axis2.
I see that Axis2 is not able to process the request as it is expecting "opration" name as the first element. So does that mean that Axis2 will never work with "document literal unwrapped" ?