Categories

Archives

ETL vs. ESB

A few months ago, prior to the release of the Apatar Community Preview, Matt Asay asked me via email whether Apatar competes with ESB products, specifically with MuleSource and ServiceMix. After I responded to Matt by email, I thought about posting my response to a blog, and Matt said, “Go ahead.” In fact, two people asked me a similar question during the MySQL User Conference, which reminded me about the email that I’m posting below.

The short answer is that Apatar is an ETL (Extract, Transform, and Load) technology; ETL does not compete, but compliments ESB products across different information integration scenarios. ServiceMix and Mule are ESB products. ESB is a standards-oriented (in the SOA age) EAI/EII technology. Therefore, I will address the comparison question, “ETL vs. ESB” from an ETL vs. EAI/EII point of view. EAI, by the way, is a term coined by Dave Linthicum in his book (published in 1999) called, “Enterprise Application Integration.”

ETL is geared toward data movement, typically in batch modes across the enterprise. It is “pull” technology and works on user’s demand or on schedule.

ESB is a “push” technology, sending messages when they occur.

ETL is a “pull” technology, works on demand/on schedule.
ESB is a “push” technology.
ETL cannot time-out, decay, or issue transactions to front-office applications during transformation processes.
ESB is capable of timing and decaying data in queues, escalating information content to the right decision-maker on that piece of content.
ETL is fully scalable, capable of loading massive batches of data in parallel.
ESB is not suitable for massive volumes of data because of its service bus architecture (by network, and source system speed to X transactions per second).
ETL can hook to ESB/EAI middleware as just another feed, if desired.
ESB’s primary job is to integrate applications, opposed to Data Migration, Replication, Data Warehousing, and BI.
I can also refer you to a comparison table I found at http://www.coreintegration.com/solutions/di.asp:

Feature
ETL
EAI
EII
Timing
Batch snapshots
Real time
Real time
Unit of work
Set of transactions committed within an ETL cycle interval
Single business transaction
Single business transaction
Historical Record
Yes
No
No
Persistent Auxiliary Tables
Yes
No. Transactions applied directly to applications’ tables
No. Virtual database
Application
Managerial reporting, trend analysis, multi-dimensional aggregation
Near real time synchronization of operational data where transaction commitment is dependent on state of related transactions
Near real time decision making based on most current information in operational systems. No update.
What it’s not
Not source of record. Does not support transaction processing
Not appropriate for ad-hoc analysis and reporting
Not a virtual data warehouse
At the end of the day, I think that business users have to consider their unique requirements, and pick the technology accordingly. There are different horses for different courses. Be it Apatar, Mule ESB, DataStage, Yahoo! Pipes–whatever works.
In my opinion, as more vendors enter the information integration space, they are confusing things even more because of the differences in technology they are offering. No two are alike, but all are calling themselves data integration vendors. Go figure.

Free vs. Cheap: It’s NOT About Shrinking a Market

Josh Kopelman of First Round Capital, an early stage investor in many Web 2.0 emerging leaders (including Wikia, Mashery, and Odeo), posted an excellent article on the emerging business models across many industries:

“Here at First Round Capital, we see a lot of business plans for consumer-facing internet services. Most assume a significant portion of their revenue comes through advertising — but almost all of them have a “premium/subscription” option. Typically that subscription revenue accounts for 20-40% of total revenue, and is based on a very low ($1-5/month) subscription fee. However, that is rarely how things play out. Most entrepreneurs fall into the trap of assuming that there is a consistent elasticity in price - that is, the lower the price of what you’re selling, the higher the demand will be. It happened in music. It happened in movies. And it’s happening in directory assistance. Now I’m looking for other industries that are going to be converted. If you’ve got a plan that uses the free model to get that first penny and disrupt an industry, I’d love to hear about it. It’s a great way to shrink a market.”

When reading this, IT folks think, “open source.” Looks like that’s exactly what is happening in enterprise software. Open source vendors play the price elasticity game in the demand/price equation. Commercial open source vendors are sizing down their markets by offering free to low-cost solutions where the buyers get far more value for the dollar. But that’s only on the surface.
What I would like to describe is why open source/free software is not about shrinking the markets. Of course, there will be many cases when market leaders will have to step down.

Middleware

IBM has $2B running on WebSphere. Looks like for every dollar of revenue JBoss made, it took two dollars of revenue from BEA and one from IBM. JBoss made its fortune from shrinking the middleware market.

Data Integration

Informatica SOLELY DEPENDS on license revenue from ETL tools.
IDE Tools Borland has had its IDE business on the auction block for a while and is realizing it’s not easy to find a buyer for a sinking ship. Did Eclipse play a role in replacing Borland IDE tools? Sure it did.
For Informatica and BEA to transition to cheap/near-free subscription and OSS is a systemic shock that might literally kill them.There is no doubt that the existing software leaders would have to shrink their appetite significantly, since the disruptive newcomers have totally different cost structure and distribution business models. When you look at the drivers of the seismic shifts going on in many markets, the disruptive business models are not about shrinking markets. It’s not about “evil communists” dreaming about getting the behemoths of the software business to die. I don’t think they will.

What really happening is that new value is being created in the marketplace. Open source is so successful because it’s creating new markets. That’s the key for a true market disruption.

Just a few years ago JBoss, now a RedHat company, and MySQL were not going after the Fortune 1000 enterprise market. They are best at enabling underserved customers who previously could not afford expensive proprietary software. Now, JBoss is a part of a public company, MySQL is VC-backed, and they both have external shareholders. The latter demand returns on their investment thus pushing portfolio companies into a replacement business where the money is. But what these companies are best at is not a replacement business.

Open source is so powerful that it can destroy a license-driven software business. But the truly disruptive businesses are successful as long as they create new markets in the first place, not shrink the existing.

Did Oracle Scare the BI Community with Hyperion’s Acquisition?

Oracle is chasing SAP like never before. Now, thousands of Hyperion customers who run SAP will view their business through an Oracle/Hyperion BI “lens.” Tony Baer posted a good review about the transaction. I can’t disagree with Tony about further market consolidation and Business Objects/Cognos being the two BI leaders left out there. Two years ago, IBM acquired Ascential Software to round up its data integration offerings, and the BI acquisition is only a matter of time. My prediction is that Cognos will be the target.

Search

  • Links

  •  Subscribe by RSS/ATOM

    Find Me Online

    • View Renat Khasanshyn's profile on LinkedIn

    Enter your email address:





    Apatar is a Proud Sponsor of: