Data Integration Services on SQL Azure platform

They say that any knowledge never goes waste, but in IT parlance this saying can be redefined as any knowledge or DATA never goes waste. In my career till date, my experience has been that any successful business would have lot of external data processing as a part of its business functions. Competitive Sales Intelligence for example is one of the category of data that many IT organizations would keep of processing to defines best analytical insights for their sales team. Companies like Facebook and Google accumulate hoards of data and makes a fortune out of advertising business. But even these giants depend on external data providers for their business functions. Example of one such data provider is Factual, that provides data to Facebook. Monetizing on carefully curated and certified business databases has become a very big business.

Cloud takes this platform of sharing and trading data one step ahead. Windows Azure Marketplace provides DataMarket for the same purpose where applications can share and trade data.

1) Private cloud and public cloud comes into question, as organizations might want to share their datasets but limited to the scope of the organization only. Microsoft codename "Data Hub" claims to provide a flavor of managed self-service enterprise data integration on the cloud, which generally takes a huge team and data centers to serve the same needs of an organization. This platform is expected to provide private data-market to enterprises which can be very interesting in terms of agility and cost-savings.

2) Any sizeable organization would generate and consume lot of internal as well as external data. Integration and sharing of data is implementation of the solution after the source of data has been recognized. Data discovery from within and outside organization for business needs, is a bigger challenge in itself. Microsoft Codename "Data Explorer" can be seen as self-service SSIS on Azure platform. It provides data discovery from the windows azure marketplace as well as provides features for self-service data mashups from a variety of standard data sources. Hadoop is not yet included in the supported data source list, but if its gets included in the future, this platform can reap immense value and can acts organizations private Google blended with SSIS to create self-service data mashups and again publish the same as a source of data using Data Hub.

Power of Hadoop combined with cloud based tools like Data Hub and Data Explorer can generate business for lot of data providers as well as bring immense value to organizations. Also it would enable better use of data and provide cost-savings in enterprise data integration.