How to use MS BI with Hadoop and Why to use Hadoop with SSAS

IT Professionals who use DBMS, SQL, ETL, Reporting and/or Advanced Reporting, and Analytics consider this as the end of data ecosystems. But this is just mainstream IT sphere in the world of data. I do not intend to emphasize of the potential riding on Hadoop, as there are tons of reference material available for the same. If you want to quick check the direction of wind, you can simply fly a kite, you dont need a satellite weather report. Translating it into plain terms, if you want to get a hint of Hadoop's potential, just google on what data and analytics related companies are upto these days. You would find that database giants like TeraData, Microsoft, Informatica and others are ramping up big efforts to provide support for Hadoop. The big businesses that run on Hadoop are Facebook, Yahoo, LinkedIn, Twitter and others. This suffices to conclude that if you are a vetern opportunist in industry, Hadoop is one of the most promising targets.

The challenge of Hadoop starts with bringing it to mainstream IT, which is mostly warehousing data, reporting it and providing analytics. This methodology generally requires activites like data profiling, data cleansing, ETL, and creating warehouses / marts.

1) From the mainstream database world, Hadoop is a source as well as destination. Its more like a content management system functioning in the the form of a database. Hadoop is a MPP system that can run on parallel nodes reaping peta-byte scale data processing speeds. Cloud is one of the most appropriate infrastructure for the same. Windows Azure was already supporting Hadoop VM installations and now a new offering in underway which is known as Hadoop based services for Windows Azure.

2) Why to use Hadoop when we already have SSAS with the power of BISM ? Well any analytics professional would have this question. Theory does not wet the apetite of a practitioner, so the best answer is a case study video featuring how Klout leverages Hadoop and Microsoft BI Technologies to manage BIG Data.

3) Microsoft has announced a connector SQL Server Connector for Apache Hadoop, its a old news. But if you pay attention to detail it says its Sqoop based which is an open source tool provided by Cloudera which imports data from SQL Database into Hadoop Clusters. It can be a very nice to learn tool to start building your skill stakes in the Hadoop world. Any application would have to pump-in and pump-out data from Hadoop, so import export of data from SQL based databases to Hadoop is an inevitable process.

I plan to make MS BI, Analytics and Visual Business Intelligence coupled with BIG Data and cloud as my new regime. I would be sharing my experiences, thoughts and views on the way through my journey. I have introduced a new section on my blog titled Hadoop, BIG Data and Cloud and added a few useful links under the same. I would be adding more to this section, to keep a watch on the same.

In my views, a rolling stone gathers no moss. I intend to earn the same amount of money and respect and recognition I earn in a year, in a months time. I believe that if one has got a dream like this, one needs to be insightful and embrace the change, be a part of the change and make efforts to change the world thats not ready to change.

For my regular blog readers: My blog has remained silent for around 6 months, and my authoring presence has been going south. To my surprise from the blog statistics I was able to make out that the site visits have remained constant and at times have gone even high that it was ever, even without any activity on my blog. This gives me motivation to keep moving on and being an MVP I also feel an obligation on my shoulders to keep sharing my experiences. The reason for my low authoring activity have been my personal life, and I am turning on the lights of my blog after 6 months straight.

In the time that I took almost a break from blogging, two platforms that I have evidenced influencing the IT Industry limited to the scope of my perspective as a Solutions Architect for Business Intelligence and Analytics, are Android and Hadoop. I would discuss about Android at some other time, this post is about Hadoop.