Microsoft Roslyn and SQL CLR 2012 : Impact analysis

Few weeks back Microsoft announced CTP release of a new application development framework codenamed Microsoft Roslyn. To define it in one line for database professionals, it can be perceived as "Compiler as a Service" and gives more granular control of application code compilation. An in-depth explanation about the same can be read from here.

Database professionals would be ready to leave this post by now thinking what do we have to do with application development frameworks ? What's new in that ? Any application would request connection to the database through a managed driver, send queries and return results which can be very well governed by database administrators. And that's true to a great extent too !

But here is the catch. An ambassador of .NET framework sits in the world of SQL Server, i.e. SQL CLR. If I ask a question to DB professionals, what do you hate to have in your database server environment, most of the professionals would answer "dynamism, ad-hoc queries, surprises...". And Roslyn brings lot of dynamism never seen before, which makes it very powerful as the assembly behavior can be very much unpredictable. Hence tasks such as security regulation and dependency analysis would become more complex due to this granular division of the complier services.

In a typical MVC framework, Data Access Layer assemblies generally used to make calls to database. From an another angle, this framework is developed for agile application development targeted for cloud environment. But if application developed with these framework and stuffed with lots of dynamism is hosted on SQL environment, it would interesting to see how much acceptance it gains from data professionals. As of now its not seamlessly supported on SQL Server 2012, but if Microsoft has a long roadway for it, then future cummulative updates can be expected to bring integration features in SQL Server 2012 for Microsoft Roslyn.

An interesting example of Microsoft Roslyn that demostrates a flavor of my thoughts can be read from here.

How to use MS BI with Hadoop and Why to use Hadoop with SSAS

IT Professionals who use DBMS, SQL, ETL, Reporting and/or Advanced Reporting, and Analytics consider this as the end of data ecosystems. But this is just mainstream IT sphere in the world of data. I do not intend to emphasize of the potential riding on Hadoop, as there are tons of reference material available for the same. If you want to quick check the direction of wind, you can simply fly a kite, you dont need a satellite weather report. Translating it into plain terms, if you want to get a hint of Hadoop's potential, just google on what data and analytics related companies are upto these days. You would find that database giants like TeraData, Microsoft, Informatica and others are ramping up big efforts to provide support for Hadoop. The big businesses that run on Hadoop are Facebook, Yahoo, LinkedIn, Twitter and others. This suffices to conclude that if you are a vetern opportunist in industry, Hadoop is one of the most promising targets.

The challenge of Hadoop starts with bringing it to mainstream IT, which is mostly warehousing data, reporting it and providing analytics. This methodology generally requires activites like data profiling, data cleansing, ETL, and creating warehouses / marts.

1) From the mainstream database world, Hadoop is a source as well as destination. Its more like a content management system functioning in the the form of a database. Hadoop is a MPP system that can run on parallel nodes reaping peta-byte scale data processing speeds. Cloud is one of the most appropriate infrastructure for the same. Windows Azure was already supporting Hadoop VM installations and now a new offering in underway which is known as Hadoop based services for Windows Azure.

2) Why to use Hadoop when we already have SSAS with the power of BISM ? Well any analytics professional would have this question. Theory does not wet the apetite of a practitioner, so the best answer is a case study video featuring how Klout leverages Hadoop and Microsoft BI Technologies to manage BIG Data.

3) Microsoft has announced a connector SQL Server Connector for Apache Hadoop, its a old news. But if you pay attention to detail it says its Sqoop based which is an open source tool provided by Cloudera which imports data from SQL Database into Hadoop Clusters. It can be a very nice to learn tool to start building your skill stakes in the Hadoop world. Any application would have to pump-in and pump-out data from Hadoop, so import export of data from SQL based databases to Hadoop is an inevitable process.

I plan to make MS BI, Analytics and Visual Business Intelligence coupled with BIG Data and cloud as my new regime. I would be sharing my experiences, thoughts and views on the way through my journey. I have introduced a new section on my blog titled Hadoop, BIG Data and Cloud and added a few useful links under the same. I would be adding more to this section, to keep a watch on the same.

In my views, a rolling stone gathers no moss. I intend to earn the same amount of money and respect and recognition I earn in a year, in a months time. I believe that if one has got a dream like this, one needs to be insightful and embrace the change, be a part of the change and make efforts to change the world thats not ready to change.

For my regular blog readers: My blog has remained silent for around 6 months, and my authoring presence has been going south. To my surprise from the blog statistics I was able to make out that the site visits have remained constant and at times have gone even high that it was ever, even without any activity on my blog. This gives me motivation to keep moving on and being an MVP I also feel an obligation on my shoulders to keep sharing my experiences. The reason for my low authoring activity have been my personal life, and I am turning on the lights of my blog after 6 months straight.

In the time that I took almost a break from blogging, two platforms that I have evidenced influencing the IT Industry limited to the scope of my perspective as a Solutions Architect for Business Intelligence and Analytics, are Android and Hadoop. I would discuss about Android at some other time, this post is about Hadoop.