Chef for Microsoft Azure, Amazon, OpenStack, Rackspace, Google Compute Engine, or Linode

What is DevOps ? For beginner, who are not aware of what is DevOps, can read this page to gain an idea on the same.

DevOps is a branch of architecture design, that is often considered trivial by many architects or development leads. For traditional applications, infrastructure provisioning, capacity management, monitoring, and operations support generally gets taken care by dedicated IT teams bound by pre-agreed SLAs.

But when architects are dealing with cloud scale applications, devops is no longer a trivial area or outside of the solution definition. Automating infrastructure management using script based templates, on all the major cloud vendors, is one of the standard industry practices and supported by almost all the cloud vendors as well. It came to my surprise when I found one of the Microsoft Azure cloud trainers not aware of what is devops automation, and I had to educate the trainer on the same.

Some of the major players in this area are as mentioned below, and Chef is leading the way in this area.

• Vagrant
• Chef
• Ansible
• Capistrano
• Travis CI
• Jenkins
• TeamCity
• NodeTime
• New Relic

Vagrant still stands a choice for VMWare lovers, but Chef is much more sophisticated compared to Vagrant. A good place for beginners is to start getting an overview of Chef, and pursuing some free webinars and free trainings provided by Chef. Chef in itself is a comprehensive framework with concepts like Knife, Cookbook, Chef-repo, Ohai etc. A picture is worth thousand words.Below mentioned is the architecture diagram of Chef.

Chef Architecture

Picking up a cloud automation vendor is not the end of  devops. Often huge businesses have hybrid infrastructure environments formed of private datacenter, physical and virtual environments, multi-tenant cloud environments. Companies like RightScale too offer specialized solutions to deal with such use-cases. Below is an interesting architecture diagram of righscale solution model.

RightScale MultiCloud Platform

Data layer requirements for applications with cloud hosting and NoSQL databases - Cloudant

MVC is one of the most famous and widely adopted application development design patterns. Typically data layer of applications starts off with features that requests actions related to data manipulations. 

• If an application is public facing where users can sign-up on the app and consume services, there are bright changes that with the success of the application, user data would grow linearly. 
• On the top of it if the application is related to social media / networking / large scale e-commerce, then the app is a future candidate to mature in the big data processing requirements. This puts an obligation on the data layer to be more mature, if you want to keep your operational costs (incurred by dedicated DBAs, infrastructure procurement and maintenance, etc) under check. 
• As data grows big, and tends towards being unstructured from structured, database choices tends to be towards specialized NoSQL databases from generalized relational RDBMS.
• To manage data load, data management methods like replication, sharding, master-slave configuration, partitioning etc are employed. As most of these techniques require flexibly scalable infrastructure, data is hosted mostly on cloud and this means that data access layer should be mature enough to work with cloud and almost manage the infrastructure administration using the cloud provider api. 
• Different cloud providers have their own pros and cons. An application downtime is directly proportional to equivalent loss is business in today's world. So you might want your data layer to be flexible enough, such that it can access data from some of the leading cloud hosting environments.

These specifications are too ambitious for an application to develop just for it's own use.  But at the same time to keep frequent revamping of data layer at bay or to avoid create duplicate data layers for different optimal data access and management methods, one needs a data layer that can manage the above mentioned challenges.

One interesting company provider a data layer solution that can meet the above specifications with even more features - Cloudant. It provides a data layer to 

• Manage data in CouchDB NoSQL database, 
• Hosted on choice of cloud providers like AWS, Azure, RackSpace etc
• RESTful CouchDB compliant APIs
• JSON based data exchange format
• Lucene based text search and much more.

NoSQL databases, Cloud providers of the scale of Azure, AWS etc, MPP systems like Hadoop etc are quite famous. But the data layer that manages data access along with data management on cloud can be a very interesting option, than starting your app from scratch with a fat sized cloud account. Checkout the technical whitepaper on Cloudant for a detailed understanding of this service and this article.

Career in Windows Azure and Career challenges due to Windows Azure

Recently Microsoft made announcement about new feature releases and additions to Windows Azure. Scott Guthrie gave a keynote speech about the guided tour of Windows Azure enhancements, which can be downloaded from here. Some of the very impressive enhancements are as below:

1) IaaS offering: This aspect of cloud platform is facilitated by provisioning a persistent, completely cross premise portable, .VHD format, Windows / Linux based Virtual Machine hosted on windows azure computing platform. This VM can be created with just a few clicks using the web based azure management portal, powered by a thorough monitoring, auditing, metering, scaling and administrative control including immediate ability to RDC the VM. A few ready VM images are also available in the gallery to start with. Also an innovative set of virtual networking capabilities have been offered that seems to seamlessly bridge the gap between private in-premise networks and cloud.

One of the ready available image is of SQL Server 2012 evaluation edition. Others are vanilla server images. If you use this wisely, you have almost a mobile server at your fingertips. Say you are travelling, and you get a call from your office to modify a report and revert in an hour. You can simply connect to the cloud, provided you have a prototype DB schema, you can use this ready SQL Server installation, modify your report and wash off your hands from an issue that can result into some serious office tensions.

2) Rich and collaborative application development and hosting platform: If you read the above section carefully or if you watch the above screenshot carefully, you would find non-microsoft OSes. Microsoft has collaborated with a lot of open-source and third-party application partners and have made a variety of application development platforms available as a service. Apart from SQL Server, Microsoft seems to have adopted mySQL too which is a supported database for asp.net powered windows azure websites. A sad part is that lot of CMS (Content Management System) frameworks are available, but not Sharepoint. Hopefully that should be on the plan.

The happy part here is that IaaS as well as SaaS can work in an integrated manner. Distributed caching, Unified Deployment using standard methods of TFS including cross-platform based methods like Git, cloud based DNS and identity management services are available in the form of Cloud Services which is the PaaS offering from Azure.

Career in Windows Azure: Azure is an aggressively developing cloud computing platform. Many foresighted professionals would definitely get a thought, can this be the new ride towards fortune pay packages ? Azure has many dimensions on its platform like Data Market, IaaS and virtualization and virtual networking, application hosting and development frameworks, hadoop on azure, upcoming services like media streaming, data explorer and data hub are some of the major freeways that gives you your ticket to get into the business of cloud. Few certifications are also available like 70-583 : PRO : Designing and Developing Windows Azure applications. Honestly, I wont opt appearing this exam as it got released in Feb 2011 and Azure drastically changed in June 2012. So I would still take an interest to pursue it, but wait to appear the final certification till the product gets stable.

Azure skills are definitely niche skills, but not so rich in demands that you would be weighted in gold for having it. To make a career you need to sell your thoughts at the right platform. If you can pull out few interesting POCs or Azure based solutions on 3-month free trial version of Azure, and flaunt around by sharing the same on Codeplex that can gain you some real good value. Dumb thesis of white papers describing genesis of any aspect of Azure won't give you any real business in Azure, thou it can give you some recognition in the technical journalism arena.

Career Challenges due to Windows Azure: This section is for those who believe that cloud poses threat to practitioners who are not ready to adapt to change. Adaptive attitude towards technology is one of the phenomenon that any successful technology practitioner would have experienced throughout one's career. RFPs (Request for Proposals) are getting more and more competitive, OPEX (Operational expenditure) and CAPEX (Capital Expenditure) are being measured edge to edge for delivery excellence, and with services like IaaS and SaaS, cloud is inducing a mind shifting game in the business.

In earlier days and may be even today, application developers consider themselves expert if they know caching and SOA. With azure platform, distributed cache that geo-replicates and synchronizes itself using SDKs like Data Sync, CDN and app fabric is a simple matter of config that can be achieved using clicks. What should I do with your caching expertise that works only in-premise, takes tons of resources, licenses and non-flexible solutioning that forces me to use all the products from microsoft stack instead of a hybrid cost-effective hybrid-platform solutioning ??

Many DBAs still bank on their hyper-V, SCOM, virtualization and clustering skills as their magic business. I created a SQL 2012 ready image with all standard HADR config within minutes. If I own a SMB scale application, do I really need you who just knows the IaaS part, if azure promises to manage it for me with 99.95% operational availability ? With Microsofts support for cross-platform and open source, this market has become more competitive than ever. Party has just started and more guests from non-microsoft world are invited with open hands. As per a report, by 2014, 80% of the new softwares would be traded on cloud as a service. Those who still have not got the point are marching their way to museums :)

How to use MS BI with Hadoop and Why to use Hadoop with SSAS

IT Professionals who use DBMS, SQL, ETL, Reporting and/or Advanced Reporting, and Analytics consider this as the end of data ecosystems. But this is just mainstream IT sphere in the world of data. I do not intend to emphasize of the potential riding on Hadoop, as there are tons of reference material available for the same. If you want to quick check the direction of wind, you can simply fly a kite, you dont need a satellite weather report. Translating it into plain terms, if you want to get a hint of Hadoop's potential, just google on what data and analytics related companies are upto these days. You would find that database giants like TeraData, Microsoft, Informatica and others are ramping up big efforts to provide support for Hadoop. The big businesses that run on Hadoop are Facebook, Yahoo, LinkedIn, Twitter and others. This suffices to conclude that if you are a vetern opportunist in industry, Hadoop is one of the most promising targets.

The challenge of Hadoop starts with bringing it to mainstream IT, which is mostly warehousing data, reporting it and providing analytics. This methodology generally requires activites like data profiling, data cleansing, ETL, and creating warehouses / marts.

1) From the mainstream database world, Hadoop is a source as well as destination. Its more like a content management system functioning in the the form of a database. Hadoop is a MPP system that can run on parallel nodes reaping peta-byte scale data processing speeds. Cloud is one of the most appropriate infrastructure for the same. Windows Azure was already supporting Hadoop VM installations and now a new offering in underway which is known as Hadoop based services for Windows Azure.

2) Why to use Hadoop when we already have SSAS with the power of BISM ? Well any analytics professional would have this question. Theory does not wet the apetite of a practitioner, so the best answer is a case study video featuring how Klout leverages Hadoop and Microsoft BI Technologies to manage BIG Data.

3) Microsoft has announced a connector SQL Server Connector for Apache Hadoop, its a old news. But if you pay attention to detail it says its Sqoop based which is an open source tool provided by Cloudera which imports data from SQL Database into Hadoop Clusters. It can be a very nice to learn tool to start building your skill stakes in the Hadoop world. Any application would have to pump-in and pump-out data from Hadoop, so import export of data from SQL based databases to Hadoop is an inevitable process.

I plan to make MS BI, Analytics and Visual Business Intelligence coupled with BIG Data and cloud as my new regime. I would be sharing my experiences, thoughts and views on the way through my journey. I have introduced a new section on my blog titled Hadoop, BIG Data and Cloud and added a few useful links under the same. I would be adding more to this section, to keep a watch on the same.

In my views, a rolling stone gathers no moss. I intend to earn the same amount of money and respect and recognition I earn in a year, in a months time. I believe that if one has got a dream like this, one needs to be insightful and embrace the change, be a part of the change and make efforts to change the world thats not ready to change.

For my regular blog readers: My blog has remained silent for around 6 months, and my authoring presence has been going south. To my surprise from the blog statistics I was able to make out that the site visits have remained constant and at times have gone even high that it was ever, even without any activity on my blog. This gives me motivation to keep moving on and being an MVP I also feel an obligation on my shoulders to keep sharing my experiences. The reason for my low authoring activity have been my personal life, and I am turning on the lights of my blog after 6 months straight.

In the time that I took almost a break from blogging, two platforms that I have evidenced influencing the IT Industry limited to the scope of my perspective as a Solutions Architect for Business Intelligence and Analytics, are Android and Hadoop. I would discuss about Android at some other time, this post is about Hadoop.

Azure Design Patterns

I have been very busy these days with personal matters, and have not been able to extract time for blogging. I apologize to my readers for the same. Microsoft Azure cloud environment is a rapidly growing cloud platform. With the release of Hadoop connectors for SQL Server, support for Hadoop implementations and Azure Table Storage, Azure provides a reasonable support for unstructured data which is going to be the need of the future. But along with this one more area is growing largely that churns our lot of business and effectively lot of data too - Social Media.

Microsoft has newly released Windows Azure Toolkit for Social Games version 1.0. Integration with social media has become a necessity for almost every other sizable organization today. LinkedIn is one of the best examples where organizations are trying to hook in the social connections of employees for a more intelligent recruitment. Going social is one of the best moves by Azure team.

Azure is classified more from a compute and storage criteria, but from different perspectives of development if would make sense to categorize it into more detailed criteria. When you speak of development, design pattern eventually comes into picture, thou in database world it is used less compared to application world. Buck Woody has put up a very nice website to study these design patterns, and I feel its definitely worth checking out. Its called Azure Design Patterns.

Using Graph Database on Windows Azure

Unstructured data is the newest and most vibrant source of data, that organizations want to mine for rich business intelligence. SQL Server and Sharepoint and the front runners from the Microsoft platform in the field of BI. SQL Server being a RDBMS, is not the right choice to contain unstructured data and Sharepoint itself can generate and contain huge volumes of unstructured data. New categories of databases like Document database, Key-Value pairs, Graph databases etc are suited for these purpose and from here starts the territory of NoSQL movement.

Microsoft Azure platform is supporting Hadoop implementations, SQL Server interoperability drivers for Hadoop has also been announced, Microsoft Research is trying to develop project Dryad, which are all movement towards building capabilities to support unstructured data. Graph databases are one of the prominently used database types in the world of unstructured data. Many would argue why relational database cannot be used to achieve the same what graph databases are used for ? Here is one of the answers for the same. Graph databases apply graph theory and once you understand the same you would find the reason why RDBMS cannot cater what Graph Databases can. Neo4j is one of the leaders in this area. Industry leaders like Google also have their own implementation of graph database know as Pregel.

sones GraphDB is one of the graph databases of choice for Microsoft professionals, as it is developed using .NET and is easily supported on Azure platform. Huge volume of unstructured data needs flexible compute and storage platforms like Azure cloud platform, and as it is using .NET framework behind the scenes, it is ideal to be hosted on Azure platform. You can access the technical datasheet from here and below is the architecture diagram of the same. Interestingly this brings a new query language for DB professionals, GraphQL !!

SSIS on Cloud with SQL / Windows Azure : Future Applications

Generally cloud and ETL have a typically known application of facilitating ETL, which mostly harnesses the elasticity of computing resources. ETL on cloud is ideal for applications where data is already stored on the cloud. Regular line of business applications would process data from OLTP sources and load the data on another data repository on the cloud. For ETL to source and load data from in-premise data sources, WCF and related RIA services are employed. Using Amazon EBS cloud and customized VM images, you can setup your ETL on cloud and Amazon EBS out-of-box supports SSIS standard version. Right when Azure was in it's CTP, I had authored two articles on SQL Azure, on how to read and write data to SQL Azure using SSIS and SSRS 2008 R2. But this is something that is already well-known. What's new ?

Semantic Web, Unstructured Data and technologies that store, process, analyze, warehouse such data and extract intelligence out of the same is the new challenge that is at the horizon of the IT industry. Few front line IT majors have tsunami sized data generated everyday due to the virtue of popularity of social media, and such organizations like Google, Yahoo, Facebook, etc have already started wrestling with these challenges. The benefits of processing unstructured data, and driving your business based on the extracted intelligence are very clear from the example of these companies where they within a duration less than a decade, their advertising revenues are worth billions and still skyrocketing. All standard business house have lots of unstructured data like emails, discussion forums, corporate blogs, recorded chat conversations with clients etc. Organizations generally have aspirations to build a knowledge base for all the different areas of business functions, but when they start seeking consulting on the strategy to implement the same, they find themselves in a whirlpool of processes and financial burdens. Unstructured data has the potential to generate data for such knowledge base.

Still the question in your mind would be, what has this to do with SSIS and Azure? Regular applications of SSIS are known to everyone, and more than that professionals would be knowing that SSIS is not supported on Azure cloud platform, which might be the motivation of reading this post as the subject line reflects that is might be supported now. SSIS is an in-memory processing architecture, and implementing the same of shared / dedicated cloud environments has its own challenges. I am also sure that SSIS Team must be on its way to bring it to the crowds in the time to come. But my interest in on the future application of SSIS on unstructured data, and hosting it on Azure cloud platform.

When SSIS would gain such capability, applications like Extractiv would be common across enterprises. To build application like Extractiv using SSIS, solution design would be too crippled as SSIS is not inherently blended with cloud. One day I would like to see SSIS packages getting executed as a crawler service in Sharepoint, which would crawl entire site and data on Sharepoint portals like FAST Server, extract entities from unstructured data and populate next generation of warehouse on the Azure cloud platform, that would be queried using LINQ for HPC kind of technologies. For me, applications like Extractiv are very fascinating, as they inspire to foresee ideas, that are window to opportunities which most are not able to envision right now.

Data Warehousing and Analytics on Unstructured Data / Big Data using Hadoop on Microsoft platform

A major community of data warehousing professionals grow up from the old school of Kimball and Inmon methods of data warehousing. Lots of professionals do boast on virtualization, complex MDX querying, performance tuning OLAP engines and managing data warehouse environments of the size of a few hundred GBs or several TBs, as the most niche and challenging jobs they have on their resume. But there is another world of data warehousing and analytics which most would not have explored, and slowly this revolutionary and emerging wave is reaching SMBs which would effectively challenge the world of data warehousing as we practice today. You might come across a question while reading this post, that what has Microsoft to do with it and answer to this question is towards the end of this post.

Data warehouses and data marts developed using Kimball, Inmon or any hybrid methodology can deal with structured data and have scalability challenges too. Appliance solutions such as Parallel Data Warehouse are Microsoft's candidate to deal with such challenges. Some might think that this is the answer to warehouse largest volume of data and build analytical capabilities on the top of it. But this data volume is just a very small piece of the ecosystem. According to Gartner, enterprise data would grow by 650% in 2014 and 85% of the same would be unstructured data, which is also termed as BIG Data.

Have you ever thought of how organizations like Yahoo, Google, Facebook etc organize their data? Which databases do they use? Whether they have data warehousing and analytics? These organizations have some of the largest data volumes in the world. For example, Facebook is heard to have 12 TB of compressed data added per day and 800 TB of compressed data scanned per day. Can you imagine structuring such volume of data using ETL, storing it in data warehouses, aggregating it using OLAP engines in data marts and extracting analytics out of the same ? To handle such volumes of data for data warehousing and analytics, innovative technologies and infrastructure design are required that can support massively parallel processing, and the one I am talking about is named "Hadoop" which is an open-source distributed computing technology and "Hadoop Distributed File System" which is the storage mechanism for handling unstructured data.

Cloud environments like Amazon are already supporting Hadoop, organizations like Cloudera and IBM are supporting commercial distributions of Hadoop, and a lot of big and famous international business majors are already using Hadoop implementation. The biggest implementation is used by Yahoo with 100,000+ CPUs running on 40,000+ computers running Hadoop. An exhaustive list of organizations using Hadoop can be read from here. Organizations are using Hadoop to implement data warehousing and analytics for purposes like Event Analytics, Click Stream Analytics, Text Analytics and more.

For those who are completely afresh to this part of the world, can go through some very interesting reference material mentioned below:

1) The Google File System

2) Data warehousing and Analytics Infrastructure at Facebook

3) Apache Hadoop Wiki

4) Apache Hadoop MapReduce Implementation at Yahoo

5) Setting up Hadoop on VM

Microsoft is aware of the challenges using unstructured data and Hadoop, and is gearing up slowly for the same.

1) Microsoft Research is developing Project Daytona on Azure platform and Project Dryad, which is perceived by the industry as Microsoft's candidate as an alternative for Apache Hadoop.

2) Those who believe that MDX is the top query language that can deal with huge amount of data from OLAP engines, should check out LINQ to HPC to update their GK.

3) With the increasing popularity and success of Hadoop, Microsoft is also supporting Hadoop on Azure platform. You can get an idea of how to deploy Hadoop cluster on Azure platform from here.

The way Microsoft professionals felt that cloud is something new when Azure was introduced, same would be the case when Microsoft would start supporting Hadoop commercially or introduce a commercial alternative for the same. But neither cloud is a recent invention nor technologies like Hadoop to handle, ware house and analyze unstructred data. In my viewpoint, architects and organizations should develop their readiness to deal with the emerging winds of change and upcoming potential business opportunites that unstructured data can offer.

SSIS , Azure , Sync Framework = Economic solution for synchronization business

SSIS is independently famous as an ETL tool, but often a tool is a good as the purpose it can cater. Often integration is just a piece of the entire process, that hands over the delivered output to the next runner of the relay race.

In today's competitive and budget economy, one of the biggest challenges that architecture designs as well as project costs face are CAPEX ( Capital Expenditure ) and OPEX ( Operational Expenditure ). If you have underutilized resources and your architecture design is such that it can raise CAPEX and/or OPEX over a period of time, it's definitely going to fail when a RFP (Request for Proposal) is sent to the client. To gain competitive edge, today's architecture design propositions necessarily needs to have dual flavors - one in the traditional way and another which includes a cloud based economy design. Cloud involvement as a platform or as a component in the solution design is the only most promising way to provide a competitive edge to your propositions. In a typical microsoft platform solution, in my views, the primary cloud choice is Amazon EC2 and Azure platform is gaining adoption and popularity gradually.

Sync Framework is one of the robust mechanisms to synchronize data across boundaries spanning networks / systems / machines. This framework can be integrated very well with .NET applications, effectively synchronization of a variety of data can be catered by this framework. A very common example is to synchronize backup of a database across geographically separate systems where each terminal need to synchronize from a common master backup of the database.

Now if we join this separate piece of the puzzle, it can form a solution to a very big business that can be seen in almost every enterprise. Huge multinational corporations often have various companies operating under a single umbrealla, for eg. CitiGroup may have CitiBank, CitiCorp, CitiSecurities etc.. each having their own business, and each company might be implementing different IT systems from legacy to cloud based. A few of these might be occasionally connected system to a central CitiGroup headquaters which might be storing centralized key data summarizing all business and geographies.

1) SSIS can be used to provide the first level of integration for each independent child company. The central controlling office might need to gather data and even the child companies might need to receive different kinds of data from the central office. These child companies might be occasionally connected or dedicatedly connected to the head office.

2) After SSIS has kept the parcel ready, Azure platform i.e. cloud can be used as a distribution network. Cloud is an economic choice, considering the time for which storage might be required for occasionally connected systems. Also Windows Azure Content Distribution Network can be utilized to replicate this data near to the geography of the child offices / businesses.

3) Sync Framework supports cloud as a data source (to the best of my knowledge). Storage and retrieval on the cloud can be controlled by sync framework, to customize the kind of replication and synchronization required for each particular businesses.

4) One can hook a .NET application and keep a monitor on these processes. Also if required, Sharepoint can be used for a collaborative control and monitoring of these processes.

Every enterprise need one or another kind of replication and synchronization of data, and using SSIS, Azure and Sync Framework (optionally), solution providers can exploit the hidden businesses that lies in every enterprise. I would be interested to hear on this viewpoint from Sales folks, especially those who deal in MS BI business.

Project Atlanta and Quest Software's Azure Services : Cloud based services related to SQL Server

Entire blogosphere related to SQL Server is cheering the release of SQL Server Denali CTP1, and everyone seems to be in a race to figure out the new features in this CTP. Some of the road blocks for developers to test this CTP are limited supported operating systems and number of issues during installation. SSIS seems to be loaded with a lot of enhancements with some great sounding support for data quality with new tools and controls.

Coming to the topic of this post, after the availability of Database Manager for SQL Azure, SQL Azure Reporting is one step forward in bringing SaaS model in SQL Server ecosystem. Project Atlanta and Quest Software's Azure Services are two of the cloud based SQL Server related SaaS / PaaS (as I have not analyzed it thoroughly) services.

Microsoft's official description for Project Atlanta is as follows: "Microsoft Codename Atlanta is a cloud-based service that analyzes installations of Microsoft SQL Server 2008 (and later) and provides proactive alerts to help you avoid downtime and adhere to Microsoft's best practices. Detection of potential issues is based on knowledge provided by experienced Microsoft Support engineers who keep it up to date to leverage the most recent experience in supporting SQL Server deployments around the world". Though it sounds very exciting, there is another dimension to this offering, which Brent has described in his post. Do not miss the conversation between Brent and the Program Manager of Project Atlanta. This is one case study where one can take few lessons, that can be helpful in developing proprietary solution offerings on similar lines.

Seems like Quest software already had the same idea in mind. An excerpt about the video where executives from Quest Software shares their views and perspectives about their cloud based IT services on Windows Azure platform in as follows: "Quest is an early adopter of the Windows Azure platform. They’ve built out an extensive services framework as well as a few initial service offerings that sits on top of Windows Azure Framework and Windows Identity Framework. One of the key elements in the design of Quest’s framework is secure communication and authentication between all the service components and layers, whereby encryption is based on certificates stored within the dedicated certificate store provided by Windows Azure". This is one another case study from which lessons can be taken for your future solution offering on Windows Azure / SQL Azure platform.

If you analyze the trend of the wind, you can easily make out the SQL Server product vendors are gearing up to provide cloud based solutions, some of which might be effective and some might be not. From a solution provider point of view, when your client might request a PaaS / SaaS solution on Windows Azure or SQL Azure, case studies like these can provide good insight from an architecture as well as designing of solution offering standpoint. I am wondering what would be Red Gate 's answer to these services, as SQL Toolsbelt might not be sufficient when cloud comes into picture.

Data storage on Windows Azure and SQL Azure

Windows Azure, SQL Azure, Blob storage, Local storage, Table storage, Queue storage, web role, worker role, Content Delivery Network, Dev Sync etc.. are some of the buzz words which indicates that there are Azure bees around you. If you are not keeping the pace with the advancements that are constantly being made available on this platform, it might feel like a very tough time when you are faced with a situation when you need to develop a POC or an architecture design that works on Azure platform. Fortunately there are a lot of whitepapers and MSDN library that can help you to get started.

But the first thing that clicks my mind when I start the architecture / solution design, is with some basic questions like, how will I capture data and where will I store data. Maintaining session state, implementing security, auditing, logging, error control and reporting, need for a collaboration platform, scalability and performance considerations, migration considerations etc.. are all vertical placeholders in a architecture design than runs across the solution. But when I need to figure out my horizontal figures, my source and destination are the basic blocks. If you have figured out the data storage as Windows Azure either as a destination / staging location / hosting platform, the answer for data storage is not as simple as "enterprise edition of 64-bit SQL Server", whether it's a BI solution where you might use Azure as a temporary / permanent staging location or an Application solution where you might use Azure in place of on-premise SQL Server. With cloud this gets quite more complex, and factors like pricing and storage capacity comes into consideration.

Recently there's a very nice article published on technet, that is worth keeping in your arsenal as a part of intelligence building exercise on Windows Azure platform, to tackle your first encounter with Windows Azure environment. This article provides nice overview on data storage offerings on Windows Azure platform, and the same can be read from here.