Cloud Video and Cloud Computing Infographic

Ever wondered how a datacenter looks like and the kind of equipments used to power the same ? What is the metal that sits beneath the service we address as Cloud ? Watch this short 10 mins video that takes you on a guided tour of Microsoft Data Centers which hosts the data stored in Windows Azure cloud.

Career in Windows Azure and Career challenges due to Windows Azure

Recently Microsoft made announcement about new feature releases and additions to Windows Azure. Scott Guthrie gave a keynote speech about the guided tour of Windows Azure enhancements, which can be downloaded from here. Some of the very impressive enhancements are as below:

1) IaaS offering: This aspect of cloud platform is facilitated by provisioning a persistent, completely cross premise portable, .VHD format, Windows / Linux based Virtual Machine hosted on windows azure computing platform. This VM can be created with just a few clicks using the web based azure management portal, powered by a thorough monitoring, auditing, metering, scaling and administrative control including immediate ability to RDC the VM. A few ready VM images are also available in the gallery to start with. Also an innovative set of virtual networking capabilities have been offered that seems to seamlessly bridge the gap between private in-premise networks and cloud.

One of the ready available image is of SQL Server 2012 evaluation edition. Others are vanilla server images. If you use this wisely, you have almost a mobile server at your fingertips. Say you are travelling, and you get a call from your office to modify a report and revert in an hour. You can simply connect to the cloud, provided you have a prototype DB schema, you can use this ready SQL Server installation, modify your report and wash off your hands from an issue that can result into some serious office tensions.

2) Rich and collaborative application development and hosting platform: If you read the above section carefully or if you watch the above screenshot carefully, you would find non-microsoft OSes. Microsoft has collaborated with a lot of open-source and third-party application partners and have made a variety of application development platforms available as a service. Apart from SQL Server, Microsoft seems to have adopted mySQL too which is a supported database for asp.net powered windows azure websites. A sad part is that lot of CMS (Content Management System) frameworks are available, but not Sharepoint. Hopefully that should be on the plan.

The happy part here is that IaaS as well as SaaS can work in an integrated manner. Distributed caching, Unified Deployment using standard methods of TFS including cross-platform based methods like Git, cloud based DNS and identity management services are available in the form of Cloud Services which is the PaaS offering from Azure.

Career in Windows Azure: Azure is an aggressively developing cloud computing platform. Many foresighted professionals would definitely get a thought, can this be the new ride towards fortune pay packages ? Azure has many dimensions on its platform like Data Market, IaaS and virtualization and virtual networking, application hosting and development frameworks, hadoop on azure, upcoming services like media streaming, data explorer and data hub are some of the major freeways that gives you your ticket to get into the business of cloud. Few certifications are also available like 70-583 : PRO : Designing and Developing Windows Azure applications. Honestly, I wont opt appearing this exam as it got released in Feb 2011 and Azure drastically changed in June 2012. So I would still take an interest to pursue it, but wait to appear the final certification till the product gets stable.

Azure skills are definitely niche skills, but not so rich in demands that you would be weighted in gold for having it. To make a career you need to sell your thoughts at the right platform. If you can pull out few interesting POCs or Azure based solutions on 3-month free trial version of Azure, and flaunt around by sharing the same on Codeplex that can gain you some real good value. Dumb thesis of white papers describing genesis of any aspect of Azure won't give you any real business in Azure, thou it can give you some recognition in the technical journalism arena.

Career Challenges due to Windows Azure: This section is for those who believe that cloud poses threat to practitioners who are not ready to adapt to change. Adaptive attitude towards technology is one of the phenomenon that any successful technology practitioner would have experienced throughout one's career. RFPs (Request for Proposals) are getting more and more competitive, OPEX (Operational expenditure) and CAPEX (Capital Expenditure) are being measured edge to edge for delivery excellence, and with services like IaaS and SaaS, cloud is inducing a mind shifting game in the business.

In earlier days and may be even today, application developers consider themselves expert if they know caching and SOA. With azure platform, distributed cache that geo-replicates and synchronizes itself using SDKs like Data Sync, CDN and app fabric is a simple matter of config that can be achieved using clicks. What should I do with your caching expertise that works only in-premise, takes tons of resources, licenses and non-flexible solutioning that forces me to use all the products from microsoft stack instead of a hybrid cost-effective hybrid-platform solutioning ??

Many DBAs still bank on their hyper-V, SCOM, virtualization and clustering skills as their magic business. I created a SQL 2012 ready image with all standard HADR config within minutes. If I own a SMB scale application, do I really need you who just knows the IaaS part, if azure promises to manage it for me with 99.95% operational availability ? With Microsofts support for cross-platform and open source, this market has become more competitive than ever. Party has just started and more guests from non-microsoft world are invited with open hands. As per a report, by 2014, 80% of the new softwares would be traded on cloud as a service. Those who still have not got the point are marching their way to museums :)

Migrating SQL Server and application server deployments across cloud environments

Big names in the cloud business are Amazon, RackSpace, Azure (which is still catching up) and others. For those who don't know, Amazon makes more money from it's cloud business than selling books on the web. Another interesting fact is that Amazon had built its vast infrastructure for its e-commerce based business and then started optimally using the same infrastructure by using the same for its cloud business. Google seems to be following the same path, and recently joined the cloud business. Like other cloud players including Azure, google offers cloud services like Google App Engine, Google Compute Engine, Google Cloud Storage, and Google BigQuery. The talk of the town and star of the fleet is IaaS offering of Google, i.e. Google Compute Engine, which is powered using 600,000 cores and can process queries in parallel on such huge computing power using its Exacycle technology. Its still in beta and to the best of my knowledge it supports just Linux based VMs on KVM workloads.

This news is interesting to learn, but how does it affect us ? Systems Portability from one cloud to another cloud is a key factor in considering any cloud environment. At times a part of the system architecture might needs to be ported to any particular cloud environment due to its suitability for the architecture for reasons like geographic location, services offered, pricing, proximity with the application hosting environment etc. Managing SQL Server instances and deployments across multiple clouds is a part of this problem statement.

Like Data Market available on the Azure platform, there also exists a cloud marketplace. Any business can become huge with the right idea, one should just have an eye to figure out that idea. System or software configuration is typically considered a DBA or Server admin or side activity. Can you think of a business just out of it ? Can you think of migrating a part of full solution from one cloud environment to another in a matter of minutes or hours ? Can you think of migrating your application and/or database servers from private cloud to public cloud and vice versa in a matter of minutes or hours ?

If the answer is "not really", RightScale is a company that one should take a look. This company deals in the business of providing ready server templates which are basically configuration and scripts that you can readily submit to the cloud vendors, and your server along with software installation would get setup in minutes. Almost every standard and non-standard server based technologies as well as operating systems are supported. So with these server templates, one does not have to worry about the targeted cloud environment. A sever template having any particular configuration can be applied to Amazon cloud environment in the same way as it can be applied to Google cloud environment. Clients are abstracted from the details of converting the configuration syntax, methods and operational details from one cloud environment to another. The potential of this such companies can be capped from the fact that even companies like Google depend on such third-party cloud translators to help clients smoothly migrate from one cloud environment to another.

Supported cloud environment include: Amazon Web Services, Data Pipe, Google Cloud Platform, IDCF - Yahoo Japan, Korea Telecom, LogicWorks, Microsoft Azure, RackSpace, SoftLayer, Tata and others private cloud environments. Server portability along with software installation, configuration, monitoring, as well as auditing across such a huge variety of cloud environments is a silver bullet for any cloud based architecture design for various aspects. It opens up new doors that can't be even thought of if this cloud portability is not on the table. Still challenges like data migration/replication/synchronization have to be dealt with, which has another solutions too.

For a better Azure adoption, it is very necessary to have such translation mechanism inherently available on the Azure platform itself. There are many players just in this business of selling configurations and images. They are not visible to Architects till you fall in need of them in time of crisis to migrate your server and systems deployments from one cloud to another. 

Data warehouse certification , Business Intelligence and Analytics certification

In Data warehousing and Analytics, lack of standard certifications have made it very hard for recruiters to identify "A" league professionals from impostors. When the topic of certification comes, there are basically two questions, why to get certified and how and what to get certified. I would first answer what and then answer why. 

Irrespective of technology, The Data Warehousing Institute (TDWI) has been a provider of Certified Business Intelligence Professional (CBIP) certification where professionals can choose their core skills and appear for the exam that suits the same. The flip side that professionals see to it is that it's not associated with any product in specific and just theory can be good at Architect profile but at career levels where professionals need to implement solutions this might not be able to impress recruiters with this certifications.

Being a Microsoft patron, I would discuss about certifications related to MS BI. Microsoft has recently introduced two certifications for Data Warehousing and Business Intelligence.

1) Exam 70-463 : Implementing a Data Warehouse with Microsoft SQL Server 2012

Skills Measured:

• Design and implement dimensions
• Design and implement fact tables
• Define connection managers
• Design data flow
• Implement data flow
• Manage SSIS package execution
• Implement script tasks in SSIS
• Design control flow
• Implement package logic by using SSIS variables and parameters
• Implement control flow
• Implement data load options
• Implement script components in SSIS
• Troubleshoot data integration issues
• Install and maintain SSIS components
• Implement auditing, logging, and event handling
• Deploy SSIS solutions
• Configure SSIS security settings
• Install and maintain Data Quality Services
• Implement master data management solutions
• Create a data quality project to clean data

2) Exam 70-467 : Designing Business Intelligence Solutions with Microsoft SQL Server 2012

Skills Measured:

Keep in view this exam has approx 30% weightage on designing and planning of BI Infrastructure.

• Plan for performance
• Plan for scalability
• Plan and manage upgrades
• Maintain server health
• Design a security strategy
• Design a SQL partitioning strategy
• Design a backup strategy
• Design a logging and auditing strategy
• Design a Reporting Services dataset
• Manage Microsoft Excel Services/Reporting for SharePoint
• Design a data acquisition strategy
• Plan and manage reporting services configuration
• Design BI reporting solution architecture
• Design the data warehouse
• Design a schema
• Design cube architecture
• Design fact tables
• Design BI semantic models
• Design and create MDX calculations
• Design SSIS package execution
• Plan to deploy SSIS solutions
• Design package configurations for SSIS packages

Why to get certified on Business Intelligence platform ? There are two groups of people divided by faith in certification, one who believes in certifying and benchmarking their skills, others who do not feel there is any value in investing and getting certified.

1) For those who are not in the favor of certifications, the prominent reasons are:

• Product version changes after few years and the certification is seen as outdated by recruiters
• Software required for the same for practicing is not available easily
• Many professionals indulge into malpractices, use exam dumps and pass exams with full score even without any knowledge of the subject
• Its hard for them to self-study and prepare for certifications, and they don't have enough resources to invest into a professional training programme.

2) For those who are in the favor of certifications, the prominent reasons are:

• Certifying your skills with changing product versions reflects your attitude to your employer, about how seriously you take your skills that earns your bread and butter.
• With cheap developer editions coupled with free virtualization software like VirtualBox and readily installed images in VHD format, resource management is possible for them.
• Professionals who pass exam using corrupt methods are digging a backfire gunshot for themselves as they are raising expectations from them, and inviting their interviewer to screen them more thoroughly as they are certified professionals.
• Those who can't self-study in IT and can't manage in investing for resources and training programmes for themselves, have already surrendered to the thought that one or other day they would go obsolete in technology. And delivery management or other avenues are right for them than remaining technical. Even in that area, certifications like ITIL or PMP or PGMP would be required.
• Certifications adds bargain power to your resume to negotiate better for your skills as you go up the ladder. With a couple of years of experience and few certifications you can't ask for 1.5 times the salary that your peers get paid, but the benefit is when your experience meter hits double digit in number of years, you won't be appearing your first ever certification at that age and experience !!

Summary: Invest in your career. Many would ask what certifications have I appeared till date. I am certified MCSD in Microsoft .Net, MCTS in SQL Server Implementation and Maintenance, MCTS in Business Intelligence and MCTS in Performancepoint Office Applications and now working as a senior architect with Accenture Services Private Ltd at Mumbai office. Still I am interested and positive to take up above mentioned certifications.

Data Integration Services on SQL Azure platform

They say that any knowledge never goes waste, but in IT parlance this saying can be redefined as any knowledge or DATA never goes waste. In my career till date, my experience has been that any successful business would have lot of external data processing as a part of its business functions. Competitive Sales Intelligence for example is one of the category of data that many IT organizations would keep of processing to defines best analytical insights for their sales team. Companies like Facebook and Google accumulate hoards of data and makes a fortune out of advertising business. But even these giants depend on external data providers for their business functions. Example of one such data provider is Factual, that provides data to Facebook. Monetizing on carefully curated and certified business databases has become a very big business.

Cloud takes this platform of sharing and trading data one step ahead. Windows Azure Marketplace provides DataMarket for the same purpose where applications can share and trade data.

1) Private cloud and public cloud comes into question, as organizations might want to share their datasets but limited to the scope of the organization only. Microsoft codename "Data Hub" claims to provide a flavor of managed self-service enterprise data integration on the cloud, which generally takes a huge team and data centers to serve the same needs of an organization. This platform is expected to provide private data-market to enterprises which can be very interesting in terms of agility and cost-savings.

2) Any sizeable organization would generate and consume lot of internal as well as external data. Integration and sharing of data is implementation of the solution after the source of data has been recognized. Data discovery from within and outside organization for business needs, is a bigger challenge in itself. Microsoft Codename "Data Explorer" can be seen as self-service SSIS on Azure platform. It provides data discovery from the windows azure marketplace as well as provides features for self-service data mashups from a variety of standard data sources. Hadoop is not yet included in the supported data source list, but if its gets included in the future, this platform can reap immense value and can acts organizations private Google blended with SSIS to create self-service data mashups and again publish the same as a source of data using Data Hub.

Power of Hadoop combined with cloud based tools like Data Hub and Data Explorer can generate business for lot of data providers as well as bring immense value to organizations. Also it would enable better use of data and provide cost-savings in enterprise data integration.