SQL Server Data Quality Services Architecture Video + My thoughts

SQL Server Denali is going to comes up with a new service, known as Data Quality Services (SSDQS). Knowing the architecture, targeted use, integration options, cost of ownership, developer friendliness etc are the fitment tests that a product needs to pass before architects give a green flag from the initial fitment test. Principal Group Program Manager of DQS gave a detailed demonstration at Tech Ed, on the developments happening in the Microsoft SQL Server factory for DQS right from architecture discussion to different detailed demos. I am not going to reiterate the same here in this post, but you can download the video from here.

Some of my thoughts after going through this video on DQS are as follows:

1) I am a big admirer of SSIS, so I was looking forward on how SSIS can hook into DQS. As we already saw in CTP 1 of Denali that SSIS would have a transform named SSIS Data Quality Client (at least as of now and to the best of my knowledge). This would behave more or less in the same way Fuzzy lookup or web services are used with SSIS. Whatever that happens inside DQ Server should be transparent to SSIS.

2) DQS has DQ Domains, Domain Rules, Domain Management, Knowledge Base, Knowledge Discovery, DQ Projects, DQ Client, Profiling, Notification etc as its core components in its architecture. These knowledge base are like Schema with rules and attributes and each knowledge base would fall under a domain. Dataset would be passed through these reference data / knowledge bases, which would identify and rectify data that violates the rules of the knowledge base, and at the same time enrich the knowledge base. This is my summarized version, if you get confused, just follow the video.

3) SQL Azure Data Market Place is presently and potentially the most promising client as well as customer for DQS, in my personal opinion. Data Quality market is already having it's leaders and DQS might take some time to reserve a seat in that league and gain customer confidence.

4) DQS uses reference data sets and knowledge base rules, as a part of the data quality setup. It would be interesting to see if Master Data Services can bring in some help in this area. If MDS can be integrated with DQS, and that is turn can be hooked up from SSIS, it would make a good integration design. Else functionally I see some kind of overlap between MDS and Domain knowledge / Reference data setup for DQS, though the intended purpose of both the tools / services are completely different.

There's a lot to come and go before SQL Server Denali goes RTM. So till then stay tuned and enjoy the thrill !