How Alation & Snowflake Integrate the Cloud Data Management Capabilities Framework (CDMC)

Most people close to data management are aware of the EDM Council. This highly respected standards & training organization was “created to elevate the practice of data management as a business and operational priority.” As an important standards body, they developed one of the only assessment models. The Data Management Capability Assessment Method (DCAM) framework is used to pressure-test data management procedures and demonstrate good practices to regulators.

In 2019 the EDM Council decided that a new extension for managing sensitive data in the cloud was required, so they created the Cloud Data Management Capability (CDMC) working group. It consists of more than 300 individuals, representing 100 organizations, which include commercial enterprises, technology providers, and consultancies. The working group produced a new Cloud Data Management Framework for sensitive data, which was announced earlier this month.

A key part of the framework is the definition of 14 key controls and capabilities. Before the announcement, the working group needed a test case that proved these controls could be implemented; this is where Snowflake and KPMG stepped in. Snowflake offered to build the test case examples and KPMG offered to create an assessment approach.

At the time Alation was not yet a member of the working group, but because of our strong governance reputation and relationship, Snowflake asked us to participate. As a result, Alation played a key role in building and achieving signoff on all key controls.

Approach & Deliverables

We used an Alation catalog instance to categorize six Snowflake data sources and a Tableau Server. The Snowflake data sources were multi-cloud (Azure, AWS, GCP) running in different regions around the world.

We configured the catalog so Snowflake, Alation, and KPMG could collaborate, using it as our shared project environment. We recorded the results (evidence) of each control, and KPMG used those results to do their assessment. As part of the process, we also made heavy use of conversations so work could continue asynchronously.

We used every part of the Alation catalog, including Alation Analytics, APIs, and the new Policy Center capability. We also used several integrations that demonstrated the flexibility and power of the catalog platform for governance-specific tasks.

Key Accomplishments and Design Patterns that Support CDMC

The CDMC key controls and capabilities call for the implementation of necessary but very challenging requirements. These include topics such as data sovereignty and cross-border data movement, privacy impact assessments, security controls, and data sharing agreements, just to name a few.

Not only were we able to accomplish all of these successfully (as verified by KPMG) but we were able to establish and extend the implementation approach using several design patterns that we believe will be leveraged and repeated.

1. Autonomous Policy Bots

We implemented policy bots that monitor specific conditions in the metadata related to the policies and standards. The bots notify & alert stewards, create tasks, and ensure that key processes are being followed.

2. Location Identification and Policy Triggering

We implemented the capability to understand the physical location of cloud data. We tracked the association of that data with consumers through data-sharing agreements. This allowed us to trigger actions when the data changed locations or was used non-compliantly.

3. Policy and Entitlement Integration

We used our ability to bi-directionally sync with Snowflake row-based access control, masking policies, and entitlements to relate those to higher-level governance policies, standards, and the discovery of sensitive data.

4. Measurement, Monitoring, and Analytics

Many of the requirements called for audit reporting, identification of exceptions, and scoring the implementation of the standards. For these, we leveraged our built-in data mart Alation Analytics and our integrated query tool Compose. We used them to create template audit reports that were embedded in steward workspaces for ready access and alerting.

These and more will be covered in detail as part of the blog series.

Closing the Gap Between Requirements and Implementation Decisions

As with most standards, CDMC describes the objective but leaves most of the details related to scope and phasing up to each organization. That left us needing to define our own assumptions and approach based on practical experience from working with our customers. We also created a running commentary of things we think every customer would want to consider as they implement. Our hope is that this will help accelerate everyone’s thought process and planning as they embark on applying standards governing sensitive data in cloud/multi-cloud and hybrid architectures.

What’s Next?

We (Alation, Snowflake, and KPMG) are working on the final document that describes and shows examples of our approach to each control. Our understanding is that the EDM Council will make that freely available, so stay tuned.

As of now, we view the work to date as only an exciting starting point. There are tremendous possibilities to advance governance processes, approaches, and techniques in the future. We are looking forward to collaborating with members of the working group and continuing to revise and advance our best practices for the CDMC key controls and beyond.