Santa Delivers a Meshy Holidata

As Santa and his advisors gathered in their post-beach recap of the 2021 holiday season, one thing grew clear: After a couple of “sleepy” years where everyone stayed home, 2022 was shaping up to be a full-on, in-person holiday — with two years of backed-up expectations on top!

Sure, the past couple of years weren’t easy. Santa had to reinvent everything to go remote. And the supply chain issues were no joke — who can forget the candy cane surplus clogging the North Pole tunnels!

Santa has always been ahead of the game. He was an early big data adopter and just last year made a big move to rein in all that data to deliver the holiday season.

His efforts paid off. The team’s adoption of a data catalog made it possible for the North Pole to deliver a great holiday season despite all the obstacles. Using Alation, they were able to — for the first time! — see across the workshop’s many many different data sources. And they finally had common definitions for key terms like “naughty” and “nice.” They were even sharing queries and dashboards!

Data Demand Surges at the North Pole

But as the reality of a full-blown holiday season set in, my sources at the North Pole reported that things were looking challenging: The good news: data catalog usage exploded with elves in every workshop wanting access to find the data they needed.

The bad news: as each new workshop got ahold of data, they wanted answers to ever tougher questions. What were the top trending toys in Tallahassee? What size wool socks for the kids in the UK? Who’s looking for Lego in Australia? And where is the need greatest for those stellar science and technology kits? Elves also wanted to know more about specific kids. Does the canoe camp kid in Maine who also paints want a new paddle…or paint brushes?

The data-team elves were at their wits’ end: Once everyone knew they could use data to make better decisions, everyone wanted more and more data! My source reported that there were some heated exchanges when the sleigh routing team started requesting data lineage for the naughty and nice lists and the wood toy assembly line started pulling in real-time local weather data to monitor wood supplies. “We’re here to spread joy – not data!” was heard more than once in the workshop.

Santa’s Data Mesh Journey

Luckily, the elven data strategy team was ahead of the curve. They could see, right in front of their pointy noses and in the exploding usage of their data catalog, that the central workshop data team alone wouldn’t be able to keep up with the avalanche of data and data tools. They’d have to recruit more people to create and curate data while also making it easier to find and activate it. This meant rethinking nearly everything about North Pole data management to include:

Domain-oriented decentralized data ownership and architecture.

The concept of one Big List owned by one Big Boss — with data isolated in the central workshop — is as old and creaky as Ranklin/Bass animation. When an outage revealed that nearly every assembly line received a direct feed from the huge inventory system, and with a pandemic-influenced shift to distribute the workshops, decentralization now rules the day!

Data as a product — gift-wrapped or not.

The Big List identifies the naughty and the nice, but each workshop, supply chain, and delivery team uses their own version of the data to manufacture the toys (and where applicable, coal). A eureka moment arrived when a train workshop elf pondered: “What if we treated data as something reusable, reliable, and scalable — like a product!”

Federated computational governance.

“Hold on,” said the veteran data elves. Decentralizing everything will cause a candy-coated mess — or need we remind you of the powdered sugar disaster of 1884?” “Fear not,” said the elf optimization team. “The data governance standards are defined centrally, but we’ll decentralize the work to the individual domain teams to execute independently – but with shared governance guidance!” Federated computational governance is a holiday stocking anyone can wear!

Self-serve data infrastructure as a platform.

If it all went to plan, the data elves would get their Christmas wish. Gone would be the days of lining up outside the data workshop with your data request parchments. Putting the “elf” in “self-service” would reduce the centralized load and allow all data users to satisfy their own requests — faster.

With these bright ideas, the sparkly-eyed data elf architecture team got right to work. Delighted to learn that others had the same idea, they became huge fans of Zhamak Dehghani, absorbing her advice for building a data mesh. Downloads of the “meshy fabric” white paper shot up. Old, pointy-eared friends Sparkle and Frizzle showed up at our mesh-focused webinars. And the Alation marketing team noticed record-high web traffic from the North Pole!

It’s no coincidence that the Snowflake Summit was full of snow. The elves enjoyed the amazing session on democratizing data with mesh from Kayleigh Lavorini of Fifth Third, and plenty of pointy ears were spotted at the Roche Diagnostics and other data mesh sessions.

But putting these ideas into action wouldn’t be easy.

• Data ownership: Elves with the best knowledge of particular data were now responsible for delivering that data to those who needed it. The newly launched domain-oriented decentralized data architecture defined how these teams should produce the data and how to use modern data stack tools, like Fivetran and dbt, to prepare and make the data available to everyone.

• Federated governance: Meanwhile, elven stewards now needed to create the foundation for federated governance. This meant integrating policies and rules to govern data, building trust in its quality, and creating the lineage to show how it was being used.

• Data products: And to enable data consumers to shop for data as a product, the team invested in data literacy training for every workshop. They also made sure their self-service tools were up to snuff and could make use of that federated governance to ensure everyone was using the right data

The good news? They already had the one thing they needed to tie all of this together: their data catalog. The Yule Catalog (yup — Santa and his team are among customers who name their catalog!) was key to bringing order to the data chaos of last year.

Now my pointy-eared sources tell me that Alation is fast becoming the one-stop shop for everyone setting off on this new data journey. Data producers were able to register their data and curate it on their own.

The elf stewards, already immersed in the catalog, were now supercharging their work to document, tag, and curate the flood of new data products. And with new lineage data coming from the data tools like dbt and Fivetran, they are helping everyone see how their data is being used.

Finally, data consumers were rushing to the catalog to get the data they needed. My source didn’t want to divulge the numbers, but did share that catalog access was at the “tippity top of top” of every workshop’s data wish this year.

To make sure the workshop elves could put that data into action, the team upgraded their self-service game. First, they made sure all the teams were using the latest Tableau integrations so teams creating dashboards can see where the data came from. They also were early adopters of Alation Anywhere for Slack to get real-time data answers into project discussions.

One innovation that really warmed the team’s cheeks was the query forms they created to let everyone get data answers directly in Alation. My source told me that this has taken off “like an elf after 15 hot chocolates.”

The result? Data products are flowing, just like the toy production lines. And good little boys and girls around the world can look forward to thoughtful, personalized gifts on Christmas morning!

But as we all know, Santa doesn’t sleep on innovation. Next year, he’s looking to harness the power of DataOps and build even more customized solutions atop the data catalog. The elves — and the children on the nice list — can barely wait!

Wishing you a merry holiday and happy new year!