Tales from the Trenches: Why Building Conversational Analytics is Harder Than You Think

In 1880, the US Census Bureau took a glacial 7 years to process data and create a final report. Enter Herman Hollerith and his revolutionary tabulating machine, slashing that time to a mere 18 months for the 1890 census^[*]. An almost 5x improvement that transformed decision-making forever.

Fast forward to today, and the need for speed remains paramount. A deeper dive into metrics when reviewing business performances is so crucial to have in real time to get actionable insights. Stale dashboards and dusty follow ups delivered weeks after the review won’t cut it. This is where conversational analytics comes in, promising frictionless and speedy experience on how we interact with information. 

But hold your horses, building a robust conversational analytics platform is no walk in the park. We, at Numbers Station, have been in the trenches, tackling the challenges head-on, and we want to share our hard-earned wisdom.

From One-Trick Ponies to Conversational Mavericks

Let’s be honest, dashboards have limitations. They are one trick ponies, meaning each insight often needs a new dashboard. A collection of these can become overwhelming information volcanoes, erupting with data that buries users in a pile of irrelevance. Worse, they are time consuming to create and yet often lack the flexibility to answer the specific, nuanced questions business users ask. Even worse, each analyst can create their own dashboards with inconsistent metrics, making it impossible to compare data across teams. We’ve covered this in detail in our previous blog, “Overcoming Self Service BI hurdles with Generative AI“.

Teams care about a handful of North Star Metrics. These are the metrics that truly matter, the ones that tell the story of their wins and losses. But to understand why these metrics fluctuate, or how to optimize them, teams need to evaluate their hypothesis. They need to see sub-metrics and understand if the biggest drivers of change are in line with their hypothesis. 

Take, for example, a marketing team obsessed with growth. Their North Star Metric might be numbers of Marketing Qualified Leads (MQL). But to understand why unique MQLs are dropping, they need to see sub-metrics like bounce rate and average time on page. They need to know which traffic sources are driving the most engaged visitors. Until now, doing this with dashboards was a grueling and inefficient process. Dashboards do not allow the user to focus on a particular hypothesis, and further because of rigidity, users often cannot validate the hypothesis to get to the root cause and/or actionable insight.

Recognizing these limitations, Conversational Analytics emerges as an obvious solution. It transcends the traditional dashboard’s constraints by offering an interactive, flexible platform that facilitates validating different hypotheses. This shift removes the friction in accessing actionable data and spares us from making real world mistakes thus accelerating growth beyond company goals. The benefits of validating hypotheses within data far outweigh the inherent challenges in creating such a sophisticated platform. 

The Hidden Complexities of Conversational Analytics

Building a conversational analytics product requires addressing several critical aspects:

1. The Standardization Enigma:
   
   Imagine different teams using slightly different terms for a given metric, like “revenue” vs. “sales.” This inconsistency creates a data swamp, making it impossible to get a clear picture of what’s going on. To overcome this, it is essential to build semantic layers by mining SQL fragments, business jargon and policies encoded in these dashboards, data reports and data documentations. Further, similar to how your iPhone contacts app suggests merging duplicate entries, the system needs to periodically ask data analysts if metrics and dimensions should be combined and language needs to be standardized.
   
   
2. Decoding Business Context:
   
   Executives rarely ask questions like “Compare sales amount post tax in $ of perfumes sold in Chicago between March 8th and March 25th to 3 weeks before that.” They instead ask about the cause and effect of specific situations like, “ What was the impact of St. Patrick’s Day promotions”. It’s the data analyst’s job to understand the context, what data exists, derive the necessary information, and ask the right question. Conversational analytics needs to understand these nuanced objectives and help users navigate towards the correct question based on the underlying data. 
   
   

3. The Ever-Shifting Data Landscape:
   
   The data landscape within organizations is like the geological record of the earth – constantly shifting and evolving. New data sources are added, and existing ones are modified, in fact user questions on our conversational analytics platform helps prioritize the data sources that need to be included or modified. This means the data model needs to be constantly updated to reflect these changes. The definition of metrics and dimensions needs to be flexible enough to accommodate this evolving environment. This is a continuous challenge that requires robust change management systems within the conversational analytics software.
   

These are just a few of the battles we’ve fought and won, and we are continually advancing. We’ve addressed countless challenges, from overcoming the limitations of text-to-SQL to making our product readily available in the cloud.

Partnering with Numbers Station:

A frictionless experience with data is essential for enabling your organization and customers to take data driven actions. Building this powerful and user-friendly conversational analytics platform is a complex undertaking. By leveraging Numbers Station AI, you can accelerate your journey towards a truly actionable data-driven organization. 

Going from weeks to minutes to formulate and get responses for deeper dive questions is a game-changer, just like Hollerith’s tabulating machine. Let’s embrace the future of data together.

Ready to join the data revolution? Contact us today and let’s discuss how we can help you achieve faster, more insightful decision-making.