Three Myths of Agentic BI – CDO Magazine
Events
Topics
What Is a Chief Data Officer in 2026?
Many people working within the data industry are saying the same thing: the role of the Chief Data Officer (CDO) has changed – but how? CDO Maga…
Generative + Agentic AI: Moving From Experimentation to Enterprise Scale Generative and agentic AI are making their way into enterprise production, bringing new questions for Chief Data Officers (CDOs) and other data and AI…
Webinar | The Agentic Mindset Shift: How to Onboard an Agentic Data Team Three sessions, real playbooks, proven results you can replicate.
What Is a Chief Data Officer in 2026? Many people working within the data industry are saying the same thing: the role of the Chief Data Officer (CDO) has changed – but how? CDO Maga…
Generative + Agentic AI: Moving From Experimentation to Enterprise Scale Generative and agentic AI are making their way into enterprise production, bringing new questions for Chief Data Officers (CDOs) and other data and AI…
Webinar | The Agentic Mindset Shift: How to Onboard an Agentic Data Team Three sessions, real playbooks, proven results you can replicate.
What Is a Chief Data Officer in 2026? Many people working within the data industry are saying the same thing: the role of the Chief Data Officer (CDO) has changed – but how? CDO Magazine began an expert survey to find out. We recent…
Generative + Agentic AI: Moving From Experimentation to Enterprise Scale Generative and agentic AI are making their way into enterprise production, bringing new questions for Chief Data Officers (CDOs) and other data and AI leaders responsible for making these technologies…
Webinar | The Agentic Mindset Shift: How to Onboard an Agentic Data Team Three sessions, real playbooks, proven results you can replicate.
What Is a Chief Data Officer in 2026? Many people working within the data industry are saying the same thing: the role of the Chief Data Officer (CDO) has changed – but how? CDO Magazine began an expert survey to find out. We recent…
Generative + Agentic AI: Moving From Experimentation to Enterprise Scale Generative and agentic AI are making their way into enterprise production, bringing new questions for Chief Data Officers (CDOs) and other data and AI leaders responsible for making these technologies…
Webinar | The Agentic Mindset Shift: How to Onboard an Agentic Data Team Three sessions, real playbooks, proven results you can replicate.
Artificial Intelligence
Written by:
Ben Ganzfried | Senior Director, Data Platform & Governance, Hungryroot
Updated 8:00 AM EST, November 12, 2026
For the last decade or so, I have probably followed advances in poker AI more closely than most data professionals, primarily because my brother, Sam Ganzfried, built poker agents as a PhD student at Carnegie Mellon. I got a secondhand view of the major breakthroughs made by teams at Carnegie Mellon and the University of Alberta. The core problem these researchers solved was deceptively simple to state: How do you make optimal decisions when you cannot see the full state of the world? That problem explains something important about where agentic AI is headed with respect to BI and why it is not going to arrive the way many people expect.
Before going further, I want to be clear on what I mean by agentic AI and business intelligence. Agentic AI refers to systems capable of perception, reasoning, planning, and autonomous execution across workflows. I am not referring to a person interacting with a ChatGPT or Claude to generate answers but rather an autonomous bot that takes action. Business intelligence is the process of turning organizational data into decisions, which means identifying the right question, retrieving the relevant data, interpreting the result in business context, and knowing that the answer is trustworthy enough to act on. The distinction matters because conflating these concepts leads to three fairly widespread myths about BI that could prove costly and risky for many organizations in the years ahead. For chief data officers (CDOs) evaluating investments in agentic BI, understanding these distinctions is essential to setting realistic expectations and determining where automation can deliver value.
Robotic process automation (RPA) is a successful solution for back-office automation for specific types of problems, namely deterministic, high-volume workflows where the inputs are known and the logic is fixed. Unlike poker, where you never know exactly what cards your opponents hold, RPA successfully tackles perfect information games where the rules are logic-based and a correct answer exists that any carefully developed system can find.
In those workflows, the answer should always be the same, but BI doesn’t work that way. The question “Why did revenue drop in Q3?” does not have an answer that is always the same. It has an answer that depends on which metric definition you use, which upstream table you trust, what business context is available, and what an analyst already knows about the data quality issues in that pipeline. What works for solving perfect information games will not work for states of imperfect information.
Many organizations do need better data. In my experience and in conversations with data peers, what often gets overlooked is that even a rigorously maintained semantic layer and knowledge graph will not unlock BI agents in any real sense on their own. Foundations matter enormously, but they are not the panacea some believe them to be.
The breakthrough that allowed poker AI agents to beat the world’s best human players wasn’t better data. It was a fundamentally different approach to reasoning under uncertainty. One of the key advances was the counterfactual regret minimization algorithm, which essentially meant playing millions of hands, accumulating regret at each decision point where a different choice would have produced a better outcome, and gradually shifting away from those choices. A further advance was the ability of a poker agent to identify its own strategic weaknesses during play and patch them autonomously. It was modeling its own uncertainty and adapting to it in real time.
A good analyst does something structurally similar. When asked why Q3 revenue dropped, they don’t return the most statistically available explanation. They hold multiple hypotheses simultaneously. They notice when two upstream tables disagree. They ask whether finance and product teams are using the same metric definition before concluding anything. They say, “Here’s what I think happened, but let me verify the source before you take this to the board of directors.”
I saw this at scale in my prior work at Wayfair. At the time, our BI stack was serving more than a million queries per week, but because data marts had developed over many years around individual teams, the same named KPI was sometimes calculated several different ways, and reports that should have agreed did not. Each number was defensible by its own definition, which is why nobody could see the problem inside any single dashboard.
Identifying the problem required people who knew the history of each pipeline. They spent hours in alignment meetings reconciling discrepancies, renaming or redefining metrics as necessary, and communicating changes to their stakeholder teams and leadership. The solution was a single governed semantic layer and, just as important, governance over how many teams could define or change a metric.
Leading conversational analytics tools, including Databricks AI/BI Genie and Snowflake’s Cortex Analyst, currently let users rate answers, request reviews, and save corrected queries as verified examples that guide future responses. This is the right instinct. But based on what these vendors publicly document, these loops capture the outcome signal, which is not the same as the reasoning signal. A thumbs-down tells the system the answer was bad. It doesn’t identify which assumption was wrong, which table was stale at that point in time, which metric definition the agent chose when three were available, or which other answers from last week rested on the same flawed reasoning and went unchallenged.
Sophisticated regret tracking in BI would involve a system that maintains a full decision trace, meaning an auditable record of what it did (including tables queried, metric definitions chosen, data freshness, and assumptions made) rather than the model’s internal reasoning, which remains hard to inspect. When a correction arrives, the system asks whether — across all the answers it has given under similar conditions — some should also be revisited. That is the difference between a feedback loop and a learning system.
Among the most advanced organizations that I know of, the state of the art in BI as of late 2026 is conversational analytics, where users can ask natural language questions of databases and get answers. I’ll make a prediction I expect to hold: Conversational analytics is not an intermediate state on the way to full agentic replacement of analysts. Framing it that way understates what conversational analytics actually is and overstates what agentic BI is likely to become for most organizations.
Conversational analytics is useful precisely because it preserves the human in the interpretive loop. The analyst, or really anyone in an organization evaluating an answer from a database, still decides whether the result makes sense. That human checkpoint is not a limitation of the technology. It is the appropriate division of labor for a problem that is structurally one of imperfect information. The mythical North Star of full analyst replacement assumes that the judgment layer of BI can eventually be automated away.
Poker might seem to argue the opposite, since poker agents now play without human help. But poker involves imperfect information within a fixed system. The rules never change, the payoff is clear, and an agent can play against itself millions of times to learn which decisions were good. BI offers none of that. Definitions shift, pipelines quietly drop data, nobody can efficiently replay a quarter a million times, and the truth about why revenue fell may never fully arrive. In BI, part of what is hidden is the game itself, and well-modeled uncertainty usually means handing the decision to someone with the context to make it.
None of this rules out agents. Much of an analyst’s work is investigative, such as checking data freshness, reconciling a metric across sources, and testing competing explanations. Agents can increasingly take on these tasks. I see that as conversational analytics getting stronger rather than a step beyond it. The defining feature is not that a human types every question but that a human owns the interpretation and the decision. An agent that reports what it found and where the sources disagree fits that model. An agent that acts on its own conclusion does not.
For most organizations, conversational analytics done well, with a governed semantic layer, a knowledge graph if applicable, clear metric ownership, and pipelines that alert teams to failures, is not a stepping stone. It is the destination.
For CDOs weighing agentic BI investments, I have three suggestions. First, treat the foundations as the product rather than as a prerequisite for a later one. A governed semantic layer, named metric owners, and pipelines with built-in observability make conversational analytics trustworthy now, and any future agent will depend on those same assets.
Second, judge vendors by how their systems actually behave under uncertainty, not by their demos. Ask a question where two of your own metric definitions disagree and see whether the system notices, asks for clarification, or quietly picks one. Then ask what happens after a correction. If the fix only applies to that answer, you are not buying a learning system.
Third, grant autonomy by decision rather than by tool. Reversible, investigative work like monitoring pipelines or drafting a first readout on an anomaly may be appropriate for more autonomy. Anything headed for a board deck, a customer, or a forecast should keep a named human who owns the interpretation. The key question is what it costs if this particular answer is confidently wrong. I anticipate that our collective sense of where that line sits will adjust as these systems improve. But I don’t expect that question to go away.
November 19, 2026
November 19, 2026 | In Person
Cincinnati Leadership Summit
Westin Cincinnati – Downtown
Elevate Your Personal BrandShape the Data Leadership AgendaBuild a Lasting NetworkExchange Knowledge & ExperienceStay Updated & Future-Ready
source
This is a newsfeed from leading technology publications. No additional editorial review has been performed before posting.
