Why Agentic AI Projects Fail: The Data Foundation Problem

Most agentic AI projects don’t die in the model layer. They die somewhere between a half-normalised events table and a customer ID that means four different things across four different systems.

According to dbt’s analysis of failing agentic deployments, the answer to why so many AI initiatives crash and burn is almost always the same: the data. Not the LLM. Not the orchestration framework. The data that was supposed to feed the agent was never production-ready, unified, or trustworthy enough to hand over to an autonomous system.

The Agent Doesn’t Know What Your Data Team Knows

Human analysts carry an invisible layer of institutional knowledge. They know that the user_id field in your Shopee connector is unreliable before a session persists, that the Bangkok warehouse events lag by 40 minutes, or that a spike in null values on Tuesday mornings is a known ETL quirk, not a real signal.

An agentic system has none of that. It reads what’s in the table and acts accordingly. dbt’s Daniel Poppy makes the point plainly: agents that fail tend to be operating on data that was never designed for autonomous consumption. Brittle schemas, undocumented transformations, and inconsistent entity resolution don’t surface as errors — they surface as quietly wrong decisions. In a Southeast Asian context, where customer journeys frequently span LINE, Grab, Lazada, and a brand’s own app within a single purchase cycle, the cross-platform entity resolution problem is acute. Stitching those touchpoints into a coherent profile before an agent touches them isn’t optional infrastructure — it’s the entire prerequisite.

Trust Hierarchies in the Data Layer Are Non-Negotiable

One pattern separating successful agentic deployments from failed ones is the presence of explicit data trust hierarchies — what dbt frames as distinguishing between data an agent is allowed to act on versus data it should only read.

In practice, this means your semantic layer needs to encode confidence. A behavioural event fired from a mobile SDK with full consent and a valid user match should carry a different operational weight than an inferred attribute from a third-party enrichment vendor. If your CDP doesn’t currently surface that distinction, your agentic layer will treat a probabilistic audience segment with the same certainty as a verified transaction record.

The fix isn’t glamorous: documented data contracts at the transformation layer, lineage tracking that agents can query before acting, and tiered activation rules that gate high-stakes decisions behind higher-confidence data. Teams using dbt Core or dbt Cloud already have the scaffolding — the gap is usually in treating those contracts as a product, not a side effect of the pipeline build.

---

Distributed Execution Changes the Query Optimisation Calculus

There’s a quieter architectural shift happening underneath the agentic conversation that’s worth flagging for teams building or rebuilding their data infrastructure. Thomas Reid’s experiment on Towards Data Science — running concurrent SQL across three remote DuckDB instances using Quack — illustrates something practically important: the assumption that your analytical queries run in one place, against one database, is increasingly obsolete.

As brands in Southeast Asia regionalise their data infrastructure to meet data residency requirements (Thailand’s PDPA, Indonesia’s PDP Law, Singapore’s PDPA each carry distinct obligations), query federation becomes less of an engineering curiosity and more of a compliance necessity. What Reid’s experiment surfaces is that concurrent remote execution is achievable with lightweight tooling — but it requires deliberate schema consistency across nodes and careful handling of connection state. An agent firing queries across fragmented regional stores without that consistency will produce results that are locally accurate and globally meaningless.

For CDP architects, the practical implication is this: if regionalisation is on your roadmap, build your semantic layer to abstract the physical data location now, before your agentic layer is trying to reconcile it at runtime.

When the Agent Goes Adversarial — And Why That’s Instructive

Jaemin Han’s piece on Towards Data Science — training an LLM to dynamically attack his Minecraft house — reads like a thought experiment, but it carries a sharp operational insight. The model wasn’t just executing pre-scripted attacks; it was observing state, adapting strategy, and exploiting gaps in real time. The adversarial framing is deliberate: the agent succeeded precisely because it had high-fidelity, real-time environmental data to work with.

Transpose that to a personalisation or retention agent operating across your customer base. An agent with clean, real-time behavioural data can adapt — catch the exit signal before churn, shift the offer before the cart is abandoned. An agent working from batch-processed, 24-hour-old session data is essentially operating blindfolded. The Minecraft analogy sounds absurd until you map it: the gap between a successful agentic deployment and a failed one is often just the freshness and fidelity of the environmental signals the agent can see.

For brands running real-time activation on Shopee or via LINE OA, the stream processing layer — Kafka, Flink, or even Pub/Sub connected to your CDP — isn’t a nice-to-have. It’s what gives your agent something worth acting on.

---

Key Takeaways

• Audit your data trust hierarchy before deploying any agentic layer — agents cannot intuit which data is reliable; that logic must be encoded in your semantic or transformation layer.
• If regional data residency is a factor, abstract physical data location behind a federated semantic layer now, not after your agentic architecture is already in production.
• Real-time signal freshness isn’t a streaming infrastructure vanity project — it’s the variable that most directly determines whether your agent can actually intervene at the right moment.

The brands that will get the most from agentic AI in 2027 are the ones quietly doing unglamorous data engineering work right now — resolving entity graphs, writing data contracts, and building semantic layers that an autonomous system can actually trust. The question worth sitting with: if you handed your current data infrastructure to an agent tomorrow, what decisions would it get quietly, confidently wrong?