Before You Trust an AI Agent, Trust Your Data Layer

Abstract illustrated stacked data layers in blue and red representing the AI agent data layer foundation for ITSM trust

Summary

The data layer is why AI agents in ITSM get overridden – not the model. Salil Kulkarni has watched the pattern repeat across hotel revenue management, casino operations, and enterprise IT: a capable model makes a defensible recommendation, a practitioner overrides it, and the reason, when traced back, is always the same – the data the model acted on had already gone stale, or was missing a relationship that any informed human would have caught. His argument is that confidence in an AI agent’s output should never outpace confidence in the data behind it, and he sets out four conditions a data layer must meet before an AI agent earns the right to act rather than merely advise.

I have run across this situation many times over years of managing large scale hospitality technology operations: reviewing the overnight revenue management report, seeing the pricing system make a series of calls that look reasonable on the surface, with occupancy projections healthy and competitive rate signals clean, and yet finding that the revenue manager, someone with twenty years of experience reading a property, has already overridden three of the recommendations before the morning standup. It comes down to data layer issues.

Why Data Quality Is the Foundation of Successful AI Agents

I have asked why enough times to know the pattern, and the answer is rarely a math problem with the model itself. It is that the data feeding the model was stale, incomplete, or disconnected from something that happened on the floor the previous night, whether a group booking that had not yet been confirmed in the system, a rate agreement extended verbally but not yet recorded, or a competitor property that went under renovation at midnight. The model performed correctly in each case, but it was operating on a version of reality that had already moved past, which is how I have come to think about confidence in the outcome outpacing confidence in the data behind it.

I have carried that observation into every technology leadership role since, including my time as EVP and CIO at Caesars Entertainment and Las Vegas Sands, and prior at John Harland and Equifax. I have watched the pattern hold whether I was managing technology platforms for hotel inventory, financial risk models, or, now, artificial intelligence (AI) agents within IT service management (ITSM) and IT asset management (ITAM) workflows. Once my confidence in an outcome outruns my confidence in the data behind it, I start overriding the system myself because the data layer has not earned my trust, even though the system is “functioning.”

Trust Your Data layer

AI Doesn’t Fail – Poor Data Does

In my experience, the AI agent issue in ITSM is a data-layer issue more than a model issue, and I have seen organizations reach it the same way each time.

A team deploys an AI agent to handle first-line incident triage, change-impact assessment, and asset lifecycle recommendations. Early results are promising, with the model spotting patterns, reducing mean time to resolution for certain incident categories, and freeing up practitioner time. Then the exceptions begin to accumulate, one at a time. An AI agent recommends a change window for a system decommissioned in the last refresh cycle but never removed from the configuration management database (CMDB), another routes an incident to an owner who transferred departments six months ago, and a third escalates on a dependency relationship that was accurate as of the last discovery scan but reflects a configuration that has since changed.

Each failure looks small on its own, but together, as I have observed across multiple deployments, they produce something more corrosive than a visible outage: a slow, institutional loss of confidence in the agent’s judgment. Practitioners build informal workarounds, leaders treat agent recommendations as first drafts to review rather than actions to take, and the override becomes policy. What I set out to build as agentic automation eventually becomes a manual process with an expensive interface bolted on the front.

The Warning Signs Your Data Layer Isn’t AI Ready

I saw the same dynamic play out at a casino resort company I worked for, on the revenue management side rather than in ITSM. The predictive analytics, revenue management, and profit optimization platforms would generate pricing recommendations for hotel rooms, and, in the vast majority of cases, executives overrode them anyway. That override was not an occasional judgment call. It was constant, creating ongoing friction between Finance, Analytics, and Revenue Management, each with its own view of what the number should be and why the system had gotten it wrong. The CEO, meanwhile, never had a clear answer to a simple question: were we actually optimized? Every competing property across the market was wrestling with the identical issue. Hence, the ground we lost was relative rather than absolute, and it was easy to look at the competitive set and conclude we were fine. But relative parity is not the same as being the best we could have been, and I have never been comfortable mistaking the first for the second.

The important understanding to recognize is that the AI agent did not fail in these cases. The data layer failed the AI agent.

Four Conditions a Data Layer Must Meet

Having watched this pattern play out across multiple organizations and industries, I have landed on four things I believe a data layer must do before I can trust an AI agent to act on it.

1. Currency, measured at the point of decision

I have learned that the most relevant question is whether it reflects the environment’s state at the moment the AI agent acts. Today, it typically only reflects when a record was last updated and that gap is where I have seen most AI agent decisions go wrong. A CMDB refreshed nightly may be accurate enough for reporting and adequate for my own human review. However, it is not accurate enough for an AI agent making a real-time change impact call. I look for a data layer that carries a point of synchronization along with every value, noting when that state was last verified, by which source, and under which conditions.

2. Completeness across the decision-relevant relationships

I have found that an AI agent making an incident-routing decision needs more than just a list of assets. It needs ownership, dependency, service context, and blast radius, and an incomplete relationship map does not tend to surface as an obvious error but instead produces a recommendation that is technically defensible and contextually wrong. I have come to believe that the objects alone are not enough, and that the connections between them need the same discipline.

3. Explainability at the point of action

When I override an AI agent’s recommendation, I treat that override as diagnostic. If I cannot trace which record, which relationship, or which data point triggered it, I have no way to close the feedback loop between AI agent behavior and data quality. I require a data layer that makes its own provenance legible, so when the AI agent acts on it, I can follow the reasoning as a traceable chain rather than take it on faith.

4. Governance that travels with the data

I have seen AI agents operate in regulated environments, during change freeze periods, and in contexts where certain configuration items have specific handling requirements. In these cases, I want the AI agent to always learn these constraints from the data itself, not from a separate policy layer it may never consult. When I embed governance in the data layer rather than bolt it on at the point of human review, I find the AI agent can carry that context into its own recommendations without requiring me to intercept every decision.

What Trusted Runtime Truth Means for ITSM and ITAM

What I am describing is an operating standard I believe the data layer must meet before I can trust agentic automation to act rather than merely advise.

At Virima, I call this Trusted Runtime Truth: live, explainable, policy-aware runtime data that gives AI agents the ground truth they need to act safely, and I hold this standard true regardless of which platform delivers it. The question to ask before extending AI agent authority is whether the data the AI agent works from has earned the right to be acted on.

Building AI Agent Trust Through Better Data

Organizations where agentic AI struggles are those where capable models sit atop a data layer that was not ready for them, and the pattern repeats often enough that I no longer look for the fix in the model.

I do not treat the revenue manager who overrides the system as the issue. I treat that override as information. Am I listening at the level of the data or explaining it away at the level of the outcome?

Data Layer and AI Agent FAQs

Why do AI agents fail in ITSM, according to Salil Kulkarni?

Rarely because of the model itself. Usually, it’s because the data feeding the AI agent was stale, incomplete, or disconnected from something that had already happened. Hence, the AI agent acts on a version of reality that’s already moved past.

What does a data layer need to give an AI agent to be trustworthy?

Salil Kulkarni names four conditions: currency measured at the point of decision, completeness across decision-relevant relationships like ownership and dependency, explainability that lets someone trace a recommendation back to the record that triggered it, and governance that travels with the data itself rather than sitting in a separate policy layer.

Why isn’t a nightly-refreshed CMDB good enough for AI agent decisions?

Because it typically only reflects when a record was last updated, not whether it matches the environment’s actual state at the moment the AI agent acts. That gap is fine for human review but not for a real-time change-impact call.

What is “Trusted Runtime Truth”?

It’s Salil Kulkarni’s term for live, explainable, policy-aware runtime data that gives an AI agent the ground truth it needs to act safely, a standard he says holds regardless of which platform delivers it.

Does this mean AI agents in ITSM don’t work?

No. Salil Kulkarni argues that AI agents struggle because the data layer beneath them wasn’t ready, not because the model is flawed. Fix the data layer, and the AI agent can be trusted to act rather than just advise.

What should you check before extending an AI agent’s authority?

Whether the data the AI agent works from has earned the right to be acted on, based on the four conditions above, rather than assuming a functioning system means trustworthy data.

Further Reading

Salil Kulkarni
Salil Kulkarni
Chief Strategy and Information Officer at Virima

Salil Kulkarni is Chief Strategy and Information Officer at Virima. He has previously served as EVP and CIO at Caesars Entertainment, CIO at Las Vegas Sands, and in C-Level roles at Equifax and John Harland, and other publicly and privately traded companies. He serves as a Board member and advisor to a number of AI-focused technology companies.

LinkedIn: https://www.linkedin.com/in/kulkarnisalil/

Want ITSM best practice and advice delivered directly to your inbox? Why not sign up for our newsletter? This way you won't miss any of the latest ITSM tips and tricks.

nl subscribe strip imgage

More Topics to Explore

Leave a Reply

Your email address will not be published. Required fields are marked *