The demo always works. It works because the demo tenant has clean data: every account maps to a person, every person has a manager, every job title means what it says, and every termination arrived on the day it happened.
Then the capability meets your environment, and the recommendations come back subtly wrong in ways nobody can quite explain. Someone can spend weeks tracing it and find the actual cause: three systems disagree about who a given population reports to, and the model picked one of them.
This is the least discussed problem in identity right now, and it is the one most likely to determine whether an AI investment returns anything. Every AI-assisted identity capability on the market, whether it recommends access, detects outliers, mines roles, or summarizes a certification, is reasoning over your identity data. Without a deliberate check for it, they cannot tell the difference between a pattern and an artifact. (Who gets to touch that data is a separate question, and I have written about the questions to ask an IAM consultant who uses AI. This piece is about whether the data is true.)
Identity data is bad for structural reasons
It is worth being clear that this is not a story about careless administrators. Identity data is difficult in ways that are baked into how organizations actually work.
HR systems are systems of record for employment, not for identity. They were built to run payroll, manage benefits, and satisfy labor reporting. The fact that they became the authoritative source for identity is a historical accident, and it shows in what they capture and what they do not.
Large parts of your population are frequently not in HR at all. Contractors, contingent staff, vendor technicians, and temporary workers often arrive through a different process, or no process. In healthcare this is especially pronounced, where physicians with privileges may be credentialed rather than employed, and where an identity program built only around the HR feed will simply not see them.
Mergers leave behind multiple sources that each legitimately claim primacy for their own population. Reorganizations change what department codes and cost centers mean without changing the codes themselves, so historical data quietly stops meaning what it used to. And many feeds are snapshots rather than event streams, which means you learn about a change on the next scheduled run rather than when it occurred.
None of this is anyone's fault. All of it shapes what a model will conclude.
The five things that actually break
Correlation. The foundational question is whether every account in every connected system maps to a known identity. In practice a meaningful share never do, and they accumulate into a pile of orphaned accounts that gets deferred from project to project. This matters more under AI than it did before. A recommendation engine reasoning about what people in a given role typically hold is working from the accounts it can see. Accounts it cannot attribute are simply absent from its picture, and it has no mechanism to tell you that its view is partial. NIST SP 800-53 treats this as a control for good reason: AC-2 governs account management, and its enhancement AC-2(3) requires disabling accounts that are no longer associated with a user or individual.
Accounts that will never have a person behind them. Service accounts, application identities, shared clinical workstations, and break-glass credentials have no HR record by design and never will. They sit permanently in the unmatched population, quietly distorting every correlation figure that includes them, which is why the orphan number and the never-had-an-owner number need to be tracked separately. They also break the assumption sitting underneath most access recommendations, which is that similar people should hold similar access. A service account is not similar to a person in any way a model can use. A system that cannot distinguish the two will reason about both with equal confidence.
Authority conflicts. When two sources disagree about an attribute, precedence is usually settled by a rule someone wrote during the original implementation and nobody has revisited since. The manager attribute is the one that hurts most, because it drives approval routing and certification assignment. If the wrong manager is authoritative for part of your population, then every access request those people submit is approved by someone without the context to judge it, and every certification they appear in is reviewed by the wrong person. That is a governance failure, and it is invisible until someone goes looking.
Lifecycle event fidelity. Terminations whose effective dates move after the fact. Rehires that enter as brand new identities, orphaning prior access. Leaves of absence that look like terminations to a rule that only reads status. Internal transfers processed as a termination plus a hire, which strips and re-grants access and leaves a gap or an overlap depending on timing. Each of these produces a wrong automated action in your joiner, mover and leaver processes. Under HIPAA the termination case in particular is not a matter of operational preference. Termination procedures, 45 CFR 164.308(a)(3)(ii)(C), are an addressable implementation specification under the required Workforce Security standard, and addressable does not mean optional: you implement them, or you document why they are not reasonable and appropriate for your environment and implement an equivalent alternative measure where that is reasonable and appropriate (45 CFR 164.306(d)(3)).
Entitlement semantics. Your entitlement catalog records what permissions are named, not what they allow. A group called FIN-RPT-USERS may grant read access to a reporting tool, or it may have been repurposed three years ago to also grant write access to the underlying data. Anything reasoning over entitlement names is reasoning over labels, and labels drift away from behavior over time.
What the feed never sees
Even a perfect HR feed only governs the access that flows through it. A good deal of access does not. Administrators grant it directly in the target system to close a ticket. Emergency access is granted and never removed. Access granted for a project outlives the project. Every transfer adds the new job's access and rarely removes the old, so long-tenured people slowly accrete entitlements no current role explains.
None of that produces an error. It produces a population that looks normal, and that is the problem for AI. A model that learns "similar people hold similar access" from accreted access learns the accretion, and recommends it to the next person in the same job. And when the governed path is slow, people take the ungoverned one, so the gap widens on its own.
Why AI makes this worse rather than better
Here is the part that deserves the most attention, because it inverts the usual assumption.
Traditional rule-based automation fails loudly when the data it needs is missing. A provisioning rule that cannot resolve a manager throws an error. The job fails, someone gets paged, and the underlying data problem surfaces because the system refused to proceed without it. The failure is annoying, and it is also a control.
A model does not fail that way. It is built to produce an output, and it will produce one from whatever it is given. Ambiguous or contradictory input yields a confident, plausible, well-formatted answer with no error state attached. The data problem does not surface. It gets absorbed into a recommendation that looks exactly like every correct recommendation.
Scale compounds it. A broken rule affects the population it matches, which is usually bounded and eventually noticed. A recommendation model touches every decision in its scope, and it arrives with an implied authority that reviewers respond to. Given a suggestion, most reviewers accept it, which is the same rubber-stamping dynamic that access certifications have struggled with for twenty years, now with something that appears more objective sitting behind it.
Then the errors set. Approved recommendations become precedent. They shape what the next recommendation looks like, they get cited as evidence that the access is appropriate, and they are pointed to during audits as the basis for a decision. A wrong answer that gets approved stops looking like a wrong answer and starts looking like policy.
What ready actually means
The useful exercise is not evaluating AI capabilities. It is establishing whether your identity data can support them, and the answer is measurable.
Know your correlation coverage. What percentage of accounts in each connected source maps to a known identity? The specific number matters less than whether you can produce it on request. An organization that cannot state its correlation rate per source does not yet know what its identity program can see.
Resolve the orphan pile. Every unmatched account either gets an owner or gets closed. A permanent orphan population is a decision your organization has made by default rather than on purpose.
Declare one authority per attribute, not per system. Write it down. Manager, department, employment status, and termination date may each legitimately come from different places, and precedence should be an explicit, reviewed decision rather than an artifact of implementation order.
Measure your lifecycle detection lag. How long after an actual termination does your identity platform know about it? That is a number, it is usually worse than people expect, and it is the single most audit-relevant figure in this list.
Put the non-employee population in scope. If contractors, vendors, and affiliates are not in the feed, you do not have an identity program. You have an employee identity program, and the gap is exactly where unreviewed standing access tends to live. Non-employee lifecycle management is its own discipline for that reason.
Make entitlement descriptions say what access does. Not what it is called. This is tedious, it is the prerequisite for meaningful certification regardless of AI, and it is the input quality ceiling for anything that reasons about entitlements.
The unglamorous conclusion
None of this is interesting work. It is correlation rules, source precedence, event handling, and entitlement documentation, and it does not demo well.
It is also the entire job. Organizations that do it get real value from AI-assisted identity capabilities, because the models are reasoning over something true. Organizations that skip it have purchased a faster and more confident way to be wrong, and they will not find out for a while.
The genuinely good news is that none of this work is contingent on buying anything. Correlation coverage, clear authority, accurate lifecycle events, and entitlement descriptions that mean something are the same foundations that make certifications defensible, audits survivable, and access requests route to someone who can actually judge them. That value exists whether or not an AI capability ever enters the picture.
Fix the feed first. Everything else in identity gets easier, and the AI question answers itself.