AI Strategy

Whoever Writes the Eval Owns the Product

The eval set is not a quality artifact. It is the operative specification of an AI product. Whoever curates the failure cases makes the product decisions, regardless of what the PRD says.

Quick take. The eval set is not a quality artifact. It is the operative specification of an AI product. Whoever curates the failure cases makes the product decisions, regardless of what the PRD says. If you cannot name the person responsible for your eval set, you cannot name the person responsible for your product.

The demo is fine. The investor deck is fine. The behavior in staging is fine, most of the time. Production keeps slipping by another sprint because nobody on the team can finish the sentence "the product is correct when ___" without opening a notebook and showing you ten test cases. That is the moment the product has a specification problem, and the specification is not in the PRD.

Most AI product teams have crossed this line without noticing. The demo-to-production stall pattern looks like a confidence problem, then a tooling problem, then a model-choice problem. Each diagnosis points back at the same blind spot. Somewhere in the team, somebody started writing down "what good output looks like" in a file that runs as code. That file is now the spec. The PRD is now a marketing document.

This article is about who owns that file, what they actually own when they own it, and how to make the ownership match the org chart instead of the other way around.

When the spec moves and nobody tells the org chart

Three months into a serious AI product effort, a pattern shows up. The product manager describes the feature in capabilities ("the assistant should summarize legal contracts and flag risk clauses"). The engineer describes the feature in regressions ("we got the contract-length-over-50-pages case back to passing, but Spanish-language inputs broke again"). The two descriptions stop overlapping. They are describing different things.

The PM is describing the marketed product. The engineer is describing the operative product, which is the set of inputs the team has decided count as in-scope, the set of outputs that count as correct, and the set of failure cases the team has decided are tolerable for now. That second description is the eval set. It is a structured artifact, usually living in a Jupyter notebook or a YAML file or a tests/eval/ directory, and it answers the question the PRD does not answer: what is the product supposed to do, with what tolerances, on which inputs.

The eval set is the product specification because the model is the executor. When you write classical software, the engineer reads the PRD and translates intent into code; the engineer is the interpreter. When you build an AI product, the model is the interpreter, and the model interprets the eval set, not the PRD. Every failure case curated into the eval is the team telling the model "this matters, optimize for it." Every failure case left out is the team telling the model "this does not matter." Whoever decides which cases go in and which cases come out is the person specifying the product.

That person is almost never the PM. It is almost never the org-chart product owner. In most AI product teams, it is an ML engineer or a QA lead, picked because they happened to know how to set up an eval harness when the team first needed one. The product authority moved with the eval set. The org chart never updated.

The mechanism: why the model executes the eval, not the PRD

The reason this shift is invisible is that it feels like a tooling decision. Setting up an eval framework looks like the same kind of work as setting up unit tests. You pick a runner. You write a few cases. You wire it into CI. None of that looks like a product-ownership move.

The mechanism that makes it a product-ownership move runs underneath, in three connected ways. First, the eval set defines "correct output." A non-AI product has correctness defined by behavior matching a deterministic spec. An AI product has correctness defined by output passing a set of eval cases at a chosen threshold. Below the threshold the product fails; above it the product ships. Whoever picks the cases picks the definition of correctness.

Second, the eval set defines the optimization target. A team using eval-driven workflows iterates by changing prompts, fine-tuning, or swapping models, then running the eval. The change ships if the score goes up. The score is the eval set. Every iteration optimizes against whatever the eval includes, and silently sacrifices anything the eval excludes. The Spanish-language contract case left out of the eval gets worse with every shipped iteration, because nothing in the optimization loop is looking at it.

Third, the eval set defines the acceptable-failure boundary. Every AI product has a probability distribution of bad outputs. The eval set is where the team writes down which bad outputs are tolerable, which are unacceptable, and at what rate. The PRD might say "the assistant should not give legal advice"; the eval set is where someone has decided whether "this looks like it could be construed as legal advice if read uncharitably" counts as a failure or a pass. That decision is product policy. It has legal implications. It has trust implications. It is usually being made by whoever wrote the eval, which is usually not whoever the org chart has decided is accountable.

The published research on production AI eval methodology converges on the same pattern. Anthropic's "Demystifying evals for AI agents" makes the distinction operationally explicit: capability evals ask what an agent can do well, regression evals ask whether the agent still handles all the tasks it used to, and the two have different artifacts, different update cadences, and different purposes. The OpenAI Evals framework, open-sourced in 2023 and now the cornerstone for community-contributed benchmarks, takes the same position from the practitioner direction: evals evaluate the behavior of any system, including prompt chains and tool-using agents, which is a different evaluation surface than the test suite that verifies the deterministic plumbing around the model. Behavioral-testing work in the broader academic literature reinforces the same direction. The eval set is its own artifact, not a test suite and not a requirements document. The literature has caught up to what production teams already know: the eval set is the spec. What the literature has not yet named, and what the operating-model layer needs to name, is who owns it.

An eval-set spec page headed "MECHANISM SPECIFICATION" lists three numbered layers: "CORRECT OUTPUT", "OPTIMIZATION TARGET", "ACCEPTABLE-FAILURE BOUNDARY".

What I have stopped recommending to AI product teams

I have stopped recommending that AI product teams "add evals" as a maturity step. The recommendation is technically correct and operationally useless. Teams add evals; the question of authority does not get touched; six months later the team has a working eval pipeline and an absent product owner. Adding evals without naming the eval owner is the equivalent of installing a control system without naming the operator.

What I now recommend instead is that any team about to install eval-driven workflows answer one question first, in writing, before the harness is even chosen: who curates the failure cases. The answer must be a specific named role, not a function. "The product team" does not count. "The ML team" does not count. "QA" does not count. The role has to be the person who, when a customer complaint surfaces a behavior nobody had thought to test, decides whether that behavior gets added to the eval set, what threshold it needs to hit, and whether the next release ships before or after it does. If you cannot answer that, you do not have an AI product owner, regardless of what your org chart says.

The second thing I recommend is that the documented product owner read the eval set quarterly, in full, in a meeting, with the person who maintains it. The point of the meeting is not to do code review. The point is to surface the policy decisions hiding inside the failure-case choices. "Why is the medical-domain input flagged as out-of-scope?" is a product question. "Why is the threshold for hallucination set at 3% rather than 1%?" is a product question. If those questions get answered by an ML engineer alone, the ML engineer is the product owner. If they get answered by the documented PM, the meeting did its job.

The third thing I recommend is that the eval set's commit history get treated as a product-decision log. Every change to the failure cases is a change to the product specification. Most teams already have this history; almost none of them treat it as a governance artifact. Naming it that way changes how the changes get made.

The two ways eval ownership defaults, and what it costs

Eval ownership defaults in two patterns when the team does not assign it explicitly. Both patterns are common; both cost more than the team realizes.

In the first pattern, ownership defaults to the ML or platform team. This is the most common case. The eval harness was originally written by an ML engineer who needed to evaluate a model swap; the eval cases grew organically from that engineer's understanding of what the product needed to do. Six months later, that engineer is the product specifier whether or not anyone has said so. The cost is not visible until the eval starts diverging from the product strategy, which it does, because ML engineers optimize for technically interesting failures and shrug at boring ones (legal-risk edge cases, low-volume language pairs, edge cases that only affect named enterprise accounts). The product floats because the optimization function is pointing at something the business has not chosen to optimize for.

In the second pattern, ownership defaults to nobody. The eval exists, in a notebook, run irregularly, maintained by whichever engineer last had to debug a regression. Failure cases get added when someone files a bug. The product policy hiding inside the eval is the accumulated residue of whichever bugs have been filed loudly enough to trigger a fix. The team thinks of this as informal; what it actually is, is the product being shaped by customer-complaint volume rather than by deliberate strategy. The cost shows up as the inability to make any large product decision, because the team cannot tell whether a proposed change will improve or break the operative spec.

Both patterns produce the same headline symptom, which is the "our AI product keeps almost-shipping" complaint that has become the trigger phrase of this cohort of products. The deeper diagnosis is not that the product is technically close to ready and just needs another sprint. The deeper diagnosis is that the team has no agreed specification of done, because the team has not named the person who decides what done means.

This is what good product discipline looked like in classical software and what it has to look like in AI product work. The PRD was the spec because the engineer was the executor; the PM owned the PRD. The eval set is the spec because the model is the executor; somebody has to own the eval set. The role label is open. The accountability is not.

Two paired cards labeled "PATTERN 1: ML/PLATFORM DEFAULT" and "PATTERN 2: NOBODY DEFAULT" each name a cost, under the shared verdict "our AI product keeps almost-shipping".

The diagnostic questions to take to the next product review

The fastest way to surface the eval-ownership question without it sounding theoretical is to bring four questions to the next product review and ask them in order. They are designed to fail in a useful way when the ownership is misaligned. If the team cannot answer them or starts answering them by deferring to different people, the misalignment is now visible to everyone in the room.

The questions:

Who curates the failure cases in the eval set, and is that the same person we have named as the product owner for this feature? If the names diverge, the documented owner does not actually own the product.
When a new failure mode surfaces from customer support or from an internal review, what is the path by which it gets evaluated for inclusion in the eval set, and who signs off on that decision? If there is no path, failures are getting added by whoever happens to notice them, and the eval is drifting from product strategy.
What is the current acceptable-failure rate for each output category, and where is that decision documented? If the decision lives only in the eval threshold value, the policy is not auditable, and the legal-and-trust implications are not visible to the people accountable for them.
When the eval score improves, what specifically improved, and what trade-off did we accept? Every optimization in AI work trades something. If the team cannot name what got worse in exchange for what got better, the optimization loop is operating outside product governance.

None of these questions require technical depth to ask. They require somebody in the room to want to ask them. That is the operating-model decision.

A silhouetted hand holds a checklist titled "PRODUCT REVIEW: EVAL OWNERSHIP DIAGNOSTIC" with four numbered questions, footed "Bring these to the next product review."

The implication for the operating model

The eval set is the new product specification. This is not a metaphor; it is the operational reality of how AI products get built and shipped. The PRD describes intent; the eval describes correctness; the model executes the eval, not the PRD. If your operating model has not caught up to this, your product authority is sitting wherever the eval set is being maintained, which is almost certainly not where your org chart says it is.

The fix is not technical. It is an operating-model decision: name the eval owner, give them the title that matches the authority, and make the documented product owner accountable for reading the eval set as the spec it is. The role might be the PM. It might be a new "AI product owner" function. It might be a senior engineer with explicit product authority. The role label matters less than the alignment between authority and accountability.

Two failure modes will tempt the operating-model fix into a shape that does not work. The first is to leave the eval ownership where it defaulted (usually with ML or platform) and add a "review process" on top. Review processes do not change ownership; they add friction without moving accountability. The second is to assign eval ownership to the PM by edict without giving them the authority to curate the cases. Authority without capability is its own failure mode; the PM either becomes a bottleneck or quietly delegates the substantive work back to the engineer who was doing it before.

The version of the fix that works names a single person as accountable for the eval set, gives them the authority to add and remove failure cases, and makes them sit in the product-strategy conversation rather than the eval-tooling conversation. The role can be a PM with a technical extension, or a senior IC with a product extension, or a hybrid role created for this purpose. What matters is that the person specifying the product is the person the org has decided is specifying the product.

Take one question to your next product review: who owns our eval set, and does that match who we said owns the product. If the two names match, you have an operating model. If they diverge, you have a product that is being shaped by an org chart you have not actually written.

A brass nameplate engraved "AI PRODUCT OWNER" and "Accountable for the eval set" sits beside the "EVAL SET v3.2" binder on a walnut desk.

Frequently Asked Questions

Who owns AI evals, the product manager or the engineer?▸

The product manager owns the eval set because the eval set IS the product specification, but in most AI product teams the ownership has silently moved to whichever engineer first set up the eval harness, and the org chart has not caught up. The common industry framing of "PM owns the what, engineer owns the how" describes the intended division of labor, not the operational reality on most teams.

The mechanism is straightforward. Every failure case in the eval set is a product-policy decision: what counts as in-scope, what threshold counts as passing, what trade-off the team is accepting between accuracy and coverage. When those decisions are being made by an ML engineer or a QA lead because they happened to know how to set up the runner, the product authority moved with them, regardless of what the documented product owner's title says. The fix is to name a single accountable role for failure-case curation, give that role the authority to add and remove cases without escalation, and make sure that role sits in the product-strategy conversation, not just the eval-tooling one.

What is eval-driven development, and what does it change at the operating-model layer?▸

Eval-driven development is the discipline of writing down what "correct output" means for an AI product as a set of evaluable test cases before iterating on prompts, models, or pipelines. Every change ships only if it raises the eval score; every failure mode that surfaces in production gets added to the eval before it gets fixed. The eval set becomes the operative specification of the product.

What it changes at the operating-model layer is who specifies the product. In classical software, the PM owns the PRD and the engineer translates intent into code. In AI products, the model is the interpreter and it interprets the eval set, not the PRD. The PRD becomes a marketing document. The eval set becomes the spec. Whichever role is curating the failure cases is, in operational terms, the product owner, even when that role has no product title. Eval-driven development without naming an eval owner is the equivalent of installing a control system without naming the operator.

What is the AI product manager's role when the team adopts evals?▸

The AI product manager's role is to own the failure-case curation as a first-class product decision, not to delegate it to the engineer who set up the harness. The PM decides which behaviors get added to the eval set, what thresholds the product ships at, and what trade-off the team accepts every time the eval score improves. The technical depth of setting up the harness is delegable; the product authority is not.

This is a different skill from ML engineering and from classical product management. The PM does not write the eval runner or pick the model family. The PM reads the eval set as the operative spec of the product, asks why each failure case is in or out, surfaces the policy decisions hiding inside the threshold values, and makes sure the eval reflects what the business has chosen to optimize for rather than what the engineer finds technically interesting. Classical PMs learned to read API contracts and reason about service-level objectives without writing the underlying code; AI PMs need to learn to read failure cases and reason about thresholds the same way.

How do I know if eval ownership has silently moved to ML or QA in my product team?▸

Three diagnostic signals, in increasing severity. First: when you ask the team "what does correct output look like for this feature," the answer comes from an engineer rather than the documented PM. Second: the PM cannot describe the failure cases currently in the eval set or explain why each one is in or out. Third: when the eval score improves between releases, nobody in the product-strategy conversation can name what trade-off the team accepted to get there.

Any one of those signals is suggestive. Two or three together is diagnostic. The pattern beneath them is the same: the eval set is being curated by whoever set up the runner, and the failure-case decisions, which are product-policy decisions, are being made outside product governance. The org chart says one thing; the eval set's commit history says another. The commit history is the more reliable signal.

How do I start eval-driven development without ceding product authority to the engineer who sets up the harness?▸

Answer one question first, in writing, before the eval harness is even chosen: who curates the failure cases. The answer must be a specific named role, not a function. "The product team" does not count. "The ML team" does not count. "QA" does not count. The role has to be the specific person who decides, when a customer complaint surfaces a behavior nobody had thought to test, whether that behavior gets added to the eval set, what threshold it needs to hit, and whether the next release ships before or after it does.

Once the named role is in writing, the rest of the setup follows. The engineer builds the harness. The named role curates the cases. The documented product owner reads the eval set quarterly with the person who maintains it, surfaces the policy decisions hiding inside the failure-case choices, and treats the eval's commit history as a product-decision log. Skipping the naming step is the documented failure mode. Teams that add evals without naming the eval owner end up six months later with a working eval pipeline and an absent product owner.

What does "correct output" actually mean for an AI product?▸

Correct output is defined by the eval set, specifically, the set of failure cases the team has decided are unacceptable and the threshold the team has decided is shippable. Classical software has correctness defined by behavior matching a deterministic spec. An AI product has correctness defined by output passing a chosen set of eval cases at a chosen threshold. Below the threshold the product fails; above it the product ships.

This matters because the PRD cannot define correct output the way it does in classical software. The PRD says "the assistant should not give legal advice." The eval set is where someone decides whether "this looks like it could be construed as legal advice if read uncharitably" counts as a failure or a pass. That decision is product policy. It has legal implications. It has trust implications. It is usually being made by whoever wrote the eval, which is usually not whoever the org chart has decided is accountable. The operating-model fix is to make sure those two are the same person.

Does adding an "AI product owner" role solve the eval-ownership problem?▸

Adding the role solves the problem only when the role is given actual authority to curate failure cases, not when it is given the title and asked to "coordinate" while the engineer keeps writing the eval. Two failure modes commonly defeat the role-creation fix. The first is leaving eval ownership where it defaulted (usually with ML or platform) and layering a "review process" on top. Review processes do not change ownership; they add friction without moving accountability. The second is assigning eval ownership to a PM by edict without giving them the authority to add and remove cases, in which case the PM becomes a bottleneck or quietly delegates the substantive work back to the engineer.

The version of the role that works names a single person as accountable for the eval set, gives them the authority to add and remove failure cases without escalation, and makes them sit in the product-strategy conversation rather than the eval-tooling conversation. The role label is open, it can be a PM with a technical extension, a senior engineer with explicit product authority, or a hybrid AI-product-owner function. What matters is that authority and accountability point to the same name.

What if our eval set is small or informal? Do we still have an eval-ownership question?▸

Yes, and the smaller the eval, the more urgent the question. A small eval means each case carries proportionally more weight in the optimization function, so the case-selection decisions are larger product decisions, not smaller ones. A small eval also tends to mean the policy decisions hiding inside it, what is in scope, what threshold counts as passing, what counts as a failure, are concentrated in fewer hands and are therefore less visible at the operating-model layer.

Informality is not the same as low-stakes. A team with no formal eval set still has someone deciding, case by case, what counts as shippable output and what counts as a regression. That person is the eval owner whether or not anyone has used the term. Naming the role before the eval grows beyond informal is cheaper than retrofitting it later, when the eval has accumulated months of implicit product decisions that nobody documented.

Does this mean PMs need to learn ML to own the eval set?▸

No. It means PMs need to learn to read and reason about an eval set as the operative spec of the product, which is a different skill from ML engineering. The same way classical PMs learned to read API contracts and reason about service-level objectives without writing the underlying code, AI PMs need to learn to read failure cases and reason about thresholds without necessarily setting up the harness themselves. The technical depth is delegable. The product authority is not.

What the PM does need: the discipline to read the eval set quarterly in full, the willingness to ask why each failure case is in or out, the judgment to spot when the failure-case selection is drifting from product strategy, and the authority to add or remove cases without going through the engineer who maintains the runner. These are product skills applied to a new kind of spec, not engineering skills.

How does this change if a major model release makes evals less necessary?▸

It does not, in the direction the question implies. Improvements in model capability change the shape of what fails, not the existence of failure. A more capable model produces fewer obvious failures and more subtle ones, which makes failure-case curation more skilled work, not less. The eval set is the discipline that captures whatever the current generation of failures actually looks like, at whatever altitude the current model operates.

Until the team is willing to ship an AI product without a definition of correct output, somebody owns that definition, and the operating-model question is who. Model capability shifts what gets evaluated; it does not eliminate the need for evaluation, and it does not eliminate the product-ownership question hiding inside it. The kill criterion for this thesis would be a model release where the system self-corrects against natural-language specs reliably enough that human failure-case curation becomes unnecessary, a future state, not the current one.