The AI/PM Collision: Understanding Each Other

Your PM thinks the model is perfect. You know it's hallucinating on 30% of queries. Here's how to align.

The Core Tension

PM mindset: Confident output makes users happy. "Ship it, we'll iterate if needed."

Engineer mindset: 30% error rate is unacceptable. "We can't ship until retrieval accuracy is 95%."

Both are partially right. But you're talking past each other.

Translating Between Worlds

When the PM says: "Users don't care about accuracy, they care about speed."

What they mean: "Our customer explicitly values fast responses even if imperfect."

Your response: "Understood. What's the threshold where errors become a problem? 20% wrong? 50%?" (Get specific.)

When you say: "The model is hallucinating."

What the PM hears: "You're asking me to ship magic that we can't control."

Better framing: "In our tests, 30% of answers aren't grounded in the source documents. That means 3 of 10 users get wrong information. Should we add a confidence score or use a different model?"

Setting Expectations

Have these conversations before building:

1. What's acceptable error rate? Not zero. Real number: 5%? 10%? 20%? Connected to business impact.
2. What's the fallback? When the model is uncertain, what do we do? Show "I'm not sure"? Escalate to human? Offer alternatives?
3. How do we measure success? Not just accuracy. User satisfaction, usage rates, business impact.
4. What's the cost/speed trade-off? Better model = higher latency + higher cost. Where's the line?

When You Disagree

Don't fight about the model. Fight about metrics.

Instead of: "This model isn't good enough."

Say: "According to our test set, this model gives wrong answers 25% of the time. Given our target of 10% error, we have options: [A] use a better model, [B] implement retrieval filtering, [C] lower user expectations. Which trade-off makes sense?"

Metrics are objective. Vibes are subjective.

Building Trust

• Do what you say you'll do
• Propose multiple options, not just "no"
• Measure impact of changes
• Celebrate wins together
• Don't let perfect be the enemy of good

PMs and engineers want the same thing: to ship something users love. You're just seeing different dimensions of the problem.

The friction usually comes from misaligned understanding of the trade-offs, not actual disagreement.