AI Safety for Product Engineers

You shipped an LLM feature to production. Now what can go wrong?

The Real Risks

Hallucination. The model confidently generates false information. Risk: misinformed users, reputational damage, legal liability depending on domain.

Prompt injection. Users craft inputs to override system instructions. Risk: security breach, unintended behavior, data leaks.

Bias and fairness. Outputs reflect training data biases. Risk: discriminatory results, regulatory violation.

Token exhaustion. Expensive API calls with no limit. Risk: runaway costs, DOS attacks.

Practical Safeguards

For hallucination:

• Require sources. Never let the LLM answer without grounding in retrieved documents.
• Fact-check against your data before returning to user.
• Show confidence scores. If no relevant documents, say "I don't know."

For prompt injection:

• Use structured inputs. Don't concatenate user text directly into prompts.
• Separate system instructions from user content in API calls.
• Monitor for unusual patterns in user inputs.

For bias:

• Test outputs on diverse inputs.
• Log outputs for audit trails.
• Document known limitations.

For token exhaustion:

• Set token limits per request and per user.
• Rate limit API calls.
• Monitor spend and alert on spikes.

The Mindset

Safety in AI isn't special — it's the same as safety in any production system. Expect failures, design for graceful degradation, monitor, and alert.

Assume the model will hallucinate. Build guardrails, not trust.