Guardrails for AI Features That Touch Customer Data
Adding an AI feature means deciding what data leaves your systems, who can see it, and what the model may do with it. A checklist for doing it safely.
Saji Nammari3 min read
The fastest way to add AI to a product is to send whatever the user has on screen to a model API and display the answer. The fastest way to lose a customer's trust is to discover, later, exactly what was sent.
Every AI feature that touches customer data needs a short list of decisions made on purpose. Here's the checklist our AI engineers work through with client teams before anything ships.
Know what leaves your systems
- Map exactly which fields are sent to the model, and why each one is needed.
- Strip or mask what isn't needed: identifiers, contact details, payment data.
- Check your provider's terms on data retention and training, and choose settings or contracts that match your obligations to customers.
- Know where the data is processed if your customers have residency requirements.
Respect existing permissions
An assistant must never become a way around access controls. If a user can't open a record in the app, the AI shouldn't be able to summarize it for them. Enforce permissions when data is retrieved, before anything reaches the model.
Treat model input as untrusted
Documents, emails, and web pages can contain instructions aimed at the model, a technique known as prompt injection. Assume any content the model reads may try to redirect it. Limit which tools and actions the model can trigger, require confirmation for anything irreversible, and never run model output as code or queries without validation.
Check what comes back
Validate outputs before showing them: check their structure, block content that shouldn't appear, and make sure the answer doesn't echo sensitive data from another context. Give users a clear way to report a bad response.
Log responsibly
You need logs to debug and improve AI features. You don't need raw personal data in them. Redact before logging, set retention limits, and restrict who can read the logs.
Test the guardrails, not just the feature
Guardrails that have never been tested are assumptions. Add cases to your evaluation set that try to break them: a user asking for someone else's record, a document with hidden instructions, a request to reveal the system prompt. Run them on every change, the same way you'd run a security regression suite. When a provider updates its model, these are the tests to watch first.
Tell users what's happening
Label AI-generated content, explain in plain language what data the feature uses, and offer a way to opt out where it makes sense. Transparency costs little and prevents most uncomfortable conversations.
Guardrails don't slow AI work down. They're what let a company say yes to the next AI feature instead of debating whether the last one was safe.
About the author
Saji Nammari
Founder & Chief Executive Officer
Saji serves as the Founder and Chief Executive Officer, having successfully pivoted from five years as a highly effective freelance software and data professional. His leadership is rooted in the pragmatic, results-driven mentality required to deliver all types of projects, driving a culture of agility and technical rigor. Outside of work, Saji enjoys the creative disciplines of literature, music, and the satisfaction of pure problem solving.
