Suppose a support team wants an AI-generated summary of each customer conversation. The first demo is straightforward: paste a conversation into a prompt and read the result. The work gets more interesting when that summary becomes part of someone’s daily job.
I’d use that small example to check the whole feature, from choosing the input to correcting a bad result.
Give it a specific job
For a support summary, I’d define the fields the agent needs: the customer’s problem, what has already been tried and what remains unresolved. I’d keep links to the source messages so the agent can check a claim.
A fluent paragraph can hide a wrong date or a promise nobody made. The person using the summary needs a quick way to notice and correct that. I’d review the interface with them before spending much time on model selection.
Keep permissions in application code
The application should choose which conversations this user can access before sending anything to the model. A conversation may contain instructions, jokes or text pasted from somewhere else. Those words must not change the user’s permissions.
I’d also keep saving a summary separate from sending it to a customer. If the feature eventually sends messages, recipients and confirmation need their own checks in code. Putting the instruction “be careful” in a prompt does not provide those checks.
Keep a small set of difficult examples
I’d save a test set with an ordinary conversation, a very long one, an unresolved case and a conversation containing contradictory dates. Synthetic examples are useful here; production messages need appropriate handling before they become test data.
For each case, I’d record the facts the summary must preserve and anything it must leave unanswered. Then I’d rerun the set after changing the prompt, model or retrieval. Reading a few convincing outputs is an easy way to miss a regression.
Plan for a failed request
If the model times out, the agent should still have the conversation and their own notes. I’d offer a retry without making them start over. For a summary that only reads data, retrying is usually straightforward.
An action that writes data needs more care. A timeout may happen after the server saved the result. The application should track whether that happened before repeating the operation, or a retry can create duplicate records.
Count the cost of the finished task
I’d measure the cost of producing a usable summary, including long conversations and retries. I’d also look at how often someone has to correct it. A cheap model call can become expensive work if every result needs rewriting.
The release needs an owner, a way to inspect failures and a switch to turn the feature off. I’d start with a small group of users and review what went wrong before widening access.