AI is useful inside a workflow when the work genuinely requires judgment: summarising a conversation, classifying an unusual request, comparing messy evidence, or drafting a response for review.
It becomes dangerous when the surrounding system treats a plausible answer as a verified fact.
The pattern I use is to give the model room to reason while keeping the boundaries around that reasoning deterministic.
Before the model
The first boundary controls what the model is allowed to see and what question it is being asked to answer.
Inputs should be scoped to the task. Secrets, unrelated customer data, private conversations, and unsupported assumptions should not be included simply because they are nearby.
The system should also define the expected output clearly. If the next step needs a category, confidence score, source reference, and review reason, those fields should be part of the contract rather than left to prose interpretation.
After the model
Model output needs validation before another system relies on it.
Useful checks include:
- Is the response in the required structure?
- Are all required fields present?
- Does the answer cite the supplied evidence?
- Is the confidence high enough for this next step?
- Does the proposed action stay inside the workflow’s permissions?
- Is a human decision required?
This does not make the model deterministic. It makes the consequences of uncertainty manageable.
Before the action
The most important boundary sits between a recommendation and a live-system change.
Drafting a follow-up is different from sending it. Identifying two records as a possible match is different from merging them. Recommending an account change is different from changing the account.
When an action affects a customer, money, access, production data, or an external relationship, the workflow needs either a deterministic rule that has been deliberately approved or a visible human gate.
“The model was confident” is not an approval mechanism.
After the action
The workflow should record what it attempted, what actually happened, and what needs attention next.
That means preserving enough evidence to answer basic questions later:
- Which source records influenced the decision?
- What did the model return?
- Which validator accepted or rejected it?
- Who approved the action?
- What changed in the destination system?
- How would we reverse or repair it?
Without that trail, failures become arguments between tools instead of diagnosable events.
A practical division of labour
I think of the system in three layers:
- Deterministic software handles identity, permissions, required fields, thresholds, idempotency, and state changes.
- AI judgment handles interpretation, synthesis, classification, and drafting where rules alone are too brittle.
- People handle ambiguity, exceptions, policy decisions, and consequential approval.
The exact boundary changes with the risk of the workflow. A private draft can tolerate more uncertainty than a payment, an access decision, or a message sent in someone’s name.
The goal is not to remove AI from important work. It is to make sure uncertain judgment never quietly becomes an irreversible action.