Agent governance is neither one policy nor an admin dashboard. It connects business accountability, technical authorization, data access, quality measurement and response to failure. The more a system can act, the more explicit these layers must become.
Name a business owner
Every agent process needs a person accountable for purpose, permitted outcomes and exceptions. IT or platform teams operate technology but cannot decide alone whether an action is allowed in the business. Also record owners for data sources, tools and incident decisions.
Enforce data boundaries
An agent does not receive broad access to “company knowledge.” Sources are approved by process, tenant and role. Sensitive fields can be removed or masked before model processing. Logs hold only the data required for security and diagnosis.
Minimize tool permissions
OWASP guidance recommends server-side authorization on every request and least privilege. For agents, that means separate credentials and rights per tool. Read, internal write, external and irreversible actions receive different control levels.
The tool validates input independently of the model. An agent cannot force a prohibited action merely through persuasive text.
Design meaningful approvals
Human in the loop must not become a ceremonial click. An approval shows the source, proposed action, affected data, uncertainty and consequence. Define when approval is mandatory and when low-impact automation is acceptable.
Evaluate with fixed cases
Evaluation sets include normal, rare, deliberately misleading and security-critical cases. Measure domain correctness, source grounding, permitted tool choice, escalation and cost. Re-run relevant suites after changes to model, prompt, tools or knowledge.
Prepare for incidents
The runbook covers stop, permission revocation, fallback, notification, investigation and recovery. A kill switch works only when someone owns the decision and the process can continue afterward.
The NIST AI RMF treats governance as a cross-cutting function over mapping, measurement and management. That fits agents: control is not a one-time go-live document, but part of every change and operating cycle.