Many large companies have spent decades developing controls to manage the risks associated with giving their employees authority and discretion. Those controls include defined responsibilities, technical limitations on access to confidential materials, layers of supervision, required approvals for certain actions, ongoing monitoring for higher risk activities, training on firm policies, and mandatory escalation if certain conduct is observed.
Some of these companies are starting to delegate a growing number of tasks that were previously performed by their human employees to AI agents. At the same time, they are trying to place restrictions on what the agents can do. For example, in financial services, an AI agent may be permitted to research a potential investment, draft a client communication, identify a potentially suspicious transaction, or prepare trade instructions. But a firm may decide that the agent is not allowed to make the ultimate investment decision, communicate with individuals outside the firm, determine compliance outcomes, or execute transactions.
Replacing a human employee with an AI agent does not reduce a company’s legal and regulatory obligations. As we have discussed here, FINRA’s 2026 Annual Regulatory Oversight Report identified the risks of agentic AI deployment—such as data mishandling and unauthorized agent conduct—as a key supervisory consideration for broker-dealers deploying this technology. Similarly, while the federal banking agencies have not issued risk management guidance specifically addressing agentic AI, the April 2026 Guidance on Model Risk Management issued by the Federal Reserve, Office of the Comptroller of the Currency, and the Federal Deposit Insurance Corporation makes clear that existing risk management and governance obligations apply when determining the appropriate controls for agentic AI. International regulators have echoed this theme. For example, Singapore’s 2026 Model AI Governance Framework for Agentic AI emphasizes that firms deploying AI agents and the personnel responsible for overseeing them remain accountable for the actions of AI agents.
To illustrate the point, suppose a financial services institution hires an employee and gives her access to the firm’s trading systems, internal email, customer information, and sensitive financial data in connection with her employment functions. Yet the company fails to maintain a reasonably designed set of policies and procedures that govern her role, and fails to train, supervise, and monitor her work. If the employee engages in conduct that violates the federal securities laws, the firm could arguably face regulatory scrutiny for its failures to establish reasonable policies and procedures and to supervise her. A firm that has replaced that employee with an AI agent could potentially encounter the same questions about a lack of controls for the agent’s conduct.
It therefore makes sense that some companies are now developing control architectures for their AI agents that resemble the compliance structures that they created for their employees.
Leveraging Employee Governance Concepts for Agentic AI Oversight and Governance
Designing an agentic AI supervision framework involves several considerations analogous to employee governance.
First, as is the case with employees, an AI agent’s access should not be confused with authority: an agent’s access to, or integration with, a system does not itself constitute authorization to act.
Second, employees are subject to controls to ensure that misconduct is harder to carry out; to increase the likelihood that the firm can detect misconduct quickly; and to discourage misconduct through disciplinary measures. These controls also allow a company to demonstrate to regulators that the employee’s misconduct had not been condoned or encouraged, was outside her given authority, and violated established policies. Having specific controls in place that are designed to prevent AI agents or employees from engaging in certain conduct that is illegal or in violation of company policy allows a company to demonstrate that it acted responsibly, even if the misconduct nonetheless occurred. While written instructions for an agent may be effective in establishing the line between an agent’s permitted and prohibited conduct, such instructions should be supplemented, where feasible, by technical controls through permissions, gateways, logging, approval steps, and other similar guardrails as additional controls to prevent an agent from taking certain actions without human review and approval.
Third, depending on the use case, agentic AI controls may need to be tested periodically, which is a common compliance practice for controls that apply to employees. For example, in the cybersecurity context, most companies do not simply tell employees once not to click suspicious links and assume that the risk has been mitigated. Instead, they conduct simulated phishing exercises on a regular cadence, routinely testing whether employees understand and can follow the company’s policies and procedures under realistic conditions. Periodic testing enables companies to identify weaknesses, retrain employees where necessary, and adjust controls based on lessons learned.
Agents with substantial access or authority may need to be stress-tested in a similar manner. Firms may need to run a pilot during which the agent encounters confusing, adversarial, or unexpected situations to see if, for example, the agent adheres to its controls when it is told to ignore them. Does it wait for human approval? Can it be induced to disclose sensitive information it should protect? What happens when its instructions conflict? Testing should also ask whether any scenario that the agent encounters, such as a claim of urgency or contrary instructions from someone who claims to be senior to the agent’s owner, can cause it to expand its own authority or bypass applicable controls.
Finally, when an agent impermissibly exceeds its authority, it should be subject to corrective action, which may involve requiring a specified human owner for the agent, restricting permissions, suspending or reconfiguring the agent, retraining it, or decommissioning it for the particular use case.
Conclusion
For agents—like human employees—the existence of a robust control framework does not guarantee that misconduct won’t happen. However, if agentic misconduct occurs, the existence of such controls may support a company’s arguments that it should not be held liable for the agent’s actions.
As with employees, a risk exists that AI agent controls will initially be too restrictive and thereby stifle creativity, innovation, and productivity. But many companies have figured out how to equip employees with the necessary authority to exercise independent judgment within a system that has sufficient controls to significantly reduce problematic or unauthorized conduct. Replicating this same balance for controls over AI agents will require time and investment, but we are working with our clients on these issues and starting to see some promising results.
*****
To subscribe to the Data Blog, please click here.
The cover art used in this blog post was generated by GPT-5.6 Sol Pro
The Debevoise STAAR (Suite of Tools for Assessing AI Risk) is a monthly subscription service that provides Debevoise clients with an online suite of tools to help them responsibly fast-track their AI adoption. Please contact us at STAARinfo@debevoise.com for more information.
