AI Security Is Becoming an Engineering Problem as AI Agents Gain More Power

AI agents are moving beyond generating text and answers. They can read files, use tools, access enterprise systems and take actions based on information they encounter. That growing autonomy is forcing companies to rethink how AI systems are secured.

NVIDIA is arguing that the answer should not depend solely on better prompts or model-level safeguards. In a September 21, 2026 security post, NVIDIA security researcher Saša Zdjelar described AI security as an engineering discipline built around defined requirements, enforceable controls, accountable owners and evidence that protections actually work.

The approach places security controls throughout the AI agent stack, from models and application harnesses to runtime environments, identities, networks and infrastructure.

Why AI Agents Create a New Security Challenge

Traditional software security principles remain relevant even as AI changes how software operates.

Organizations still need to establish identity, control access, restrict exposure and verify that security measures work. But AI agents introduce another layer of complexity because they can reason through tasks, interact with tools and adjust their actions based on data they encounter.

That means an agent can potentially encounter untrusted instructions while performing an otherwise legitimate task.

For example, an agent authorized to update a customer record could encounter malicious instructions in an attached document and attempt to send customer information to an unauthorized destination.

The security system should not rely on the agent recognizing that the action is dangerous.

Instead, infrastructure-level controls should be able to block the unauthorized transfer regardless of what the agent decides to do.

Security Has to Cover the Full Agent Stack

AI agents depend on several interconnected layers.

Models provide the underlying capabilities. Harnesses organize context, tools and workflows. Runtime environments determine where those actions actually execute.

Each layer introduces potential security responsibilities.

NVIDIA’s recent work on agent security emphasizes establishing the security boundary before an agent starts operating. Its OpenShell runtime is designed to provide sandboxed execution and enforce policies around filesystem access, network activity, processes and credentials.

This distinction is important because instructions are not the same as enforcement.

An agent can potentially ignore an instruction. A runtime-level security policy can prevent the action from occurring in the first place.

Permissions Should Be Limited to the Agent’s Job

An AI agent should have an identifiable identity and credentials appropriate to its assigned task.

Access to one system should not automatically provide access to unrelated systems or data.

For example, permission to modify a customer record does not necessarily justify permission to export the customer’s information.

NVIDIA’s OpenShell documentation describes a model in which agents operate inside sandboxed environments with declarative policies that restrict access to files, credentials and external networks.

Organizations also need procedures for handling requests for additional permissions. An agent can request additional access, but it should not be able to authorize that access itself when the action carries meaningful security consequences.

Monitoring Is Part of the Security Boundary

Blocking an unauthorized action is only one part of the problem.

Security teams also need evidence showing what the agent attempted to do, which policy was applied and what happened afterward.

NVIDIA’s OpenShell documentation describes audit capabilities covering agent activity and policy decisions, while its architecture places enforcement at the runtime boundary rather than relying exclusively on the agent itself.

That evidence can become particularly important during an incident.

If an agent attempts to contact an unauthorized destination, security teams need to know which process made the request, what destination was involved and whether the request was allowed or denied.

Security Testing Cannot Stop at Deployment

NVIDIA also argues that AI security needs continuous testing.

Before an agent is deployed, teams should test whether it can obtain credentials outside its authorized scope, send sensitive information to an unauthorized destination, modify permissions or interfere with security monitoring.

Those tests should be repeated when models, tools or workflows undergo significant changes.

When a security test fails, the failure should be reproduced and investigated. Once a fix is implemented, the original failure can become a repeatable security test for future releases.

NVIDIA points to tools such as CrowdStrike’s SafeMind for attack simulation and Palo Alto Networks’ Prisma AIRS for ongoing red-team testing as examples of this broader approach.

Open and Closed AI Models Can Serve Different Security Needs

NVIDIA also highlights the role of model choice in defensive security work.

Closed models can provide managed capabilities and services, while open models can give security teams more control over how systems are inspected, adapted and operated.

That control can matter during incident response. A security team may need to reproduce an AI-related failure inside its own environment, test a mitigation and work with sensitive evidence without moving that information outside its infrastructure.

AI itself can also assist defenders by identifying vulnerabilities, investigating attacks and helping validate fixes.

The key measurement, according to NVIDIA’s framework, should be practical evidence: reproducible findings, verifiable fixes and faster incident response rather than simply the presence of an AI security tool.

NVIDIA OpenShell Pushes Security Outside the Agent

One of the most concrete technologies in NVIDIA’s approach is OpenShell.

NVIDIA describes OpenShell as an open-source runtime for autonomous AI agents that uses sandboxed execution and declarative policies to restrict what an agent can access and do. Its documentation says the system can control filesystem access, network activity, processes and runtime credentials.

The concept is straightforward: the agent can make decisions, but the surrounding infrastructure determines which actions are actually permitted.

Cisco is building on this approach with DefenseClaw, which combines security tools for agent skills, Model Context Protocol servers and AI assets and integrates with NVIDIA OpenShell for runtime-level controls.

JFrog is also integrating with OpenShell to help scan and verify agent skills and enforce policies governing which skills agents can access, according to NVIDIA.

AI Security Is Becoming a Shared Engineering Effort

NVIDIA is also promoting greater information sharing among AI security teams.

The Open Secure AI Alliance, which launched in 2026, has brought together organizations working on open tools and security practices for AI systems. NVIDIA said in August that the alliance had grown to more than 120 organizations and was working with the Linux Foundation on the proposed Shared AI Findings Exchange, or SAFE.

The goal is to make security findings more useful beyond the organization where an incident originally occurred.

If defenders can share what failed, which controls stopped an attack and how a fix was verified, other organizations can turn those findings into their own security tests.

What Happens Next for AI Agent Security

The growing use of autonomous agents means security controls will increasingly need to operate alongside the systems they protect.

For businesses deploying AI agents, that means establishing clear identities and permissions, restricting network and data access, isolating agent execution, protecting audit records and continuously testing security boundaries.

The broader direction outlined by NVIDIA is that AI security should be treated much like other engineering disciplines: define the requirements, enforce the controls, assign responsibility and produce evidence that the controls work.

As agents gain more ability to act independently, those principles could become increasingly important for determining how safely organizations can deploy them at scale.