Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Models for Agentic AI

• Categories of agentic threats, including misuse, escalation, data leakage, and supply-chain risks.
• Adversary profiles and attacker capabilities specific to autonomous agents.
• Mapping assets, trust boundaries, and critical control points for agents.

Governance, Policy, and Risk Management

• Governance frameworks for agentic systems, covering roles, responsibilities, and approval gates.
• Policy design encompassing acceptable use, escalation rules, data handling, and auditability.
• Compliance considerations and evidence collection for audits.

Non-Human Identity & Authentication for Agents

• Designing identities for agents using service accounts, JWTs, and short-lived credentials.
• Least-privilege access patterns and just-in-time credentialing.
• Identity lifecycle management, including rotation, delegation, and revocation strategies.

Access Controls, Secrets, and Data Protection

• Fine-grained access control models and capability-based patterns for agents.
• Secrets management, encryption in transit and at rest, and data minimization.
• Safeguarding sensitive knowledge sources and PII from unauthorized agent access.

Observability, Auditing, and Incident Response

• Designing telemetry for agent behavior, including intent tracing, command logs, and provenance.
• SIEM integration, alerting thresholds, and forensic readiness.
• Runbooks and playbooks for handling agent-related incidents and containment.

Red-Teaming Agentic Systems

• Planning red-team exercises, defining scope, rules of engagement, and safe failover.
• Adversarial techniques such as prompt injection, tool misuse, chain-of-thought manipulation, and API abuse.
• Conducting controlled attacks and measuring exposure and impact.

Hardening and Mitigations

• Engineering controls like response throttles, capability gating, and sandboxing.
• Policy and orchestration controls, including approval flows, human-in-the-loop, and governance hooks.
• Model and prompt-level defenses, such as input validation, canonicalization, and output filters.

Operationalizing Safe Agent Deployments

• Deployment patterns for agents, including staging, canary, and progressive rollout.
• Change control, testing pipelines, and pre-deploy safety checks.
• Cross-functional governance involving security, legal, product, and ops playbooks.

Capstone: Red-Team / Blue-Team Exercise

• Execute a simulated red-team attack against a sandboxed agent environment.
• Defend, detect, and remediate as the blue team using established controls and telemetry.
• Present findings, remediation plans, and policy updates.

Summary and Next Steps