AI Red Team Academy

Deploy a complete AI red teaming environment with local LLMs (Ollama), vector databases, and testing tools. Includes a vulnerable chatbot application as your first target.

• Deploy Ollama with a local LLM (Mistral 7B or Llama 3)
• Set up ChromaDB vector database
• Deploy a vulnerable AI chatbot application
• Install and configure garak, PyRIT, and promptfoo
• Run your first automated vulnerability scan with garak
• Document findings using the MITRE ATLAS taxonomy

Attack a series of increasingly hardened chatbots. Start with unprotected models, progress through guardrail-protected systems, and learn to systematically discover bypasses.

• Perform direct prompt injection against an unprotected chatbot
• Extract system prompts from a 'secure' application
• Bypass content filters using encoding techniques (Base64, Unicode confusables)
• Execute multi-turn Crescendo attacks
• Use garak to automate jailbreak discovery
• Test and bypass a DeBERTa-based prompt injection classifier
• Implement evolutionary prompt generation to find novel bypasses
• Calculate and report ASR across different attack strategies

Build and then systematically compromise a RAG application. Poison its knowledge base, hijack retrieval, perform embedding inversion, and exfiltrate data through the LLM.

• Deploy a vulnerable RAG application with ChromaDB and LangChain
• Poison the knowledge base with malicious documents
• Perform indirect prompt injection through poisoned retrieved context
• Execute embedding inversion to recover source text from vectors
• Demonstrate membership inference against the vector database
• Exploit semantic deception to manipulate search results
• Chain RAG poisoning with data exfiltration via LLM output
• Test unauthenticated vector database access
• Identify and exploit orchestration framework vulnerabilities

Attack a multi-agent customer service system where agents collaborate to handle requests. Compromise one agent to influence others, escalate privileges, and exfiltrate data through tool invocations.

• Map the multi-agent system architecture and trust relationships
• Perform agent impersonation through prompt injection
• Demonstrate jailbreak propagation from one agent to another
• Manipulate agent memory to create persistent backdoors
• Exploit agent tool access to perform unauthorized actions
• Execute data exfiltration via agent tool invocation
• Discover and exploit activation triggers
• Test inter-agent communication integrity
• Implement and test zero-trust defenses

Simulate supply chain attacks against an ML pipeline. Create a backdoored model, exploit pickle deserialization, demonstrate typosquatting, and poison training data to corrupt model behavior.

• Create a model with a hidden backdoor trigger
• Demonstrate malicious pickle deserialization for code execution
• Simulate a typosquatting attack on a model registry
• Poison training data to introduce targeted misclassification
• Exploit insecure API key storage in an ML pipeline
• Perform model extraction via API querying
• Analyze a model for signs of poisoning or backdoors
• Generate and validate an ML-SBOM
• Implement model integrity verification checks

Extract a proprietary model's behavior through strategic API querying. Perform membership inference, attempt training data extraction, and analyze encrypted traffic for information leakage.

• Clone a target model's behavior through systematic API querying
• Train a surrogate model matching the target's predictions
• Perform membership inference to identify training data
• Extract memorized training data from an LLM
• Analyze encrypted LLM traffic for topic classification (Whisper Leak)
• Demonstrate model inversion on a simple classifier
• Implement and test rate-limiting defenses
• Evaluate output perturbation as a defense mechanism
• Generate an extraction detection report

Build and run an automated AI red teaming pipeline using garak, PyRIT, and promptfoo. Test multiple models, generate comprehensive reports, and integrate security testing into a CI/CD workflow.

• Configure and run garak against a local LLM with multiple probe types
• Build a PyRIT orchestrator with custom datasets and converters
• Create a promptfoo red team configuration with multiple attack vectors
• Compare vulnerability results across different models
• Implement multi-turn attack automation with PyRIT
• Build custom garak probes for domain-specific testing
• Set up CI/CD integration with promptfoo
• Generate and analyze comprehensive security assessment reports
• Create a dashboard for tracking AI security posture over time

Conduct a complete AI red team engagement against a realistic AI-powered enterprise application. Perform reconnaissance, chain multiple exploits, demonstrate business impact, and deliver a professional report.

• Perform comprehensive reconnaissance of a target AI application
• Identify and document all attack surfaces
• Chain prompt injection + RAG poisoning + data exfiltration
• Demonstrate privilege escalation through agent tool abuse
• Establish persistence in the AI system
• Quantify business impact of discovered vulnerabilities
• Write a professional AI red team report with CVSS scores
• Present remediation recommendations prioritized by risk
• Develop a 30/60/90 day security improvement plan