BlacksmithAI: Open-source AI-powered penetration testing framework

BlacksmithAI is an open-source penetration testing framework that uses multiple AI agents to execute different stages of a security assessment lifecycle.

A multi-agent structure for offensive workflows

BlacksmithAI runs as a hierarchical system in which an orchestrator coordinates task execution across specialized agents.

Each agent maps to a common penetration testing function. The recon agent handles attack surface mapping and information gathering. The scan and enumeration agent performs service discovery. A vulnerability analysis agent evaluates weaknesses and potential exposure. An exploit agent executes proof of concept activity. A post-exploitation agent examines impact and potential lateral movement.

The orchestrator assigns tasks across these roles and compiles outputs into final reporting. This model distributes reasoning and execution across defined functions that align with established offensive testing practices.

“Most AI security tools, and there aren’t many, rely on a single ‘super agent’ to do everything. But that’s not how real penetration testing works. While it can be performed by one person, it is often conducted by a team to maximize effectiveness and depth. That’s why we model agents based on real-world practice using a hierarchical multi-agent approach,” Yohannes Gebrekirstos, author of BlacksmithAI, told Help Net Security.

He explained orchestrator agent acts as the process lead, interacting with the user and managing the other agents. This authority allows it to decompose complex tasks into sub-goals and delegate them to specialized sub-agents. Each sub-agent has its own domain expertise and toolset. For example, a reconnaissance agent performs target analysis using tools such as Whois and Dig. This mirrors how real penetration testing teams operate, each specialist contributes focused expertise, improving efficiency and outcomes.

“The system is designed to be flexible and simple. It’s lightweight and easily extensible, allowing contributors to add new specialized agents and tools without friction. BlacksmithAI also uses a shared container, a mini-Kali environment. Instead of spinning up new containers for each task, which consumes time and memory, agents operate within a pre-configured environment with the necessary tools already installed. This improves resource efficiency and enables tool caching, while access controls prevent agents from adding or removing tools, ensuring consistency across runs. Built once, it runs continuously, and with FastAPI, it is fast as well,” Yohannes added.

Integration with existing security tooling

The framework integrates with established security tools through a containerized environment. BlacksmithAI includes:

• Preconfigured Docker images
• Industry-standard security utilities
• CLI-based tools designed for automated execution

Execution takes place in controlled environments to support automated assessment activity.

Deployment requirements include Docker, Python 3.12, Node.js runtime components, and the uv package manager. Supported operating environments include Linux, macOS, and Windows through WSL2.

Flexible AI backend support

BlacksmithAI supports multiple large language model providers through configurable backends.

Current integrations include OpenRouter, vLLM, and custom provider endpoints. This setup allows agent reasoning to run on internal infrastructure or external model services depending on deployment preferences.

Interfaces for operational use

The framework provides both terminal and web interfaces for operational use.

Use cases include automated security assessments, continuous monitoring, vulnerability discovery, and validation workflows. The system also supports educational testing environments and research activity.

Reporting capabilities generate structured outputs with supporting evidence drawn from executed tasks.

Future plans and download

Yohannes said, “Plans for the future include adding support for interactive tools such as Metasploit and BeEF, which will make the system more powerful. We also plan to introduce browser support so agents can test website functionality. Currently, tools like Nikto and Gobuster can discover paths and known vulnerabilities, but they cannot click buttons or fill out forms.

I aim to allow users to add additional tools using MCPs, making the platform easily scalable. Examples include an MCP Playwright browser or MCP Shodan integration. Another priority is enabling users to add skills. Beyond tool documentation, this would allow agents to learn best practices by combining multiple tools.”

BlacksmithAI is available for free on GitHub.

Must read:

• 40 open-source tools redefining how security teams secure the stack
• Firmware scanning time, cost, and where teams run EMBA

Subscribe to the Help Net Security ad-free monthly newsletter to stay informed on the essential open-source cybersecurity tools. Subscribe here!