Show HN: I spent 8 months trying to make LLMs Hack
May 24, 2025 - news.ycombinator.com
The article discusses the development of an AI Security Tester designed to identify vulnerabilities in web applications. Over the past eight months, the creator has built a system comprising 29 agents using a custom LLM Orchestration framework that employs a task-subtask architecture. This framework is entirely custom-built for penetration testing, without relying on tools like Auto-Gen, Langchain, or Crew AI. Each test is conducted in an isolated Kali Linux environment on AWS Fargate, allowing the agents full access to the environment and the internet for research purposes. The testing process can take between 2 to 12 hours, depending on the complexity of the target.
Upon completion of the test, the system, named Peneterrer, provides a comprehensive Vulnerability Management portal and an AI-generated Pentest report, which can be over 30 pages long. The article invites readers to test the system at the provided website link and welcomes feedback on its performance and effectiveness.
Key takeaways:
The AI Security Tester consists of 29 agents using a custom LLM Orchestration framework for task-subtask architecture.
Each test runs in an isolated Kali Linux environment on AWS Fargate, allowing agents full access to hack web applications and find vulnerabilities.
Agents have internet access through Tavily to research content during tests, which can last from 2 to 12 hours.
After testing, Peneterrer provides a full Vulnerability Management portal and an AI-generated Pentest report, sometimes exceeding 30 pages.
