How to train your clanker?
Building a generic pentest harness seems far more difficult than building one for a specific codebase because it requires an additional layer of context building in order to formulate a plan of attack.
I'm trying something new now, which is feeding a corpus of recently found vulnerabilities in open source codebases to a training clanker that then runs the pentest clanker without giving it any hints, and then the training clanker analyzes the output of the pentest and tries to work backwards to figure out why the vulnerability wasn't found so that it can improve the pentest harness in a generic fashion.
Jameson Lopp
npub17u…wt4tp
2026-09-27 16:43:28 UTC
Author Public Key
npub17u5dneh8qjp43ecfxr6u5e9sjamsmxyuekrg2nlxrrk6nj9rsyrqywt4tpPublished at
2026-09-27 16:43:28 UTCEvent JSON
{
"id": "ee2bb274ab0c5af897c7571b4503331abef63225237af6b2782f84e7876ff4c6",
"pubkey": "f728d9e6e7048358e70930f5ca64b097770d989ccd86854fe618eda9c8a38106",
"created_at": 1790527408,
"kind": 1,
"tags": [
[
"client",
"Amethyst"
]
],
"content": "How to train your clanker?\n\nBuilding a generic pentest harness seems far more difficult than building one for a specific codebase because it requires an additional layer of context building in order to formulate a plan of attack.\n\nI'm trying something new now, which is feeding a corpus of recently found vulnerabilities in open source codebases to a training clanker that then runs the pentest clanker without giving it any hints, and then the training clanker analyzes the output of the pentest and tries to work backwards to figure out why the vulnerability wasn't found so that it can improve the pentest harness in a generic fashion.",
"sig": "8e3878de0b6f9458cc703ed8aaaf608fba38898792eeba200672aaecec8f27299920514570ac861c8fd0853d10edfaa27d7946d43d005739a7b8b18f35f56194"
}