Dr. Bruce Simpson (npub12vv…nhf0) Michael Downey :notAI: (npub1ufx…u2t8)
The thing that’s hidden when projects get reports from Anthropic is how much human triage is needed.
I had someone send me a code review of one of my projects done with Claude 4.6 (which, apparently, is as good at Mythos at finding bugs but less good at producing PoC exploits). Of the top ten bugs, most were not bugs (e.g. missing null checks on things where the API contract requires non-null arguments). Two were intentional design choices and the proposed changes would have made things slower. One was a bug that needed fixing, but there was already an open PR to fix it before Claude looked at the project.
The signal to noise ratio is worse than Coverity, and FreeBSD hasn’t had the resources to triage / fix all of the issues the free Coverity scans found in 15 or so years of having access to it.
David Chisnall (*Now with 50% more sarcasm!*)
npub13g…dk50d
2026-04-22 07:45:01 UTC
Author Public Key
npub13gcwraghdcw9xzkg3tky24feu0lzkhtlt57wpdn58y4m4tyr9qdsxdk50dSeen on
wss://relay.momostr.pinkPublished at
2026-04-22 07:45:01 UTCEvent JSON
{
"id": "888b62d3b8cf8fd7a1f39c2abbaedb5724fa2f823555422369207d6a0e9c42d5",
"pubkey": "8a30e1f5176e1c530ac88aec455539e3fe2b5d7f5d3ce0b674392bbaac83281b",
"created_at": 1776843901,
"kind": 1,
"tags": [
[
"p",
"e24dffdbc7e58718036754f6dd931940f7830aeb540904e47f661213cec0f8f9"
],
[
"proxy",
"https://infosec.exchange/@david_chisnall/116447241950476980",
"web"
],
[
"p",
"531977456882d729256f893bcb4959e613a78a496e686a5cbc55f0e29c076305"
],
[
"proxy",
"https://infosec.exchange/users/david_chisnall/statuses/116447241950476980",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://infosec.exchange/users/david_chisnall/statuses/116447241950476980",
"pink.momostr"
],
[
"-"
]
],
"content": "nostr:npub12vvhw3tgsttjjft03yaukj2eucf60zjfde5x5h9u2hcw98q8vvzs5knhf0 nostr:npub1ufxllk78ukr3sqm82nmdmycegrmcxzht2sysferlvcfp8nkqlrus75u2t8 \n\nThe thing that’s hidden when projects get reports from Anthropic is how much human triage is needed.\n\nI had someone send me a code review of one of my projects done with Claude 4.6 (which, apparently, is as good at Mythos at finding bugs but less good at producing PoC exploits). Of the top ten bugs, most were not bugs (e.g. missing null checks on things where the API contract requires non-null arguments). Two were intentional design choices and the proposed changes would have made things slower. One was a bug that needed fixing, but there was already an open PR to fix it before Claude looked at the project.\n\nThe signal to noise ratio is worse than Coverity, and FreeBSD hasn’t had the resources to triage / fix all of the issues the free Coverity scans found in 15 or so years of having access to it.",
"sig": "149342f6bd9a9e08e069039acd35a30430aba78fc4785e6e9da1aab31f1c2ec716f51ac9fd44439f43d9e90340bae5aea9e59935c8cbd04df1414a430af77388"
}