jonny (nonvenomous) on Nostr: i am so fucking tired. if the LLM invents a tool to call, it first tells itself to ...

i am so fucking tired. if the LLM invents a tool to call, it first tells itself to call another tool to check if the tool was actually real but the fucking nightmare code failed to pick it up in its necronomically guided wander of the environmental catacombs.

The ToolSearchTool then invalidates all its caches, checks for "deferred tools" (which are an INCREDIBLY AWESOME IDEA that allow tools to be injected in the prompt text, will get to that later), and then performs an old school regex-based scoring against all the tools that exist and their descriptions to find candidates. remember this is A LANGUAGE MODEL whose ENTIRE EXISTENCE is based on SOPHISTICATED TEXT AND INTENT MATCHING.

so yes. there is a chance that your LLM can hallucinate a tool and then end up calling some real tool if there is some regex overlap in their descriptions.