Join Nostr
2026-05-15 08:03:14 UTC
in reply to

bleeptrack on Nostr: wouldn't it be more effective to get the LLM to output as many tokens as possible? ...

wouldn't it be more effective to get the LLM to output as many tokens as possible? More output tokens, more inference runs 😸🔥
The input is usually padded 🤔 (at least with the vanilla transformer I used)