jcoglan on Nostr: somehow I only recently learned that LLMs need the whole conversation fed back to ...

somehow I only recently learned that LLMs need the whole conversation fed back to them on each prompt so their i/o cost scales as O(n^2), something that would be considered completely unacceptable in almost any other production network-accessible software