I think AGI is pretty unlikely. But for AI I think the calculator is a perfect metaphor.
I see the scenario you're speaking about as an example of specialization and I think specialization is the magic that makes market economies work.
Why should we all be able to solve a complex math problem on paper if there is a more efficient method? And in the hands of the people specializing in math, the calculator has turbo charged productivity and advancements.
I see LLMs enabling the exact same kind of specialization but in the realm of language, which will be a good thing for the species