Yes, using our existing DC infrastructure may not be as efficient as once again surrendering our sovereignty to the same tech bros we regularly complain about. And I would argue that more than 90% of today's typical workload is inference rather than training.
And to your last point: give any two software devs the same task and you'll get two different results. Have them do it again and you'll get another set of different solutions. Absolutely non-deterministic. No different with LLMs.