I agree with this. It is ultimately a utilitarian argument, and so the pro-AI side is subject to the same logical errors and harmful outcomes as any utilitarian.
There is one other point I'd like to bring up, if you don't mind (I'll try to keep it on topic).
If you are anti-AI for ethical reasons, shouldn't you be opposed to nearly all technology for those same reasons? Computers are generally seen as a good thing because of their utility, but to construct computer chips requires the use of extremely expensive infrastructure and the mining of rare earth metals. The supply chains that support the construction of computer chips are inherently extractive, exploitative, and colonial in nature.
So on the one hand, yes AI (or computers in general) are unethical. On the other hand, there is not just utility to AI but the potential to be useful for the underprivileged as well, that is if the technology in question can be made available to a larger number of people, which I think it can.
I can picture LLMs trained on small data sets consisting of only inputs provided with consent of the human authors, trained in data centers powered by solar energy and air cooled, producing LLMs only 3 gigabytes in size which are small enough to run inference locally on a cell phone. Not all of these conditions have been met so far, but they theoretically could be met.
