What I like to do when building the other way around (x86_64 target and arm64 host) is just adding the architecture I build for as a foreign architecture (plus the correct repos) and then install all the required libraries directly (by specifying `:arm64`).
However, I of course do that in a disposable container, since it basically turns the system into a complete mess.
