HPKE has the benefit of being standardized, and imperfect forward secrecy is the best you can do with 0-RTT (which probably fits the fediverse model better).
Just be very careful about the "use correctly" and it's "fine". I'm partial to Noise (https://noiseprotocol.org/) for these sort of use cases because it has less foot+guns and there's more flexibility, but I wouldn't freak out or scream if I saw HPKE in use.
As a side note, it is indeed safe to convert ed25519 keys to X25519 keys assuming a common sense step is taken when using the latter (https://eprint.iacr.org/2021/509).
