No, what I mean is unless the video length is changed, ffmpeg can easily swap it.
Even if it changed, you can rerender only changed parts and splice it in with minimal reencode to base video then overlay on top the audio. Need to have some sort of anchor to splice though but probably easy
