Peertube now supports remote runners for the transcoding jobs! That's the one thing I was missing, I had no idea it was in the pipeline... (I had looked into adding it myself, but found it required substantial changes to how jobs are configured, which looks like they now did).

If only I wasn't on an old Debian version which blocks upgrading...