Ill maybe work on a cleaner schematic and maybe a bit of a write up later but here are the basics.
Use ARTS with associated CW ID between 817 and vx5r
Use 440 mhz
Squelch out detect of 817 is wired into a transistor switch to key PTT of 100d
Squelch out detect of 100d is wired into transistor switch to key PTT of 817
Transmit and receive are crossed over between 100d and 817 with pots for control
A switch enables or disables Ptt to the 817
Set squelches on the 817 and the 100d and it should work