My relay VMs are provisioned with Terraform. Once the VMs are up, Terraform copies over a shell script which installs and configures everything. Relay keys, configs, etc live on a separate volume so that relays keep the same identities even when I rebuild the VMs from scratch. I can destroy all the VMs and the automation will bring them back up with the same fingerprints.
To make working on the config easier, the shell script is idempotent so I can re-run it on an existing VM without reprovisioning.