On the matter of onion key rotation, I had the idea of making the onion key files read-only. Roger did some source code investigation and said that it might work to prevent onion key rotation, with some minor side effects. I plan to give the idea a try on a different bridge. The possible side effects are that tor will continue trying and failing to rotate the onion key every hour, and "force a router descriptor rebuild, so it will try to publish a new descriptor each hour."
Making secret_onion_key and secret_onion_key_ntor read-only does not quite work, because tor first renames them to secret_onion_key.old and secret_onion_key_ntor.old before writing new files. (Making the *.old files read-only does not work either, because the `tor_rename` function first unlinks the destination.) https://gitweb.torproject.org/tor.git/tree/src/feature/relay/router.c?h=tor-...
But a slight variation does work: make secret_onion_key.old and secret_onion_key_ntor.old *directories*, so that tor_rename cannot rename a file over them. It does result in an hourly `BUG` stack trace, but otherwise it seems effective.
I did a test with two tor instances. The rot1 instance had the directory hack to prevent onion key rotation. The rot2 had nothing to prevent onion key rotation.
```plain # tor-instance-create rot1 # tor-instance-create rot2 ```
/etc/tor/instances/rot1/torrc: ```plain Log info file /var/lib/tor-instances/rot1/onionrotate.info.log BridgeRelay 1 AssumeReachable 1 BridgeDistribution none ORPort 127.0.0.1:auto ExtORPort auto SocksPort 0 Nickname onionrotate1 ```
/etc/tor/instances/rot2/torrc: ```plain Log info file /var/lib/tor-instances/rot2/onionrotate.info.log BridgeRelay 1 AssumeReachable 1 BridgeDistribution none ORPort 127.0.0.1:auto ExtORPort auto SocksPort 0 Nickname onionrotate2 ```
Start rot1, copy its keys to rot2, then start rot2:
```plain # service tor@rot1 start # cp -r /var/lib/tor-instances/rot1/keys /var/lib/tor-instances/rot2/ # chown -R _tor-rot2:_tor-rot2 /var/lib/tor-instances/rot2/keys # service tor@rot2 start ```
Stop the two instances, check that the onion keys are the same, and that `LastRotatedOnionKey` is set in both state files:
```plain # service tor@rot1 stop # service tor@rot2 stop # ls -l /var/lib/tor-instances/rot*/keys/secret_onion_key* -rw------- 1 _tor-rot1 _tor-rot1 888 Jan 28 22:57 /var/lib/tor-instances/rot1/keys/secret_onion_key -rw------- 1 _tor-rot1 _tor-rot1 96 Jan 28 22:57 /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor -rw------- 1 _tor-rot2 _tor-rot2 888 Jan 28 23:05 /var/lib/tor-instances/rot2/keys/secret_onion_key -rw------- 1 _tor-rot2 _tor-rot2 96 Jan 28 23:05 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor # md5sum /var/lib/tor-instances/rot*/keys/secret_onion_key* fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot1/keys/secret_onion_key 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot2/keys/secret_onion_key 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor # grep LastRotatedOnionKey /var/lib/tor-instances/rot*/state /var/lib/tor-instances/rot1/state:LastRotatedOnionKey 2022-01-28 22:57:14 /var/lib/tor-instances/rot2/state:LastRotatedOnionKey 2022-01-28 23:11:04 ```
Set `LastRotatedOnionKey` 6 weeks into the past to force an attempt to rotate the keys the next time tor is restarted:
```plain # sed -i -e 's/^LastRotatedOnionKey .*/LastRotatedOnionKey 2021-12-15 00:00:00/' /var/lib/tor-instances/rot*/state # grep LastRotatedOnionKey /var/lib/tor-instances/rot*/state /var/lib/tor-instances/rot1/state:LastRotatedOnionKey 2021-12-15 00:00:00 /var/lib/tor-instances/rot2/state:LastRotatedOnionKey 2021-12-15 00:00:00 ```
Create the secret_onion_key.old and secret_onion_key_ntor.old directories in the rot1 instance.
```plain # mkdir -m 700 /var/lib/tor-instances/rot1/keys/secret_onion_key{,_ntor}.old ```
Check the identity of keys before starting:
```plain # md5sum /var/lib/tor-instances/rot*/keys/secret_onion_key* fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot1/keys/secret_onion_key 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor md5sum: /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor.old: Is a directory md5sum: /var/lib/tor-instances/rot1/keys/secret_onion_key.old: Is a directory fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot2/keys/secret_onion_key 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor ```
Start both instances:
```plain # service tor@rot1 start # service tor@rot2 start ```
Verify that the rot1 instance is still using the same onion keys, while rot2 has rotated them:
```plain # ls -ld /var/lib/tor-instances/rot*/keys/secret_onion_key* -rw------- 1 _tor-rot1 _tor-rot1 888 Jan 28 23:45 /var/lib/tor-instances/rot1/keys/secret_onion_key -rw------- 1 _tor-rot1 _tor-rot1 96 Jan 28 23:45 /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor drwx--S--- 2 root _tor-rot1 4096 Jan 28 23:44 /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor.old drwx--S--- 2 root _tor-rot1 4096 Jan 28 23:44 /var/lib/tor-instances/rot1/keys/secret_onion_key.old -rw------- 1 _tor-rot2 _tor-rot2 888 Jan 28 23:47 /var/lib/tor-instances/rot2/keys/secret_onion_key -rw------- 1 _tor-rot2 _tor-rot2 96 Jan 28 23:47 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor -rw------- 1 _tor-rot2 _tor-rot2 96 Jan 28 23:05 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor.old -rw------- 1 _tor-rot2 _tor-rot2 888 Jan 28 23:05 /var/lib/tor-instances/rot2/keys/secret_onion_key.old # md5sum /var/lib/tor-instances/rot*/keys/secret_onion_key* fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot1/keys/secret_onion_key 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor md5sum: /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor.old: Is a directory md5sum: /var/lib/tor-instances/rot1/keys/secret_onion_key.old: Is a directory fb8a5e8787141dba4e935267f818cc2a /var/lib/tor-instances/rot2/keys/secret_onion_key 2c3f7d81e96641e2c04fb9c452296337 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor.old fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot2/keys/secret_onion_key.old ```
The rot1 instance's `LastRotatedOnionKey` remains the same, while rot2's is updated:
```plain # grep LastRotatedOnionKey /var/lib/tor-instances/rot*/state /var/lib/tor-instances/rot1/state:LastRotatedOnionKey 2021-12-15 00:00:00 /var/lib/tor-instances/rot2/state:LastRotatedOnionKey 2022-01-28 23:47:02 ```
The rot1 instance's log shows the failure to rotate the keys:
/var/lib/tor-instances/rot1/onionrotate.info.log ```plain Jan 28 23:46:59.000 [info] rotate_onion_key_callback(): Rotating onion key. Jan 28 23:46:59.000 [warn] Couldn't rotate onion key. Jan 28 23:46:59.000 [info] router_rebuild_descriptor(): Rebuilding relay descriptor (forced) ... Jan 28 23:46:59.000 [info] check_onion_keys_expiry_time_callback(): Expiring old onion keys. ```
While the rot2 rotation was successful:
/var/lib/tor-instances/rot2/onionrotate.info.log ```plain Jan 28 23:47:02.000 [info] rotate_onion_key_callback(): Rotating onion key. Jan 28 23:47:02.000 [info] rotate_onion_key(): Rotating onion key Jan 28 23:47:02.000 [info] mark_my_descriptor_dirty(): Decided to publish new relay descriptor: rotated onion key ```
After 1 hour, the rot1 instance tries to rebuild its relay descriptor, and triggers a `BUG` non-fatal assertion failure in [`router_rebuild_descriptor`](https://gitweb.torproject.org/tor.git/tree/src/feature/relay/router.c?h=tor-...). I let it run for 1 more hour after that, and it happened again.
/var/lib/tor-instances/rot1/onionrotate.info.log ```plain Jan 29 00:46:59.000 [info] router_rebuild_descriptor(): Rebuilding relay descriptor (forced) Jan 29 00:46:59.000 [warn] The IPv4 ORPort address 127.0.0.1 does not match the descriptor address 172.105.3.197. If you have a static public IPv4 address, use 'Address <IPv4>' and 'OutboundBindAddress <IPv4>'. If you are behind a NAT, use two ORPort lines: 'ORPort <PublicPort> NoListen' and 'ORPort <InternalPort> NoAdvertise'. Jan 29 00:46:59.000 [info] extrainfo_dump_to_string_stats_helper(): Adding stats to extra-info descriptor. Jan 29 00:46:59.000 [info] read_file_to_str(): Could not open "/var/lib/tor-instances/rot1/stats/bridge-stats": No such file or directory Jan 29 00:46:59.000 [warn] tor_bug_occurred_(): Bug: ../src/feature/relay/router.c:2452: router_rebuild_descriptor: Non-fatal assertion !(desc_gen_reason == NULL) failed. (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: Tor 0.4.5.10: Non-fatal assertion !(desc_gen_reason == NULL) failed in router_rebuild_descriptor at ../src/feature/relay/router.c:2452. Stack trace: (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(log_backtrace_impl+0x57) [0x5638b9538047] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(tor_bug_occurred_+0x16b) [0x5638b954327b] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(router_rebuild_descriptor+0x13d) [0x5638b94f4e1d] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(+0x21f163) [0x5638b9665163] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(+0x83577) [0x5638b94c9577] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /lib/x86_64-linux-gnu/libevent-2.1.so.7(+0x239ef) [0x7f701bae49ef] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /lib/x86_64-linux-gnu/libevent-2.1.so.7(event_base_loop+0x52f) [0x7f701bae528f] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(do_main_loop+0x101) [0x5638b94b1321] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(tor_run_main+0x1d5) [0x5638b94acdd5] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(tor_main+0x49) [0x5638b94a92e9] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(main+0x19) [0x5638b94a8ec9] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f701b391d0a] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [warn] Bug: /usr/bin/tor(_start+0x2a) [0x5638b94a8f1a] (on Tor 0.4.5.10 ) Jan 29 00:46:59.000 [info] router_upload_dir_desc_to_dirservers(): Uploading relay descriptor to directory authorities Jan 29 00:46:59.000 [info] directory_post_to_dirservers(): Uploading an extrainfo too (length 822) Jan 29 00:46:59.000 [info] rep_hist_note_used_internal(): New port prediction added. Will continue predictive circ building for 3332 more seconds. Jan 29 00:46:59.000 [info] connection_ap_make_link(): Making internal anonymized tunnel to [scrubbed]:9001 ... Jan 29 00:46:59.000 [info] connection_ap_make_link(): ... application connection created and linked. Jan 29 00:46:59.000 [info] check_onion_keys_expiry_time_callback(): Expiring old onion keys. ```
Stopping and restarting the tor1 instance keeps the same onion keys, and the first rotation does not hit the assertion failure:
```plain # service tor@rot1 stop # service tor@rot1 start # md5sum /var/lib/tor-instances/rot*/keys/secret_onion_key* fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot1/keys/secret_onion_key 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor md5sum: /var/lib/tor-instances/rot1/keys/secret_onion_key_ntor.old: Is a directory md5sum: /var/lib/tor-instances/rot1/keys/secret_onion_key.old: Is a directory fb8a5e8787141dba4e935267f818cc2a /var/lib/tor-instances/rot2/keys/secret_onion_key 2c3f7d81e96641e2c04fb9c452296337 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor 2066ab7e01595adf42fc791ad36e1fc5 /var/lib/tor-instances/rot2/keys/secret_onion_key_ntor.old fb2a8a8f9de56f061eccbb3fedd700c4 /var/lib/tor-instances/rot2/keys/secret_onion_key.old ```
/var/lib/tor-instances/rot1/onionrotate.info.log ```plain Jan 29 02:06:13.000 [info] rotate_onion_key_callback(): Rotating onion key. Jan 29 02:06:13.000 [warn] Couldn't rotate onion key. Jan 29 02:06:13.000 [info] router_rebuild_descriptor(): Rebuilding relay descriptor (forced) ... Jan 29 02:06:13.000 [info] check_onion_keys_expiry_time_callback(): Expiring old onion keys. ```