-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Thanks; this is incredibly helpful!
I've started a branch to do a test case to demonstrate all these bugs ; it's called "ed25519_keygen" in my public repository. It also adds a couple more features to '--keygen'. It does cases 2...4 so far; I want to make it cover 5..10.
Once I've got it demonstrating all the cases, I'll try to clean it up and start hammering down the bugs.
Finished tests on branch ed25519_keygen, commit: b71dafcb3358fac7
Things are indeed much much better. I am sending you the new (hopefully last) cases which need adjustments. Note that the case numbers do not match the ones in my previous email, since I am completely removing the cases where now Tor does what it should.
We have cases with two possible status codes: LOG MESSAGE - where we need to change the log message and not die with [err] do_main_loop(): Bug: - I am also trying to provide suggestions for log messages, but we certainly need to improve them.
DEFECT - where we also need to adjust the behavior (only one defect fortunately).
- ----------------------------------------------------------
1. If ed25519_master_id_secret_key is missing or encrypted and we only have ed25519_master_id_public_key available in $datadirectory/keys (no ed25519_signing_cert and 25519_signing_secret_key):
What Tor should do: - - Don't start at all and print log messages requiring user action.
What Tor currently does: - - Doesn't start at all: Aug 06 15:15:01.000 [warn] We needed to load a secret key from /root/torsrc/data/keys/ed25519_master_id_secret_key, but couldn't find it. Aug 06 15:15:01.000 [warn] Missing identity key Aug 06 15:15:01.000 [err] do_main_loop(): Bug: Error initializing keys; exiting (on Tor 0.2.7.2-alpha-dev b71dafcb3358fac7)
The behavior is correct, it detects the file ed25519_master_id_public_key, it doesn't see ed25519_signing_cert and ed25519_signing_secret_key so it tries to look for ed25519_master_id_secret_key (unencrypted) for generating them.
STATUS: LOG MESSAGE -> Change the log message, it is not a bug: Failed to start: Missing a valid signing key and certificate. Please generate manually a valid signing key and certificate and move them to $datadirectory/keys or move the plaintext (unencrypted) master key in $datadirectory/keys so Tor can do this automatically. - ----------------------------------------------------------
2. If only ed25519_master_id_secret_key_encrypted exists in $datadirectory/keys (ed25519_master_id_public_key, ed25519_signing_secret_key and ed25519_signing_cert are missing):
What Tor should do: - - We assume the relay identity existed previously and ed25519_master_id_secret_key_encrypted was restored form a backup. Do not start Tor at all and print explicit warnings that action is required - Tor cannot decrypt the ed25519_master_id_secret_key_encrypted file. Generate ed25519_master_id_public_key, ed25519_signing_secret_key and ed25519_signing_cert or decrypt the ed25519_master_id_secret_key_encrypted in $datadirectory/keys so Tor can do it by itself.
What Tor currently does: - - Doesn't start: Aug 06 15:03:25.000 [warn] Found an encrypted secret key, but not public key file /root/torsrc/data/keys/ed25519_master_id_public_key! Aug 06 15:03:25.000 [warn] Missing identity key Aug 06 15:03:25.000 [err] do_main_loop(): Bug: Error initializing keys; exiting (on Tor 0.2.7.2-alpha-dev b71dafcb3358fac7)
ed25519_master_id_public_key missing is not the issue here, the issue is actually not finding valid ed25519_signing_cert and ed25519_signing_secret_key.
STATUS: LOG MESSAGE -> Change the log message, it is not a bug: Failed to start: The master key in $datadirectory/keys is encrypted and we don't have a valid signing key and certificate. Please generate manually a valid signing key and certificate and move them to $datadirectory/keys or move the plaintext (unencrypted) master key in $datadirectory/keys so Tor can do this automatically. - ----------------------------------------------------------
3. If ed25519_master_id_secret_key_encrypted and ed25519_master_id_public_key both exist in $datadirectory/keys (ed25519_signing_secret_key and ed25519_signing_cert are missing):
What Tor should do: - - We assume the relay identity existed previously and ed25519_master_id_secret_key_encrypted was restored form a backup. Do not start Tor at all and print explicit warnings that action is required - Tor cannot decrypt the ed25519_master_id_secret_key_encrypted file. Generate ed25519_master_id_public_key, ed25519_signing_secret_key and ed25519_signing_cert or decrypt the ed25519_master_id_secret_key_encrypted in $datadirectory/keys so Tor can do it by itself.
STATUS: LOG MESSAGE -> Change the log message, it is not a bug: Failed to start: The master key in $datadirectory/keys is encrypted and we don't have a valid signing key and certificate. Please generate manually a valid signing key and certificate and move them to $datadirectory/keys or move the plaintext (unencrypted) master key in $datadirectory/keys so Tor can do this automatically. - ----------------------------------------------------------
4. If only valid ed25519_signing_secret_key and valid ed25519_signing_cert exist in $datadirectory/keys (ed25519_master_id_secret_key and ed25519_master_id_public_key missing):
What Tor should do: - - Compute ed25519_master_id_public_key from ed25519_signing_secret_key, validate and start. Proceed as normal after (regenerate automatically after expiration if master key unencrypted is still available).
What Tor currently does: - - It starts (this is good), but does not compute and save to disk ed25519_master_id_public_key from ed25519_signing_cert:
Aug 06 15:33:21.000 [warn] No key found in /root/torsrc/data/keys/ed25519_master_id_secret_key or /root/torsrc/data/keys/ed25519_master_id_public_key. Aug 06 15:33:21.000 [warn] Master public key was absent; inferring from public key in signing certificate
STATUS: DEFECT -> Tor should also save to disk in $datadirectory/keys ed25519_master_id_public_key (computed from ed25519_signing_cert) not just start, in order to avoid future warnings for the same reason.
We have to compute and save to disk *ed25519_master_id_public_key of ed25519_master_id_secret_key*, not the public key of ed25519_signing_secret_key. - ----------------------------------------------------------
5. If we have in $datadirectory/keys an ed25519_master_id_secret_key and ed25519_master_id_public_key which doesn't belong to the secret key:
What Tor should do: - - Don't start at all - something is obviously wrong and user has to clean it up.
What Tor currently does: - - Doesn't start: Aug 06 15:45:36.000 [warn] /root/torsrc/data/keys/ed25519_master_id_public_key does not match /root/torsrc/data/keys/ed25519_master_id_secret_key! Aug 06 15:45:36.000 [warn] Missing identity key Aug 06 15:45:36.000 [err] do_main_loop(): Bug: Error initializing keys; exiting (on Tor 0.2.7.2-alpha-dev b71dafcb3358fac7)
STATUS: LOG MESSAGE -> Change the log message, it is not a bug: Failed to start: The ed25519_master_id_public_key and ed25519_master_id_secret_key do not match. If you are trying to restore this relay from a backup, make sure you didn't accidentally mix the key files. If you are sure the ed25519_master_id_secret_key is the right identity file for this relay, delete ed25519_master_id_public_key. - ----------------------------------------------------------
6. ed25519_signing_cert and ed25519_signing_secret_key are valid but do not validate against ed25519_master_id_public_key:
What Tor should do: - - Don't start at all - something is wrong.
What Tor currently does: - - Doesn't start: Aug 06 15:54:20.000 [warn] The signing cert we have was not signed with the master key we loaded! Aug 06 15:54:20.000 [err] do_main_loop(): Bug: Error initializing keys; exiting (on Tor 0.2.7.2-alpha-dev b71dafcb3358fac7)
STATUS: LOG MESSAGE -> Change the log message, it is not a bug: Change the log message, not a bug: Failed to start: The certificate was not signed with the master key we loaded! Maybe we have in $datadirectory/keys the signing key and certificate generated with another relay's master key, please double check! - ----------------------------------------------------------
7. Only expired ed25519_signing_cert and ed25519_signing_secret_key:
What Tor should do: - - Don't start.
What Tor currently does: - - Doesn't start: Aug 06 15:48:56.000 [warn] No key found in /root/torsrc/data/keys/ed25519_master_id_secret_key or /root/torsrc/data/keys/ed25519_master_id_public_key. Aug 06 15:48:56.000 [warn] Missing identity key Aug 06 15:48:56.000 [err] do_main_loop(): Bug: Error initializing keys; exiting (on Tor 0.2.7.2-alpha-dev b71dafcb3358fac7)
The behavior is correct, Tor first checks the certificate's expiration date, sees that it is expired and proceeds looking for ed25519_master_id_secret_key unencrypted to generate fresh new cert and key (very good).
STATUS: LOG MESSAGE -> Change the log message, it is not a bug: Failed to start: the signing key and certificate are expired. Generate a fresh valid signing key and certificate and move them to $datadirectory/keys or move the ed25519_master_id_secret_key unencrypted to $datadirectory/keys so Tor can do this automatically.