Filename: 315-update-dir-required-fields.txt Title: Updating the list of fields required in directory documents Author: Nick Mathewson Created: 23 April 2020 Status: Open
1. Introduction
When we add a new field to a directory document, we must at first describe it as "optional", since older Tor implementations will not generate it. When those implementations are obsolete and unsupported, however, we can safely describe those fields as "required", since they are always included in practice.
Making fields required is not just a matter of bookkeeping: it helps prevent bugs in two ways. First, it simplifies our code. Second, it makes our code's requirements match our assumptions about the network.
Here I'll describe a general policy for making fields required when LTS versions become unsupported, and include a list of fields that should become required today.
This document does not require to us to make all optional fields required -- only those which we intend that all Tor instances should always generate and expect.
When we speak of making a field "required", we are talking about describing it as "required" in dir-spec.txt, so that any document missing that field is no longer considered well-formed.
2. When fields should become required
We have three relevant kinds of directory documents: those generated by relays, those generated by authorities, and those generated by onion services.
Relays generate extrainfo documents and routerdesc documents. For these, we can safely make a field required when it is always generated by all relay versions that the authorities allow to join the network. To avoid partitioning, authorities should start requiring the field before any relays or clients do.
(If a relay field indicates the presence of a now-required feature, then instead of making the field mandatory, we may change the semantics so that the field is assumed to be present. Later we can remove the option.)
Authorities generate authority certificates, votes, consensus documents, and microdescriptors. For these, we can safely make a field required once all authorities are generating it, and we are confident that we do not plan to downgrade those authorities.
Onion services generate service descriptors. Because of the risk of partitioning attacks, we should not make features in service descriptors required without a phased process, described in the following section.
2.1. Phased addition of onion service descriptor changes
Phase one: we add client and service support for the new field, but have this support disabled by default. By default, services should not generate the new field, and clients should not parse it when it is present. This behavior is controlled by a pair of network parameters. (If the feature is at all complex, the network parameters should describe a _minimum version_ that should enable the feature, so that we can later enable it only in the versions where the feature is not buggy.)
During this phase, we can manually override the defaults on particular clients and services to test the new field.
Phase two: authorities use the network parameters to enable the client support and the service support. They should only do this once enough clients and services have upgraded to a version that supports the feature.
Phase three: once all versions that support the feature are obsolete and unsupported, the feature may be marked as required in the specifications, and the network parameters ignored.
Phase four: once all versions that used the network parameters are obsolete and unsupported, authorities may stop including those parameters in their votes.
3. Directory fields that should become required.
These fields in router descriptors should become required: * identity-ed25519 * master-key-ed25519 * onion-key-crosscert * ntor-onion-key * ntor-onion-key-crosscert * router-sig-ed25519 * proto
These fields in router descriptors should become "assumed present": * hidden-service-dir
These fields in extra-info documents should become required: * identity-ed25519 * router-sig-ed25519
The following fields in microdescriptors should become required: * ntor-onion-key
The following fields in votes and consensus documents should become required: * pr