Filename: 315-update-dir-required-fields.txt Title: Updating the list of fields required in directory documents Author: Nick Mathewson Created: 23 April 2020 Status: Open
1. Introduction
When we add a new field to a directory document, we must at first describe it as "optional", since older Tor implementations will not generate it. When those implementations are obsolete and unsupported, however, we can safely describe those fields as "required", since they are always included in practice.
Making fields required is not just a matter of bookkeeping: it helps prevent bugs in two ways. First, it simplifies our code. Second, it makes our code's requirements match our assumptions about the network.
Here I'll describe a general policy for making fields required when LTS versions become unsupported, and include a list of fields that should become required today.
This document does not require to us to make all optional fields required -- only those which we intend that all Tor instances should always generate and expect.
When we speak of making a field "required", we are talking about describing it as "required" in dir-spec.txt, so that any document missing that field is no longer considered well-formed.
2. When fields should become required
We have three relevant kinds of directory documents: those generated by relays, those generated by authorities, and those generated by onion services.
Relays generate extrainfo documents and routerdesc documents. For these, we can safely make a field required when it is always generated by all relay versions that the authorities allow to join the network. To avoid partitioning, authorities should start requiring the field before any relays or clients do.
(If a relay field indicates the presence of a now-required feature, then instead of making the field mandatory, we may change the semantics so that the field is assumed to be present. Later we can remove the option.)
Authorities generate authority certificates, votes, consensus documents, and microdescriptors. For these, we can safely make a field required once all authorities are generating it, and we are confident that we do not plan to downgrade those authorities.
Onion services generate service descriptors. Because of the risk of partitioning attacks, we should not make features in service descriptors required without a phased process, described in the following section.
2.1. Phased addition of onion service descriptor changes
Phase one: we add client and service support for the new field, but have this support disabled by default. By default, services should not generate the new field, and clients should not parse it when it is present. This behavior is controlled by a pair of network parameters. (If the feature is at all complex, the network parameters should describe a _minimum version_ that should enable the feature, so that we can later enable it only in the versions where the feature is not buggy.)
During this phase, we can manually override the defaults on particular clients and services to test the new field.
Phase two: authorities use the network parameters to enable the client support and the service support. They should only do this once enough clients and services have upgraded to a version that supports the feature.
Phase three: once all versions that support the feature are obsolete and unsupported, the feature may be marked as required in the specifications, and the network parameters ignored.
Phase four: once all versions that used the network parameters are obsolete and unsupported, authorities may stop including those parameters in their votes.
3. Directory fields that should become required.
These fields in router descriptors should become required: * identity-ed25519 * master-key-ed25519 * onion-key-crosscert * ntor-onion-key * ntor-onion-key-crosscert * router-sig-ed25519 * proto
These fields in router descriptors should become "assumed present": * hidden-service-dir
These fields in extra-info documents should become required: * identity-ed25519 * router-sig-ed25519
The following fields in microdescriptors should become required: * ntor-onion-key
The following fields in votes and consensus documents should become required: * pr
Hi Nick,
This proposal is missing the "bridge" case.
Bridges are more complicated, because we have at least 3 kinds of bridges: * bridges distributed by BridgeDB * bridges distributed with apps (such as Tor Browser) * private bridges
Bridge option transitions are also more complicated, because clients download bridge descriptors directly from their configured bridges.
T
On 24 Apr 2020, at 00:45, Nick Mathewson nickm@torproject.org wrote:
Filename: 315-update-dir-required-fields.txt Title: Updating the list of fields required in directory documents Author: Nick Mathewson Created: 23 April 2020 Status: Open
- Introduction
When we add a new field to a directory document, we must at first describe it as "optional", since older Tor implementations will not generate it. When those implementations are obsolete and unsupported, however, we can safely describe those fields as "required", since they are always included in practice.
Making fields required is not just a matter of bookkeeping: it helps prevent bugs in two ways. First, it simplifies our code. Second, it makes our code's requirements match our assumptions about the network.
Here I'll describe a general policy for making fields required when LTS versions become unsupported, and include a list of fields that should become required today.
This document does not require to us to make all optional fields required -- only those which we intend that all Tor instances should always generate and expect.
When we speak of making a field "required", we are talking about describing it as "required" in dir-spec.txt, so that any document missing that field is no longer considered well-formed.
- When fields should become required
We have three relevant kinds of directory documents: those generated by relays, those generated by authorities, and those generated by onion services.
Relays generate extrainfo documents and routerdesc documents. For these, we can safely make a field required when it is always generated by all relay versions that the authorities allow to join the network. To avoid partitioning, authorities should start requiring the field before any relays or clients do.
(If a relay field indicates the presence of a now-required feature, then instead of making the field mandatory, we may change the semantics so that the field is assumed to be present. Later we can remove the option.)
Authorities generate authority certificates, votes, consensus documents, and microdescriptors. For these, we can safely make a field required once all authorities are generating it, and we are confident that we do not plan to downgrade those authorities.
Onion services generate service descriptors. Because of the risk of partitioning attacks, we should not make features in service descriptors required without a phased process, described in the following section.
2.1. Phased addition of onion service descriptor changes
Phase one: we add client and service support for the new field, but have this support disabled by default. By default, services should not generate the new field, and clients should not parse it when it is present. This behavior is controlled by a pair of network parameters. (If the feature is at all complex, the network parameters should describe a _minimum version_ that should enable the feature, so that we can later enable it only in the versions where the feature is not buggy.)
During this phase, we can manually override the defaults on particular clients and services to test the new field.
Phase two: authorities use the network parameters to enable the client support and the service support. They should only do this once enough clients and services have upgraded to a version that supports the feature.
Phase three: once all versions that support the feature are obsolete and unsupported, the feature may be marked as required in the specifications, and the network parameters ignored.
Phase four: once all versions that used the network parameters are obsolete and unsupported, authorities may stop including those parameters in their votes.
- Directory fields that should become required.
These fields in router descriptors should become required: * identity-ed25519 * master-key-ed25519 * onion-key-crosscert * ntor-onion-key * ntor-onion-key-crosscert * router-sig-ed25519 * proto
These fields in router descriptors should become "assumed present": * hidden-service-dir
These fields in extra-info documents should become required: * identity-ed25519 * router-sig-ed25519
The following fields in microdescriptors should become required: * ntor-onion-key
The following fields in votes and consensus documents should become required: * pr _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On Thu, Apr 23, 2020 at 5:26 PM teor teor@riseup.net wrote:
Hi Nick,
This proposal is missing the "bridge" case.
Bridges are more complicated, because we have at least 3 kinds of bridges:
- bridges distributed by BridgeDB
- bridges distributed with apps (such as Tor Browser)
- private bridges
Bridge option transitions are also more complicated, because clients download bridge descriptors directly from their configured bridges.
Thank you!
I think that the transition isn't too bad, since the partitioning that a bridge _could_ do by maintaining an obsolete descriptor format is limited in its impact, since the bridge already (typically) sees the client's IP address. So the only difference is that we need to be a little more careful about when we start to require the fields, since bridges sometimes lag the versions supported by the rest of the network.
I've update the proposal with these paragraphs:
Bridge relays have their descriptors processed by clients without necessarily passing through authorities. We can make fields mandatory in bridge descriptors once we can be confident that no bridge lacking them will actually connect to the network-- or that all such bridges are safe to stop using.
For bridges, when a field becomes required, it will take some time before all clients require that field. This would create a partitioning opportunity, but partitioning at the first-hop position is not so strong: the bridge already knows the client's IP, which is a much better identifier than the client's Tor version.
cheers,
-
Nick Mathewson -
Nick Mathewson -
teor