Much-revised draft, RFC: removing current obsolete clients from the network
[This is a significantly revised version of the last version of this proposal draft, sent here for comment. The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html . This is more or less a complete rewrite, after I convinced myself that the original ideas wouldn't work. Part of the functionality of that old draft has been subsumed in proposal 264.] 1. Introduction Frequently, we find that very old versions of Tor should no longer be supported on the network. To remove relays is easy enough: we simply update the directory authorities to stop listing relays that advertise versions that are too old. But to disable clients is harder. In another proposal I describe a system for letting future clients go gracefully obsolete. This proposal explains how we can safely disable the obsolete clients we have today (and all other client versions of Tor to date, assuming that they will someday become obsolete). 1.1. Why disable clients? * Security. Anybody who hasn't updated their Tor client in 5 years is probably vulnerable to who-knows-what attacks. They aren't likely to get much anonymity either. * Withstand zombie installations. Some Tors out there were once configured to start-on-boot systems that are now unmaintained. (See 1.4 below.) They put needless load on the network, and help nobody. * Be able to remove backward-compatibility code. Currently, Tor supports some truly ancient protocols in order to avoid breaking ancient versions or Tor. This code needs to be maintained and tested. Some of it depends on undocumented or deprecated or non-portable OpenSSL features, and makes it hard to produce a conforming Tor server implementation. * Make it easier to write a conforming Tor relay. If a Tor relay needs to support every Tor client back through the beginning of time, that makes it harder to develop and test compatible implementations. 1.2. Is this dangerous? I don't think so. This proposal describes a way to make older clients gracefully disconnect from the network only when a majority of authorities agree that they should. A majority of authorities already have the ability to inflict arbitrary degrees of sabotage on the consensus document. 1.3. History The earliest versions of Tor checked the recommended-versions field in the directory to see whether they should keep running. If they saw that their version wasn't recommended, they'd shut down. There was an "IgnoreVersion" option that let you keep running anyway. Later, around 2004, the rule changed to "shut down if the version is _obsolete_", where obsolete was defined as "not recommended, and older than a version that is recommended." In 0.1.1.7-alpha, we made obsolete versions only produce a warning, and removed IgnoreVersion. (See 3ac34ae3293ceb0f2b8c49.) We have still disabled old tor versions. With Tor 0.2.0.5-alpha, we disabled Tor versions before 0.1.1.6-alpha by having the v1 authorities begin publishing empty directories only. In version 0.2.5.2-alpha, we completely removed support for the v2 directory protocol used before Tor 0.2.0; there are no longer any v2 authorities on the network. Tor versions before 0.2.1 will currently not progress past fetching an initial directory, because they believe in a number of directory authority identity keys that no longer sign the directory. Tor versions before 0.2.4 are (lightly) throttled in multihop circuit creation, because we prioritize ntor CREATE cells over TAP ones when under load. 1.4. The big problem: slow zombies and fast zombies It would be easy enough to 'disable' old clients by simply removing server support for the obsolete protocols that they use. But there's a problem with that approach: what will the clients do when they fail to make connections, or to extend circuits, or whatever else they are no longer able to do? * Ideally, I'd like such clients to stop functioning _quietly_. If they stop contacting the network, that would be best. * Next best would be if these clients contacted the network only occasionally and at different times. I'll call these clients "slow zombies". * Worse would be if the clients contact the network frequently, over and over. I'll call these clients "fast zombies". They would be at their worst when they focus on authorities, or when they act in synchrony to all strike at once. One goal of this proposal is to ensure that future clients to not become zombies at all; and that ancient clients become slow zombies at worst. 2. Some ideas that don't work. 2.1. Dropping connections based on link protocols. Tor versions before before 0.2.3.6-alpha use a renegotiation-based handshake instead of our current handshake. We could detect these handshakes and close the connection at the relay side if the client attempts to renegotiate. I've tested these changes on versions maint-0.2.0 through maint-0.2.2. They result in zombies with the following behavior: The client contact each authority it knows about, attempting to make a one-hop directory connection. It fails, detects a failure, then reconnects more and more slowly ... but one hour later, it resets its connection schedule and starts again. In the steady state this appears to result in about two connections per client per authority per hour. That is probably too many. (Most authorities would be affected: of the authorities that existed in 0.2.2, gabelmoo has moved and turtles has shut down. The authorities Faravahar and longclaw are new. The authorities moria1, tor26, dizum, dannenberg, urras, maatuska and maatuska would all get hit here.) (We could simply remove the renegotiation-detection code entirely, and reply to all connections with an immediate VERSIONS cell. The behavior would probably be the same, though.) If we throttled connections rather than closing them, we'd only get one connnection per authority per hour, but authorities would have to keep open a potentially huge number of sockets. 2.2. Blocking circuit creation under certain circumstances In tor 0.2.5.1-alpha, we began ignoring the UseNTorHandshake option, and always preferring the ntor handshake where available. Unfortunately, we can't simply drop all TAP handshakes, since clients and relays can still use them in the hidden service protocol. But we could detect these versions by: Looking for use of a TAP handshake from an IP not associated with with any known relay, or on a connection where the client did not authenticate. (This could be from a bridge, but clients don't build circuits that go to an IntroPoint or RendPoint directly after a bridge.) This would still result in clients not having directories, however, and retrying once an hours. 3. Ideas that might work 3.1. Move all authorities to new ports We could have each authority known to older clients start listening for connections at a new port P. We'd forward the old port to the new port. Once sufficiently many clients were using the new ports, we could disable the forwarding. This would result in the old clients turning into zombies as above, but they would only be scrabbling at nonexistent ports, causing less load on the authorities. [This proposal would probably be easiest to implement.] 3.2. Start disabling old link protocols on relays We could have new relays start dropping support for the old link protocols, while maintaining support on the authorities and older relays. The result here would be a degradation of older client performance over time. They'd still behave zombieishly if the authorities dropped support, however. 3.3. Changing the consensus format. We could allow 'f' (short for "flag") as a synonym for 's' in consensus documents. Later, if we want to disable all Tor versions before today, we can change the consensus algorithm so that the consensus (or perhaps only the microdesc consensus) is spelled with 'f' lines instead of 'f' lines. This will create a consensus which older clients and relays parse as having all nodes down, which will make them not connect to the network at all. We could similarly replace "r" with "n", or replace Running with Online, or so on. In doing this, we could also rename fresh-until and valid-until, so that new clients would have the real expiration date, and old clients would see "this consensus never expires". This would prevent them from downloading new consensuses. [This proposal would result in the quietest shutdown.]
07.01.2016, 18:12 Nick Mathewson: Mostly spelling corrections and one question at the bottom. I have not enough insight to comment on the proposed ideas.
[...]
One goal of this proposal is to ensure that future clients to not
*do not*
become zombies at all; and that ancient clients become slow zombies at worst.
2. Some ideas that don't work.
2.1. Dropping connections based on link protocols.
Tor versions before before 0.2.3.6-alpha use a renegotiation-based handshake instead of our current handshake. We could detect these handshakes and close the connection at the relay side if the client attempts to renegotiate.
I've tested these changes on versions maint-0.2.0 through maint-0.2.2. They result in zombies with the following behavior:
The client contact each authority it knows about, attempting to
An s is missing to make 'contacts'.
make a one-hop directory connection. It fails, detects a failure, then reconnects more and more slowly ... but one hour later, it resets its connection schedule and starts again.
In the steady state this appears to result in about two connections per client per authority per hour. That is probably too many.
(Most authorities would be affected: of the authorities that existed in 0.2.2, gabelmoo has moved and turtles has shut down. The authorities Faravahar and longclaw are new. The authorities moria1, tor26, dizum, dannenberg, urras, maatuska and maatuska would all get hit here.)
maatuska is listed twice.
(We could simply remove the renegotiation-detection code entirely, and reply to all connections with an immediate VERSIONS cell. The behavior would probably be the same, though.)
If we throttled connections rather than closing them, we'd only get one connnection per authority per hour, but authorities would have to
connection
keep open a potentially huge number of sockets.
2.2. Blocking circuit creation under certain circumstances
In tor 0.2.5.1-alpha, we began ignoring the UseNTorHandshake option, and always preferring the ntor handshake where available.
Unfortunately, we can't simply drop all TAP handshakes, since clients and relays can still use them in the hidden service protocol. But we could detect these versions by:
Looking for use of a TAP handshake from an IP not associated with with any known relay, or on a connection where the client
'with' is there twice.
did not authenticate. (This could be from a bridge, but clients don't build circuits that go to an IntroPoint or RendPoint directly after a bridge.)
This would still result in clients not having directories, however, and retrying once an hours.
3. Ideas that might work
3.1. Move all authorities to new ports
We could have each authority known to older clients start listening for connections at a new port P. We'd forward the old port to the new port. Once sufficiently many clients were using the new ports, we could disable the forwarding.
This would result in the old clients turning into zombies as above, but they would only be scrabbling at nonexistent ports, causing less load on the authorities.
[This proposal would probably be easiest to implement.]
3.2. Start disabling old link protocols on relays
We could have new relays start dropping support for the old link protocols, while maintaining support on the authorities and older relays.
The result here would be a degradation of older client performance over time. They'd still behave zombieishly if the authorities dropped support, however.
3.3. Changing the consensus format.
We could allow 'f' (short for "flag") as a synonym for 's' in consensus documents. Later, if we want to disable all Tor versions before today, we can change the consensus algorithm so that the consensus (or perhaps only the microdesc consensus) is spelled with 'f' lines instead of 'f' lines. This will create a consensus which
'f' lines instead of 's' lines.
older clients and relays parse as having all nodes down, which will make them not connect to the network at all.
We could similarly replace "r" with "n", or replace Running with Online, or so on.
In doing this, we could also rename fresh-until and valid-until, so that new clients would have the real expiration date, and old clients would see "this consensus never expires". This would prevent them from downloading new consensuses.
[This proposal would result in the quietest shutdown.]
What would be repeatable in 5 years? I imagine that you would like to get rid of old clients again and again. Regards, Sebastian G.
On 8 Jan 2016, at 04:12, Nick Mathewson <nickm@torproject.org> wrote: ...
2. Some ideas that don't work.
...
2.2. Blocking circuit creation under certain circumstances
In tor 0.2.5.1-alpha, we began ignoring the UseNTorHandshake option, and always preferring the ntor handshake where available.
Unfortunately, we can't simply drop all TAP handshakes, since clients and relays can still use them in the hidden service protocol. But we could detect these versions by:
Looking for use of a TAP handshake from an IP not associated with with any known relay, or on a connection where the client did not authenticate. (This could be from a bridge, but clients don't build circuits that go to an IntroPoint or RendPoint directly after a bridge.)
I wonder how this affects existing Tor2web clients (as they make one-hop client intro and rendezvous connections), and the upcoming rendezvous single onion services, which make one-hop intro and rendezvous server connections. (I also wonder about single onion service extends, but I think they're ok.)
This would still result in clients not having directories, however, and retrying once an hours.
3. Ideas that might work
3.1. Move all authorities to new ports
We could have each authority known to older clients start listening for connections at a new port P. We'd forward the old port to the new port. Once sufficiently many clients were using the new ports, we could disable the forwarding.
This would result in the old clients turning into zombies as above, but they would only be scrabbling at nonexistent ports, causing less load on the authorities.
[This proposal would probably be easiest to implement.]
This might not work for clients that are always on: Changing authority ports would only affect clients that have yet to bootstrap, or clients that are off for long enough for their consensus to expire. Otherwise, they can continue to contact directory mirrors in their current consensus to obtain their next consensus. For similar reasons, changing ports only works for tor versions *without* default Fallback Directory mirrors, which likely means 0.2.7 and earlier. This might have unintended consequences: Some clients have FascistFirewall set, which restricts the initial directory connection (to the authorities) to port 80. If we switched some authorities away from DirPort 80, we'd need to switch a similar number to DirPort 80. (Or make sure a sufficient number of default Fallback Directory mirrors are on DirPort 80 from 0.2.8 onwards.) (FascistFirewall also restricts OR connections to port 443, but by the time a client makes an OR connection, it has the full consensus.)
3.2. Start disabling old link protocols on relays
We could have new relays start dropping support for the old link protocols, while maintaining support on the authorities and older relays.
The result here would be a degradation of older client performance over time. They'd still behave zombieishly if the authorities dropped support, however.
If a relay fails to support a client's protocol, does the client continue to contact relays until it is successful? If so, does this place extra load on (new) relays as the proportion of new relays increases? (Or extra load on old relays which still support the old protocols?)
3.3. Changing the consensus format.
We could allow 'f' (short for "flag") as a synonym for 's' in consensus documents. Later, if we want to disable all Tor versions before today, we can change the consensus algorithm so that the consensus (or perhaps only the microdesc consensus) is spelled with 'f' lines instead of 'f' lines. This will create a consensus which older clients and relays parse as having all nodes down, which will make them not connect to the network at all.
We could similarly replace "r" with "n", or replace Running with Online, or so on.
In doing this, we could also rename fresh-until and valid-until, so that new clients would have the real expiration date, and old clients would see "this consensus never expires". This would prevent them from downloading new consensuses.
[This proposal would result in the quietest shutdown.]
Are we aiming to do this for 0.2.8? I think it would be a good idea, as adding default fallback directories makes it harder to implement some authority-only strategies for shutting down old clients. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F
On Thu, Jan 7, 2016 at 12:12 PM, Nick Mathewson <nickm@torproject.org> wrote:
[This is a significantly revised version of the last version of this proposal draft, sent here for comment.
The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html . This is more or less a complete rewrite, after I convinced myself that the original ideas wouldn't work.
Part of the functionality of that old draft has been subsumed in proposal 264.]
This is now proposal 266.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello, I think the solution to change the consensus format - section 3.3 is by far superior vs changing the ports on directory authorities or disabling old link protocols at relays side. Changing the consensus format is the cleanest way to do it and it has smallest chances to degrade performance on working up to date clients. Might as well make add the synonym as fast as possible, when we also add the shared randomness value in the consensus document, because it'll take quite some time until we are ready to say 'let's disable all Tor versions before today'. On 1/14/2016 6:26 PM, Nick Mathewson wrote:
On Thu, Jan 7, 2016 at 12:12 PM, Nick Mathewson <nickm@torproject.org> wrote:
[This is a significantly revised version of the last version of this proposal draft, sent here for comment.
The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html
. This is more or less a complete rewrite, after I convinced myself
that the original ideas wouldn't work.
Part of the functionality of that old draft has been subsumed in proposal 264.]
This is now proposal 266. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBCAAGBQJWl897AAoJEIN/pSyBJlsRLd4H/02MUaOVyHxM/O0iWVcX2qMm jnwGvlotVpKhCitb6eA19ah0R5iaX4u87VHxPB7+Z7TrJv1Hshe5cFrztNdkKLR2 5JJ/X2pAuC+kiY+LoPBEOSZroIK9/XIt8M4e/RMZeLHtmeiJevlwcx494FiYmcaz mn2aBSaqy+ncecvg2q23wNUbanga6pCR52I5xIy4hcENl3RzUCUeDWCkLI31KV/U 09evLvByphP/mQormo/iQj/JDpPUtb9deaLSDhegemrIzdutaSyn0Qyzk9c9PWTs lYkbduw7HOwacGjIrEDmfdbplCiXfVhQxO/q5QYyvpgor4N2etFw2EMqfqrXoDw= =J1Oy -----END PGP SIGNATURE-----
Hi,
Nick Mathewson: [This is a significantly revised version of the last version of this proposal draft, sent here for comment.]
Questions?
The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html
I asked some questions to this draft [0] that you may have forgotten to answer that still seem relevant; I have sniped them here.
Nick Mathewson: Draft proposal -- no number yet: How to safely drop support for old clients.
I had made an observation about the title's "fluffy and not reflective of the proposal" nature and offered some options but I feel now like the current title still isn't as clear as it is intended to be. 'Current' suggests there are past or future clients that are being overlooked. 'Obsolete' suggests that the clients are no longer used or have fallen into disuse, which is quite a presumption given the encouragement behind "upgrading" that kinda forces people to show up as current-clients-in-the-wild data points. I still recommend: - How to depreciate support for old clients Though 'Network' is a valuable descriptor (:
Frequently, we find that very old versions of Tor should no longer be supported on the network.
Where can we find research on the impact?
Disabling all versions that don't support this proposal
With all due respect, doesn't Microsoft do stuff like this? Is the impact so large that they require this level of action?
if we want to disable all Tor versions before today that do not support this proposal.
Is the proposal for 5 years in the past, pre this version, or can/will the cutoff be specified willy-nilly?
It might be good to make any disable-switches advisory rather than mandatory.
Ultimately, if this is the approach, and it is in the hands of each client operator, this will be good. Though it would be great to hear from those who use previous versions to thwart limitations in "upgrades" to Firefox, such as the way media is streamed, windows are re-sized, and so on (:
s7r: it'll take quite some time until we are ready to say 'let's disable all Tor versions before today'.
How long do you think? Wordlife, Spencer [0]: https://lists.torproject.org/pipermail/tor-dev/2015-September/009601.html
On Sun, Jan 17, 2016 at 12:32 PM, Spencer <spencerone@openmailbox.org> wrote:
Hi,
Nick Mathewson: [This is a significantly revised version of the last version of this proposal draft, sent here for comment.]
Questions?
The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html
I asked some questions to this draft [0] that you may have forgotten to answer that still seem relevant; I have sniped them here.
Hi, Spencer! Indeed, these are relevant, and since the discussion meeting for the latest incarnation of these proposals is soon (see https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/MeetingS... ) I should try to clarify! However, a bunch of your questions don't seem to apply to proposal 266.
Nick Mathewson: Draft proposal -- no number yet: How to safely drop support for old clients.
I had made an observation about the title's "fluffy and not reflective of the proposal" nature and offered some options but I feel now like the current title still isn't as clear as it is intended to be.
'Current' suggests there are past or future clients that are being overlooked.
'Obsolete' suggests that the clients are no longer used or have fallen into disuse, which is quite a presumption given the encouragement behind "upgrading" that kinda forces people to show up as current-clients-in-the-wild data points.
I still recommend: - How to depreciate support for old clients
Though 'Network' is a valuable descriptor (:
I've tried to split the first version of the proposal into 2. Section 4 of Proposal 264, "Putting version numbers on the Tor subprotocols" might also be called "How we can make clients that follow this proposal stop connecting to the network." Proposal 266 might be called "How to make clients that exist today stop connection to the Tor network.
Frequently, we find that very old versions of Tor should no longer be supported on the network.
Where can we find research on the impact?
I'm not aware of anything published. There are a few important reasons why very old clients shouldn't be supported. 1) A non-updated Tor is insecure. Nobody backports bugfixes back to 0.2.0, for example. 2) For some old versions, the bulk of deployed versions appear to be one or more defunct botnets; have a look around tor-dev around 2 years ago for threads about dealing with said botnets. 3) Some TLS protocol features used in very old Tor versions (such as SSL renegotiation and tricky games with the ciphersuites listed in the ClientHello), force us to tie our implementation to OpenSSL and its close derivatives, and make our code significantly harder to maintain seceurly so long as they remain.
Disabling all versions that don't support this proposal
With all due respect, doesn't Microsoft do stuff like this? Is the impact so large that they require this level of action?
Yes, the impact is so large it requires this level of action. Microsoft, on the other hand, has millions of Windows XP clients still running today, making the internet less secure. (Also, the actual content of proposal is far milder than this heading makes it sound. My goal with proposal 266 is to work under the assumption that every current Tor MAY eventually prove so broken it needs to go away; not to take as a given that (eg) 0.2.7 will eventually need to get deprecated this hard.)
if we want to disable all Tor versions before today that do not support this proposal.
Is the proposal for 5 years in the past, pre this version, or can/will the cutoff be specified willy-nilly?
See above; proposal 266 attempts to describe a way to disable all clients that do not support proposals 264 and 266. The extent to which this will ever be necessary, or the time at which any of this will be necessary, is unknown. To do this would require the cooperation of a majority of directory authorities. And I'm not planning to advocate that anything be deprecated without good reason. Right now, for example, I think we should stop 0.2.3 and earlier from using Tor: their use of RSA1024 / DH1024 makes their security quite questionable, and the zombie vestigial botnet on 0.2.2, even though it's slowly decaying, is doing nobody any favors. I'd suggest that everybody actually _running_ a client or server should really be on 0.2.6 or later, but I'm not aware of any current need to kick 0.2.4 or 0.2.5 off the network. hth, -- Nick
Hi,
Nick Mathewson: I should try to clarify!
Awesome!
questions don't seem to apply to proposal 266
They are about the central control of a [somewhat] distributed network, specifically, the execution of clients on behalf of the operator. So, #264 & #266.
I've tried to split the first version of the proposal into 2.
I understand the proposals as:
prop#264 is for how things _should_ work ; prop#266 is what we do in the absence of client-side support in existing Tor versions.
anybody who doesn't know how to die via prop264 will be killable in whatever way we choose for prop266.
And would recommend the titles [though obviously not as relevant as the contents]: 'How to ensure client death' 'How to kill clients that wont die'
I'm not aware of anything published.
Bummer ):
reasons:
1) A non-updated Tor is insecure. 2) the bulk of [some older] deployed versions appear to be defunct botnets 3) [Depreciated] features
Word.
impact is so large it requires this level of action
Where can this impact be studied? Given there is no research, there must be a way to visualize the impact.
Windows XP clients still running today, making the internet less secure.
Business clients pay money to keep MS supporting XP systems, though that doesn't weaken the internet as a whole.
every current Tor MAY eventually prove so broken it needs to go away
Word. It feels like a decision that the operator should make but I kind of see the issue with abandoned clients. The poison consensus seems fun. Thanks for taking the time to write, it means a lot (: Wordlife, Spencer
The list of server/client -versions seems long and unuseful... 0.2.4.23 0.2.4.24 0.2.4.25 0.2.4.26 0.2.4.27 0.2.5.8-rc 0.2.5.9-rc 0.2.5.10 0.2.5.11 0.2.5.12 0.2.6.5-rc 0.2.6.6 0.2.6.7 0.2.6.8 0.2.6.9 0.2.6.10 0.2.7.1-alpha 0.2.7.2-alpha 0.2.7.3-rc 0.2.7.4-rc 0.2.7.5 0.2.7.6 0.2.8.1-alpha I'd cut it to a fixed falloff pattern more like... 0.2.4.27 0.2.5.11 0.2.5.12 0.2.6.8 0.2.6.9 0.2.6.10 0.2.7.3-rc 0.2.7.4-rc 0.2.7.5 0.2.7.6 0.2.8.1-alpha ... to be made shorter by dropping any with security issues, refill with future releases. Point: stay current.
participants (7)
-
grarpamp -
Nick Mathewson -
Nick Mathewson -
s7r -
Sebastian G. <bastik.tor> -
Spencer -
Tim Wilson-Brown - teor