- tor-dev - lists.torproject.org

Proposal 337: A simpler way to decide, "Is this guard usable?"
by Nick Mathewson 23 Oct '21

23 Oct '21

``` Filename: 337-simpler-guard-usability.md Title: A simpler way to decide, "Is this guard usable?" Author: Nick Mathewson Created: 2021-10-22 Status: Open ``` # Introduction The current `guard-spec` describes a mechanism for how to behave when our primary guards are unreachable, and we don't know which other guards are reachable. This proposal describes a simpler method, currently implemented in [Arti](https://gitlab.torproject.org/tpo/core/arti/). (Note that this method might not actually give different results: its only advantage is that it is much simpler to implement.) ## The task at hand For illustration, we'll assume that our primary guards are P1, P2, and P3, and our subsequent guards (in preference order) are G1, G2, G3, and so on. The status of each guard is Reachable (we think we can connect to it), Unreachable (we think it's down), or Unknown (we haven't tried it recently). The question becomes, "What should we do when P1, P2, and P3 are Unreachable, and G1, G2, ... are all Unknown"? In this circumstance, we _could_ say that we only build circuits to G1, wait for them to succeed or fail, and only try G2 if we see that the circuits to G1 have failed completely. But that delays in the case that G1 is down. Instead, the first time we get a circuit request, we try to build one circuit to G1. On the next circuit request, if the circuit to G1 isn't done yet, we launch a circuit to G2 instead. The next request (if the G1 and G2 circuits are still pending) goes to G3, and so on. But (here's the critical part!) we don't actually _use_ the circuit to G2 unless the circuit to G1 fails, and we don't actually _use_ the circuit to G3 unless the circuits to G1 and G2 both fail. This approach causes Tor clients to check the status of multiple possible guards in parallel, while not actually _using_ any guard until we're sure that all the guards we'd rather use are down. ## The current algorithm and its drawbacks For the current algorithm, see `guard-spec` section 4.9: circuits are exploratory if they are not using a primary guard. If such an exploratory circuit is `waiting_for_better_guard`, then we advance it (or not) depending on the status of all other _circuits_ using guards that we'd rather be using. In other words, the current algorithm is described in terms of actions to take with given circuits. For Arti (and for other modular Tor implementations), however, this algorithm is a bit of a pain: it introduces dependencies between the guard code and the circuit handling code, requiring each one to mess with the other. # Proposal I suggest that we describe an alternative algorithm for handing circuits to non-primary guards, to be used in preference to the current algorithm. Unlike the existing approach, it isolates the guard logic a bit better from the circuit logic. ## Handling exploratory circuits When all primary guards are Unreachable, we need to try non-primary guards. We select the first such guard (in preference order) that is neither Unreachable nor Pending. Whenever we give out such a guard, if the guard's status is Unknown, then we call that guard "Pending" until the attempt to use it succeeds or fails. We remember when the guard became Pending. > Aside: None of the above is a change from our existing specification. After completing a circuit, the implementation must check whether its guard is usable. A guard is usable according to these rules: Primary guards are always usable. Non-primary guards are usable for a given circuit if every guard earlier in the preference list is either unsuitable for that circuit (e.g. because of family restrictions), or marked as Unreachable, or has been pending for at least `{NONPRIMARY_GUARD_CONNECT_TIMEOUT}`. Non-primary guards are unusable for a given circuit if some guard earlier in the preference list is suitable for the circuit _and_ Reachable. Non-primary guards are unusable if they have not become usable after `{NONPRIMARY_GUARD_IDLE_TIMEOUT}` seconds. If a circuit's guard is neither usable nor unusable immediately, the circuit is not discarded; instead, it is kept (but not used) until it becomes usable or unusable. > I am not 100% sure whether this description produces the same behavior > as the current guard-spec, but it is simpler to describe, and has > proven to be simpler to implement. ## Implications for program design. (This entire section is implementation detail to explain why this is a simplification from the previous algorithm. It is for explanatory purposes only and is not part of the spec.) With this algorithm, we cut down the interaction between the guard code and the circuit code considerably, but we do not remove it entirely. Instead, there remains (in Arti terms) a pair of communication channels between the circuit manager and the guard manager: * Whenever a guard is given to the circuit manager, the circuit manager receives the write end of a single-use channel to report whether the guard has succeeded or failed. * Whenever a non-primary guard is given to the circuit manager, the circuit receives the read end of a single-use channel that will tell it whether the guard is usable or unusable. This channel doesn't report anything until the guard has one status or the other. With this design, the circuit manager never needs to look at the list of guards, and the guard manager never needs to look at the list of circuits. ## Subtleties concerning "guard success" Note that the above definitions of a Reachable guard depend on reporting when the _guard_ is successful or failed. This is not necessarily the same as reporting whether the _circuit_ is successful or failed. For example, a circuit that fails after the first hop does not necessarily indicate that there's anything wrong with the guard. Similarly, we can reasonably conclude that the guard is working (at least somewhat) as long as we have an open channel to it.

2 1

Proposal 336: Randomized schedule for guard retries
by Nick Mathewson 23 Oct '21

23 Oct '21

``` Filename: 336-randomize-guard-retries.md Title: Randomized schedule for guard retries Author: Nick Mathewson Created: 2021-10-22 Status: Open ``` # Introduction When we notice that a guard isn't working, we don't mark it as retriable until a certain interval has passed. Currently, these intervals are fixed, as described in the documentation for `GUARDS_RETRY_SCHED` in `guard-spec` appendix A.1. Here we propose using a randomized retry interval instead, based on the same decorrelated-jitter algorithm we use for directory retries. The upside of this approach is that it makes our behavior in the presence of an unreliable network a bit harder for an attacker to predict. It also means that if a guard goes down for a while, its clients will notice that it is up at staggered times, rather than probing it in lock-step. The downside of this approach is that we can, if we get unlucky enough, completely fail to notice that a preferred guard is online when we would otherwise have noticed sooner. Note that when a guard is marked retriable, it isn't necessarily retried immediately. Instead, its status is changed from "Unreachable" to "Unknown", which will cause it to get retried. For reference, our previous schedule was: ``` {param:PRIMARY_GUARDS_RETRY_SCHED} -- every 10 minutes for the first six hours, -- every 90 minutes for the next 90 hours, -- every 4 hours for the next 3 days, -- every 9 hours thereafter. {param:GUARDS_RETRY_SCHED} -- -- every hour for the first six hours, -- every 4 hours for the next 90 hours, -- every 18 hours for the next 3 days, -- every 36 hours thereafter. ``` # The new algorithm We re-use the decorrelated-jitter algorithm from `dir-spec` section 5.5. The specific formula used to compute the 'i+1'th delay is: ``` Delay_{i+1} = MIN(cap, random_between(lower_bound, upper_bound)) where upper_bound = MAX(lower_bound+1, Delay_i * 3) lower_bound = MAX(1, base_delay). ``` For primary guards, we set base_delay to 30 seconds and cap to 6 hours. For non-primary guards, we set base_delay to 10 minutes and cap to 36 hours. (These parameters were selected by simulating the results of using them until they looked "a bit more aggressive" than the current algorithm, but not too much.) The average behavior for the new primary schedule is: ``` First 1.0 hours: 10.14283 attempts. (Avg delay 4m 47.41s) First 6.0 hours: 19.02377 attempts. (Avg delay 15m 36.95s) First 96.0 hours: 56.11173 attempts. (Avg delay 1h 40m 3.13s) First 168.0 hours: 83.67091 attempts. (Avg delay 1h 58m 43.16s) Steady state: 2h 36m 44.63s between attempts. ``` The average behavior for the new non-primary schedule is: ``` First 1.0 hours: 3.08069 attempts. (Avg delay 14m 26.08s) First 6.0 hours: 8.1473 attempts. (Avg delay 35m 25.27s) First 96.0 hours: 22.57442 attempts. (Avg delay 3h 49m 32.16s) First 168.0 hours: 29.02873 attempts. (Avg delay 5h 27m 2.36s) Steady state: 11h 15m 28.47s between attempts. ```

2 1

Proposal 335: An authority-only design for MiddleOnly
by Nick Mathewson 17 Oct '21

17 Oct '21

``` Filename: 335-middle-only-redux.md Title: An authority-only design for MiddleOnly Author: Nick Mathewson Created: 2021-10-08 Status: Open ``` # Introduction This proposal describes an alternative design for a `MiddleOnly` flag. Instead of making changes at the client level, it adds a little increased complexity at the directory authority's voting process. In return for that complexity, this design will work without additional changes required from Tor clients. For additional motivation and discussion see proposal 334 by Neel Chauhan, and the related discussions on tor-dev. # Protocol changes ## Generating votes When voting for a relay with the `MiddleOnly` flag, an authority should set all flags indicating that a relay is unusable for a particular purpose, and against all flags indicating that the relay is usable for a particular position. These flags SHOULD be set in a vote whenever `MiddleOnly` is present, and only when the authority is configured to vote on the `BadExit` flag. * `BadExit` These flags SHOULD be cleared in a vote whenever `MiddleOnly` is present. * `Exit` * `Guard` * `HSDir` * `V2Dir` ## Computing a consensus This proposal will introduce a new consensus method (probably 32). Whenever computing a consensus using that consensus method or later, authorities post-process the set of flags that appear in the consensus after flag voting takes place, by applying the same rule as above. That is, with this consensus method, the authorities first compute the presence or absence of each flag on each relay as usual. Then, if the `MiddleOnly` flag is present, the authorities set `BadExit`, and clear `Exit`, `Guard`, `HSDir`, and `V2Dir`. # Configuring authorities We'll need a means for configuring which relays will receive this flag. For now, we'll just reuse the same mechanism as `AuthDirReject` and `AuthDirBadExit`: a set of torrc configuration lines listing relays by address. We'll call this `AuthDirMiddleOnly`. We'll also add an `AuthDirListsMiddleOnly` option to turn on or off voting on this option at all. # Notes on safety and migration Under this design, the MiddleOnly option becomes useful immediately, since authorities that use it will stop voting for certain additional options for MiddleOnly relays without waiting for the other authorities. We don't need to worry about a single authority setting MiddleOnly unilaterally for all relays, since the MiddleOnly flag will have no special effect until most authorities have upgraded to the new consensus method.

3 2

Arti report 7: September 30 through October 13
by Nick Mathewson 14 Oct '21

14 Oct '21

# Arti Report 7: September 30 through October 13 ## Guards are working! Finally, at long last, we've got Guard nodes working in Arti! When we first made our roadmap, we set our first milestone as "Arti has all the features necessary for basic anonymity." Now that Guards are implemented, all of those features are done! Does this mean that Arti is now production-ready for anonymous use? No, not yet. Just because we no longer have _missing_ privacy features doesn't mean that those privacy features are actually implemented correctly. We'll need a lot more testing, debugging, and experimentation. Also, it's very possible we've forgotten something. We'll need careful auditing to be sure. ### Interesting guard issues There are more issues to fix and features to add to the guard implementation down the road. For a current list, have a look at [this comment](https://gitlab.torproject.org/tpo/core/arti/-/issues/58#note_27549…, which lists them. ## What's next Over the next couple of weeks, we'll be working on smaller issues, bugfixes, and miscellaneous cleanups for an upcoming "0.0.1" release, targeting November 1. You can see our latest progress [here]((https://gitlab.torproject.org/tpo/core/arti/-/milestones/6). ## Thanks! Thanks to Jani Monoses for progress towards a WASM port, and Trinity Pointard for updates to dependencies.

1 0

IANA well-known URI suffix registration for tor-relay-fingerprints file
by nusenu 10 Oct '21

10 Oct '21

Hi, to reduce the risk of certain kinds of attacks I'm aiming to submit a short spec (like [1]) in accordance with RFC8615 https://tools.ietf.org/html/rfc8615#section-3.1 for the registration of of a suffix for the verifyurl from https://github.com/nusenu/ContactInfo-Information-Sharing-Specification#ver… The name I have in mind is: tor-relay-fingerprints The file contains comments (lines starting with #) and relay fingerprints. https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml Let me know if you have any feedback on this and whether you have any opinion on whether you (The Torproject) wants to be the change controller. The only question that came up was: Will there be two types of relay fingerprints in the future (Ed25519)? thanks, nusenu -- https://mastodon.social/@nusenu [1] https://keybase.io/docs/keybase_well_known

5 22

tor Prometheus Metrics Documentation
by nusenu 10 Oct '21

10 Oct '21

Hi, since Neel is working on unix socket support [1] for MetricsPort I'm preparing to add support for it to relayor. I don't like to add additional attack surface to a relay but since tor does not support TLS and basic auth on the MetricsPort I'll likely install nginx that takes care of TLS and authentication. If you have any opinion on that I'd also like to hear it. It would be nice to have a documentation for all tor prometheus metrics in the man page or on a website, some information can be found at [3] but not all. Is there already a complete documentation for tor prometheus metrics data? Here is an example of how other projects have documented their metrics [5]. I don't know if it is too late for that but it would be great to use the well known IANA RCODE [4] naming for the DNS return codes where possible instead of the custom naming. examples (just guessing): tor_relay_exit_dns_error_total{... reason="format"} -> FormErr? tor_relay_exit_dns_error_total{... reason="notexist"} -> NXDomain? kind regards, nusenu [1] https://gitlab.torproject.org/tpo/core/tor/-/merge_requests/453 [3] https://support.torproject.org/relay-operators/relay-bridge-overloaded/ [4] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… [5] https://doc.powerdns.com/recursor/metrics.html#metricnames related feature request: [2] https://gitlab.torproject.org/tpo/core/tor/-/issues/40194 -- https://nusenu.github.io

1 0

ed25519_master_id_public_key -> ed25519 id
by nusenu 10 Oct '21

10 Oct '21

Hi, given a relay's ed25519_master_id_public_key file, is there a simple way to generate the 43 chars long ed25519 identity string (also found in fingerprint-ed25519)? thanks, nusenu -- https://nusenu.github.io

2 3

Re: [tor-dev] Proposal 335: An authority-only design for MiddleOnly
by ezhigp＠Safe-mail.net 08 Oct '21

08 Oct '21

> ``` > Filename: 335-middle-only-redux.md > Title: An authority-only design for MiddleOnly > Author: Nick Mathewson > Created: 2021-10-08 > Status: Open > ... > > These flags SHOULD be set in a vote whenever `MiddleOnly` is > present, and only when the authority is configured to vote on the > `BadExit` flag. > > * `BadExit` > > These flags SHOULD be cleared in a vote whenever `MiddleOnly` is > present. > > * `Exit` I believe that BadExit is supposed to be given together with Exit, to mark that technically it's possible to exit from this relay, but it is not recommended unless you know what you do. > * `Guard` > * `HSDir` > * `V2Dir` It looks like we don't fear such a relay at Intro? Or it is a sign that this proposal is only a set of quick actions before #334?

1 0

v2 Onions, IPv6 BiDir P2P Capability Regression... [re: Relays running unsupported EOL ver]
by grarpamp 07 Oct '21

07 Oct '21

On that, and regardless of what TPO Inc says re... tor is a bit decentralized, and freely licensed, as such... DirAuths, HSDirs, Relays are all free to choose to continue signing over and running older versions in order to support the community of tor clients that are happily using useful features that the Tor Project on its high horse demands with its blanket fiat fist and refusing to consider and support users... to remove from such users, herein as noted before, v2 onions and simple OnionCat IPv6 /48+80 UDP transport P2P E2E BiDir addressing and thus all current IPv6 capable apps incl VOIP Bittorrent comms status chat game coins etc and future apps that needing such tech to operate across onionland. That's bad, and a big regression in service, both active now, and preventing potential development on interesting new uses and applications in the future. These users, these features, apps, and their recognized tradeoffs, their free choices made therein, are of merit and import. And Tor Project is choosing to censor and ban v2 and them without regard. Perhaps some v2 users will likewise email all the relays seeking not their censorship, but instead their support. But perhaps due to any needs they may have to remain unknown they may not be able to voice out to lists etc, as such the surfaceweb cannot make dis-use assumptions and must indeed hold v2 in their stead for them. Kudos to those distributed DirAuths, HSDirs, and Relays who refuse to comply with Tor Project Inc demands to crush all v2 onion users and their legitimate use cases. The better path is for TPO to either recognize and continue, or to modularize v2 onion support out to community stewardship such that the features that it provides may continue until a future vN or a seamless scalable externally attached solution for the same features and uses is found. And to make v3 the default choice "advertised" in docs and configs, with v2 a lesser noted option yet for just providing above capabilities. Tor Project still doesn't have a version feature matrix table, nor an honest open balanced and free to all authors wiki education page detailing use case and tradeoff notes for users about the v2+OnionCat vs v3 tech capabilities, thereby allowing tor users to freely make educated choices therein on their own. Instead it has a bricked up and censored prop blog and lists. Like TPO's censorship, that's unfair, unwarranted, questionable, and needs to end. ps: Yes, upgrade, if merely behind on versions sake, lots of good fixes, improvements, performance, features, security, etc come with those. But when it comes to a big and permanent regression of overall network capability issue such as this, that's different, and needs consideration beyond the vague handwavy universal whitewash "reason" being uttered by TPO such as "v2 insecure"... which is false re educated tradeoffs and uses that users are free to evaluate and choose on their own. In fact, users are in better more intelligent aware position having learned a bit of something and made that themselves. Users are also free to eval by bring forward to today sense of the NSA's 10+ year old slides that noted "Tor Stinks". What conditions, designs, capabilities, etc have changed since then. Cheers all. On 10/5/21, Georg Koppen <gk(a)torproject.org> wrote: > Relays running unsupported Tor versions is a problem we have never > really dealt with in a systematic way in the way. Some of you might > recall that we (with the help of volunteers) tried back in 2019/2020 to > get operators, running an unsupported Tor version, to upgrade[1] but > then we dropped the ball. Alas. > > We just started that process again by contacting every relay operator > running an outdated Tor version (any version not 0.3.5.x or 0.4.5.x or > 0.4.6.x or 0.4.7.x) by email where possible. Additionally, we created a > wiki page outlining the current process and things we still need to > figure out.[2] On that page we plan to make statistics related to the > EOL relay removal available as well, including the final list of relays > we'll reject. Thus, stay tuned. Feedback, as always, is very much welcome! > > We plan to keep this topic on our radar this time while refining the > process as we go. Meanwhile, if you are running a relay with an > unsupported Tor version, please upgrade for the sake of our users' safety. > > If you need help, join us on #tor-relays or #tor-relays:matrix.org if > you use Element. > > [1] https://blog.torproject.org/removing-end-life-relays-network > [2] https://gitlab.torproject.org/tpo/network-health/team/-/wikis/Relay-EOL-pol…

1 0

Updated Proposal 324: RTT-based Congestion Control for Tor
by Mike Perry 06 Oct '21

06 Oct '21

As a heads up, I have updated Proposal 324: RTT-based Congestion Control for Tor - https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/324-rt… The updated version has a new congestion control algorithm, detailed specifics on how to accurately measure RTT and BDP, and descriptions of algorithm parameter values and tuning experiments, among other improvements. A C implementation of these algorithms has been merged to tor.git orgin/main. They live in src/core/or/congestion_control_*.[ch]. These algorithms are not enabled yet; we still need to implement flow control and negotiation, as well as do shadow experimentation to determine good candidate parameter values to further tune on live. Here is a full changelog since the proposal was last published to tor-dev: - Correct algorithms to update once per congestion window - Add orconn blocking and edge connection checks as signals - Specify BDP estimators based on onion service testing and eval. - Specify a pure BDP tracking congestion control algorithm (TOR_NOLA) - Update consensus parameters with tuning notes - Document what we learned so far in live onion service experimentation - Mention that we need to test intermittent downloads in Shadow/live - Describe clock jump and stall detection during RTT measurement - Mention if we calculate package window, it can become negative - Break off the backwards ECN idea into a future idea draft: https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/ideas/… - Update old trac URLs - Add Acknowledgements Review and comments are welcome! -- Mike Perry

1 1