tor-dev October 2021

tor-dev@lists.torproject.org

11 participants
18 discussions

the consequences of deprecating debian alpha repos with every new major branch
by nusenu 21 Dec '21

21 Dec '21

Hi, tldr: - more outdated relays (that is a claim I'm making and you could easily proof me wrong by recreating the 0.3.3.x alpha repos and ship 0.3.3.7 in them and see how things evolve after a week or so) - more work for the tpo website maintainer - less happy relay operators [3][4] - more work for repo maintainers? (since a new repo needs to be created) When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23 I was about to submit a PR for the website to include it in the sources.list generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first. The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to update the repo generator for 0.3.4.x nonetheless and here is why. Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search) (> 3.2% CW fraction) Then I realized that this was the last version the tor-experimental-0.3.3.x-* repos were shipping before they got abandoned due to the new 0.3.4.x-* repos (I can no longer verify it since they got removed by now). Peter made it clear in the past that the current way to have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie) will not change [2]: > If you can't be bothered to change your sources.list once or twice a > year, then you probably should be running stable. but maybe someone else would be willing to invoke a "ln" commands everytime a new new alpha repo is born. tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie once 0.3.5.x repos are created the link would point to tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie It is my opinion that this will help reduce the amount of relays running outdated versions of tor. It will certainly avoid having to update the tpo website, which isn't a big task and could probably be automated but it isn't done currently. "..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas (and break setups)!" Yes, and I think that is better than relays stuck on an older version because the former repo no longer exists and operators still can choose the old repos which will not jump to newer major versions. [1] https://www.torproject.org/docs/debian.html.en#ubuntu [2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3 [3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html [4] https://trac.torproject.org/projects/tor/ticket/26474 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

4 7

Client identification for authenticated onions
by cho8jeiv4aus＠paperboats.net 12 Dec '21

12 Dec '21

Hi there. I had an idea recently for an onion service to improve the UX of sites that require a login. The site would have two onions: one for those who want to use onion auth and another for those who don't or are still setting it up. A user would first sign in with a username+password on the unauthenticated onion and click a button to generate a certificate associated with their account. Then they would add the public key to their browser and visit the authenticated onion. The application server would then match the pubkey used to authenticate with an account in the database, and log them in automatically. I've looked in the mailing list archives and `man 1 tor` but didn't find anything that would facilitate this. The closest, it seems, is HiddenServiceExportCircuitID, but that is for *circuit* IDs, not *client* IDs. Is this possible to implement, either as an operator or as a Tor developer? As an operator, an alternative would be to generate one (authenticated) onion service per user and route them all to the same place with different Host headers, but that seems rather inefficient, and I don't know how well the tor daemon scales up to hundreds of onion services anyway. P.S. I didn't find an easy way to do full text search on the mailing list archives, so I wrote a little script to download them all. I've attached it in case it ends up useful. It requires python3.8+ and you'll need to `pip install aiohttp anyio BeautifulSoup4` first. After that you can run `./pipermail_fetch.py https://lists.torproject.org/pipermail/tor-dev/` and then something like `rg --context 3 --search-zip '^[^>].*search term here'` will do the trick.

3 2

Accessing Shared Random Values as a Protocol on top of Tor
by Sebastian Hoffmann 07 Dec '21

07 Dec '21

Hi, I'm wondering if I can access the shared random value[1] while developing a protocol/application on top of Tor onion services. The application is still in early development, but it would be great if I could depend on the shared random value. If this is not the correct mailing list for this question, I would be glad if you could point me to one. --Sebastian [1]: PUB-SHAREDRANDOM in https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt

2 1

A Simple Web of Trust for Tor Relay Operator IDs
by nusenu 05 Nov '21

05 Nov '21

Hi, I wrote down a spec for a simple web of trust for relay operator IDs: https://gitlab.torproject.org/nusenu/torspec/-/blob/simple-wot-for-relay-op… This is related to: https://gitlab.torproject.org/tpo/network-health/metrics/relay-search/-/iss… https://lists.torproject.org/pipermail/tor-relays/2020-July/018656.html kind regards, nusenu -- https://nusenu.github.io

4 6

proposal 328 status
by nusenu 01 Nov '21

01 Nov '21

Hi, I'm wondering if the current version of the text is the latest available version of it or if there is somewhere a newer version that hasn't been pushed yet? https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/328-re… "Status: Draft" but it is already in released tor versions. also in the context of this change: https://gitlab.torproject.org/tpo/core/tor/-/issues/40364 the proposal still mentions extra-info documents. thanks, nusenu -- https://nusenu.github.io

2 3

Re: [tor-dev] [tor-relays] Relays running an unsupported (EOL) Tor version
by grarpamp 28 Oct '21

28 Oct '21

> mpan: >> Is there any data available that sheds light on >> why operators run outdated versions Besides latent OS packages, or being busy, or simply not updating things, those are natures of computing world anyway... or having no network operations crypto-monetization model as some nextgen p2p overlay-exit networks are now working with... there can be other reasons people may run different versions... tor the software is different from from Tor Project Inc. The software is opensource and BSD licensed. Thus anyone may copy, run, share, redistribute, fork, modify, support it, create and vote in any consensus, run relays, run more networks than just tor on their nodes, run services, etc, completely without involvement from, and indeed in full disregard of whatever Tor Project Inc and its people may say. The tor protocol, code, and operational network are not the property of Tor Project Inc, nor are its operators and users under its command and control. tor's users may run whichever of the many tor client implementations they wish to run, and may run whatever protocols, applications, and uses over the tor network that they wish. tor's relay operators and dirauths are free to run and support the running of older, different, forked, or project protocol addition enhancement or migration versions, in part in order to support features that some of the entire global userbase of tor are using, and have no good replacement for, or may wish to develop apps to, and run them upon or over, such features and capabilities in the future. Such as the censored topic of v2 onions with OnionCat, OnionVPN, etc in order to continue or design, deploy, and run new p2p apps comms, etc over those, ie... https://www.onioncat.org/ So a large number of relays may be freely and rightly choosing, as is their right if they wish, to continue running a v2 onion version for that. And overlay nets have some great uses, some examples of which Tor Project advertises, such that those users or even operators may prefer not to post, so you may want to consider some of those good uses for them in their stead, even ones that are dependent on such versions. And another topical fact re versions, which will be censored, is that attackers do falsify their version strings, and a lot more, in order to run whatever custom exploits they want against the network. And that Traffic Analysis and Sybil attacks nodes are real and in use. And that there is no longer sufficient warning and ongoing visibly posted education on these and other matters, in particular at point of download, install, splashpage, and frontpage... even some warnings were removed... versioned away. These are the sort of lack of info that can put users at severe risk. Tor Project Inc is censoring embarassing or simple facts / info. And is now attempting to silence and kill useful and in use features and future application possibilities based on them, whitewashing them with handwavy vague absolutist claims such as "old" or "insecure", instead of creating detailed comparisons for users. tor's users and operators are the ones who get to choose their own versions, and use appropriate features / freedom / security tradeoffs, not by the sole dictate of Tor Project Inc. Which by the way is funded to $Millions per year so that is not an argument to killing otherwise modularizable features either, but may be why there are insufficient disclaimers, and censors. On 10/28/21, Georg Koppen <gk(a)torproject.org> wrote: > I don't think we have [...] feedback Well when Tor's hypocrites, liars, and frauds are censoring feedback etc off all their "bricked up" fora, full stop. Thus their users and operators are missing information and topical discussions may hardly flourish can they in an environment of censor, chilled, and steered speech. The censorship of conversation on tor by Tor Project Inc and its people... while publicizing claims to cherish and uphold Free Speech ideals... is blatantly hypocritical, and it must end.

1 0

Arti report 8: October 14 through October 27
by Nick Mathewson 27 Oct '21

27 Oct '21

# Arti Report 7: October 14 through October 27 # Towards Arti 0.0.1 On Monday November 1, we're planning to put out Arti version 0.0.1. Our goal is to include all the features that are needed for reasonable security, and a top-level API that isn't _so_ far from completely wrong. Right now, it looks like we're in good shape for that timeline. Since the last update, we've fixed a lot of smaller issues, including: * running out of files on OSX * making it harder to accidentally leak DNS requests * lots of API refactoring * restored the ability for multiple Arti instances to share a single set of local directories * easier setup * easier API for stream isolatoin * more reliable tests for asynchronous timeout-based logic * and more! We're now working on documentation and usability. There's now an example program in [`crates/arti-client/examples`](https://gitlab.torproject.org/tpo/core/arti/-/blob/main/crates/arti-client/examples/hyper.rs) that uses [`hyper`](https://hyper.rs/) to download an HTTP document, and we're adding examples elsewhere in our top-level API documentation. # Followup from our guard implementation Our work on guards (completed in the last round) has kicked off a new batch of guard research. As with other parts of the Tor protocols, implementing our specification in Rust has exposed some weaknesses or confusing points in our design. In response to issues that we found, we've opened a few issues and written a couple of proposals to improve our code. For example, see: * [Proposal 336](https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/3…, to randomize guard retry timing. * [Proposal 337](https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/3…, to describe a simpler formulation of our guard behavior. * [torspec#67](https://gitlab.torproject.org/tpo/core/torspec/-/issues/67), to treat guards as usable more eagerly. * [torspec#68](https://gitlab.torproject.org/tpo/core/torspec/-/issues/68), to improve our behavior when our primary guards are down. # What's next Once the next release is out, we'll turn our attention to Arti 0.1.0. For that release, our priority target is beta testing and experimental embedding. We'll be working to improve performance and to stabilize a set of stable APIs to provide a wider set of functionality for applications that want to embed Arti in different scenarios. We'll need your help, though: we will need feedback from everybody who's interested in using our code, in order to make sure that we're providing solid, usable interfaces that do what you need. Please try it out, and let us know what's bad or missing! We're hoping to get Arti 0.1.0 released some time in March of 2022, with multiple smaller update releases between now and then. At our bi-weekly meeting next Wednesday, we'll be prioritizing tasks for the next set of releases, and refining our roadmap for the coming months. Why not join us? Information is on the [meeting pad](https://pad.riseup.net/p/arti-meeting-pad-keep). # Thanks! Thanks to Jani Monoses and Dimitris Apostolou for their patches in this release! And special thanks and welcome to eta, our new team member! She's started with Tor last Tuesday, and has hit the ground running, already fixing substantial bugs and refactoring a bunch of tricky code. It's great to watch the pace and quality of our development improve as more people join the team!

1 0

Proposal 337: A simpler way to decide, "Is this guard usable?"
by Nick Mathewson 22 Oct '21

22 Oct '21

``` Filename: 337-simpler-guard-usability.md Title: A simpler way to decide, "Is this guard usable?" Author: Nick Mathewson Created: 2021-10-22 Status: Open ``` # Introduction The current `guard-spec` describes a mechanism for how to behave when our primary guards are unreachable, and we don't know which other guards are reachable. This proposal describes a simpler method, currently implemented in [Arti](https://gitlab.torproject.org/tpo/core/arti/). (Note that this method might not actually give different results: its only advantage is that it is much simpler to implement.) ## The task at hand For illustration, we'll assume that our primary guards are P1, P2, and P3, and our subsequent guards (in preference order) are G1, G2, G3, and so on. The status of each guard is Reachable (we think we can connect to it), Unreachable (we think it's down), or Unknown (we haven't tried it recently). The question becomes, "What should we do when P1, P2, and P3 are Unreachable, and G1, G2, ... are all Unknown"? In this circumstance, we _could_ say that we only build circuits to G1, wait for them to succeed or fail, and only try G2 if we see that the circuits to G1 have failed completely. But that delays in the case that G1 is down. Instead, the first time we get a circuit request, we try to build one circuit to G1. On the next circuit request, if the circuit to G1 isn't done yet, we launch a circuit to G2 instead. The next request (if the G1 and G2 circuits are still pending) goes to G3, and so on. But (here's the critical part!) we don't actually _use_ the circuit to G2 unless the circuit to G1 fails, and we don't actually _use_ the circuit to G3 unless the circuits to G1 and G2 both fail. This approach causes Tor clients to check the status of multiple possible guards in parallel, while not actually _using_ any guard until we're sure that all the guards we'd rather use are down. ## The current algorithm and its drawbacks For the current algorithm, see `guard-spec` section 4.9: circuits are exploratory if they are not using a primary guard. If such an exploratory circuit is `waiting_for_better_guard`, then we advance it (or not) depending on the status of all other _circuits_ using guards that we'd rather be using. In other words, the current algorithm is described in terms of actions to take with given circuits. For Arti (and for other modular Tor implementations), however, this algorithm is a bit of a pain: it introduces dependencies between the guard code and the circuit handling code, requiring each one to mess with the other. # Proposal I suggest that we describe an alternative algorithm for handing circuits to non-primary guards, to be used in preference to the current algorithm. Unlike the existing approach, it isolates the guard logic a bit better from the circuit logic. ## Handling exploratory circuits When all primary guards are Unreachable, we need to try non-primary guards. We select the first such guard (in preference order) that is neither Unreachable nor Pending. Whenever we give out such a guard, if the guard's status is Unknown, then we call that guard "Pending" until the attempt to use it succeeds or fails. We remember when the guard became Pending. > Aside: None of the above is a change from our existing specification. After completing a circuit, the implementation must check whether its guard is usable. A guard is usable according to these rules: Primary guards are always usable. Non-primary guards are usable for a given circuit if every guard earlier in the preference list is either unsuitable for that circuit (e.g. because of family restrictions), or marked as Unreachable, or has been pending for at least `{NONPRIMARY_GUARD_CONNECT_TIMEOUT}`. Non-primary guards are unusable for a given circuit if some guard earlier in the preference list is suitable for the circuit _and_ Reachable. Non-primary guards are unusable if they have not become usable after `{NONPRIMARY_GUARD_IDLE_TIMEOUT}` seconds. If a circuit's guard is neither usable nor unusable immediately, the circuit is not discarded; instead, it is kept (but not used) until it becomes usable or unusable. > I am not 100% sure whether this description produces the same behavior > as the current guard-spec, but it is simpler to describe, and has > proven to be simpler to implement. ## Implications for program design. (This entire section is implementation detail to explain why this is a simplification from the previous algorithm. It is for explanatory purposes only and is not part of the spec.) With this algorithm, we cut down the interaction between the guard code and the circuit code considerably, but we do not remove it entirely. Instead, there remains (in Arti terms) a pair of communication channels between the circuit manager and the guard manager: * Whenever a guard is given to the circuit manager, the circuit manager receives the write end of a single-use channel to report whether the guard has succeeded or failed. * Whenever a non-primary guard is given to the circuit manager, the circuit receives the read end of a single-use channel that will tell it whether the guard is usable or unusable. This channel doesn't report anything until the guard has one status or the other. With this design, the circuit manager never needs to look at the list of guards, and the guard manager never needs to look at the list of circuits. ## Subtleties concerning "guard success" Note that the above definitions of a Reachable guard depend on reporting when the _guard_ is successful or failed. This is not necessarily the same as reporting whether the _circuit_ is successful or failed. For example, a circuit that fails after the first hop does not necessarily indicate that there's anything wrong with the guard. Similarly, we can reasonably conclude that the guard is working (at least somewhat) as long as we have an open channel to it.

2 1

Proposal 336: Randomized schedule for guard retries
by Nick Mathewson 22 Oct '21

22 Oct '21

``` Filename: 336-randomize-guard-retries.md Title: Randomized schedule for guard retries Author: Nick Mathewson Created: 2021-10-22 Status: Open ``` # Introduction When we notice that a guard isn't working, we don't mark it as retriable until a certain interval has passed. Currently, these intervals are fixed, as described in the documentation for `GUARDS_RETRY_SCHED` in `guard-spec` appendix A.1. Here we propose using a randomized retry interval instead, based on the same decorrelated-jitter algorithm we use for directory retries. The upside of this approach is that it makes our behavior in the presence of an unreliable network a bit harder for an attacker to predict. It also means that if a guard goes down for a while, its clients will notice that it is up at staggered times, rather than probing it in lock-step. The downside of this approach is that we can, if we get unlucky enough, completely fail to notice that a preferred guard is online when we would otherwise have noticed sooner. Note that when a guard is marked retriable, it isn't necessarily retried immediately. Instead, its status is changed from "Unreachable" to "Unknown", which will cause it to get retried. For reference, our previous schedule was: ``` {param:PRIMARY_GUARDS_RETRY_SCHED} -- every 10 minutes for the first six hours, -- every 90 minutes for the next 90 hours, -- every 4 hours for the next 3 days, -- every 9 hours thereafter. {param:GUARDS_RETRY_SCHED} -- -- every hour for the first six hours, -- every 4 hours for the next 90 hours, -- every 18 hours for the next 3 days, -- every 36 hours thereafter. ``` # The new algorithm We re-use the decorrelated-jitter algorithm from `dir-spec` section 5.5. The specific formula used to compute the 'i+1'th delay is: ``` Delay_{i+1} = MIN(cap, random_between(lower_bound, upper_bound)) where upper_bound = MAX(lower_bound+1, Delay_i * 3) lower_bound = MAX(1, base_delay). ``` For primary guards, we set base_delay to 30 seconds and cap to 6 hours. For non-primary guards, we set base_delay to 10 minutes and cap to 36 hours. (These parameters were selected by simulating the results of using them until they looked "a bit more aggressive" than the current algorithm, but not too much.) The average behavior for the new primary schedule is: ``` First 1.0 hours: 10.14283 attempts. (Avg delay 4m 47.41s) First 6.0 hours: 19.02377 attempts. (Avg delay 15m 36.95s) First 96.0 hours: 56.11173 attempts. (Avg delay 1h 40m 3.13s) First 168.0 hours: 83.67091 attempts. (Avg delay 1h 58m 43.16s) Steady state: 2h 36m 44.63s between attempts. ``` The average behavior for the new non-primary schedule is: ``` First 1.0 hours: 3.08069 attempts. (Avg delay 14m 26.08s) First 6.0 hours: 8.1473 attempts. (Avg delay 35m 25.27s) First 96.0 hours: 22.57442 attempts. (Avg delay 3h 49m 32.16s) First 168.0 hours: 29.02873 attempts. (Avg delay 5h 27m 2.36s) Steady state: 11h 15m 28.47s between attempts. ```

2 1

Proposal 335: An authority-only design for MiddleOnly
by Nick Mathewson 17 Oct '21

17 Oct '21

``` Filename: 335-middle-only-redux.md Title: An authority-only design for MiddleOnly Author: Nick Mathewson Created: 2021-10-08 Status: Open ``` # Introduction This proposal describes an alternative design for a `MiddleOnly` flag. Instead of making changes at the client level, it adds a little increased complexity at the directory authority's voting process. In return for that complexity, this design will work without additional changes required from Tor clients. For additional motivation and discussion see proposal 334 by Neel Chauhan, and the related discussions on tor-dev. # Protocol changes ## Generating votes When voting for a relay with the `MiddleOnly` flag, an authority should set all flags indicating that a relay is unusable for a particular purpose, and against all flags indicating that the relay is usable for a particular position. These flags SHOULD be set in a vote whenever `MiddleOnly` is present, and only when the authority is configured to vote on the `BadExit` flag. * `BadExit` These flags SHOULD be cleared in a vote whenever `MiddleOnly` is present. * `Exit` * `Guard` * `HSDir` * `V2Dir` ## Computing a consensus This proposal will introduce a new consensus method (probably 32). Whenever computing a consensus using that consensus method or later, authorities post-process the set of flags that appear in the consensus after flag voting takes place, by applying the same rule as above. That is, with this consensus method, the authorities first compute the presence or absence of each flag on each relay as usual. Then, if the `MiddleOnly` flag is present, the authorities set `BadExit`, and clear `Exit`, `Guard`, `HSDir`, and `V2Dir`. # Configuring authorities We'll need a means for configuring which relays will receive this flag. For now, we'll just reuse the same mechanism as `AuthDirReject` and `AuthDirBadExit`: a set of torrc configuration lines listing relays by address. We'll call this `AuthDirMiddleOnly`. We'll also add an `AuthDirListsMiddleOnly` option to turn on or off voting on this option at all. # Notes on safety and migration Under this design, the MiddleOnly option becomes useful immediately, since authorities that use it will stop voting for certain additional options for MiddleOnly relays without waiting for the other authorities. We don't need to worry about a single authority setting MiddleOnly unilaterally for all relays, since the MiddleOnly flag will have no special effect until most authorities have upgraded to the new consensus method.

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev October 2021