- tor-dev - lists.torproject.org

onionoo overload_general_timestamp (prop 328)
by nusenu 10 Feb '22

10 Feb '22

Hi, in onionoo all timestamps used to be in the format YYYY-MM-DD hh:mm:ss Proposal 328 has timestamps in this same format https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/328-re… The newly added prop328 fields in oninoo appear to break with that convention https://metrics.torproject.org/onionoo.html#details_relay_overload_general_… Here is an example for an onionoo overload_general_timestamp which appears to be at millisecond granularity (the source has a granularity of an hour): 1636038000000 Was there a particular motivation for this format change and granularity? And what do you think about changing it to use the YYYY-MM-DD hh:mm:ss format for consistency and having a direct human readable format here as well? related: Karsten used to maintain onionoo protocol documentation/changelog and versions: https://metrics.torproject.org/onionoo.html#versions Is that and the 'version' field in onionoo no longer maintained? (since it didn't change with the new fields) kind regards, nusenu -- https://nusenu.github.io

2 3

onbasca - the next-generation bandwidth scanner
by Georg Koppen 10 Feb '22

10 Feb '22

Hello everyone! As many of you know we have been working on a simple bandwidth scanner (sbws) over the last years to deal with the old and unmaintained Torflow code on our bandwidth authorities. We learned a lot during that process, especially after we started to replace Torflow with the new sbws code on bandwidth authorities (today sbws is powering a majority of them, yay!). We realized that we needed to improve the robustness and testability of the code significantly, in particular as we want to build exciting features like loadbalancing capability on top of it and decided it would be easier to do so in a new tool: welcome onbasca. Onbasca takes the best out of sbws and makes the code easier to test and more robust for future improvements by deploying a database management system where we previously dealt with JSON files only.[1] The code repository lives in our network-health group[2] and is open for contributions. Get in touch in case you are interested, everyone is welcome! Georg [1] For a more detailed overview over the bandwidth scanner history, design issues we encountered in sbws, and decisions we made, see: https://tpo.pages.torproject.net/network-health/onbasca/history.html. [2] https://gitlab.torproject.org/tpo/network-health/onbasca

2 1

HAROI: Human Readable Authenticated Relay Operator Identifier
by nusenu 27 Dec '21

27 Dec '21

Hi, below is a partial proposal draft for human readable relay operator IDs that are authenticated by directory authorities. If there is any interest in implementing something like this I'll complete the draft and submit it via gitlab. kind regards, nusenu # HAROI: Human Readable Authenticated Relay Operator Identifier A HAROI is a DNS FQDN that can be chosen by a relay operator and linked to one or more relays in a verifiable way. The link is authenticated in the sense that it requires some control over the relay and the FQDN (on the DNS or webroot level). A relay can only be linked to a single HAROI. Relays publish their HAROI via their descriptor. Directory authorities verify the link and publish the verification result in their vote. ## Motivation Many tools in the tor ecosystem display at least some relay (operator) information (nickname, ContactInfo, ...), some examples are: - Tor Browser - Relay Search [1] - Onioncircuits by tails [2] - ExoneraTor [3] - allium [4] unfortunatelly there is no authenticated operator ID available, these tools could display, so they might display misleading information, something that has been exploited in the wild for impersonation attacks. There is a value in giving relay operators the possibility to declare an identifier that is globally unique, consistent and can not be easily spoofed by adversaries so they do not become easy victims of impersonation attacks. The tor ecosystem would benefit from an operator ID that can not be spoofed. This proposal is not replacing the relay ContactInfo. ## Design * A HAROI is optional and not mandatory. * A HAROI is verified by directory authorities in one of two ways, depending on the proof type. In the distant future where relays have no RSA1024 keys, Ed25519 proof types are added. * The used proof type can be selected by the relay operator. Successfully verified proofs are cached by directory authorities for 30 days. After 30 days proofs are verified again. Relay operators that want to make use of HAROI can participate without requiring a domain since many free services offer custom free subdomains (example: GitLab or GitHub pages). ## Proof Types Relay operators, that chose to set a HAROI, can select their preferred a proof type. ### uri-rsa This proof type can be verified by fetching the list of relay RSA SHA1 fingerprints from the FQDN via HTTPS using the well-known URI as defined in proposal 326 https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/326-to… Example: if the FQDN is example.com, the url to fetch the list of fingerprints is: https://example.com/.well-known/tor-relay/rsa-fingerprint.txt The TLS certificate used on the webserver must be from a trusted CA and logged in a public certificate transparency log. For N relays using the same FQDN only a single HTTPS GET request is needed. ### dns-rsa This proof type can be verified by performing DNS lookups. To make use of this proof type the DNS zone MUST be DNSSEC signed. The DNS query is constructed by combining the relay's fingerprint and the FQDN: relay-fingerprint.<FQDN> example: relay-fingerprint.example.com value: “we-run-this-tor-relay” relay-fingerprint is the 40 character RSA SHA1 fingerprint of the tor relay. Each relay has its own DNS record, a single TXT record MUST be returned per relay only. content of the TXT record MUST match: "we-run-this-tor-relay” Each relay requires a single DNS lookup (less scalable than the uri-rsa option). ## Relay Configuration Relay operators can specify their identifier via a torrc option. OperatorIdentifier <FQDN> <proof-type> example: OperatorIdentifier example.com dns-rsa ## Length Limit Although DNS FQDNs can be rather long, we limit HAROIs to 40 characters. (As of December 2021 the longest observed HAROI is 28 characters long.) Operators will not be able to specify a HAROI longer than 40 characters in their torrc. Directory authorities refuse to accept them as well. ## Relays Descriptor Changes This proposal defines a new optional descriptor field that is automatically verified and voted on by tor directory authorities. operatorid <FQDN> <proof-type> ## Directory Authorities Directory authorities refuse to accept domains on the public suffix list [5]. XXX TODO ## Security Considerations Relay operators can trick directory authorities into performing DNS lookups and HTTPS GETs on arbitrary domains. Possible solutions: Directory authorities could required PTR+A DNS records on the same domain as the given FQDN for the relays IP address. [1] https://metrics.torproject.org/rs.html [2] https://gitlab.tails.boum.org/tails/onioncircuits [3] https://metrics.torproject.org/exonerator.html [4] https://yui.cat/ [5] https://publicsuffix.org/list/public_suffix_list.dat -- https://nusenu.github.io

2 4

the consequences of deprecating debian alpha repos with every new major branch
by nusenu 21 Dec '21

21 Dec '21

Hi, tldr: - more outdated relays (that is a claim I'm making and you could easily proof me wrong by recreating the 0.3.3.x alpha repos and ship 0.3.3.7 in them and see how things evolve after a week or so) - more work for the tpo website maintainer - less happy relay operators [3][4] - more work for repo maintainers? (since a new repo needs to be created) When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23 I was about to submit a PR for the website to include it in the sources.list generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first. The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to update the repo generator for 0.3.4.x nonetheless and here is why. Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search) (> 3.2% CW fraction) Then I realized that this was the last version the tor-experimental-0.3.3.x-* repos were shipping before they got abandoned due to the new 0.3.4.x-* repos (I can no longer verify it since they got removed by now). Peter made it clear in the past that the current way to have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie) will not change [2]: > If you can't be bothered to change your sources.list once or twice a > year, then you probably should be running stable. but maybe someone else would be willing to invoke a "ln" commands everytime a new new alpha repo is born. tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie once 0.3.5.x repos are created the link would point to tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie It is my opinion that this will help reduce the amount of relays running outdated versions of tor. It will certainly avoid having to update the tpo website, which isn't a big task and could probably be automated but it isn't done currently. "..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas (and break setups)!" Yes, and I think that is better than relays stuck on an older version because the former repo no longer exists and operators still can choose the old repos which will not jump to newer major versions. [1] https://www.torproject.org/docs/debian.html.en#ubuntu [2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3 [3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html [4] https://trac.torproject.org/projects/tor/ticket/26474 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

4 7

Re: [tor-dev] The case for Tor-over-QUIC
by Ali Clark 12 Dec '21

12 Dec '21

Sections 3 and 6 of the Quux paper have some relevant discussion [1] > Unfortunately, it appears that the Quux QUIC paper studied QUIC at the > wrong position - between relays, and the QuTor implementation is > unclear. This means that it may still be an open question as to if > QUIC's congestion control algorithms will behave optimally in a > Tor-like network under heavy load. As Reardon and Goldberg noted in concluding remarks, approaches other than hop-by-hop will incur an extra cost for retransmissions, since these must be rerouted through a larger part of the network [RG09]. As Tschorsch and Scheuermann discuss [TS12], due to the longer RTT of TCP connections, end-to-end approaches will also take longer to “ramp up” through slow start and up to a steady state. Both of these factors (not to mention increased security risk of information leakage [DM09]) suggest that hop-by-hop designs are likely to yield beer results. In fact, the hop-by-hop approach may be viewed as an instance of the Split TCP Performance-Enhancing Proxy design, whereby arbitrary TCP connections are split in two to negate the issues noted above. > Unfortunately, the Quux implementation appears to use QUIC at a > suboptimal position -- they replace Tor's TLS connections with QUIC, > and use QUIC streams in place of Tor's circuit ID -- but only between > relays. This means that it does not benefit from QUIC's end-to-end > congestion control for the entire path of the circuit. Such a design > will not solve the queuing and associated OOM problems at Tor relays, > since relays would be unable to drop cells to signal backpressure to > endpoints. Drops will instead block every circuit on a connection > between relays, and even then, due to end-to-end reliability, relays > will still need to queue without bound, subject to Tor's current (and > inadequate) flow control. A fully QUIC relay path (with slight modication to fix a limit on internal buffer sizes) would allow end-to-end backpressure to be used from the client application TCP stream up to the exit TCP stream. Leaving aside Tor’s inbound rate limit mechanism but retaining the global outbound limit, this design would allow max-min fairness to be achieved in the network, as outlined by Tschorsch and Scheuermann [TS11]. ... Once implemented however, backpressure would allow Tor to adopt a signicantly improved internal design. In such a design, a Tor relay could read a single cell from one QUIC stream’s read buffer, onion crypt it, and immediately place it onto the write buffer of the next stream in the circuit. This process would be able to operate at the granularity of a single cell because the read and write operations for QUIC are very cheap user-space function calls and not syscalls as for host TCP. The schedule of this action would be governed by the existing EWMA scheduler for circuits that have both a readable stream and a writeable stream (and as allowed by a global outgoing token bucket), allowing optimal quality of service for circuits. It’s expected that backpressure implemented in this way will yield signicant performance and fairness gains on top of the performance improvement found in this thesis. One issue for Quux was that it used the Chromium demo QUIC server code as the basis for its implementation, which was fine for performance research but not such a good choice for Tor's networking stack. Several Rust implementations have been released with server-side (not just client-side) usage, so I expect that to be much less of an issue today. io_uring is also a significant development since Quux was developed, as it can reduce the performance hit for host-TCP syscalls, or for using recvmsg instead of recvmmsg with QUIC if the implementation makes it difficult to use recvmmsg on the listener side. [1] https://www.benthamsgaze.org/wp-content/uploads/2016/09/393617_Alastair_Cla… The following paper has in-depth discussion, but I don't have a copy to hand unfortunately: Ali Clark. Tor network performance — transport and flow control. Technical report, University College London, April 2016

1 0

Client identification for authenticated onions
by cho8jeiv4aus＠paperboats.net 12 Dec '21

12 Dec '21

Hi there. I had an idea recently for an onion service to improve the UX of sites that require a login. The site would have two onions: one for those who want to use onion auth and another for those who don't or are still setting it up. A user would first sign in with a username+password on the unauthenticated onion and click a button to generate a certificate associated with their account. Then they would add the public key to their browser and visit the authenticated onion. The application server would then match the pubkey used to authenticate with an account in the database, and log them in automatically. I've looked in the mailing list archives and `man 1 tor` but didn't find anything that would facilitate this. The closest, it seems, is HiddenServiceExportCircuitID, but that is for *circuit* IDs, not *client* IDs. Is this possible to implement, either as an operator or as a Tor developer? As an operator, an alternative would be to generate one (authenticated) onion service per user and route them all to the same place with different Host headers, but that seems rather inefficient, and I don't know how well the tor daemon scales up to hundreds of onion services anyway. P.S. I didn't find an easy way to do full text search on the mailing list archives, so I wrote a little script to download them all. I've attached it in case it ends up useful. It requires python3.8+ and you'll need to `pip install aiohttp anyio BeautifulSoup4` first. After that you can run `./pipermail_fetch.py https://lists.torproject.org/pipermail/tor-dev/` and then something like `rg --context 3 --search-zip '^[^>].*search term here'` will do the trick.

3 2

compiling torsocks
by lejeczek 12 Dec '21

12 Dec '21

Hi guys A complete non-dev/programmer here so please go easy on me. I'm trying to compile torsocks with 'mock', while 'tor' compiles okey for 'torsock' I fail with: ... configure.ac:20: comes from Automake 1.16.1. You should recreate configure.ac:20: aclocal.m4 with aclocal and run automake again. WARNING: 'automake-1.16' is probably too old. You should only need it if you modified 'Makefile.am' or 'configure.ac' or m4 files included by 'configure.ac'. The 'automake' program is part of the GNU Automake package: <http://www.gnu.org/software/automake> It also requires GNU Autoconf, GNU m4 and Perl in order to run: <http://www.gnu.org/software/autoconf> <http://www.gnu.org/software/m4/> <http://www.perl.org/> make[1]: *** [Makefile:600: Makefile.in] Error 63 make: *** [Makefile:466: all-recursive] Error 1 ... My setup is Centos 9 and obviously I do it with an rpm/spec. Would you guys be kind to guide me to troubleshoot this? many thanks, L.

1 0

Accessing Shared Random Values as a Protocol on top of Tor
by Sebastian Hoffmann 07 Dec '21

07 Dec '21

Hi, I'm wondering if I can access the shared random value[1] while developing a protocol/application on top of Tor onion services. The application is still in early development, but it would be great if I could depend on the shared random value. If this is not the correct mailing list for this question, I would be glad if you could point me to one. --Sebastian [1]: PUB-SHAREDRANDOM in https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt

2 1

proposal 328: consensus parameters
by nusenu 21 Nov '21

21 Nov '21

Hi, https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/328-re… > For DNS timeouts, the X and Y are consensus parameters > (overload_dns_timeout_scale_percent and overload_dns_timeout_period_secs) > defined in param-spec.txt. should one be able to find these parameters (overload_dns_timeout_scale_percent and overload_dns_timeout_period_secs) on https://consensus-health.torproject.org/#consensusparams or in https://collector.torproject.org/recent/relay-descriptors/consensuses/ or is this not yet deployed? What are the values of these parameters currently? thanks, nusenu -- https://nusenu.github.io

2 2

prometheus relay label
by nusenu 09 Nov '21

09 Nov '21

Hi, when adding MetricsPort support to ansible-relayor I realized that many operators that run more than one tor instance per server will run into an issue because tor's relay prometheus metrics has no identifying label like fingerprint= or similar to tell tor instances appart. The instance= default label can have the same value for all tor instances on a given server so that can not be used. To avoid using nickname (might not be set) the easiest option is probably to use the relay's SHA1 fingerprint or alternatively the IP:ORPort combination which is unique per server but not necessarily globally unique (RFC1918 IPs). Another neat option for operators is to use node_exporter's textfile collector to collect tor's MetricsPort content to avoid having to run an additional webserver for TLS and authentication (because unlike tor's exporter node_exporter comes with TLS and authentication builtin). In that case the suggested solution would be even more needed because in that case relabling via prometheus' scrape config is no longer possible. What do you think about this suggestion? kind regards, nusenu -- https://nusenu.github.io

1 0