- tor-dev - lists.torproject.org

Network Status APIs
by Mattia Righetti 22 Jun '23

22 Jun '23

Greetings Tor devs, In an effort to improve current resource utilisation, the Network Health Status team is developing a new version of their pipeline [1]. Previously, much of the data was stored on files, this made many operations slow due to their I/O bound nature. The new pipeline will transfer much of the data related to Tor nodes and bridges from files stored on a single server's disk to two separate databases: Postgres and Victoria Metrics. With this new approach, there's the need to develop a new service that is going to replace the current onionoo web protocol [2]. The main objective of this project is to design a RESTful API service, using the actix_web framework, that is going to be integrated in the new pipeline v2.0 to support data retrieval of bridges and relays from the two databases and additionally provide new features such as historic data search. The project will be developed over the next few weeks as part of the GSoC sponsored program. Please do reach out to me, hiro or GeKo if you have any ideas or feedback you'd like to share with us. Cheers, Matt [1] https://gitlab.torproject.org/tpo/network-health/team/-/wikis/metrics/colle… [2] https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/

1 0

Two features that would help load-balanced bridges
by David Fifield 01 Jun '23

01 Jun '23

After the blocking of Tor in Russia in December 2022, the number of Snowflake users rapidly increased. Eventually the tor process became the limiting factor for performance, using all of one CPU core. In a thread on tor-relays, we worked out a design where we run multiple instances of tor on the same host, all with the same identity keys, in order to effectively use all the server's CPU resources. It's running on the live bridge now, and as a result the bridge's bandwidth use has roughly doubled. Design thread https://forum.torproject.net/t/tor-relays-how-to-reduce-tor-cpu-load-on-a-s… Installation instructions https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Survival-Gui… Two details came up that are awkward to deal with. We have workaround for them, but they could benefit from support from core tor. They are: 1. Provide a way to disable onion key rotation, or configure a custom onion key. 2. Provide a way to set a specific authentication cookie for ExtORPort SAFE_COOKIE authentication, or a new authentication type that doesn't require credentials that change whenever tor is restarted. I should mention that, apart from the load-balancing design we settled on, we have brainstormed some other options for scaling the Snowflake bridge or bridges. At this point, none of these ideas can immediately be put into practice, because there's no way to tell tor "connect to one of these bridges at random, but only one," or "connect to this bridge, but accept any of these fingerprints." https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl… # Disable onion key rotation Multiple tor instances with the same identity keys will work fine for the first 5 weeks (onion-key-rotation-days + onion-key-grace-period-days), but after that time the instances will have independently rotated their onion keys, and clients will have connection failures unless the load balancer happens to connect them to the instance whose descriptor they have cached. This post investigates what the failure looks like: https://lists.torproject.org/pipermail/tor-relays/2022-January/020238.html Examples of what could work here are a torrc option to set onion-key-rotation-days to a large value, an option to disable onion key rotation, an option to set a certain named file as the onion key. What we are doing now is a bit of a nasty hack: we create a directory named secret_onion_key.old, so that a failed replace_file causes an early exit from rotate_onion_key. https://gitweb.torproject.org/tor.git/tree/src/feature/relay/router.c?h=tor… There are a few apparently benign side effects, like tor trying to rebuild its descriptor every hour, but it's effective at stopping onion key rotation. https://lists.torproject.org/pipermail/tor-relays/2022-January/020277.html # Stable ExtORPort authentication ExtORPort (extended ORPort) is a protocol that lets a pluggable transport attach transport and client IP metadata to a connection, for metrics purposes. In order to connect to the ExtORPort, the pluggable transport needs to authenticate using a scheme like ControlPort authentication. https://gitweb.torproject.org/torspec.git/tree/proposals/217-ext-orport-aut… tor generates a secret auth cookie and stores it in a file. When the pluggable transport process is managed by tor, tor tells the pluggable transport where to find the file by setting the TOR_PT_AUTH_COOKIE_FILE environment variable. In the load-balanced configuration, the pluggable transport server (snowflake-server) is not run and managed by tor. It is an independent daemon, so it doesn't have access to TOR_PT_AUTH_COOKIE_FILE (which anyway would be a different path for every tor instance). The bigger problem is that tor regenerates the auth cookie and rewrites the file on every restart. All the tor instances have different cookies, and snowflake-server does not know which it will get through the load balancer, so it doesn't know what cookie to use. Examples of what would work here are an option to use a certain file as the auth cookie, an option to leave the auth cookie file alone if it already exists, or a new ExtORPort authentication type that can use the same credentials across multiple instances. What we're doing now is using a shim program, extor-static-cookie, which presents an ExtORPort interface with a static auth cookie for snowflake-server to authenticate with, then re-authenticates to the ExtORPort of its respective instance of tor, using that instance's auth cookie. https://lists.torproject.org/pipermail/tor-relays/2022-January/020183.html

2 2

txtorcon v23.5.0
by meejah 19 May '23

19 May '23

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I'm happy to announce txtorcon 23.5.0 with the following changes: * twisted.web.client.Agent instances now use the same HTTPS policy by default as twisted.web.client.Agent. It is possible to override this policy with the tls_context_factory= argument, the equivalent to Agent's contextFactory= (Thanks to Itamar Turner-Trauring) * Added support + testing for Python 3.11. * No more ipaddress dependency You can download the release from PyPI or GitHub (or of course "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/23.5.0 https://github.com/meejah/txtorcon/releases/tag/v23.5.0 Releases are also available from the hidden service: http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/txtor… http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/txtor… You can verify the sha256sum of both by running the following 4 lines in a shell wherever you have the files downloaded: cat <<EOF | sha256sum --check 93fd80a9dd505f698d0864fe93db8b6a9c1144b5feb91530820b70ed8982651c dist/txtorcon-23.5.0.tar.gz 987f0a91184f98cc3f0a7eccaa42f5054063744d6ac15e325cfa666403214208 dist/txtorcon-23.5.0-py3-none-any.whl EOF thanks, meejah -----BEGIN PGP SIGNATURE----- iQFFBAEBCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmRm0DoRHG1lZWphaEBt ZWVqYWguY2EACgkQwmAoAxKAaae/+wgAw3gAm65npc7+yMdGFixNmCd+RUXorJq9 Hy76hK3BWdtNIA6TZF20QFYs3CX5Vepa0vCJOK1N40fYgxoZTb1/828Zp6Zq2+Gn piJGvQ0Z1S95ww7lwSV77o67Xf7PozhLR+k7DaOdY8ugvLb/0Rdp15BykF5DWIo8 PRgqB8uZ418ebmDLLrYtqYTdlcUMxFTji4CHXc4N55/2hVHiFiuFt59os6kJ3iG1 u90lmQH8vbDyVF7N6tpgEAdWeb7OdgDbtzhVBdBWHrPg+vDO+UL7WZU8ZjDAcdEr YzzmK3fIiCH7ngG2E/VIebiJfrjAA9G4eZXltIm7VcWh5css9MXY1Q== =TeQp -----END PGP SIGNATURE-----

1 0

Hacks to reduce Tor's initial download on slow Internet, sacrificing privacy?
by Holmes Wilson 08 May '23

08 May '23

Hi everyone, Is there a way right now to get Tor hidden service functionality (hosting a hidden service, connecting to hidden services) on a connection where the Internet is so slow and unreliable that the initial download of network information currently takes ~forever, provided one is willing to sacrifice metadata protection? Is there a way to download, say, 100x less network information on startup and still effectively host and connect to hidden services? Or is there a way to hardcode network information with the client, since that can be installed before going into the slow Internet zone, from a CDN that is less impacted, or from a source on the local network? (I read that this is how Tor worked in the past?) The context is the following: I have a p2p messaging app that uses Tor and hidden services (Quiet) in a way similar to Ricochet or Onionshare. I'm going to a conference where last year the Internet was so slow that Tor's initial download of network information took too long and kept timing out, rendering Quiet unusable. The Internet was, however, fairly reliable and usable for e.g. web browsing and messaging. I'd like to be able to use Quiet at this conference. It would be used purely as a demo for a few days, and we could warn everyone that our use of Tor did not offer its typical security properties. (Then in future years we might support p2p connectivity over local wifi, like Briar does!) My understanding is that the network information download step is to protect users against epistemic attacks. My intuition is that for situations where this doesn't matter there is some way to use Tor with a small subset of the network information and that the initial download could be skipped. Is this true? What's the best way to do it in the Tor client we ship? (I'm familiar with the Walking Onions paper, but looking for something that is ready now. There isn't already an implementation of Walking Onions, is there?) Thanks! Holmes

3 2

New Proposal - CAA Extensions for the Tor Rendezvous Specification
by Q Misell 08 May '23

08 May '23

Hi all, I've spent some time working on ACME for Tor hidden services (you may have seen discussion of this work on the onion-advisors mailing list). Full details of the project are available at https://e.as207960.net/w4bdyj/AX8Ffqsd Attached is my proposal for a change to the Tor Rendezvous Specification to support the inclusion of CAA records in hidden service descriptors. My fork of Tor implementing publishing these records is available at https://e.as207960.net/w4bdyj/XMN03dmD Thanks, Q ------------------------------ Any statements contained in this email are personal to the author and are not necessarily the statements of the company unless specifically stated. AS207960 Cyfyngedig, having a registered office at 13 Pen-y-lan Terrace, Caerdydd, Cymru, CF23 9EU, trading as Glauca Digital, is a company registered in Wales under № 12417574 <https://find-and-update.company-information.service.gov.uk/company/12417574>. ICO register №: ZA782876 <https://ico.org.uk/ESDWebPages/Entry/ZA782876>. UK VAT №: GB378323867. EU VAT №: EU372013983. Turkish VAT №: 0861333524. South Korean VAT №: 522-80-03080. Glauca Digital and the Glauca logo are registered trademarks in the UK, under № UK00003718474 and № UK00003718468, respectively.

5 6

Suggestions for Arti Example Programs
by Saksham Mittal 08 May '23

08 May '23

Hello there! I was recently selected as a GSoC contributor to help improve Arti by providing example utilities to test drive the APIs. I've come up with a couple ideas in my GSoC proposal and I wanted to share them as well as ask the community if they had any ideas of what projects they may want to see included in examples to help developers use Arti APIs in the future. 1. A download manager in Rust which will download the Tor Browser Bundle through a bridge connection. 2. A simple DNS lookup tool like dig which uses Tor relay to resolve hostnames anonymously, reducing the need to trust a DNS server. (This will use DNS over TLS) 3. A small script to just print out the current connection info on the circuit that had been chosen (like in Tor Browser) and to test relay and bridge connectivity at the user’s end for diagnostic purposes. 4. A bridge reachability monitor using Arti. This tool would hold connections to many obfs4 bridges open and detect when one of them goes down. (More info about this can be found at https://gitlab.torproject.org/tpo/core/arti/-/issues/717) 5. A small proxy server and client which uses just the pluggable transport code to create obfuscated connection between the two (doesn't go through the Tor network, it is just intended to showcase how to setup connections through PTs) I'd be happy to know of any cool ideas you may have to build with Arti that I could work on this summer! Thanks! -Saksham

1 0

Re: [tor-dev] New Proposal - CAA Extensions for the Tor Rendezvous Specification
by Q Misell 27 Apr '23

27 Apr '23

Hi Raph, Whilst I agree that in an ideal world CAs would be irrelevant, we do not live in an ideal world. My proposal is one of many ways that a certificate could be issued to hidden services. Issuing standard TLS certificates to .onion domains allows HTTPS without modification to the browser. This allows non Tor Browsers user agents to access the Tor network via a proxy (SOCKS etc), doing otherwise would require all browsers to understand Tor. It also opens up new opportunities such as payment processing as current PCI DSS requirements do not allow non-standard TLS. Current hidden services with HTTPS such as the BBC already use standard TLS certificates, however the process for these is extremely involved with current CAs. My IETF draft aims to make this a much simpler process via the already well-proven ACME. Thanks, Q ------------------------------ Any statements contained in this email are personal to the author and are not necessarily the statements of the company unless specifically stated. AS207960 Cyfyngedig, having a registered office at 13 Pen-y-lan Terrace, Caerdydd, Cymru, CF23 9EU, trading as Glauca Digital, is a company registered in Wales under № 12417574 <https://find-and-update.company-information.service.gov.uk/company/12417574>. ICO register №: ZA782876 <https://ico.org.uk/ESDWebPages/Entry/ZA782876>. UK VAT №: GB378323867. EU VAT №: EU372013983. Turkish VAT №: 0861333524. South Korean VAT №: 522-80-03080. Glauca Digital and the Glauca logo are registered trademarks in the UK, under № UK00003718474 and № UK00003718468, respectively. On Thu, 27 Apr 2023 at 13:59, Raphaël Fabre <contact(a)fabco.tech> wrote: > Hey > > I've read your proposal, although I'm not sure to grap the totality of it. > I'm Raph the lead dev of this project -> https://e.as207960.net/w4bdyj/YG6dkkyr > > I have a question, since .onion is basically a non ICANN domain name > system, why do you care about using Certificate Authorities ? Could you > instead store the TLS certificates in the name system, as TXT / CERT > records (similar to DANE) ? > > Doing this would allow TOR service providers to not rely on certificate > authorities, control their TLS certificates directly (self signed) and *do > not need at all to renew it*. > > happy to chat further > Raph > > ------- Original Message ------- > Le jeudi 27 avril 2023 à 14:45, Q Misell via tor-dev < > tor-dev(a)lists.torproject.org> a écrit : > > Hi all, > > I've spent some time working on ACME for Tor hidden services (you may have > seen discussion of this work on the onion-advisors mailing list). Full > details of the project are available at https://e.as207960.net/w4bdyj/xqviywZW > <https://e.as207960.net/w4bdyj/yBzJUPTT>. > > Attached is my proposal for a change to the Tor Rendezvous Specification > to support the inclusion of CAA records in hidden service descriptors. > > My fork of Tor implementing publishing these records is available at > https://e.as207960.net/w4bdyj/ItcJLhNd <https://e.as207960.net/w4bdyj/LmAkW3uG>. > > Thanks, > Q > ------------------------------ > > Any statements contained in this email are personal to the author and are > not necessarily the statements of the company unless specifically stated. > AS207960 Cyfyngedig, having a registered office at 13 Pen-y-lan Terrace, > Caerdydd, Cymru, CF23 9EU, trading as Glauca Digital, is a company > registered in Wales under № 12417574 > <https://e.as207960.net/w4bdyj/pIFzAYXo>. ICO register №: ZA782876 > <https://e.as207960.net/w4bdyj/pZ2mD5UQ>. UK VAT №: GB378323867. EU VAT > №: EU372013983. Turkish VAT №: 0861333524. South Korean VAT №: > 522-80-03080. Glauca Digital and the Glauca logo are registered trademarks > in the UK, under № UK00003718474 and № UK00003718468, respectively. > > >

1 0

goptlib moved to gitlab.torproject.org
by David Fifield 15 Apr '23

15 Apr '23

I have moved the goptlib repository from git.torproject.org to gitlab.torproject.org. If you want to use it at the new location, change this import: import "git.torproject.org/pluggable-transports/goptlib.git" to this: import "gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/goptlib" and then run `go get` and `go mod tidy`. Here is the issue about moving anti-censorship team projects from git.torproject.org to gitlab.torproject.org generally: https://bugs.torproject.org/tpo/anti-censorship/team/86#note_2823122 The old git.torproject.org URL should continue to work for the time being.

1 0

MSVC compile failure
by Gisle Vanem 21 Mar '23

21 Mar '23

Hello list. Anybody out there? Trying to compile Tor with 'cl' ver. 19.36.32323 from yesterdays 'git master', I got this error in 'src/feature/hs/hs_metrics_entry.c': hs/hs_metrics_entry.c(98,3): error C2099: initializer is not a constant { ^ hs/hs_metrics_entry.c(106,3): error C2099: initializer is not a constant { ^ ---------- This MSVC compiler is rather stupid; it fails to parse that 'static const size_t hs_metrics_circ_build_time_buckets_size' is truly 'const'. No problem with 'clang-cl'. But this patch works: --- a/feature/hs/hs_metrics_entry.c 2023-03-17 18:00:57 +++ b/feature/hs/hs_metrics_entry.c 2023-03-18 13:56:58 @@ -28,8 +28,7 @@ 60000 /* 60s */ }; -static const size_t hs_metrics_circ_build_time_buckets_size = - ARRAY_LENGTH(hs_metrics_circ_build_time_buckets); +#define hs_metrics_circ_build_time_buckets_size ARRAY_LENGTH(hs_metrics_circ_build_time_buckets) ----------------- Just letting you know here. -- --gv

2 3

Cross-user TLS traffic mixing in snowflake-server until 2023-03-13
by David Fifield 15 Mar '23

15 Mar '23

Between 2022-10-01 and 2023-03-13, there was a bug in the software deployed at Snowflake bridges that could cause parts of a user's stream to be overwritten by parts of other users' streams. Though the Snowflake team believes the privacy risks of the bug are minor, we are treating it as a security bug and therefore making this announcement. The fix is already deployed on the server, and no action is required by users. ## The cause The source of the bug was issue https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl… and specifically commit https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow…, which was intended as a performance optimization to reduce memory allocation during the large influx of Snowflake users in late September 2022. The analysis surrounding the performance improvement incorrectly concluded that certain memory buffers were not reused and therefore did not need to be copied. But in fact, the buffers were reused, with the effect that data meant for one user could be unpredictably overwritten by data for another user. ## The effect There is analysis at issue https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl…. If our understanding of the bug is correct, it did not reveal plaintext data or IP addresses to other users. The Snowflake bridge is a Tor bridge, not an exit relay, so even a catastrophic failure could not leak users' plaintext data or what destinations they were accessing. A Snowflake connection consists of a layered set of protocols, and the bug was in a layer that does not have access to IP addresses or any other persistent identifiers. * IP * TCP * HTTPS (WebSocket) * session/retransmission ← The bug was here * Tor TLS * user data At most, users and proxies would see fragments of other users' Tor TLS streams. We believe the degree of information disclosure is less than what an eavesdropper listening at the Snowflake bridge's HTTPS port would see, which we already believe to be safe. The practical effect, more than a loss of privacy, was excessive retransmissions at the session/retransmission layer, and corrupted TLS data resulting in a terminated connection. ## Remediation We reverted the erroneous commit and added a test to check for the erroneous buffer reuse. We deployed the update to both Snowflake bridges on 2023-03-13. https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl… ## Required action No action is required by users. The bug was in code that was only used by the Snowflake server, not proxies or clients. ## Thanks The problem was discovered and first investigated by Haz Æ 41 (https://gitlab.torproject.org/hazae41) and reported on HackerOne: https://hackerone.com/reports/1880610.

1 0