June 2022 - tor-dev - lists.torproject.org

Two features that would help load-balanced bridges
by David Fifield 01 Jun '23

01 Jun '23

After the blocking of Tor in Russia in December 2022, the number of Snowflake users rapidly increased. Eventually the tor process became the limiting factor for performance, using all of one CPU core. In a thread on tor-relays, we worked out a design where we run multiple instances of tor on the same host, all with the same identity keys, in order to effectively use all the server's CPU resources. It's running on the live bridge now, and as a result the bridge's bandwidth use has roughly doubled. Design thread https://forum.torproject.net/t/tor-relays-how-to-reduce-tor-cpu-load-on-a-s… Installation instructions https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Survival-Gui… Two details came up that are awkward to deal with. We have workaround for them, but they could benefit from support from core tor. They are: 1. Provide a way to disable onion key rotation, or configure a custom onion key. 2. Provide a way to set a specific authentication cookie for ExtORPort SAFE_COOKIE authentication, or a new authentication type that doesn't require credentials that change whenever tor is restarted. I should mention that, apart from the load-balancing design we settled on, we have brainstormed some other options for scaling the Snowflake bridge or bridges. At this point, none of these ideas can immediately be put into practice, because there's no way to tell tor "connect to one of these bridges at random, but only one," or "connect to this bridge, but accept any of these fingerprints." https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowfl… # Disable onion key rotation Multiple tor instances with the same identity keys will work fine for the first 5 weeks (onion-key-rotation-days + onion-key-grace-period-days), but after that time the instances will have independently rotated their onion keys, and clients will have connection failures unless the load balancer happens to connect them to the instance whose descriptor they have cached. This post investigates what the failure looks like: https://lists.torproject.org/pipermail/tor-relays/2022-January/020238.html Examples of what could work here are a torrc option to set onion-key-rotation-days to a large value, an option to disable onion key rotation, an option to set a certain named file as the onion key. What we are doing now is a bit of a nasty hack: we create a directory named secret_onion_key.old, so that a failed replace_file causes an early exit from rotate_onion_key. https://gitweb.torproject.org/tor.git/tree/src/feature/relay/router.c?h=tor… There are a few apparently benign side effects, like tor trying to rebuild its descriptor every hour, but it's effective at stopping onion key rotation. https://lists.torproject.org/pipermail/tor-relays/2022-January/020277.html # Stable ExtORPort authentication ExtORPort (extended ORPort) is a protocol that lets a pluggable transport attach transport and client IP metadata to a connection, for metrics purposes. In order to connect to the ExtORPort, the pluggable transport needs to authenticate using a scheme like ControlPort authentication. https://gitweb.torproject.org/torspec.git/tree/proposals/217-ext-orport-aut… tor generates a secret auth cookie and stores it in a file. When the pluggable transport process is managed by tor, tor tells the pluggable transport where to find the file by setting the TOR_PT_AUTH_COOKIE_FILE environment variable. In the load-balanced configuration, the pluggable transport server (snowflake-server) is not run and managed by tor. It is an independent daemon, so it doesn't have access to TOR_PT_AUTH_COOKIE_FILE (which anyway would be a different path for every tor instance). The bigger problem is that tor regenerates the auth cookie and rewrites the file on every restart. All the tor instances have different cookies, and snowflake-server does not know which it will get through the load balancer, so it doesn't know what cookie to use. Examples of what would work here are an option to use a certain file as the auth cookie, an option to leave the auth cookie file alone if it already exists, or a new ExtORPort authentication type that can use the same credentials across multiple instances. What we're doing now is using a shim program, extor-static-cookie, which presents an ExtORPort interface with a static auth cookie for snowflake-server to authenticate with, then re-authenticates to the ExtORPort of its respective instance of tor, using that instance's auth cookie. https://lists.torproject.org/pipermail/tor-relays/2022-January/020183.html

2 2

GSoC 2022 - Tor Weather: Improving the Tor Network
by Sarthik Gupta 19 Sep '22

19 Sep '22

Hello everyone, I’m Sarthik Gupta (irc: sarthikg), a recent grad from Punjab Engineering College, Chandigarh & currently working as a Software Engineer at Soroco. This summer, I’ll be working on GSoC Project: “Tor Weather: Improving the Tor Network” with my mentors GeKo & Silvia. I’m extremely excited to be a part of Tor and am interested in working on this project! Briefing about the project: As of now, if any relay disappears from the tor-network, no one will know. This causes the network to lose out on bandwidth from nodes which have been down for hours because no-one knew they were down. Tor-Weather aims at solving this by creating a notification service which relay operators can subscribe to in order to get various types of updates for their relays. The tor-weather service will offer a plethora of notifications options for the relays. These include, the node being down, running on EOL/Outdated version, losing a flag, ranking in top 20/50/100, etc. These notifications can be subscribed & customised by the relay operators to fit their needs using a web-frontend. A detailed proposal for this project is available at: <https://gitlab.torproject.org/sarthikg/tor-weather/-/wikis/Proposal> https://gitlab.torproject.org/sarthikg/tor-weather/-/wikis/Proposal. I plan to keep this wiki updated with the progress & design decisions taken throughout the development of the project. Suggestions are always welcomed! Please reach out to us in irc (#tor-dev) for any ideas, questions, or suggestions you might have. Thanks, Sarthik Gupta

1 5

Counting HS descriptor uploads
by Michael Rogers 17 Aug '22

17 Aug '22

Hi, The Briar team is working on a new app that uses Tor hidden services, and we're trying to work out when the server publishing a hidden service should consider the service to be reachable by clients. We've tried counting HS descriptor uploads via control port events, but we found that when republishing a hidden service that has been published before, the number of descriptor uploads seems to vary. When republishing a hidden service, is it guaranteed that at least one copy of the descriptor will be uploaded? Or are there circumstances where Tor might decide that enough copies were previously uploaded, and still contain up-to-date information about introduction points etc, so no new copies need to be uploaded? Thanks, Michael

3 3

Shrink Tor binary size
by Sergey Ponomarev 17 Jul '22

17 Jul '22

I’m working on a firmware for routers based on OpenWrt and it needs Tor out of the box for NAT punching i.e. SSH and Web admin access. It will expose a Single Onion Service i.e. not "hidden" with just 3 hop for a better performance. In fact it just needs to connect to some random relay to have a tunnel and encrypt/decrypt traffic. The full hidden service functionality with IP/RP dances is not needed: I'm not trying to hide the location of the router. In fact most users even don't need anonymity with the 3 hop to access their router but Tor requires 3. So this is probably a kind of abuse of the Tor network but this is the only free network of proxies. At the same time a user will have an encryption while accessing the router's admin panel. I may enable HTTPS with a self signed cert but users must accept it manually which is hard for inexperienced users. The Tor is already ported to OpenWrt but its binary is more than 2mb. For 16mb routers this is not critical but a lot and I need to keep space. So is it possible to compile Tor without some features? As far as I understood from autoconf I can disable Relay functionality. But maybe I can also disable some admin api parts and SOCKS proxy. Also OpenWrt out of the box has WolfSSL so is it possible to compile tor with it instead of OpenSSL? Another problem is files sizes inside of /var/lib/tor/: 20,442 cached-certs 2,303,443 cached-microdesc-consensus 8,725,598 cached-microdescs.new 3,531 state I think that all these files may be minimized. E.g. cached-certs contains keys in PEM format which is just Base64 form of DER. So switching to plain DER certs can significantly reduce size. As a side question: is it possible to make a small relay proxy that can work part time when I sleep? Imagine that each router already has a Tor. This is potentially thousands of relay nodes and all of them have a motivation to support the network which they are using themselves. But users don’t want to lose bandwidth. I see that I can set some traffic or bandwidth limits. But maybe I can write some script that will enable or disable the relay by schedule in the evening. As far I know relays must be always online so this will make the relay unstable and it won’t be used by Tor. Is it technically possible? Regards, Sergey Ponomarev, stokito.com

2 1

Bootstrapping with a very inaccurate clock
by Michael Rogers 28 Jun '22

28 Jun '22

Hello again, Apologies for starting two threads in quick succession. First of all I want to say thank you for all the improvements in handling clock skew between 0.3.5 and 0.4.5. We recently migrated Briar to 0.4.5, and some situations that are common on mobile, such as the system clock being configured with the wrong timezone, are now much less likely to affect Tor connectivity (except when using obfs4). When the system clock is very far behind real time, Tor's clock skew events enable us to warn the user to check their time and date settings. When the clock is ahead of real time by up to a day, there are no clock skew events but Tor can bootstrap. But when the clock is several days ahead of real time, there are no clock skew events and Tor gets stuck boostrapping at 14%. At the moment we're having trouble distinguishing this situation from other situations that might cause bootstrapping to get stuck, such as access to the Tor network being blocked. So ideally we'd like to be able to detect that the clock is the problem and ask the user to fix it. I should say that this is probably a rare situation. Old devices that haven't been turned on for a while sometimes reset their clocks to a point in the distant past, but I've never seen a device spontaneously change its clock to a point in the distant future. Still, people do mess around with the time and date settings so we'd like to detect and fix this issue if possible. Is there any way for Tor to tell, during bootstrapping, that the system clock appears to be far ahead of real time? Thanks, Michael

1 0

Metrics: Estimating fraction of reported directory-request statistics
by David Fifield 27 Jun '22

27 Jun '22

I am trying to reproduce the "frac" computation from the Reproducible Metrics instructions: https://metrics.torproject.org/reproducible-metrics.html#relay-users Which is also Section 3 in the tech report on counting bridge users: https://research.torproject.org/techreports/counting-daily-bridge-users-201… h(R^H) * n(H) + h(H) * n(R\H) frac = ----------------------------- h(H) * n(N) My minor goal is to reproduce the "frac" column from the Metrics web site (which I assume is the same as the frac above, expressed as a percentage): https://metrics.torproject.org/userstats-relay-country.csv?start=2022-04-01… date,country,users,lower,upper,frac 2022-04-01,,2262557,,,92 2022-04-02,,2181639,,,92 2022-04-03,,2179544,,,93 2022-04-04,,2350360,,,93 2022-04-05,,2388772,,,93 2022-04-06,,2356170,,,93 2022-04-07,,2323184,,,93 2022-04-08,,2310170,,,91 I'm having trouble with the computation of n(R\H) and h(R∧H). I understand that R is the subset of relays that report directory request counts (i.e. that have dirreq-stats-end in their extra-info descriptors) and H is the subset of relays that report directory request byte counts (i.e. that have dirreq-write-history in their extra-info descriptors). R and H partially overlap: there are relays that are in R but not H, others that are in H but not R, and others that are in both. The computations depend on some values that are directly from descriptors: n(R) = sum of hours, for relays with directory request counts n(H) = sum of hours, for relays with directory write histories h(H) = sum of written bytes, for relays with directory write histories > Compute n(R\H) as the number of hours for which responses have been > reported but no written directory bytes. This fraction is determined > by summing up all interval lengths and then subtracting the written > directory bytes interval length from the directory response interval > length. Negative results are discarded. I interpret this to mean: add up all the dirrect-stats-end intervals (this is n(R)), add up all the dirreq-write-history intervals (this is n(H)), and compute n(R\H) as n(R) − n(H). This seems wrong: it would only be true when H is a subset of R. > Compute h(R∧H) as the number of written directory bytes for the > fraction of time when a server was reporting both written directory > bytes and directory responses. As above, this fraction is determined > by first summing up all interval lengths and then computing the > minimum of both sums divided by the sum of reported written directory > bytes. This seems to be saying to compute h(R∧H) (a count of bytes) as min(n(R), n(H)) / h(H). This is dimensionally wrong: the units are hours / bytes. What would be more natural to me is min(n(R), n(H)) / max(n(R), n(H)) × h(H); i.e., divide the smaller of n(R) and n(R) by the larger, then multiply this ratio by the observable byte count. But this, too, only works when H is a subset of R. Where is this computation done in the metrics code? I would like to refer to it, but I could not find it. Using the formulas and assumptions above, here's my attempt at computing recent "frac" values: date `n(N)` `n(H)` `h(H)` `n(R)` `n(R\H)` `h(R∧H)` frac 2022-04-01 166584 177638. 2.24e13 125491. 0 1.59e13 0.753 2022-04-02 166951 177466. 2.18e13 125686. 0 1.54e13 0.753 2022-04-03 167100 177718. 2.27e13 127008. 0 1.62e13 0.760 2022-04-04 166970 177559. 2.43e13 126412. 0 1.73e13 0.757 2022-04-05 166729 177585. 2.44e13 125389. 0 1.72e13 0.752 2022-04-06 166832 177470. 2.39e13 127077. 0 1.71e13 0.762 2022-04-07 166532 177210. 2.48e13 127815. 0 1.79e13 0.768 2022-04-08 167695 176879. 2.52e13 127697. 0 1.82e13 0.761 The "frac" column does not match the CSV. Also notice that n(N) < n(H), which should be impossible because H is supposed to be a subset of N (N is the set of all relays). But this is what I get when I estimate n(N) from a network-status-consensus-3 and n(H) from extra-info documents. Also notice that n(R) < n(H), which means that H cannot be a subset of R, contrary to the observations above.

2 4

tor-browser-build : any way to get it to use system packages for building?
by Jason Vas Dias 15 Jun '22

15 Jun '22

Good day - I recently attempted to check-out and build tor-browser-build - MUST it download ALL the gits, if system packages are found to satisfy version dependencies ? I just don't have enough storage / network bandwidth . Why am I downloading the llvm source when I already have llvm and clang 13 (-devel packages also) on my Fedora 36 system ? Is there any interest in producing a version of tor-browser-build to use installed system packages if they meet version requirements instead of checking out GITs & building for ALL dependencies ? It could be enabled with a USE_SYSTEM_PKGS=1 make option or something. I'm going to have to develop this if I want to work on tor-browser, which I do want to do. Best Regards, Jason

3 3