December 2019 - tor-dev - lists.torproject.org

the consequences of deprecating debian alpha repos with every new major branch
by nusenu 21 Dec '21

21 Dec '21

Hi, tldr: - more outdated relays (that is a claim I'm making and you could easily proof me wrong by recreating the 0.3.3.x alpha repos and ship 0.3.3.7 in them and see how things evolve after a week or so) - more work for the tpo website maintainer - less happy relay operators [3][4] - more work for repo maintainers? (since a new repo needs to be created) When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23 I was about to submit a PR for the website to include it in the sources.list generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first. The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to update the repo generator for 0.3.4.x nonetheless and here is why. Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search) (> 3.2% CW fraction) Then I realized that this was the last version the tor-experimental-0.3.3.x-* repos were shipping before they got abandoned due to the new 0.3.4.x-* repos (I can no longer verify it since they got removed by now). Peter made it clear in the past that the current way to have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie) will not change [2]: > If you can't be bothered to change your sources.list once or twice a > year, then you probably should be running stable. but maybe someone else would be willing to invoke a "ln" commands everytime a new new alpha repo is born. tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie once 0.3.5.x repos are created the link would point to tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie It is my opinion that this will help reduce the amount of relays running outdated versions of tor. It will certainly avoid having to update the tpo website, which isn't a big task and could probably be automated but it isn't done currently. "..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas (and break setups)!" Yes, and I think that is better than relays stuck on an older version because the former repo no longer exists and operators still can choose the old repos which will not jump to newer major versions. [1] https://www.torproject.org/docs/debian.html.en#ubuntu [2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3 [3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html [4] https://trac.torproject.org/projects/tor/ticket/26474 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

4 7

tor relay process health data for operators (controlport)
by nusenu 16 Nov '20

16 Nov '20

Hi, every now and then I'm in contact with relay operators about the "health" of their relays. Following these 1:1 discussions and the discussion on tor-relays@ I'd like to rise two issues with you (the developers) with the goal to help improve relay operations and end user experience in the long term: 1) DNS (exits only) 2) tor relay health data 1) DNS ------ Current situation: Arthur Edelstein provides public measurements to tor exit relay operators via his page at: https://arthuredelstein.net/exits/ This page is updated once daily. the process to use that data looks like this: - first they watch Arthur's measurement results - if their failure rate is non-zero they try to tweak/improve/change their setup - wait for another 24 hours (next measurement) This is a somewhat suboptimal and slow feedback loop and is probably also less accurate and less valuable data when compared to the data the tor process can provide. Suggestion for improvement: Exposes the following DNS status information via tor's controlport to help debug and detect DNS issues on exit relays: (total numbers since startup) - amount of DNS queries send to the resolver - amount of DNS queries send to the resolver due to a RESOLVE request - DNS queries send to resolver due to a reverse RESOLVE request - amount of queries that did not result in any answer from the resolver - breakdown of number of responses by response code (RCODE) https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… - max amount of DNS queries send per curcuit If this causes a significant performance impact this feature should be disabled by default. 2) general relay health metrics -------------------------------- Compared to other server daemons (webserver, DNS server, ..) tor provides little data for operators to detect operational issues and anomalies. I'd suggest to provide the following stats via the control port: (most of them are already written to logfiles by default but not accessible via the controlport as far as I've seen) - total amount of memory used by the tor process - amount of currently open circuits - circuit handshake stats (TAP / NTor) DoS mitigation stats - amount of circuits killed with too many cells - amount of circuits rejected - marked addresses - amount of connections closed - amount of single hop clients refused - amount of closed/failed circuits broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n1402 https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n1994 - amount of closed/failed OR connections broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n2205 If this causes a significant performance impact this feature should be disabled by default. cell stats - extra info cell stats as defined in: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1072 This data should be useful to answer the following questions: - High level questions: Is the tor relay healthy? - is it hitting any resource limits? - is the tor process under unusual load? - why is tor using more memory? - is it slower than usual at handling circuits? - can the DNS resolver handle the amount of DNS queries tor is sending it? This data could help prevent errors from occurring or provide additional data when trying to narrow down issues. When it comes to the question: **Is it "safe" to make this data accessible via the controlport?** I assume it is safe for all information that current versions of tor writes to logfiles or even publishes as part of its extra info descriptor. Should tor provide this or similar data I'm planing to write scripts for operators to make use of that data (for example a munin plugin that connects to tor's controlport). I'm happy to help write updates for control-spec should these features seem reasonable to you. Looking forward to hearing your feedback. nusenu -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

5 7

DNS-over-HTTPS (DOH) in Firefox/Torbrowser
by nusenu 01 Jul '20

01 Jul '20

Hi, since Mozilla did tests [0] on DOH [1] in Firefox I was wondering if Torbrowser developers have put any thought into that as well? Note: I'm _not_ suggesting to use DOH in torbrowser I'm just asking because the answer probably matters for exit documentation in the relay guide if clients do DNS themselves over TCP connections instead of relying on the exit (even if torbrowser is not the only tor client). thanks, nusenu [0] https://www.ghacks.net/2018/03/20/firefox-dns-over-https-and-a-worrying-shi… [1] https://datatracker.ietf.org/doc/draft-hoffman-dns-over-https/ -- https://mastodon.social/@nusenu twitter: @nusenu_

4 4

Lets give every circuit its own exit IP?
by nusenu 18 Mar '20

18 Mar '20

The unbearable situation with Google's reCAPTCHA motivated this email (but it is not limited to this specific case). This idea came up when seeing a similar functionality in unbound (which has it for a different reason). Assumption: There are systems that block some tor exit IP addresses (most likely the bigger once), but they are not blocked due to the fact that they are tor exits. It just occurred that the IP got flagged because of "automated / malicious" requests and IP reputation systems. What if every circuit had its "own" IP address at the exit relay to avoid causing collateral damage to all users of the exit if one was bad? (until the exit runs out of IPs and starts to recycle previously used IPs again) The goal is to avoid accumulating a bad "reputation" for the single used exit IP address that affects all tor users of that exit. Instead of doing it on the circuit level you could do it based on time. Change the exit IP every 5 minutes (but do _not_ change the exit IPs for _existing_ circuits even if they live longer than 5 minutes). Yes, no one has that many IPv4 addresses but with the increasing availability of IPv6 at exits and destinations, this could be feasible to a certain extend, depending on how many IPv6 addresses the exit operator has. There are exit operators that have entire /48 IPv6 blocks. problems: - will not solve anything since reputation will shift to netblocks as well (How big of a netblock are you willing to block?) - you can tell two tor users easily apart from each other even if they use the same exit (or more generally: you can tell circuits apart). There might be all kinds of bad implications that I'm not thinking off right now. - check.tpo would no longer be feasible - how can do we still provide the list of exit IPs for easy blocking? Exits could signal their used netblock via their descriptor. What if they don't? (that in turn opens new kinds of attacks where an exit claims to be /0 and the target effectively blocks everything) - more state to track and store at the exit -... some random thoughts, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

4 6

Adding Tracing to little-t tor
by David Goulet 17 Jan '20

17 Jan '20

Greetings tor-dev! This email is a discussion on adding tracing to little-t tor. Tracing can be a very abstract notion sometimes so I'll start with a quick overview of what that is, what we can achieve and use cases within tor. Then I'll go over a last point which is safety. This email doesn't go into the technical details of userspace tracing on how and what will be done to add it to tor. That is for another discussion. 1. Overview Long story short, you can see tracing as a specific type of logging as in it records information of the application at runtime using tracepoints (similar to logging statement) so it can be used later. But the main differences from logging are in two parts: performance and API stability. Usually, tracing implies high performance as in adds very little overhead to the application in order to disrupt as little as possible the normal behavior of an application. This is extremely useful in cases where you want to catch race conditions or performance bottle necks. Tracers in userspace have usually an "inprocess library" which in short means that it records data (raw) from the application and move it to an outside buffer. Then, that buffer is emptied either on disk or network by the outside component of the tracer for which the data can be analyzed after collection. So all a tracer do is, within the application when a tracepoint is hit, copy some data into a buffer and yields back to the application. The other part is the API stability. Very often, logs (let say at DEBUG level) don't usually have strict stable requirements between released versions. But tracing events (tracepoints), are exposed to the outside for tracers to hook on, and for people to run analyzing tools on the recorded data. Thus, stability is usually strongly encouraged. In other words, what the tracepoint exposes, once released stable, should really not change that much over time. With a proper abstraction in the application, we can offer stable tracepoints for which a variety of tracers can hook themselves on at runtime. It is all about providing an interface to the outside world. 2. Why Tracing in Tor The tor software is a very complex beast. It has dozens of subsystems with various interactions between them. One of the big main job of tor is to relay data as fast as possible in order to keep the latency low. Which means, that there are code paths that are considered "fast path" implying that they must remain light and fast. One example is the crypto code that is hit at each cell. Tracing comes in extremely handy to hunt down race conditions, performance issues, or even multithreading problems. A fast relay, let say 25MB/s, if we wanted to record cell timing in order to hunt down such issues, it simply can _not_ be done with logging at debug level since it slows down considerably tor but also fills the disk in a matter of minutes. And using the control port is not a good solution for two main reasons: string formatting at each event and control port is part of the mainloop. So anything you ask to go on the control port will add an overhead to the overall behavior of tor which is not good when you hunt down races. One concrete example where tracing was used in the past in tor is with the rewrite of the cell scheduler (KIST). In order to measure cell timings within Tor so bottlenecks issues could be found, tracing had to be added so millions of events could be recorded within few minutes of using a fast relay in production. In pressure situation, this is where tracing comes handy. Tracing was also used recently to find onion service v3 reachability issues. In order to correlate connection, cell and circuit level problems with the higher level HS subsystem, we were able to record events in all those subsystems, match them with their precise timing (offered by tracing) and analyze the results later on after recording the data. 3. Safety Discussion Onto the last part I wanted to raise. Allowing anyone to record very low level data from tor, there is an obvious safety question that must be raised. Over the years, I've talked about tracing with many people in Tor and the consensus was always that it should never be enabled in production. As in, the packages shipped by Tor or by distros should _never_ build the tracepoints. In other words, it should be considered a development option only. Not only an option, but compiled _out_ in production and one has to explicitly build them into tor. For example (nothing final, just to show the idea): $ ./configure --enable-tracing I personally think that should be enough since the presence of the code upstream won't stop people from using it (bad or not) but we can prevent it to be in any legit Tor packages out there. See it a bit like the obsolete Tor2web option that was never enabled in any published packages by Tor Project or distros, one had to explicitly enable it at configure time. The ControlPort is allowed in production and if a malicious actor gets access to it, then game over. I do see tracing like that as well but at least we can control its availability as a feature where we can't for the ControlPort as of today. Any feedback is very welcome! Concerns, questions, thoughts. Cheers! David -- AOrq46damX3clZogjR9FlXTru90GV9IT5Rq/J0EzVSA=

3 4

Updates to Prop306: A Tor Implementation of IPv6 Happy Eyeballs
by Neel Chauhan 15 Jan '20

15 Jan '20

Hi tor-dev@ mailing list, Sorry for the many-months delay in updating Prop306. I have updated Prop306, which is the IPv6 Happy Eyeballs proposal. The GitHub PR is here: https://github.com/torproject/torspec/pull/98 The Trac ticket is here: https://trac.torproject.org/projects/tor/ticket/29801 Some of the older discussion on Prop306 can be seen on the thread here: https://lists.torproject.org/pipermail/tor-dev/2019-August/013959.html Could some of you please review this proposal? -Neel === https://www.neelc.org/

2 3

Stem and Nyx bug tracking moved
by Damian Johnson 21 Dec '19

21 Dec '19

Hi all. Stem and Nyx's issue tracking has moved from Trac to GitHub. In the future please file tickets at... https://github.com/torproject/stem/issues/ https://github.com/torproject/nyx/issues/ Thanks! PS. All existing tickets have moved. These trac components will be disabled to cut down on confusion... https://trac.torproject.org/projects/tor/ticket/32832

1 0

(No Subject)
by techniquea2z＠protonmail.com 14 Dec '19

14 Dec '19

I am working on nibirumrx3wruiqf and have some questions: How to setup DDOS protection on the VPS & nginx levels? Any guidelines on nginx configurations for scale? How does mirroring work? I imagine you'd open many ports on the torrc, and have nginx listen to different instances of the same app on different http ports. If this project interests you and you want to collaborate, contact me

1 0

Probability of Guessing a v3 Onion Address
by procmem＠riseup.net 11 Dec '19

11 Dec '19

Hi I was wondering what the mathematical probability of guessing an onion v3 address that is kept secret. Or asked differently: what is the entropy of v3 addresses if an adversary decides to bruteforce the entire keyspace? I am struggling to come up with a usecase for authenticated v3 services when keeping an address secret has the same effect and one can generate multiple addresses for the same server and share them with different entities. The degraded usability of v3 auth services compared to v2 is the reason I'm asking.

3 2

Onion DoS: Killing rendezvous circuits over the application layer
by George Kadianakis 10 Dec '19

10 Dec '19

Greetings! This is another thread [0] about onion service denial-of-service attacks. It has long been suggested that onion service operators should be given the option to kill spammy rendezvous circuits at will if they feel they are causing too much damage. Right now this is possible using the HiddenServiceExportCircuitID torrc option (introduced in #4700) and then using the CLOSECIRCUIT control port command to close circuits. Unfortunately, we have recently got reports that this technique is not viable for busy onion services under DoS because their control port gets overwhelmed by (useful for them) events, and it's basically rendered useless to the point that any CLOSECIRCUIT command takes several seconds to become effective. For this reason, multiple onion operators [1] have resorted to using the actual HTTP protocol as a direct channel of communication to the Tor daemon to request circuit shutdowns. This works by embedding a special string (or HTTP error code) to the HTTP responses from nginx to the Tor daemon and adding special custom code to the Tor daemon to close circuits that carry this string. This seems to work well enough for people so far. This is a thread to discuss this approach and other alternatives since it seems a useful tool against application-layer onion service denial of service attacks. Let me go through the positives and negatives of actuallym erging this defence upstream to little-t-tor: --- Positives: 1) This is a solid defense that actually helps people and has been reported as a positive countermeasure in an area that has been hard to find concrete defences (also see [0]). 2) Seems like more and more people are doing this already in a custom ad-hoc fashion, so merging this upstream will at least give them a secure way of doing it (instead of writing custom C code). 3) It's actually a pretty simple patch in terms of tech-debt and maintaining it. 4) The more we address DoS vectors like this one, the less incentive will exist for DoS actors to exist. Effectively improving the long-term health of the network. Negatives: a) It's a dirty hotfix that blends the networking layers and might be annoying to maintain in the long-term. b) It only works for HTTP (and without SSL?). --- For me, point (1) is extremely important, since we've been struggling with helping onion services that are getting DoSsed and this feature offers a solid defense against practical attacks. However, IMO the right way to do this feature, would be to improve the control port code and design so that it doesn't get so overwhelmed by multiple events. That said, I'm not sure exactly what kind of changes we would have to do to the control port to actually make it a viable option, and it seems to me like a pretty big project that serves as a medium-term to long-term solution (which we have no resources to pursue right now), whereas the hack of this thread is more of a short-term solution. I'm looking forward to constructive feedback here, since this seems like a controversial feature that users really need. Thanks! :) [0]: serving as a continuation of previous classics such as: https://lists.torproject.org/pipermail/tor-dev/2019-June/013875.html https://lists.torproject.org/pipermail/tor-dev/2019-July/013923.html etc. [1]: http://www.hackerfactor.com/blog/index.php?/archives/804-A-New-Tor-Attack.h… https://trac.torproject.org/projects/tor/ticket/32511

4 3