July 2019 - tor-dev - lists.torproject.org

the consequences of deprecating debian alpha repos with every new major branch
by nusenu 21 Dec '21

21 Dec '21

Hi, tldr: - more outdated relays (that is a claim I'm making and you could easily proof me wrong by recreating the 0.3.3.x alpha repos and ship 0.3.3.7 in them and see how things evolve after a week or so) - more work for the tpo website maintainer - less happy relay operators [3][4] - more work for repo maintainers? (since a new repo needs to be created) When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23 I was about to submit a PR for the website to include it in the sources.list generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first. The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to update the repo generator for 0.3.4.x nonetheless and here is why. Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search) (> 3.2% CW fraction) Then I realized that this was the last version the tor-experimental-0.3.3.x-* repos were shipping before they got abandoned due to the new 0.3.4.x-* repos (I can no longer verify it since they got removed by now). Peter made it clear in the past that the current way to have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie) will not change [2]: > If you can't be bothered to change your sources.list once or twice a > year, then you probably should be running stable. but maybe someone else would be willing to invoke a "ln" commands everytime a new new alpha repo is born. tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie once 0.3.5.x repos are created the link would point to tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie It is my opinion that this will help reduce the amount of relays running outdated versions of tor. It will certainly avoid having to update the tpo website, which isn't a big task and could probably be automated but it isn't done currently. "..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas (and break setups)!" Yes, and I think that is better than relays stuck on an older version because the former repo no longer exists and operators still can choose the old repos which will not jump to newer major versions. [1] https://www.torproject.org/docs/debian.html.en#ubuntu [2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3 [3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html [4] https://trac.torproject.org/projects/tor/ticket/26474 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

4 7

tor relay process health data for operators (controlport)
by nusenu 16 Nov '20

16 Nov '20

Hi, every now and then I'm in contact with relay operators about the "health" of their relays. Following these 1:1 discussions and the discussion on tor-relays@ I'd like to rise two issues with you (the developers) with the goal to help improve relay operations and end user experience in the long term: 1) DNS (exits only) 2) tor relay health data 1) DNS ------ Current situation: Arthur Edelstein provides public measurements to tor exit relay operators via his page at: https://arthuredelstein.net/exits/ This page is updated once daily. the process to use that data looks like this: - first they watch Arthur's measurement results - if their failure rate is non-zero they try to tweak/improve/change their setup - wait for another 24 hours (next measurement) This is a somewhat suboptimal and slow feedback loop and is probably also less accurate and less valuable data when compared to the data the tor process can provide. Suggestion for improvement: Exposes the following DNS status information via tor's controlport to help debug and detect DNS issues on exit relays: (total numbers since startup) - amount of DNS queries send to the resolver - amount of DNS queries send to the resolver due to a RESOLVE request - DNS queries send to resolver due to a reverse RESOLVE request - amount of queries that did not result in any answer from the resolver - breakdown of number of responses by response code (RCODE) https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… - max amount of DNS queries send per curcuit If this causes a significant performance impact this feature should be disabled by default. 2) general relay health metrics -------------------------------- Compared to other server daemons (webserver, DNS server, ..) tor provides little data for operators to detect operational issues and anomalies. I'd suggest to provide the following stats via the control port: (most of them are already written to logfiles by default but not accessible via the controlport as far as I've seen) - total amount of memory used by the tor process - amount of currently open circuits - circuit handshake stats (TAP / NTor) DoS mitigation stats - amount of circuits killed with too many cells - amount of circuits rejected - marked addresses - amount of connections closed - amount of single hop clients refused - amount of closed/failed circuits broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n1402 https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n1994 - amount of closed/failed OR connections broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n2205 If this causes a significant performance impact this feature should be disabled by default. cell stats - extra info cell stats as defined in: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1072 This data should be useful to answer the following questions: - High level questions: Is the tor relay healthy? - is it hitting any resource limits? - is the tor process under unusual load? - why is tor using more memory? - is it slower than usual at handling circuits? - can the DNS resolver handle the amount of DNS queries tor is sending it? This data could help prevent errors from occurring or provide additional data when trying to narrow down issues. When it comes to the question: **Is it "safe" to make this data accessible via the controlport?** I assume it is safe for all information that current versions of tor writes to logfiles or even publishes as part of its extra info descriptor. Should tor provide this or similar data I'm planing to write scripts for operators to make use of that data (for example a munin plugin that connects to tor's controlport). I'm happy to help write updates for control-spec should these features seem reasonable to you. Looking forward to hearing your feedback. nusenu -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

5 7

DNS-over-HTTPS (DOH) in Firefox/Torbrowser
by nusenu 01 Jul '20

01 Jul '20

Hi, since Mozilla did tests [0] on DOH [1] in Firefox I was wondering if Torbrowser developers have put any thought into that as well? Note: I'm _not_ suggesting to use DOH in torbrowser I'm just asking because the answer probably matters for exit documentation in the relay guide if clients do DNS themselves over TCP connections instead of relying on the exit (even if torbrowser is not the only tor client). thanks, nusenu [0] https://www.ghacks.net/2018/03/20/firefox-dns-over-https-and-a-worrying-shi… [1] https://datatracker.ietf.org/doc/draft-hoffman-dns-over-https/ -- https://mastodon.social/@nusenu twitter: @nusenu_

4 4

Lets give every circuit its own exit IP?
by nusenu 18 Mar '20

18 Mar '20

The unbearable situation with Google's reCAPTCHA motivated this email (but it is not limited to this specific case). This idea came up when seeing a similar functionality in unbound (which has it for a different reason). Assumption: There are systems that block some tor exit IP addresses (most likely the bigger once), but they are not blocked due to the fact that they are tor exits. It just occurred that the IP got flagged because of "automated / malicious" requests and IP reputation systems. What if every circuit had its "own" IP address at the exit relay to avoid causing collateral damage to all users of the exit if one was bad? (until the exit runs out of IPs and starts to recycle previously used IPs again) The goal is to avoid accumulating a bad "reputation" for the single used exit IP address that affects all tor users of that exit. Instead of doing it on the circuit level you could do it based on time. Change the exit IP every 5 minutes (but do _not_ change the exit IPs for _existing_ circuits even if they live longer than 5 minutes). Yes, no one has that many IPv4 addresses but with the increasing availability of IPv6 at exits and destinations, this could be feasible to a certain extend, depending on how many IPv6 addresses the exit operator has. There are exit operators that have entire /48 IPv6 blocks. problems: - will not solve anything since reputation will shift to netblocks as well (How big of a netblock are you willing to block?) - you can tell two tor users easily apart from each other even if they use the same exit (or more generally: you can tell circuits apart). There might be all kinds of bad implications that I'm not thinking off right now. - check.tpo would no longer be feasible - how can do we still provide the list of exit IPs for easy blocking? Exits could signal their used netblock via their descriptor. What if they don't? (that in turn opens new kinds of attacks where an exit claims to be /0 and the target effectively blocks everything) - more state to track and store at the exit -... some random thoughts, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

4 6

Optimistic SOCKS Data
by Tom Ritter 15 Oct '19

15 Oct '19

The attached is a draft proposal for allowing tor to lie to an application about the SOCKS connection enabling it to send data optimistically. It's going to need some fleshing out in ways I am not familiar with, but I wanted to get something out to start as we think that this is probably the best path forward for bringing back Tor Browser's optimistic SOCKS behavior. -tom

6 11

New Proposal 306: A Tor Implementation of IPv6 Happy Eyeballs
by neel＠neelc.org 06 Sep '19

06 Sep '19

Hi tor-dev@ mailing list, I have a new proposal: A Tor Implementation of IPv6 Happy Eyeballs This is to implement Tor IPv6 Happy Eyeballs and acts as an alternative to Prop299 as requested here: https://trac.torproject.org/projects/tor/ticket/29801 The GitHub pull request is here: https://github.com/torproject/torspec/pull/87 Thank You, Neel Chauhan === https://www.neelc.org/

7 18

24 hours worth of BridgeDB usage metrics
by Philipp Winter 30 Jul '19

30 Jul '19

Over at <https://bugs.torproject.org/9316>, we are working on having BridgeDB export metrics. The patch is almost done and I deployed the work-in-progress code on BridgeDB, so we can take a look at the metrics and think of ways to improve them. The metrics format encodes the approximate number of requests per distribution mechanism per transport per country per success/fail. All numbers are rounded up to the next multiple of 10. The last field, "none", will be used for an anomaly score and is currently unused. For example, the line bridgedb-metric-count email.obfs4.riseup.success.none 10 tells us that there have been 1-10 successful email requests for obfs4 coming from Riseup addresses. I attached 24 hours worth of metrics to this email. Keep the following issues in mind: * My feature branch hasn't been reviewed yet and likely still has bugs, so take all numbers with a grain of salt. * The country codes are based on Debian stretch's geoip-database, which is slightly outdated and uses Maxmind's far-from-perfect GeoLite database. * The country code "??" refers to geo-location failure or lack of IP addresses (in the cast of moat). The country code "zz" refers to a request from a Tor exit relay. Some observations: * Gmail sees much more use than Riseup. That's no surprise. * The email distributor sees more vanilla than obfs4 requests. I wonder to what degree this is caused by the poor UX of the email distributor. * For HTTPS, many countries have a fail and success bucket of 10 each. I would expect this to be at least one user who failed the captcha at least once before finally getting it right. * The captcha success rate for obfs4 over moat is 54%. That's very low and must cause lots of frustration for users. This is a known issue that's tracked in this ticket: <https://bugs.torproject.org/29695> * Ignore the large amount of HTTPS requests from "zz" -- I expect the vast majority of these to be a bot that's interacting with BridgeDB over exit relays. After a cursory look at the numbers, I would like to aggregate the data, to make it easier to compare distributors, transports, and countries. For example: how do moat, email, and HTTPS rank in popularity? I'll improve the patch to keep track of these numbers in separate metrics. Any thoughts or suggestions? Cheers, Philipp

3 4

What hidden services does my relay node know about?
by clive.jenson＠tutanota.com 29 Jul '19

29 Jul '19

I built tor from source and am running it as a relay (not exit node) and also as a directory server. It is my understanding that after a while I should experience V2 hidden services publishing their descriptors to me (each hidden service picks 6 directory servers at random to publish their descriptor to). How can I tell if this is happening? Can I (temporarily, for debugging purposes) view the descriptors being published to me? I'm thinking their might be some useful logging if I enable debug logs (but I can't find it)? Or maybe there is something I can get over the control protocol (but I couldn't find anything obvious in the stem API docs that exposes it)? Or maybe I could make a code change to log this information (I'm not sure where the code is that receives the announcements)?

2 4

Hack
by Pordeshi Pothik 27 Jul '19

27 Jul '19

Hi,I want to learn how to hack facebook.

2 1

Proposal 306: Onion Balance Support for Onion Service v3
by David Goulet 26 Jul '19

26 Jul '19

Greetings! Around 4 months ago, nickm published a proposal draft for OnionBalance to support onion services v3. Due to #29583, it turns out that we can afterall take the easy approach which is basically how OnionBalance is working today for v2 but transposed to v3. See the ticket for the reasons why for now we'll proceed this way. However, we all agreed that #29583 is a "must fix at some point" meaning that OnionBalance v3 will then become much more complicated. In order to not forget about it, I'm reviving nickm's proposal as prop306 so we have the draft in torspec.git. See attachement for the proposal. Cheers! David -- vjZrE+FHHEPpllrbcUukx7LtWFzk066L23yoggvnmso=

3 2