tor-dev November 2019

tor-dev@lists.torproject.org

15 participants
13 discussions

the consequences of deprecating debian alpha repos with every new major branch
by nusenu 21 Dec '21

21 Dec '21

Hi, tldr: - more outdated relays (that is a claim I'm making and you could easily proof me wrong by recreating the 0.3.3.x alpha repos and ship 0.3.3.7 in them and see how things evolve after a week or so) - more work for the tpo website maintainer - less happy relay operators [3][4] - more work for repo maintainers? (since a new repo needs to be created) When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23 I was about to submit a PR for the website to include it in the sources.list generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first. The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to update the repo generator for 0.3.4.x nonetheless and here is why. Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search) (> 3.2% CW fraction) Then I realized that this was the last version the tor-experimental-0.3.3.x-* repos were shipping before they got abandoned due to the new 0.3.4.x-* repos (I can no longer verify it since they got removed by now). Peter made it clear in the past that the current way to have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie) will not change [2]: > If you can't be bothered to change your sources.list once or twice a > year, then you probably should be running stable. but maybe someone else would be willing to invoke a "ln" commands everytime a new new alpha repo is born. tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie once 0.3.5.x repos are created the link would point to tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie It is my opinion that this will help reduce the amount of relays running outdated versions of tor. It will certainly avoid having to update the tpo website, which isn't a big task and could probably be automated but it isn't done currently. "..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas (and break setups)!" Yes, and I think that is better than relays stuck on an older version because the former repo no longer exists and operators still can choose the old repos which will not jump to newer major versions. [1] https://www.torproject.org/docs/debian.html.en#ubuntu [2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3 [3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html [4] https://trac.torproject.org/projects/tor/ticket/26474 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

4 7

tor relay process health data for operators (controlport)
by nusenu 16 Nov '20

16 Nov '20

Hi, every now and then I'm in contact with relay operators about the "health" of their relays. Following these 1:1 discussions and the discussion on tor-relays@ I'd like to rise two issues with you (the developers) with the goal to help improve relay operations and end user experience in the long term: 1) DNS (exits only) 2) tor relay health data 1) DNS ------ Current situation: Arthur Edelstein provides public measurements to tor exit relay operators via his page at: https://arthuredelstein.net/exits/ This page is updated once daily. the process to use that data looks like this: - first they watch Arthur's measurement results - if their failure rate is non-zero they try to tweak/improve/change their setup - wait for another 24 hours (next measurement) This is a somewhat suboptimal and slow feedback loop and is probably also less accurate and less valuable data when compared to the data the tor process can provide. Suggestion for improvement: Exposes the following DNS status information via tor's controlport to help debug and detect DNS issues on exit relays: (total numbers since startup) - amount of DNS queries send to the resolver - amount of DNS queries send to the resolver due to a RESOLVE request - DNS queries send to resolver due to a reverse RESOLVE request - amount of queries that did not result in any answer from the resolver - breakdown of number of responses by response code (RCODE) https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… - max amount of DNS queries send per curcuit If this causes a significant performance impact this feature should be disabled by default. 2) general relay health metrics -------------------------------- Compared to other server daemons (webserver, DNS server, ..) tor provides little data for operators to detect operational issues and anomalies. I'd suggest to provide the following stats via the control port: (most of them are already written to logfiles by default but not accessible via the controlport as far as I've seen) - total amount of memory used by the tor process - amount of currently open circuits - circuit handshake stats (TAP / NTor) DoS mitigation stats - amount of circuits killed with too many cells - amount of circuits rejected - marked addresses - amount of connections closed - amount of single hop clients refused - amount of closed/failed circuits broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n1402 https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n1994 - amount of closed/failed OR connections broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n2205 If this causes a significant performance impact this feature should be disabled by default. cell stats - extra info cell stats as defined in: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1072 This data should be useful to answer the following questions: - High level questions: Is the tor relay healthy? - is it hitting any resource limits? - is the tor process under unusual load? - why is tor using more memory? - is it slower than usual at handling circuits? - can the DNS resolver handle the amount of DNS queries tor is sending it? This data could help prevent errors from occurring or provide additional data when trying to narrow down issues. When it comes to the question: **Is it "safe" to make this data accessible via the controlport?** I assume it is safe for all information that current versions of tor writes to logfiles or even publishes as part of its extra info descriptor. Should tor provide this or similar data I'm planing to write scripts for operators to make use of that data (for example a munin plugin that connects to tor's controlport). I'm happy to help write updates for control-spec should these features seem reasonable to you. Looking forward to hearing your feedback. nusenu -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

5 7

DNS-over-HTTPS (DOH) in Firefox/Torbrowser
by nusenu 01 Jul '20

01 Jul '20

Hi, since Mozilla did tests [0] on DOH [1] in Firefox I was wondering if Torbrowser developers have put any thought into that as well? Note: I'm _not_ suggesting to use DOH in torbrowser I'm just asking because the answer probably matters for exit documentation in the relay guide if clients do DNS themselves over TCP connections instead of relying on the exit (even if torbrowser is not the only tor client). thanks, nusenu [0] https://www.ghacks.net/2018/03/20/firefox-dns-over-https-and-a-worrying-shi… [1] https://datatracker.ietf.org/doc/draft-hoffman-dns-over-https/ -- https://mastodon.social/@nusenu twitter: @nusenu_

4 4

Lets give every circuit its own exit IP?
by nusenu 18 Mar '20

18 Mar '20

The unbearable situation with Google's reCAPTCHA motivated this email (but it is not limited to this specific case). This idea came up when seeing a similar functionality in unbound (which has it for a different reason). Assumption: There are systems that block some tor exit IP addresses (most likely the bigger once), but they are not blocked due to the fact that they are tor exits. It just occurred that the IP got flagged because of "automated / malicious" requests and IP reputation systems. What if every circuit had its "own" IP address at the exit relay to avoid causing collateral damage to all users of the exit if one was bad? (until the exit runs out of IPs and starts to recycle previously used IPs again) The goal is to avoid accumulating a bad "reputation" for the single used exit IP address that affects all tor users of that exit. Instead of doing it on the circuit level you could do it based on time. Change the exit IP every 5 minutes (but do _not_ change the exit IPs for _existing_ circuits even if they live longer than 5 minutes). Yes, no one has that many IPv4 addresses but with the increasing availability of IPv6 at exits and destinations, this could be feasible to a certain extend, depending on how many IPv6 addresses the exit operator has. There are exit operators that have entire /48 IPv6 blocks. problems: - will not solve anything since reputation will shift to netblocks as well (How big of a netblock are you willing to block?) - you can tell two tor users easily apart from each other even if they use the same exit (or more generally: you can tell circuits apart). There might be all kinds of bad implications that I'm not thinking off right now. - check.tpo would no longer be feasible - how can do we still provide the list of exit IPs for easy blocking? Exits could signal their used netblock via their descriptor. What if they don't? (that in turn opens new kinds of attacks where an exit claims to be /0 and the target effectively blocks everything) - more state to track and store at the exit -... some random thoughts, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

4 6

HSv3 descriptor work in stem
by George Kadianakis 04 Dec '19

04 Dec '19

Hello atagar, I'm starting this thread to ask you questions about stem and the HSv3 work we've been doing over email so that we don't do it over IRC. Here is an initial question: I'm working on HSv3 descriptor encoding, and I'm trying to understand how `_descriptor_content()` works. In particular, I want to compute the signature of a descriptor, but I see that `descriptor_content()` fills it with random bytes in all the `content()` methods I managed to find: ('signature', _random_crypto_blob('SIGNATURE')), What's the right way to compute the signature for such objects? In particular, I would need a method that first generates the whole descriptor body, and then computes the signature of that with a given private key. Can I use `_descriptor_content()` to do that? Or should I call `_descriptor_content()` to generate the whole thing _without_ the sig, and then do the signature computation on its result and concatenate it after? Thanks! :)

3 10

Practracker regen in #30381
by teor 28 Nov '19

28 Nov '19

Hi George, David, It looks like you regenerated the whole practracker file in #30381: https://trac.torproject.org/projects/tor/ticket/30381 https://github.com/torproject/tor/commit/53ac9a9a91a8f2ab45c755504567160749… But we usually just add exceptions for the files that we modified. When we do a full regeneration, we lose a whole lot of warnings that tell us where our code quality is getting worse. Do you mind if I revert the unrelated changes? T -- teor ----------------------------------------------------------------------

3 4

Proposal 271 - improvements
by Florentin Rochet 26 Nov '19

26 Nov '19

Hello, I'm doing research on improving Tor security via path selection. This work is with Ryan Wails, Aaron Johnson, Prateek Mittal, and Olivier Pereira. In our work, we ran into load-balancing problems that affected our experiments. We identified the problem to originate in the specification and implementation of Proposal 271, "Another algorithm for guard selection", which was implemented in tor-0.3.0.1-alpha ~2 years ago and is thus now in tor stable. In short, Prop 271 causes guards to be selected with probabilities different than their weights due to the way it samples many guards and then chooses primary guards from that sample. We are suggesting a straightforward fix to the problem, which is, roughly speaking, to choose primary guards in the order in which they were sampled. We have created a patch implementing this fix for the case affecting our experiments, which would improve the current situation. We are further suggesting that Tor apply the technique throughout the guard-selection logic. In more detail, Prop 271 chooses guards via a multi-step process: 1. It chooses 20 distinct guards (and sometimes more) by sampling without replacement with probability proportional to consensus weight. 2. It produces two subsets of the sample: (1) "filtered" guards, which are guards that satisfy various torrc constraints, and (2) "confirmed" guards, which are guards through which a circuit has been constructed. 3. The "primary" guards (i.e. the actual guards used for circuits) are chosen from the confirmed and filtered subsets. I'm ignoring the additional "usable" subsets for clarity. This description is based on Section 4.6 of the specification (https://gitweb.torproject.org/torspec.git/tree/guard-spec.txt). The primary guards are selected uniformly at random from the filtered guards when no confirmed guards exist. No confirmed guards appear to exist until some primary guards have been selected, and so when Tor is started the first time the primary guards always come only from the filtered set. The uniformly-random selection causes a bias in primary-guard selection away from consensus weights and towards a more uniform selection of guards. As just an example of the problem, if there were only 20 guards in the network, the sampled set would be all guards and primary guard selection would be entirely uniformly random, ignoring weights entirely. This bias is worse the larger the sampled set is relative to the entire set of guards, and it has a significant effect on Tor simulations, which are typically on smaller networks. We believe that this issue has only a limited effect on Tor currently due to the relatively large number of guards. However, the issue is potentially worse when there exist confirmed guards. In this case, primary guards are chosen from the intersection of confirmed and filtered guards in the order they were added to the confirmed set. That order seems to be the order in which circuits were successfully created through the guards, which would thus be both somewhat non-deterministic (because it depends on when a circuit successfully finishes) and random (because circuits are only attempted through primary guards, which are initially uniformly-random filtered guards). The non-determinism especially could yield guard selection probabilities that deviate quite a bit from the consensus weights. This design has both performance and security implications. It potentially reduces the performance of the Tor network by making the load unbalanced. It also affects the correctness of performance analyses done with Shadow simulations, and the error is likely much larger due to the smaller size of simulation networks, which commonly include 100 or fewer guards. The design also reduces Tor's security by increasing the number of clients that an adversary running small relays can observe. In addition, an adversary has to wait less time than it should after it starts a malicious guard to be chosen by a client. This weakness occurs because the malicious guard only needs to enter the sampled list to have a chance to be chosen as primary, rather than having to wait until all previously-sampled guards have already expired. We propose a solution that fits well within the existing guard-selection algorithm. Our solution is to select primary guards in the order they were sampled. This ordering should be applied after the filtering and/or confirmed guard sets are constructed as normal. That is, primary guards should be selected from the filtered guards (if no guards are both confirmed and filtered) or from the set of confirmed and filtered guards (if such guards exist) in the order they were initially sampled. This solution guarantees that each primary guard is selected (without replacement) from all guards with a probability that is proportional to its consensus weight. We include a patch that applies this ordering only when primary guards are selected from filtered guards. This fixes the problem in Shadow simulations because confirmed guards never exist when primary guards are being selected. However, to solve all issues in the real Tor network, we recommend that you apply similar logic to the case that primary guards are selected from confirmed guards. Note that the issue of sampling skewing guard selection seems to have been raised in proposal discussions, which is documented in the proposal (although the proposed solution is less practical): > [Paul Syverson in a conversation at the Wilmington Meeting 2017 says that > we should look into how we're doing this sampling. Essentially, his > concern is that, since we are sampling by bandwidth at first (when we > choose the `sampled` set), then later there is another bias—when trying to > build circuits (and hence marking guards as confirmed) we select those > which completed a usable circuit first (and hence have the lowest > latency)—that this sort of "doubly skewed" selection may "snub" some > low-consensus-weight guards and leave them unused completely. Thus the > issue is primarily that we're not allocating network resources > efficiently. Mine and Nick's guard algorithm simulation code never > checked what percentage of possible guards the algorithm reasonably > allowed clients to use; this would be an interesting thing to check in > simulation at some point. If it does turn out to be a problem, Paul's > intuition for a fix is to select uniformly at random to obtain the > `sampled` set, then weight by bandwidth when trying to build circuits and > marking guards as confirmed. —isis] Best, Florentin

3 3

introduce some customization on the tbb
by sarpedon montecarlo 21 Nov '19

21 Nov '19

Hello to the community! my first question is that is there a solution for rapid development of tbb? i was wondering that each time a contributor wants to change something in the tbb or add extra functionality, they have to go through the build process which i just guess would be very time consuming. i was wondering that is there any containerized environment for this? any docker ecosystem available or other solutions that might help? my second question is about the changes that i want to implement. i am interested into the graphical settings page about bridges and tor process that currently tbb is exposing to the end users. if we want to add more functionality into this settings page, where should we start to change? and is there any possibility that we may have some control over the underlying tor process as well, for instance controlling it's launching or it's torrc configuration; because as i see, there must be a link between the graphical interface of tbb and the underlying tor process so that is the way we can control bridges for instance. similar to this i was planning to add more graphical options to the interface for controlling other configuration aspects of the tor process through it's torrc or other workarounds. in the past i have achieved this by writing a browser extension that benefits from native messaging for communication to the tor process; but this is not really performance friendly and an intuitive experience for the end users.i am not satisfied with the result. so i was interested on mirroring these functionalities inside tbb. i really appreciate your insights and thoughts on the matter, have a great day, Sarpedon.

3 5

转发： Which elliptic curve Tor use & use safe curve only
by Tom255 21 Nov '19

21 Nov '19

Please use Ed448-Goldilocks or named Curve448 (suggest) and Curve25519 only. Curve448 potentially offering 224 bits of security, even safer than NIST P-384. Curve25519 potentially offering 128 bits of security and safer and faster than NIST P-256. More important, they're both ECC Security, not only ECDLP Security. ECC Security doesn't equal to ECDLP Security. " Unfortunately, there is a gap between ECDLP difficulty and ECC security. None of these standards do a good job of ensuring ECC security. There are many attacks that break real-world ECC without solving ECDLP. The core problem is that if you implement the standard curves, chances are you're doing it wrong: -Your implementation produces incorrect results for some rare curve points. -Your implementation leaks secret data when the input isn't a curve point. -Your implementation leaks secret data through branch timing. -Your implementation leaks secret data through cache timing. " (https://safecurves.cr.yp.to/) And those curves are recommended by RFC 7748: https://tools.ietf.org/html/rfc7748 If possible, use Curve448 only. It's even safer than NIST P-384 and it's new.

1 0

torsocks configure bug (uClibc, maybe other nonstandard libcs)
by akater 14 Nov '19

14 Nov '19

In torsocks' configure.ac, libc_name is determined by means of ldd /usr/bin/yes which is then grepped for libc. On a uClibc system I use, ldd /usr/bin/yes yields two entries, namely libc.so.0 => ... ld64-uClibc.so.0 => ... The resulting string begins with quotation mark, ends with quotation mark and contains newline character. At some point during the configuration process (I have not figured out when exactly), the aforementioned string is trimmed erroneously. In my case, the first line is left out and the second one is discarded. The remaining chunk, with the opening quotation mark but without the closing one, ends up in another configuration file, producing a line like SOME_VARIABLE="libc.so.0 without the closing quotation mark. Build cannot proceed from there. The libc determination process thus needs to be fixed. My hotfix was to change grep 'libc\.' to grep '\slibc\.' but I can't suggest the proper solution as I'm not experienced with shell scripts. I will perform a test if provided with (a link to) a relevant standalone patch for torsocks-2.2.0 or torsocks-2.3.0.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev November 2019