tor-dev July 2020

tor-dev@lists.torproject.org

15 participants
15 discussions

the consequences of deprecating debian alpha repos with every new major branch
by nusenu 21 Dec '21

21 Dec '21

Hi, tldr: - more outdated relays (that is a claim I'm making and you could easily proof me wrong by recreating the 0.3.3.x alpha repos and ship 0.3.3.7 in them and see how things evolve after a week or so) - more work for the tpo website maintainer - less happy relay operators [3][4] - more work for repo maintainers? (since a new repo needs to be created) When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23 I was about to submit a PR for the website to include it in the sources.list generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first. The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to update the repo generator for 0.3.4.x nonetheless and here is why. Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search) (> 3.2% CW fraction) Then I realized that this was the last version the tor-experimental-0.3.3.x-* repos were shipping before they got abandoned due to the new 0.3.4.x-* repos (I can no longer verify it since they got removed by now). Peter made it clear in the past that the current way to have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie) will not change [2]: > If you can't be bothered to change your sources.list once or twice a > year, then you probably should be running stable. but maybe someone else would be willing to invoke a "ln" commands everytime a new new alpha repo is born. tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie once 0.3.5.x repos are created the link would point to tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie It is my opinion that this will help reduce the amount of relays running outdated versions of tor. It will certainly avoid having to update the tpo website, which isn't a big task and could probably be automated but it isn't done currently. "..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas (and break setups)!" Yes, and I think that is better than relays stuck on an older version because the former repo no longer exists and operators still can choose the old repos which will not jump to newer major versions. [1] https://www.torproject.org/docs/debian.html.en#ubuntu [2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3 [3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html [4] https://trac.torproject.org/projects/tor/ticket/26474 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

4 7

IANA well-known URI suffix registration for tor-relay-fingerprints file
by nusenu 10 Oct '21

10 Oct '21

Hi, to reduce the risk of certain kinds of attacks I'm aiming to submit a short spec (like [1]) in accordance with RFC8615 https://tools.ietf.org/html/rfc8615#section-3.1 for the registration of of a suffix for the verifyurl from https://github.com/nusenu/ContactInfo-Information-Sharing-Specification#ver… The name I have in mind is: tor-relay-fingerprints The file contains comments (lines starting with #) and relay fingerprints. https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml Let me know if you have any feedback on this and whether you have any opinion on whether you (The Torproject) wants to be the change controller. The only question that came up was: Will there be two types of relay fingerprints in the future (Ed25519)? thanks, nusenu -- https://mastodon.social/@nusenu [1] https://keybase.io/docs/keybase_well_known

5 22

tor relay process health data for operators (controlport)
by nusenu 16 Nov '20

16 Nov '20

Hi, every now and then I'm in contact with relay operators about the "health" of their relays. Following these 1:1 discussions and the discussion on tor-relays@ I'd like to rise two issues with you (the developers) with the goal to help improve relay operations and end user experience in the long term: 1) DNS (exits only) 2) tor relay health data 1) DNS ------ Current situation: Arthur Edelstein provides public measurements to tor exit relay operators via his page at: https://arthuredelstein.net/exits/ This page is updated once daily. the process to use that data looks like this: - first they watch Arthur's measurement results - if their failure rate is non-zero they try to tweak/improve/change their setup - wait for another 24 hours (next measurement) This is a somewhat suboptimal and slow feedback loop and is probably also less accurate and less valuable data when compared to the data the tor process can provide. Suggestion for improvement: Exposes the following DNS status information via tor's controlport to help debug and detect DNS issues on exit relays: (total numbers since startup) - amount of DNS queries send to the resolver - amount of DNS queries send to the resolver due to a RESOLVE request - DNS queries send to resolver due to a reverse RESOLVE request - amount of queries that did not result in any answer from the resolver - breakdown of number of responses by response code (RCODE) https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… - max amount of DNS queries send per curcuit If this causes a significant performance impact this feature should be disabled by default. 2) general relay health metrics -------------------------------- Compared to other server daemons (webserver, DNS server, ..) tor provides little data for operators to detect operational issues and anomalies. I'd suggest to provide the following stats via the control port: (most of them are already written to logfiles by default but not accessible via the controlport as far as I've seen) - total amount of memory used by the tor process - amount of currently open circuits - circuit handshake stats (TAP / NTor) DoS mitigation stats - amount of circuits killed with too many cells - amount of circuits rejected - marked addresses - amount of connections closed - amount of single hop clients refused - amount of closed/failed circuits broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n1402 https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n1994 - amount of closed/failed OR connections broken down by their reason value https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n2205 If this causes a significant performance impact this feature should be disabled by default. cell stats - extra info cell stats as defined in: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1072 This data should be useful to answer the following questions: - High level questions: Is the tor relay healthy? - is it hitting any resource limits? - is the tor process under unusual load? - why is tor using more memory? - is it slower than usual at handling circuits? - can the DNS resolver handle the amount of DNS queries tor is sending it? This data could help prevent errors from occurring or provide additional data when trying to narrow down issues. When it comes to the question: **Is it "safe" to make this data accessible via the controlport?** I assume it is safe for all information that current versions of tor writes to logfiles or even publishes as part of its extra info descriptor. Should tor provide this or similar data I'm planing to write scripts for operators to make use of that data (for example a munin plugin that connects to tor's controlport). I'm happy to help write updates for control-spec should these features seem reasonable to you. Looking forward to hearing your feedback. nusenu -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu

5 7

Re: [tor-dev] [RFC] Proposal: A First Take at PoW Over Introduction Circuits
by tevador 25 Sep '20

25 Sep '20

Hi all, I was asked by George to submit my comments about the proposal and to suggest suitable RandomX parameters for this PoW scheme. > For our dynamic PoW system to work, we will need to be able to compare PoW > tokens with each other. To do so we define a function: > unsigned effort(uint8_t *token) > which takes as its argument a hash output token, and returns the number of > leading zero bits on it. This definition makes the effort exponential, i.e. the computational resources required to reach one notch higher effort increase by a factor of 2 each time. I suggest to use linear effort defined as the quotient of dividing a bitstring of 1s by the hash value: == Example A: effort(00000001100010101101) = 11111111111111111111 / 00000001100010101101 or in decimal: effort(6317) = 1048575 / 6317 = 165. This definition of effort has the advantage of directly expressing the expected number of hashes that the client had to calculate to reach the effort. With the exponential definition, we could have an equivalent linear effort of either 128 (7 leading zeroes) or 256 (8 leading zeroes), while the linear definition provides smoother classification of PoW results. > The EXT_FIELD content format is: > > POW_VERSION [1 byte] > POW_NONCE [32 bytes] > I suggest to use a 16-byte nonce value, which is more than sufficient given the target security level and has the benefit of reducing the replay cache size to half. Since we expect the seed to be valid for around 3 hours (as proposed), then even if the service receives 1 million proofs per second and each proof has an effort of 1 million, then the number of submitted nonces from clients will only reach about 10^10. With a 108-bit solution space (subtracting 20 bits as the search space per client), the probability that two clients accidentally submit the same nonce is roughly 10^-13 (see [REF_BIRTHDAY]). Additionally, I suggest to add the client's effort for convenience: The updated EXT_FIELD content format would be: POW_VERSION [1 byte] POW_EFFORT [4 bytes] POW_NONCE [16 bytes] Including the effort has 2 benefits: 1. In case the Intro Priority Queue is full, the service doesn't need to waste time verifying PoW solutions that have effort lower than the last element in the queue. While an attacker can always lie about the actual effort of their nonce value, I think this field can still save some CPU cycles when the service is under heavy load. 2. The service can conveniently verify the reported effort with the following inequality: POW_EFFORT * pow_function(POW_NONCE, SEED) <= MAX_RESULT where MAX_RESULT is the highest possible output from the pow_function. In the case of Example A, that would be: 165 * 6317 = 1042305 <= 1048576 > Similar to how our cell scheduler works, the onion service subsystem will > poll the priority queue every 100ms tick and process the first 20 cells from > the priority queue (if they exist). The service will perform the rendezvous > and the rest of the onion service protocol as normal. I suggest to use a different selection method rather than always taking the first 20 requests. Selecting cells from the front of the queue actually minimizes the effort that an attacker needs to expend to completely block access to the service. The attacker can always submit the minimum effort required to occupy the first 20 slots in each 100 ms window. This can be done by submitting just 200 requests per second and observing how many circuits are successfully open and adjusting the effort until no other request can go through. This is the "Total overwhelm strategy" described in § 5.1.1. See the following examples to show how selecting the first N cells from the queue is unfair. I will use N = 4 for clarity. E denotes some value of effort. == Example B: ATTACKER LEGITIMATE CLIENTS ___________ _________ ____________ ______________ / \ / \ +--------+--------+--------+--------+----+----+----+----+----+----+----+----+ | 2E | 2E | 2E | 2E | E | E | E | E | E | E | E | E | +--------+--------+--------+--------+----+----+----+----+----+----+----+----+ ^ ^ ^ ^ selected selected selected selected Here both the attacker and the legitimate clients have expended a combined effort of 8E. All of the attacker's cells get selected, while none of the other clients get through. Instead, I suggest to use Stochastic universal sampling [REF_SUS], where the probability that a cell gets selected is proportional to its effort. == Example C: Here the total effort in the queue is 16E, so we first select a random value in the interval [0, 4E) and then select 4 evenly spaced cells: ATTACKER LEGITIMATE CLIENTS ___________ _________ ____________ ______________ / \ / \ +--------+--------+--------+--------+----+----+----+----+----+----+----+----+ | 2E | 2E | 2E | 2E | E | E | E | E | E | E | E | E | +--------+--------+--------+--------+----+----+----+----+----+----+----+----+ ^ ^ ^ ^ selected selected selected selected In this case, 2 cells are selected from each group, which is fairer considering that each group expended the same effort. == Example D: Now if the attacker wanted to block access to all legitimate clients, they would need to at least quadruple their total PoW effort (and there would still be a chance that a legitimate client gets selected from the end of the queue): ATTACKER LEGITIMATE CLIENTS __________________________ ______________________ ______________ / \ / \ +--------------+--------------+---------------+--------------+-+-+-+-+-+-+-+-+ | 8E | 8E | 8E | 8E |E|E|E|E|E|E|E|E| +--------------+--------------+--------------+---------------+-+-+-+-+-+-+-+-+ ^ ^ ^ ^ selected selected selected selected Note: With linear effort, the original selection method becomes even worse because the attacker can occupy the front of the queue with an effort of just E+1 per cell rather than 2E. In some cases, Stochastic universal sampling can select a single element multiple times, which is not a problem for genetic algorithms, but we want to avoid it. I suggest to restart the selection algorithm from the beginning of the next cell in those cases and shorten the sampling interval according to the remaining portion of the queue. See the following example. == Example E: +---------------+-------------+--------------+------------+-+-+-+-+-+-+-+-+-+-+ | 16E | 8E | 6E |E|E|E|E|E|E|E|E|E|E| +---------------+-------------+--------------+------------+-+-+-+-+-+-+-+-+-+-+ ^ ^ ^ ^ selected selected selected selected > In particular, the service starts with a default suggested-effort value of 15. > > Everytime the service handles an introduction request from the priority queue > in [HANDLE_QUEUE], the service compares the request's effort to the current > suggested-effort value. If the new request's effort is lower than the > suggested-effort, set the suggested-effort equal to the effort of the new > request. > > Everytime the service trims the priority queue in [HANDLE_QUEUE], the service > compares the request at the trim point against the current suggested-effort > value. If the trimmed request's effort is higher than the suggested-effort, > set the suggested-effort equal to the effort of the new request. I think the default suggested-effort is a bit too high. Assuming each attempt takes around 1 ms to calculate, the client would need, on average, 2^15 ms = 33 seconds to find a solution using 1 CPU core. I suggest to specify a minimum effort instead. Requests with effort < MIN_EFFORT and requests without the PROOF_OF_WORK extension would be treated as having effort = 1 for the purposes of the sampling algorithm. Secondly, the proposed method of calculating the suggested-effort is susceptible to gaming by attackers. Since the service can only update the value in the directory once every 'hs-pow-desc-upload-rate-limit' seconds, they could stop the attack for a little while just before the directory gets updated, which would cause the suggested-effort value to be too low despite an ongoing attack. I suggest to take the median effort of the selected cells during each 100 ms window. For the examples above that would be: Example C: 1.5E Example D: 8E Example E: 7E Then I would take the median of these values over the directory update period. > > 5. Attacker strategies > Additional attacks: 5.1.2 PoW Spam Attack The attacker may try to spam many requests with random values of POW_NONCE, requiring the service to waste cycles verifying the invalid proofs. No such request would make it into the Intro Queue, but it may still be a viable DoS strategy depending on proof verification time and the number of intro requests that can be practically delivered through the network. 5.1.3 Precomputed PoW attack The attacker may precompute many valid PoW nonces and submit them all at once before the current seed expires, overwhelming the service temporarily even using a single computer. > 4.2. Seed expiration issues > > As mentioned in [DESC_POW], the expiration timestamp on the PoW seed can > cause issues with clock skewed clients. Furthermore, even not clock skewed > clients can encounter TOCTOU-style race conditions here. > > The client descriptor refetch logic of [CLIENT_TIMEOUT] should take care of > such seed-expiration issues, since the client will refetch the descriptor. > I suggest to use 2 concurrent seeds, i.e. to accept PoW both from the current and the last seed epoch. We use this approach in Monero. It would however require adding the seed value into the proof of work extension field and also double the memory requirements for verification with RandomX. > The proposal suggests argon2, and Mike has been looking at Randomx. However, > after further consideration and speaking with some people (props to Alex > Biryukov), it seems like those two functions are not well fitted for this > purpose, since they are memory-hard both for the client and the service. And > since we are trying to minimize the verification overhead, so that the > service can do hundreds of verifications per second, they don't seem like > good fits. Asymmetric function like Equihash, Cuckoo cycle [REF_CUCKOO] and MTP have the advantage of being very fast to verify, but they run much faster on GPUs and specialized hardware, so I don't think they are particularly suitable for this purpose. When we designed RandomX to be used as the PoW algorithm by Monero, we selected the parameters very conservatively to maximize the ASIC resistance of the algorithm. That's because it is very difficult to change the PoW algorithm once it's deployed. TOR is not limited by this, so we can be much more aggressive when configuring RandomX. I suggest to use a configuration that gives the fastest possible verification time without completely breaking the algorithm. In particular, the following parameters should be set differently from Monero: RANDOMX_ARGON_SALT = "RandomX-TOR-v1" The unique RandomX salt means we do not need to use a separate salt as PoW input as specified in § 3.2. RANDOMX_ARGON_ITERATIONS = 1 RANDOMX_CACHE_ACCESSES = 4 RANDOMX_DATASET_BASE_SIZE = 1073741824 RANDOMX_DATASET_EXTRA_SIZE = 16777216 These 4 changes reduce the RandomX Dataset size to ~1 GiB, which allows the number of iteration to be reduced from 8 to 4. The combined effect of this is that Dataset initialization becomes 4 times faster, which is needed due to more frequent updates of the seed (Monero updates once per ~3 days). RANDOMX_PROGRAM_COUNT = 2 RANDOMX_SCRATCHPAD_L3 = 1048576 Additionally, reducing the number of programs from 8 to 2 makes the hash calculation about 4 times faster, while still providing resistance against program filtering strategies (see [REF_RANDOMX_PROGRAMS]). Since there are 4 times fewer writes, we also have to reduce the scratchpad size. I suggest to use a 1 MiB scratchpad size as a compromise between scratchpad write density and memory hardness. Most x86 CPUs will perform roughly the same with a 512 KiB and 1024 KiB scratchpad, while the larger size provides higher resistance against specialized hardware, at the cost of possible time-memory tradeoffs (see [REF_RANDOMX_TMTO] for details). Lastly, we reduce the output of RandomX to just 8 bytes: RANDOMX_HASH_SIZE = 8 64-bit preimage security is more than sufficient for proof-of-work and it allows the result to be treated as a little-endian encoded unsigned integer for easy effort calculation. RandomX would be used as follows: The service will select a 32-byte POW_SEED and initialize the cache and the dataset: randomx_init_cache(myCache, POW_SEED, 32); randomx_init_dataset(myDataset, myCache, 0, randomx_dataset_item_count()); randomx_vm *myMachine = randomx_create_vm(flags, NULL, myDataset); Then in order to validate a PoW token, we could use something like this: int validateProof(uint32_t pow_effort, void* pow_nonce) { uint64_t result; randomx_calculate_hash(myMachine, pow_nonce, 16, &result); if (mulh(pow_effort, result) == 0) { return 1; } return 0; } I suggest to set MIN_EFFORT = 10000, which takes about 1 second on my laptop. Requests with pow_effort < MIN_EFFORT would not be validated. I have collected some performance figures for the fast mode with the above RandomX configuration (~1 GiB of memory is required): H/s = hashes per second == CPUs: Intel Core i3-3220 - 1 thread 700 H/s - 3 threads 1400 H/s Intel Xeon (dual core VPS, Sandy Bridge, unknown model) - 1 thread 2000 H/s - 2 threads 4000 H/s Intel Core i5-2500K (stock) - 1 thread 2200 H/s - 4 threads 8200 H/s Intel Core i7-8550U (laptop) - 1 thread 2700 H/s - 8 threads 10000 H/s Intel Core i7-9850H (laptop) - 1 thread 3100 H/s - 12 threads 16000 H/s AMD Ryzen 1700 @ 3300MHz - 1 thread 2300 H/s - 16 threads 23800 H/s AMD Ryzen 3700X @ 3300MHz - 1 thread 2500 H/s - 16 threads 27500 H/s AMD Epyc 7702P - 1 thread 2100 H/s - 128 threads 139000 H/s == GPUs: NVIDIA GeForce GTX 1660 Ti (credits to SChernykh, see [REF_RANDOMX_CUDA]) - 3072 intensity 2600 H/s According to the above results, the time to verify a single hash is around 400-500 μs. A mid-range GPU has a similar performance as a single CPU core. Most CPUs made since 2011 have similar per-core performance except of low-end CPUs without hardware AES support. References: [REF_BIRTHDAY]: https://en.wikipedia.org/wiki/Birthday_attack#Mathematics [REF_SUS]: https://en.wikipedia.org/wiki/Stochastic_universal_sampling [REF_CUCKOO]: https://github.com/tromp/cuckoo [REF_RANDOMX_PROGRAMS]: https://github.com/tevador/RandomX/blob/master/doc/design.md#12-the-easy-pr… [REF_RANDOMX_TMTO]: https://github.com/tevador/RandomX/issues/65 [REF_RANDOMX_CUDA]: https://github.com/SChernykh/RandomX_CUDA

7 21

Safe Alternative Uses of Onion Service Keys
by Matthew Finkel 12 Aug '20

12 Aug '20

Hello everyone, Onion service version two (v2) key pairs were used for more purposes than simply facilitating the establishment of rendezvous circuits, in particular third-party applications used this key in numerous ways. Similarly, version three (v3) onion service keys are being re-used in similar (and new) ways. However, the (re)use of v3 long-term keys is not obviously safe in all situations. How should (and should not) these keys be used, such that the security of the onion service is preserved? I'll briefly summarize v3 keys (for those who don't want to read through the spec), and then I'll sketch two alternative use cases along with a straw man proposal. The long-term v3 identity keys are ed25519 keys (see [0] for a nice write up of ed25519, in general). The generated key material is serialized externally (outside of tor) in two (similar) formats: written on-disk [2] and from the control port [3]. The secret (private) key is the (64-byte) SHA-512 digest of a 32-byte seed value. The 64-byte digest is the "expanded form". The seed is thrown away. The public key is obtained by taking the first 32 bytes of that SHA-512 hash, clamping some of the bits, and performing a scalar multiplication with the (fixed) base point. This key is the onion service long-term identity key. Creating and verifying a signature using the above keys (respectively) follow standard EdDSA. However, (at this time) Tor does not use these keys directly for signing messages. All messages are signed using an ephemeral ed25519 key, and that ephemeral key is certified by a blinded ed25519 key derived from the long-term key pair. The blinded keys are computed using a specified blinding scheme [4]. All messages signed using the ephemeral key are prefixed with a context-specific string. In summary, long-term keys are used for deriving a short-term blinded key, and that short-term blinded key is used for certifying an ephemeral signing key. For computing the blinded key, the first 32 bytes of the long-term secret key (LH) are multiplied with a blinding factor (h*a mod l), see the specification for the value of **h** [4]. This becomes LH' (LH-prime). The second 32 bytes of the secret key (RH) are concatenated with a string prefix and then the SHA3-256 digest is computed of the concatenated string. The first 32 bytes of the resulting digest become RH' (RH-prime). LH' and RH' are used as regular ed25519 secret keys for signing and verifying messages following EdDSA. Tor's EdDSA signature is "R|S", R concatenated with S (the message is not included in the signature). The safest usage of the long-term keys for alternative purposes I see appears to be by deriving a (fixed/deterministic) blinded key pair using the same scheme that Tor uses, and signing/verification simply follow the same process as Tor, except the derived keys need not rotate periodically (is this true?). The derived key should be used for certifying a freshly generated ed25519 key, which is used in the application protocol. For example, if I want to use a key for code signing such that it is bound to my onion service key, then I could derive a certifying key by following Tor's derivation scheme, by substituting: BLIND_STRING = "Derive temporary signing key" | INT_1(0) N = "key-blind" | INT_8(period-number) | INT_8(period_length) with BLIND_STRING = "Derive code signing key" | INT_1(0) N = "code-sigining-key-blind" | "v0" | "YYYY-MM-DD" | INT_8(validity_period) for computing the blinding factor. Where "v0" is a version tag. YYYY-MM-DD is an arbitrary date, but it can be used for rotating signing keys in the future. INT_8(validity_period) may be used for specifying the number of days after YYYY-MM-DD at which time previously unverified signatures using this key should be considered invalid (where INT_8(0) could indicate "never expire"). And substituting RH_BLIND_STRING = "Derive temporary signing key hash input" with RH_BLIND_STRING = "Derive code signing key hash input" for computing RH'. A signature must include "v0" and the values used in "YYYY-MM-DD" and INT_8(validity_period), such that the client can derive the correct blinded public key for verification when starting from the long-term identity key. The signature should be over a certification of an independently generated ed25519 key pair. This new key pair (along with the certification) can be used for providing message integrity within the application's protocol. If, instead, the derived key is used directly for signing, and the application needs the keys online for signing messages, then this risks the security of the long-term key, as well. The blinding scheme allows for (partially) recovering the long-term secret key from the derived secret key. Another example use case comes from Jeremy Rand where the onion service key is used in a root CA certificate, and a leaf certificate (signed by the CA cert) is used by the application. Following from the previous example, (most likely) the CA certificate should not be signed directly using the onion service's long-term secret key. However, a derived key could be used in the CA certificate and the leaf cert could contain an ephemeral key (in exactly the same way that tor certifies ephemeral keys using the derived blinded signing key). This idea appears to be a concrete design of how the above (abstract) key certification could be implemented, and it could be a format that tor natively supports. The above process seems like a lot to ask from application developers. Can we make it easier for them? Open questions: 1) Going back to the long-term secret key, can LH and RH be used directly in EdDSA without reducing the security and unlinkability of the blinded keys? 2) Should other use cases of the long-term keys only derive distinct (blinded) keys, instead of using the long-term keys directly? 3) If other use cases should only use derived keys, then is there an alternative derivation scheme for when unlinkability between derived keys is not needed (without reducing the security properties of the onion service blinded keys), and is allowing linkability useful/worthwhile? 4) Is the above example derivation scheme safe if different applications tweak the above prefix strings in similar ways? 5) Should Tor simply derive one blinded key that can be used by all alternative applications? Is that safe? I'd like to thank Nick Mathewson and David Goulet for their comments on an earlier version of this mail. Thanks, Matt [0] https://blog.mozilla.org/warner/2011/11/29/ed25519-keys/ [1] https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt#n2267 [2] A tagged value: "== ed25519v1-secret: type0 =="\0\0\0 | (64-byte SHA-512 hash of the seed) [3] https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n1746 [4] https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt#n2267

4 6

CAPTCHA Monitoring Project Updates #3
by Barkin Simsek 25 Jul '20

25 Jul '20

Hi everyone, I have a few more updates since my last email (you might want to check the project's wiki page [0] first if you didn't see my previous emails): -- I have the onion service [1] of the dashboard [2] up and running. -- I was using an SQLite database and migrated it to PostgreSQL. This migration was needed to scale up the whole system. At the moment, there are 50000+ measurements in the database, and access time was very long with SQLite. -- Now there is a properly documented API [3] available for everyone's use. You can perform queries through the API if you prefer to access the raw data, instead of the visualizations on the dashboard. -- The Dockerfile [4] is updated and working again. -- I decided to give priority to the API and Dockerfile for the "reproducibility" of the results and graphs in the dashboard. Before the API, the graphs were using hardcoded SQL queries, which required people to dig in the source code to find out possible mistakes. It was also impossible for people to execute these SQL queries and see the results themselves. API allows me to put a link to the exact API calls to anything I place on the dashboard so that people can double-check with the raw data if they want. -- I'm working on adding Brave Browser's "Private Window with Tor" to the CAPTCHA Monitor. I'm very close to finishing it and should be ready at the end of this weekend. I expect to see a high amount of CAPTCHA since it is using Tor without Tor Browser's user-agent (and thus fingerprint). -- I discussed a possible integration with the metrics website on the metrics team meeting. I initially thought about placing CAPTCHA rate graphs under each relay in the "Relay Search" section. If you have any suggestions, you are welcomed to mention them in ticket tpo/metrics/website/40002 [5]. -- Finally, you can take a look at my blog posts on DIAL's blog [6] to see more detailed explanations, but I already summarized the blog posts in this email. Best, Barkin (woswos) [0] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/home [1] http://captchaufjq5m2i73up537pldaxnbp6rzcbdrzc7y5rlwtx3mwigznad.onion/ [2] https://dashboard.captcha.wtf/ [3] https://api.captcha.wtf/ [4] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/tree/master/docker [5] https://gitlab.torproject.org/tpo/metrics/website/-/issues/40002 [6] https://hub.osc.dial.community/t/tor-project-cloudflare-captcha-monitoring/…

1 0

Distributing Tor developer keys via Fedora packages
by Andrew Clausen 20 Jul '20

20 Jul '20

Hi everyone, I propose distributing the Tor developer keys inside the Fedora package distribution-gpg-keys.[1] This would give most Linux users a trustworthy chain of signatures from their own distributor (e.g. CentOS or Fedora) to Tor project downloads. I am happy to take care of this, although I am also happy if somebody who is more involved with Tor than me takes this on. I wrote a shell script (attached) to acquire and organise the keys based on https://2019.www.torproject.org/include/keys.txt. My script would install the following keys under /usr/share/distribution-gpg-keys/tor: Arm_releases/Damian_Johnson.gpg Tails_live_system_releases/The_Tails_team.gpg TorBirdy_releases/Sukhbir_Singh.gpg Tor_Browser_releases/Arthur_Edelstein.gpg Tor_Browser_releases/Georg_Koppen.gpg Tor_Browser_releases/Mike_Perry.gpg Tor_Browser_releases/Nicolas_Vigier.gpg Tor_Browser_releases/The_Tor_Browser_Developers.gpg Tor_source_tarballs/Nick_Mathewson.gpg Tor_source_tarballs/Roger_Dingledine.gpg Torsocks_releases/David_Goulet.gpg deb.torproject.org_repositories_and_archives/Tor_Project_Archive.gpg older_Tor_tarballs/Nick_Mathewson.gpg other/Peter_Palfrader.gpg Unless someone else volunteers (please do!), I will set up a weekly job to run the script and alert me to any changes. Can anyone see any potential problems with this plan? The most obvious question is: how do I know that I am distributing unadulterated keys? I think the answer is that I don't! But any attack would have to affect a large group of people, and would be detected quickly as long as many people are looking at the distribution-gpg-keys package. If this solution is unsatisfactory, then perhaps someone who is more involved with the Tor developers -- and hence able to directly check the keys -- ought to take this on. [1] See https://github.com/xsuchy/distribution-gpg-keys and https://rpmfind.net/linux/RPM/fedora/updates/32/x86_64/Packages/d/distribut…

2 2

Proposal 325: Packed relay cells: saving space on small commands
by Nick Mathewson 10 Jul '20

10 Jul '20

``` Filename: 325-packed-relay-cells.md Title: Packed relay cells: saving space on small commands Author: Nick Mathewson Created: 10 July 2020 Status: Draft ``` # Introduction In proposal 319 I suggested a way to fragment long commands across multiple RELAY cells. In this proposal, I suggest a new format for RELAY cells that can be used to pack multiple relay commands into a single cell. Why would we want to do this? As we move towards improved congestion-control and flow-control algorithms, we might not want to use an entire 498-byte relay payload just to send a one-byte flow-control message. We already have some cases where we'd benefit from this feature. For example, when we send SENDME messages, END cells, or BEGIN_DIR cells, most of the cell body is wasted with padding. As a side benefit, packing cells in this way may make the job of the traffic analyst a little more tricky, as cell contents become less predictable. # The basic design Let's use the term "Relay Message" to mean the kind of thing that a relay cell used to hold. Thus, this proposal is about packing multiple "Relay Messages" in to a cell. I'll use "Packed relay cell" to mean a relay cell in this new format, that supports multiple messages. I'll use "client" to mean the initiator of a circuit, and "relay" to refer to the parties through who a circuit is created. Note that each "relay" (as used here) may be the "client" on circuits of its own. When a relay supports relay message packing, it advertises the fact using a new Relay protocol version. Clients must opt-in to using this protocol version (see XXX below) before they can send any packed relay cells, and before the relay will send them any packed relay cells. When packed cells are in use, multiple cell messages can be concatenated in a single relay cell. Only some relay commands are supported for relay cell packing, listed here: - SENDME - DROP - DATA - BEGIN - BEGIN_DIR - END - CONNECTED - PADDING_NEGOTIATE - PADDING_NEGOTIATED If any relay message with a relay command _not_ listed above appears in a packed relay cell with another relay message, then the receiving party MUST tear down the circuit. (Note that relay cell fragments (proposal 319) are not supported for packing.) The command byte "0" is now used to explicitly indicate "end of cell". If the byte "0" appears after a relay message, the rest of the cell MUST be ignored. When generating RELAY cells, implementations SHOULD (as they do today) fill in the unused bytes with four 0-valued bytes, followed by a sequence of random bytes up to the end of the cell. If there are fewer than 4 unused bytes at the end of the cell, those unused bytes should all be filled with 0-valued bytes. # Negotiation and migration After receiving a packed relay cell, the relay know that the client supports this proposal: Relays SHOULD send packed relay cells on any circuit on which they have received a packed relay cell. Relays MUST NOT send packed relay cells otherwise. Clients, in turn, MAY send packed relay cells to any relay whose "Relay" subprotocol version indicates that it supports this protocol. To avoid fingerprinting, this client behavior should controlled with a tristate (1/0/auto) torrc configuration value, with the default set to use a consensus parameter. The parameter is: "relay-cell-packing" Boolean: if 1, clients should send packed relay cells. (Min: 0, Max 1, Default: 0) To handle migration, first the parameter should be set to 0 and the configuration setting should be "auto". To test the feature, individual clients can set the tristate to "1". Once enough clients have support for the parameter, the parameter can be set to 1. # A new relay message format (This section is optional and should be considered separately; we may decide it is too complex.) Currently, every relay message uses 5 bytes of header to hold a relay command, a length field, and a stream ID. This is wasteful: the stream ID is often redundant, and the top 7 bits of the length field are always zero. I propose a new relay message format, described here (with `ux` denoting an x-bit bitfield). This format is 2 bytes or 4 bytes, depending on its first bit. struct relay_header { u1 stream_id_included; // Is the stream_id included? u6 relay_command; // as before u9 relay_data_len; // as before u8 optional_stream_id[]; // 0 bytes or two bytes. } Alternatively, you can view the first three fields as a 16-bit value, computed as: (stream_id_included<<15) | (relay_command << 9) | (relay_data_len). If the optional_stream_id field is not present, then the default value for the stream_id is computed as follows. We use stream_id 0 for any command that doesn't take a stream ID. For commands that _do_ take a steam_id, we use whichever nonzero stream_id appeared last in this cell. This format limits the space of possible relay commands. That's probably okay: after 20 years of Tor development, we have defined 25 relay command values. But in case 2^6==64 commands will not be enough, we reserve command values 48 through 63 for future formats that need more command bits.

3 2

CAPTCHA Monitoring Project Updates & Findings
by Barkin Simsek 10 Jul '20

10 Jul '20

Hi everyone, I made progress on the Cloudflare CAPTCHA Monitoring project since my last email, and I wanted to share some of the updates & findings. This year Tor Project is participating in GSoC under the DIAL umbrella, and I have already been posting updates to the DIAL blog [1] weekly. I started mirroring these updates [2] to my project's wiki page, and I will be posting more frequent updates here. a) Updates: Firstly, I moved the wiki page that explains the project, the code base, and the issue tracker to Tor Project's GitLab. They are all in the same GitLab project. You can find detailed information about the project on the wiki page [3] and leave comments & suggestions within that repository by creating issues. Secondly, I got a fully functioning system up and running. The system fetches various URLs with Tor Browser & Firefox over Tor and checks for CAPTCHAs. The system also checks if any third-party code was injected by comparing the hash of the received page with an expected hash value. It repeats these experiments using different exit relays and records results. You can view the results on the dashboard [4] I created. I'm looking for more URLs to track for CAPTCHAs. Feel free to share the websites you frequently visit and get CAPTCHAs, so that I can track these websites with this tool as well. I want to experiment with all types of CAPTCHAs, and these URLs don't have to be fronted by Cloudflare. b) Findings: So far, I have observed that using the Tor Browser Bundle out of the box without changing its configurations doesn't lead to a high CAPTCHA rate on Cloudflare fronted websites (assuming the website owners don't explicitly block exit relays [5]). That said, modifying the user-agent or any other modifications that deviate your browser's fingerprint from a typical Tor Browser user, significantly increases the chance of getting CAPTCHAs. For example, using the regular Firefox over Tor resulted in getting CAPTCHAs in ~90% of the measurements. I believe Cloudflare is very aggressive against the "Firefox over Tor" users because many people, unfortunately, use Chromium/Firefox + Selenium + Tor to scrape web pages and bypass IP-based rate limits. That's why I'm interested in hearing about your specific browser/Tor configurations to test them with the CAPTCHA Monitor. Not everyone is affected in the same way because of these differences in the way we use Tor, but we can understand which differences affect the CAPTCHA rate more than others by experimenting. Additionally, I observed that the TLS fingerprint has a significant role in whether someone gets a CAPTCHA or not. As a part of the project, I decided to capture the HTTP headers during measurements to understand how they affect the CAPTCHA rates. Initially, I was using a Python library called seleniumwire to capture the HTTP headers by intercepting the traffic between the Tor Browser and Tor. By doing this, I got a very high CAPTCHA rate, like 98% of the time. seleniumwire forwards the traffic transparently, but it has a different TLS fingerprint than Tor Browser. I figured out that the difference in the TLS fingerprints was triggering the MITM detection on the Cloudflare side, thus, resulting in very high CAPTCHA rates. Interestingly, I tried using the exact same Tor Browser & seleniumwire setup, but without Tor and, practically, I didn't get any CAPTCHAs. I believe the MITM detection is more aggressive if the traffic is coming through an exit relay. So, I stopped using seleniumwire to capture headers because it didn't reflect what a real human Tor Browser user is usually experiencing. Please feel free to use the sample code [6] that I used to combine seleniumwire and Tor, if you are interested in doing further experimenting on this. c) Next: I will work on collecting more metrics by testing more configurations and websites. I will create a "Relay Search" section on the dashboard, where CAPTCHA statistics for the relays (exit relays for now) will be available. I will also work on using the collected data to predict the probability of getting CAPTCHAs with a given exit relay and configuration/setup. Best, Barkin [1] https://hub.osc.dial.community/t/tor-project-cloudflare-captcha-monitoring/… [2] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/Updates [3] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/home [4] https://dashboard.captcha.wtf/ [5] Cloudflare has a setting to block all traffic originating from the Tor network, but that setting is not "turned on" by default [6] https://gist.github.com/woswos/38b921f0b82de009c12c6494db3f50c5

1 0

Gitolite to Gitlab Sync Change
by Alexander Færøy 09 Jul '20

09 Jul '20

Hello folks! In an attempt to solve Gitlab#41[1] the hooks that is executed when you push to git.torproject.org to synchronize to Gitlab was modified to avoid pruning references in Gitlab that was missing in the Gitolite repository. This /should/ have the following implications: - We no longer delete the "refs/merge-requests/*" namespace each time someone pushes to your repository on git.torproject.org. This should allow people to use the `git mr` alias that can be found online which should make local code-reviews easier and also allow you to handle manual CLI merging more easily. - We no longer delete branches automatically on Gitlab: if a branch is deleted on git.torproject.org. This have to be deleted manually now on Gitlab. Please report any issues you that you might discover on the ticket. Thanks to Hiro for getting this running! All the best, Alex. [1]: https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/41 -- Alexander Færøy

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev July 2020