- tor-dev - lists.torproject.org

Can "ExcludeNodes" be used multiple times in torrc?
by nusenu 14 Aug '20

14 Aug '20

Hi, it is not clear to me whether ExcludeNodes and other similar options (ExcludeExitNodes) can be used multiple times in a torrc file. Are all of the lines merged like multiple "MyFamily" lines or how does it behave? thanks, nusenu -- https://mastodon.social/@nusenu

3 4

Safe Alternative Uses of Onion Service Keys
by Matthew Finkel 12 Aug '20

12 Aug '20

Hello everyone, Onion service version two (v2) key pairs were used for more purposes than simply facilitating the establishment of rendezvous circuits, in particular third-party applications used this key in numerous ways. Similarly, version three (v3) onion service keys are being re-used in similar (and new) ways. However, the (re)use of v3 long-term keys is not obviously safe in all situations. How should (and should not) these keys be used, such that the security of the onion service is preserved? I'll briefly summarize v3 keys (for those who don't want to read through the spec), and then I'll sketch two alternative use cases along with a straw man proposal. The long-term v3 identity keys are ed25519 keys (see [0] for a nice write up of ed25519, in general). The generated key material is serialized externally (outside of tor) in two (similar) formats: written on-disk [2] and from the control port [3]. The secret (private) key is the (64-byte) SHA-512 digest of a 32-byte seed value. The 64-byte digest is the "expanded form". The seed is thrown away. The public key is obtained by taking the first 32 bytes of that SHA-512 hash, clamping some of the bits, and performing a scalar multiplication with the (fixed) base point. This key is the onion service long-term identity key. Creating and verifying a signature using the above keys (respectively) follow standard EdDSA. However, (at this time) Tor does not use these keys directly for signing messages. All messages are signed using an ephemeral ed25519 key, and that ephemeral key is certified by a blinded ed25519 key derived from the long-term key pair. The blinded keys are computed using a specified blinding scheme [4]. All messages signed using the ephemeral key are prefixed with a context-specific string. In summary, long-term keys are used for deriving a short-term blinded key, and that short-term blinded key is used for certifying an ephemeral signing key. For computing the blinded key, the first 32 bytes of the long-term secret key (LH) are multiplied with a blinding factor (h*a mod l), see the specification for the value of **h** [4]. This becomes LH' (LH-prime). The second 32 bytes of the secret key (RH) are concatenated with a string prefix and then the SHA3-256 digest is computed of the concatenated string. The first 32 bytes of the resulting digest become RH' (RH-prime). LH' and RH' are used as regular ed25519 secret keys for signing and verifying messages following EdDSA. Tor's EdDSA signature is "R|S", R concatenated with S (the message is not included in the signature). The safest usage of the long-term keys for alternative purposes I see appears to be by deriving a (fixed/deterministic) blinded key pair using the same scheme that Tor uses, and signing/verification simply follow the same process as Tor, except the derived keys need not rotate periodically (is this true?). The derived key should be used for certifying a freshly generated ed25519 key, which is used in the application protocol. For example, if I want to use a key for code signing such that it is bound to my onion service key, then I could derive a certifying key by following Tor's derivation scheme, by substituting: BLIND_STRING = "Derive temporary signing key" | INT_1(0) N = "key-blind" | INT_8(period-number) | INT_8(period_length) with BLIND_STRING = "Derive code signing key" | INT_1(0) N = "code-sigining-key-blind" | "v0" | "YYYY-MM-DD" | INT_8(validity_period) for computing the blinding factor. Where "v0" is a version tag. YYYY-MM-DD is an arbitrary date, but it can be used for rotating signing keys in the future. INT_8(validity_period) may be used for specifying the number of days after YYYY-MM-DD at which time previously unverified signatures using this key should be considered invalid (where INT_8(0) could indicate "never expire"). And substituting RH_BLIND_STRING = "Derive temporary signing key hash input" with RH_BLIND_STRING = "Derive code signing key hash input" for computing RH'. A signature must include "v0" and the values used in "YYYY-MM-DD" and INT_8(validity_period), such that the client can derive the correct blinded public key for verification when starting from the long-term identity key. The signature should be over a certification of an independently generated ed25519 key pair. This new key pair (along with the certification) can be used for providing message integrity within the application's protocol. If, instead, the derived key is used directly for signing, and the application needs the keys online for signing messages, then this risks the security of the long-term key, as well. The blinding scheme allows for (partially) recovering the long-term secret key from the derived secret key. Another example use case comes from Jeremy Rand where the onion service key is used in a root CA certificate, and a leaf certificate (signed by the CA cert) is used by the application. Following from the previous example, (most likely) the CA certificate should not be signed directly using the onion service's long-term secret key. However, a derived key could be used in the CA certificate and the leaf cert could contain an ephemeral key (in exactly the same way that tor certifies ephemeral keys using the derived blinded signing key). This idea appears to be a concrete design of how the above (abstract) key certification could be implemented, and it could be a format that tor natively supports. The above process seems like a lot to ask from application developers. Can we make it easier for them? Open questions: 1) Going back to the long-term secret key, can LH and RH be used directly in EdDSA without reducing the security and unlinkability of the blinded keys? 2) Should other use cases of the long-term keys only derive distinct (blinded) keys, instead of using the long-term keys directly? 3) If other use cases should only use derived keys, then is there an alternative derivation scheme for when unlinkability between derived keys is not needed (without reducing the security properties of the onion service blinded keys), and is allowing linkability useful/worthwhile? 4) Is the above example derivation scheme safe if different applications tweak the above prefix strings in similar ways? 5) Should Tor simply derive one blinded key that can be used by all alternative applications? Is that safe? I'd like to thank Nick Mathewson and David Goulet for their comments on an earlier version of this mail. Thanks, Matt [0] https://blog.mozilla.org/warner/2011/11/29/ed25519-keys/ [1] https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt#n2267 [2] A tagged value: "== ed25519v1-secret: type0 =="\0\0\0 | (64-byte SHA-512 hash of the seed) [3] https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n1746 [4] https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt#n2267

4 6

Re: [tor-dev] CAPTCHA Monitoring Project's Dashboard
by Barkin Simsek 11 Aug '20

11 Aug '20

I messed up with replying to my own thread since I was receiving emails in a digest. I'm sorry for creating a duplicate thread. This email intends to reply to Georg's reply from the original thread. > Thanks for that wiki page. That's been really helpful. One thing I kept > wondering while reading through the different graph descriptions with > the default graph style in mind: [..snip..] > So, what's the tick shown per day then? One consensus somehow picked or > data for all consensuses for that day? Or...? I initially wanted to have per-day data points on the graphs, but later I realized that it complicates the calculations a lot (and increases the probability of making a mistake) since some relays show up & disappear (and their exit probabilities change) throughout the day. So, I decided to calculate the CAPTCHA rates per each hourly consensus to simplify calculations. There will be 24 data points (representing each consensus) per day and 30 days of history in a single graph. In total, there will be 24*30=720 data points in a single graph. Since having 720 labels similar to "2020-07-18 14:00" in the x-axis would be too cluttered, I decided to put ticks for each whole day and label them only rather than labeling every single data point. I hope this explanation makes it more clear. I will update the wiki and example graph on the wiki to reflect what I just explained. > How do you plan to show those graphs on the dashboard? Are they all > shown in some order? Grouped by sections (maybe along those on the > wiki)? Grouped by "importance"? I plan to have separate pages for each section mentioned in the wiki. In addition to that, I plan to have a "highlights" page to show a quick summary of all graphs. People visiting the dashboard will first see the highlights and later check more detailed graphs if they are interested in seeing them. I agree that there is a lot of information for a visitor to digest, and making this process as effortless as possible is one of my top priorities. > Somewhat related to that, I think we can try reducing the number of > graphs, at least those which are greatly related. [..snip..] > One thing that could be useful here > is having a switch/buttons next to the graph giving the user the option > to see "by exit relay age" for all CAPTCHAs, or for Cloudflare ones, or > for Akamai ones or for... depending on the switch/button the users > clicked on. The graph would update accordingly once clicked, showing the > user choice. The default could be for all CAPTCHAs or essentially > whatever we think is more important for us. That way you have related > things grouped together AND you have less graphs shown per default on > the dashboard, too, which might help not getting confused. A few other people suggested using this approach as well, and I like it! I will place buttons to achieve what you have suggested. On top of this, there will be checkboxes next to graphs for users to make decisions on what to include in the graphs, and the graphs will be updated accordingly. The default values will be configured in a way to reflect what we care about most. For example, both "Tor Browser" and "Firefox over Tor" measurements could be used to calculate the CAPTCHA rates, but only "Tor Browser" checkbox will be checked by default, and the user will have the ability to include "Firefox over Tor" and others into calculations. This gives the user the ability to compare different options easily. That said, I wonder if having too many options is another problem for users. It might lead to confusion, and determining the right balance is still an open issue. Thank you for your feedback! Best, Barkin

1 0

CAPTCHA Monitoring Project's Dashboard
by Barkin Simsek 11 Aug '20

11 Aug '20

Hi everyone, I created a wiki page [1] for "describing" the graphs that will be used to visualize the CAPTCHA Monitor dataset [2]. There are already a few graphs on the dashboard [3] and they will be replaced with the new ones described on the wiki page. I'm interested in hearing your feedback on the graph descriptions [1] and suggestions for adding new graphs. I want to understand if the graphs I proposed make sense and useful for anyone. The descriptions have a similar style to the one used in the metrics website's "Reproducible Metrics" page [4]. Thank you in advance! Best, Barkin (woswos) [1] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/Dashboard-Grap… [2] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/home [3] https://dashboard.captcha.wtf [4] https://metrics.torproject.org/reproducible-metrics.html

2 1

CentOS 8 TOR Repo
by Ladar Levison 07 Aug '20

07 Aug '20

It appears that installing TOR via the CentOS 8 DNF repo isn't working. The pre-flight configuration check fails because the repo package was built using zstd 1.4.4, and CentOS 8 is currently providing zstd 1.4.2. For any that cares to try, the repo is at: https://rpm.torproject.org/centos/8/ L~

2 2

Connection padding set to 1 vs auto
by procmem＠riseup.net 06 Aug '20

06 Aug '20

Hi. I was wondering if setting the connection padding setting in torrc to 1 instead of auto has any benefit in protecting against a passive adversary outside the Tor network.

1 0

CAPTCHA Monitoring Project Updates #3
by Barkin Simsek 25 Jul '20

25 Jul '20

Hi everyone, I have a few more updates since my last email (you might want to check the project's wiki page [0] first if you didn't see my previous emails): -- I have the onion service [1] of the dashboard [2] up and running. -- I was using an SQLite database and migrated it to PostgreSQL. This migration was needed to scale up the whole system. At the moment, there are 50000+ measurements in the database, and access time was very long with SQLite. -- Now there is a properly documented API [3] available for everyone's use. You can perform queries through the API if you prefer to access the raw data, instead of the visualizations on the dashboard. -- The Dockerfile [4] is updated and working again. -- I decided to give priority to the API and Dockerfile for the "reproducibility" of the results and graphs in the dashboard. Before the API, the graphs were using hardcoded SQL queries, which required people to dig in the source code to find out possible mistakes. It was also impossible for people to execute these SQL queries and see the results themselves. API allows me to put a link to the exact API calls to anything I place on the dashboard so that people can double-check with the raw data if they want. -- I'm working on adding Brave Browser's "Private Window with Tor" to the CAPTCHA Monitor. I'm very close to finishing it and should be ready at the end of this weekend. I expect to see a high amount of CAPTCHA since it is using Tor without Tor Browser's user-agent (and thus fingerprint). -- I discussed a possible integration with the metrics website on the metrics team meeting. I initially thought about placing CAPTCHA rate graphs under each relay in the "Relay Search" section. If you have any suggestions, you are welcomed to mention them in ticket tpo/metrics/website/40002 [5]. -- Finally, you can take a look at my blog posts on DIAL's blog [6] to see more detailed explanations, but I already summarized the blog posts in this email. Best, Barkin (woswos) [0] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/home [1] http://captchaufjq5m2i73up537pldaxnbp6rzcbdrzc7y5rlwtx3mwigznad.onion/ [2] https://dashboard.captcha.wtf/ [3] https://api.captcha.wtf/ [4] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/tree/master/docker [5] https://gitlab.torproject.org/tpo/metrics/website/-/issues/40002 [6] https://hub.osc.dial.community/t/tor-project-cloudflare-captcha-monitoring/…

1 0

Distributing Tor developer keys via Fedora packages
by Andrew Clausen 20 Jul '20

20 Jul '20

Hi everyone, I propose distributing the Tor developer keys inside the Fedora package distribution-gpg-keys.[1] This would give most Linux users a trustworthy chain of signatures from their own distributor (e.g. CentOS or Fedora) to Tor project downloads. I am happy to take care of this, although I am also happy if somebody who is more involved with Tor than me takes this on. I wrote a shell script (attached) to acquire and organise the keys based on https://2019.www.torproject.org/include/keys.txt. My script would install the following keys under /usr/share/distribution-gpg-keys/tor: Arm_releases/Damian_Johnson.gpg Tails_live_system_releases/The_Tails_team.gpg TorBirdy_releases/Sukhbir_Singh.gpg Tor_Browser_releases/Arthur_Edelstein.gpg Tor_Browser_releases/Georg_Koppen.gpg Tor_Browser_releases/Mike_Perry.gpg Tor_Browser_releases/Nicolas_Vigier.gpg Tor_Browser_releases/The_Tor_Browser_Developers.gpg Tor_source_tarballs/Nick_Mathewson.gpg Tor_source_tarballs/Roger_Dingledine.gpg Torsocks_releases/David_Goulet.gpg deb.torproject.org_repositories_and_archives/Tor_Project_Archive.gpg older_Tor_tarballs/Nick_Mathewson.gpg other/Peter_Palfrader.gpg Unless someone else volunteers (please do!), I will set up a weekly job to run the script and alert me to any changes. Can anyone see any potential problems with this plan? The most obvious question is: how do I know that I am distributing unadulterated keys? I think the answer is that I don't! But any attack would have to affect a large group of people, and would be detected quickly as long as many people are looking at the distribution-gpg-keys package. If this solution is unsatisfactory, then perhaps someone who is more involved with the Tor developers -- and hence able to directly check the keys -- ought to take this on. [1] See https://github.com/xsuchy/distribution-gpg-keys and https://rpmfind.net/linux/RPM/fedora/updates/32/x86_64/Packages/d/distribut…

2 2

Proposal 325: Packed relay cells: saving space on small commands
by Nick Mathewson 10 Jul '20

10 Jul '20

``` Filename: 325-packed-relay-cells.md Title: Packed relay cells: saving space on small commands Author: Nick Mathewson Created: 10 July 2020 Status: Draft ``` # Introduction In proposal 319 I suggested a way to fragment long commands across multiple RELAY cells. In this proposal, I suggest a new format for RELAY cells that can be used to pack multiple relay commands into a single cell. Why would we want to do this? As we move towards improved congestion-control and flow-control algorithms, we might not want to use an entire 498-byte relay payload just to send a one-byte flow-control message. We already have some cases where we'd benefit from this feature. For example, when we send SENDME messages, END cells, or BEGIN_DIR cells, most of the cell body is wasted with padding. As a side benefit, packing cells in this way may make the job of the traffic analyst a little more tricky, as cell contents become less predictable. # The basic design Let's use the term "Relay Message" to mean the kind of thing that a relay cell used to hold. Thus, this proposal is about packing multiple "Relay Messages" in to a cell. I'll use "Packed relay cell" to mean a relay cell in this new format, that supports multiple messages. I'll use "client" to mean the initiator of a circuit, and "relay" to refer to the parties through who a circuit is created. Note that each "relay" (as used here) may be the "client" on circuits of its own. When a relay supports relay message packing, it advertises the fact using a new Relay protocol version. Clients must opt-in to using this protocol version (see XXX below) before they can send any packed relay cells, and before the relay will send them any packed relay cells. When packed cells are in use, multiple cell messages can be concatenated in a single relay cell. Only some relay commands are supported for relay cell packing, listed here: - SENDME - DROP - DATA - BEGIN - BEGIN_DIR - END - CONNECTED - PADDING_NEGOTIATE - PADDING_NEGOTIATED If any relay message with a relay command _not_ listed above appears in a packed relay cell with another relay message, then the receiving party MUST tear down the circuit. (Note that relay cell fragments (proposal 319) are not supported for packing.) The command byte "0" is now used to explicitly indicate "end of cell". If the byte "0" appears after a relay message, the rest of the cell MUST be ignored. When generating RELAY cells, implementations SHOULD (as they do today) fill in the unused bytes with four 0-valued bytes, followed by a sequence of random bytes up to the end of the cell. If there are fewer than 4 unused bytes at the end of the cell, those unused bytes should all be filled with 0-valued bytes. # Negotiation and migration After receiving a packed relay cell, the relay know that the client supports this proposal: Relays SHOULD send packed relay cells on any circuit on which they have received a packed relay cell. Relays MUST NOT send packed relay cells otherwise. Clients, in turn, MAY send packed relay cells to any relay whose "Relay" subprotocol version indicates that it supports this protocol. To avoid fingerprinting, this client behavior should controlled with a tristate (1/0/auto) torrc configuration value, with the default set to use a consensus parameter. The parameter is: "relay-cell-packing" Boolean: if 1, clients should send packed relay cells. (Min: 0, Max 1, Default: 0) To handle migration, first the parameter should be set to 0 and the configuration setting should be "auto". To test the feature, individual clients can set the tristate to "1". Once enough clients have support for the parameter, the parameter can be set to 1. # A new relay message format (This section is optional and should be considered separately; we may decide it is too complex.) Currently, every relay message uses 5 bytes of header to hold a relay command, a length field, and a stream ID. This is wasteful: the stream ID is often redundant, and the top 7 bits of the length field are always zero. I propose a new relay message format, described here (with `ux` denoting an x-bit bitfield). This format is 2 bytes or 4 bytes, depending on its first bit. struct relay_header { u1 stream_id_included; // Is the stream_id included? u6 relay_command; // as before u9 relay_data_len; // as before u8 optional_stream_id[]; // 0 bytes or two bytes. } Alternatively, you can view the first three fields as a 16-bit value, computed as: (stream_id_included<<15) | (relay_command << 9) | (relay_data_len). If the optional_stream_id field is not present, then the default value for the stream_id is computed as follows. We use stream_id 0 for any command that doesn't take a stream ID. For commands that _do_ take a steam_id, we use whichever nonzero stream_id appeared last in this cell. This format limits the space of possible relay commands. That's probably okay: after 20 years of Tor development, we have defined 25 relay command values. But in case 2^6==64 commands will not be enough, we reserve command values 48 through 63 for future formats that need more command bits.

3 2

CAPTCHA Monitoring Project Updates & Findings
by Barkin Simsek 10 Jul '20

10 Jul '20

Hi everyone, I made progress on the Cloudflare CAPTCHA Monitoring project since my last email, and I wanted to share some of the updates & findings. This year Tor Project is participating in GSoC under the DIAL umbrella, and I have already been posting updates to the DIAL blog [1] weekly. I started mirroring these updates [2] to my project's wiki page, and I will be posting more frequent updates here. a) Updates: Firstly, I moved the wiki page that explains the project, the code base, and the issue tracker to Tor Project's GitLab. They are all in the same GitLab project. You can find detailed information about the project on the wiki page [3] and leave comments & suggestions within that repository by creating issues. Secondly, I got a fully functioning system up and running. The system fetches various URLs with Tor Browser & Firefox over Tor and checks for CAPTCHAs. The system also checks if any third-party code was injected by comparing the hash of the received page with an expected hash value. It repeats these experiments using different exit relays and records results. You can view the results on the dashboard [4] I created. I'm looking for more URLs to track for CAPTCHAs. Feel free to share the websites you frequently visit and get CAPTCHAs, so that I can track these websites with this tool as well. I want to experiment with all types of CAPTCHAs, and these URLs don't have to be fronted by Cloudflare. b) Findings: So far, I have observed that using the Tor Browser Bundle out of the box without changing its configurations doesn't lead to a high CAPTCHA rate on Cloudflare fronted websites (assuming the website owners don't explicitly block exit relays [5]). That said, modifying the user-agent or any other modifications that deviate your browser's fingerprint from a typical Tor Browser user, significantly increases the chance of getting CAPTCHAs. For example, using the regular Firefox over Tor resulted in getting CAPTCHAs in ~90% of the measurements. I believe Cloudflare is very aggressive against the "Firefox over Tor" users because many people, unfortunately, use Chromium/Firefox + Selenium + Tor to scrape web pages and bypass IP-based rate limits. That's why I'm interested in hearing about your specific browser/Tor configurations to test them with the CAPTCHA Monitor. Not everyone is affected in the same way because of these differences in the way we use Tor, but we can understand which differences affect the CAPTCHA rate more than others by experimenting. Additionally, I observed that the TLS fingerprint has a significant role in whether someone gets a CAPTCHA or not. As a part of the project, I decided to capture the HTTP headers during measurements to understand how they affect the CAPTCHA rates. Initially, I was using a Python library called seleniumwire to capture the HTTP headers by intercepting the traffic between the Tor Browser and Tor. By doing this, I got a very high CAPTCHA rate, like 98% of the time. seleniumwire forwards the traffic transparently, but it has a different TLS fingerprint than Tor Browser. I figured out that the difference in the TLS fingerprints was triggering the MITM detection on the Cloudflare side, thus, resulting in very high CAPTCHA rates. Interestingly, I tried using the exact same Tor Browser & seleniumwire setup, but without Tor and, practically, I didn't get any CAPTCHAs. I believe the MITM detection is more aggressive if the traffic is coming through an exit relay. So, I stopped using seleniumwire to capture headers because it didn't reflect what a real human Tor Browser user is usually experiencing. Please feel free to use the sample code [6] that I used to combine seleniumwire and Tor, if you are interested in doing further experimenting on this. c) Next: I will work on collecting more metrics by testing more configurations and websites. I will create a "Relay Search" section on the dashboard, where CAPTCHA statistics for the relays (exit relays for now) will be available. I will also work on using the collected data to predict the probability of getting CAPTCHAs with a given exit relay and configuration/setup. Best, Barkin [1] https://hub.osc.dial.community/t/tor-project-cloudflare-captcha-monitoring/… [2] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/Updates [3] https://gitlab.torproject.org/woswos/CAPTCHA-Monitor/-/wikis/home [4] https://dashboard.captcha.wtf/ [5] Cloudflare has a setting to block all traffic originating from the Tor network, but that setting is not "turned on" by default [6] https://gist.github.com/woswos/38b921f0b82de009c12c6494db3f50c5

1 0