- tor-dev - lists.torproject.org

Walking onions status update: week 3 notes
by Nick Mathewson 30 Mar '20

30 Mar '20

Walking Onions: week 3 update On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send out these updates once a week. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html Week 2: Specifying details of SNIP and ENDIVE formats, in CDDL. https://lists.torproject.org/pipermail/tor-dev/2020-March/014181.html You might like to check out those updates, and their references, if this update doesn't make sense to you. === As you might recall, the SNIPs need to be nice and small, but they need to be completely self-contained. The ENDIVE, on the other hand, needs to be diff-friendly, and compressible. Relays need to derive all the SNIPs from the ENDIVE. The ENDIVE and SNIP formats that I worked on during last week [SNIPFMT] was designed to meet these criteria. This week, I wrote up the algorithms to extract signed SNIPs from an ENDIVE. This is in section 4 [RECONST] of the specs in my work-in-progress repository [GITREPO] This was mainly straightforward consequence from the work I did last week, but there were a few things that turned up. One trickier aspect of this work is that we want to move most of the decision-making to the authorities, and have the relays simply do an expansion algorithm. Updating all the relays would be slow, so relays need to be able to handle new index types and data types without fully understanding them. That means that we need to specify an algorithm that can work flexibly with many future kinds of data. Another tricky aspect is that the data which the relays need to put into SNIPs all needs to be signed, so all of that data needs to come out the same when relays calculate it as the when the authorities calculate it. Thus, all the algorithms for building Merkle trees and indices and SNIPRouterData bodies need to be deterministic. I think that I got all of this relatively right in [RECONST], but there are probably some mistakes hiding in there. I'm hoping that there is a cleaner alternative to the weighted-index reconstruction algorithm -- the one that's there right now puts a lot of requirements on the input data, in order to avoid floating-point operations. === After working on ENDIVE expansion, I moved on to the problem of authorities voting. There's a lot of uncertainty here and decisions I'll need to make. The biggest decision is how the new voting process for ENDIVEs should relate to the current voting process for consensus documents. The choices seem to be: * Have two processes with different kinds of votes. * Expand the information that goes into current votes, and generate ENDIVEs as a new kind of consensus from the current voting process. * Design the ENDIVE voting process so that it can also be used to generate current-style consensuses. Using separate processes might be simplest, but it does carry risk for duplicated code, and for the two voting processes reaching divergent results. It would also be potentially bad if an authority could vote differently in each process. The second approach -- where ENDIVEs are another kind of consensus generated from current-style voting documents -- has some charm to it, but it perpetuates our current consensus-voting system, and makes us specify a second encoding of CBOR objects as text, if we want to vote on them meaningfully. This is a bit nasty, since the standard-specified text encoding of cbor is not a bijection, and doesn't mesh too well with our old directory format. Therefore the third approach seems a little better to me now. In it, we'd specify our votes as a CBOR-based format loosely based on the current vote format, and describe a way to extract from them the same data that is currently used for our voting algorithm. This would let us encode CBOR within the document naturally, while not having to throw away too much of our existing voting specification and voting code. I've got a few more ideas here, though I don't have any specs yet. I've been collecting the ideas in a notes file [VNOTES] so I don't forget them. === While working on voting notes, I found some areas where the ENDIVE and SNIP formats might be better. First, I hadn't thought about authority certificates. Those should go in, so that we can have the ENDIVE-signing keys be signed themselves with the authorities' long-lived identities. Second, there seem to be some opportunities for transmitting indices in a simpler way. Instead of assigning a weight to each router separately, for example, we could derive one weighted index from another weighted index by re-weighting it, as we do with the "bandwidth-weights" values in the current consensus. Teor also made some good suggestions for improving the diff format; I incorporated one and made a note to check the other. === I also spent a little while this week reading about BLS signatures, since they are the leading candidate for how we might do threshold signatures here. === Next week I'm going to try to get voting all specified. This will be a pretty deep dive, since I will need to maintain some backward compatibility with existing voting algorithms. There may be opportunities for simplification, however. === [GITREPO] https://github.com/nmathewson/walking-onions-wip [RECONST] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/04-recon… d [SNIPFMT] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/02-endiv… [VNOTES] https://github.com/nmathewson/walking-onions-wip/blob/master/notes/voting.t…

3 3

Walking onions status update: week 4 notes
by Nick Mathewson 30 Mar '20

30 Mar '20

Walking Onions: week 4 update On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send out these updates once a week. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html Week 2: Specifying details of SNIP and ENDIVE formats, in CDDL. https://lists.torproject.org/pipermail/tor-dev/2020-March/014181.html Week 3: Expanding ENDIVEs into SNIPs. https://lists.torproject.org/pipermail/tor-dev/2020-March/014194.html This week was pretty rough for me; the impact of being isolated at home is starting to kick in, and my kid is still trying to find a schedule that works for him. I got a lot less done this week than I had planned. That said, I _did_ get some good design done, in two areas: voting, and circuit extension. == Voting I'm trying to generalize our voting algorithms to minimize the need for new consensus formats in the future. I want most new fields to be self-describing in terms of how to tabulate votes for them. Trying to work this out has taken a bunch of energy, though and I'm not quite sure I'm doing it right. The work here is unfinished. I still think it's reasonable to encode existing votes and new votes in a new cbor-based structure. By using compression, a little redundancy in votes shouldn't hurt us too badly -- though that's something I need to benchmark eventually before we build this. I've come up with a way for voters to use diffs and compression when _uploading_ votes: first, the voter asks for a summary of what mechanisms are supported by the recipient and which older votes it has. Next, the voter can use that information to upload a vote diff with an appropriate compression format. Right now I believe that I have most of the fields for the votes figured out, but I need to clean them up a _lot_. SNIP bodies can work similarly to how microdescriptors work now, if we like -- or they could involve individual votes on input fields. I think I'm going to write up what I can here, and then revisit SNIP bodies once I've worked some on section 6 (the "fancy" client behavior). While working on voting, I found a need for various changes in the already written sections. For one example, the previous description of the ENDIVE format included the SNIP signatures as part of the signed ENDIVE. But we can't actually generate that as part our voting protocol: the authorities can't sign SNIPs until they know the complete ENDIVE contents, and so putting the SNIP signatures in the signed part of the ENDIVE would add an extra round of voting and signatures. Instead, I've moved the SNIP signatures (and related stuff) into the signature header of the ENDIVE. I've also worked somewhat on the so-called "root document" holding the network configuration parameters. I'm hoping that this document can have an effective lifetime of weeks or months for clients, and that clients can download only the parts that they need. The format is in section 02, but needs more description. The work-in-progress voting writeup is in [VOTING], but it is not so close to done. I think it will need a lot of writing before it's ready for even preliminary review; it doesn't make sense as it is, and it contradicts itself in places. == Extending circuits There are a few wrinkles here that we hadn't actually specified in earlier walking onions work! When one relay is told to extending a circuit to a target relay by index, the extending relay actually needs to include part of a SNIP for that second relay: otherwise, the extending relay won't know which onion key the client expects to receive for it. We don't want to send an entire SNIP -- only the parts that the client will expect to match. In addition to wide EXTEND cells, we'll also need wide RENDEZVOUS and INTRODUCE1 and INTRODUCE2 and TRUNCATED cells. All together, that means that having a general "cell fragmentation" mechanism is probably warranted; I've started a separate proposal for that [WIDE_EVERYTHING]. We're going to need a general extension format for CREATE and EXTEND cells, to signal other information, like SNIPs, requests for SNIPs, and so on. We also need a way to ask for SNIPs without actually performing a handshake. I've tried to fit these all together in a fairly logical way in [EXTENDING], but there's more to write here, including examples. == Fun facts The walking onion proposal is now the second-longest that we have ever had, behind only proposal 224 (v3 hidden services). == Next steps In this coming week I plan to try to wrap up section 3 on voting and section 5 on extending circuits. I'm going to go back to the start of section 2 and start revising all I've written so far for consistency. Time permitting, I want to sketch out all the knowns and unknowns for section 6 (on tricky client behavior). [EXTENDING] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/05-exten… [VOTING] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/02-votin… [WIDE_EVERYTHING] https://github.com/nmathewson/walking-onions-wip/blob/master/other-proposal…

2 1

Walking Onions status update: week 2 notes
by Nick Mathewson 29 Mar '20

29 Mar '20

Walking onions -- week 2 update Hi! On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send these out thee updates once a week, in case anybody is interested. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html You might like to have a look at that update, and its references, if this update doesn't make sense to you. === This week, I worked specifying the nitty-gritty of the SNIP and ENDIVE document formats. I used the CBOR meta-format [CBOR] to build them, and the CDDL specification language [CDDL] to specify what they should contain. As before, I've been working in a git repository at [GITHUB]; you can see the document I've been focusing on this week at [SNIPFMT]. (That's the thing to read if you want to send me patches for my grammar.) There were a few neat things to do here: * I had to define SNIPs so that clients and relays can be mostly agnostic about whether we're using a merkle tree or a bunch of signatures. * I had to define a binary diff format so that relays can keep on downloading diffs between ENDIVE documents. (Clients don't download ENDIVEs). I did a quick prototype of how to output this format, using python's difflib. * To make ENDIVE diffs as efficient as possible, it's important not to transmit data that changes in every ENDIVE. To this end, I've specified ENDIVEs so that the most volatile parts (Merkle trees and index ranges) are recomputed on the relay side. I still need to specify how these re-computations work, but I'm pretty sure I got the formats right. Doing this calculation should save relays a bunch of bandwidth each hour, but cost some implementation complexity. I'm going to have to come back to this choice going forward to see whether it's worth it. * Some object types are naturally extensible, some aren't. I've tried to err on the size of letting us expand important things in the future, and using maps (key->value mappings) for object that are particularly important. In CBOR, small integers are encoded with a little less space than small strings. To that end, I'm specifying the use of small integers for dictionary keys that need to be encoded briefly, and strings for non-tor and experimental extensions. * This is a fine opportunity to re-think how we handle document liveness. Right now, consensus directories have an official liveness interval on them, but parties that rely on consensuses tolerate larger variance than is specified in the consensus. Instead of that approach, the usable lifetime of each object is now specified in the object, and is ultimately controlled by the authorities. This gives the directory authorities more ability to work around network tolerance issues. Having large lifetime tolerances in the context of walking onions is a little risky: it opens us up to an attack where a hostile relay holds multiple ENDIVEs, and decides which one to use when responding to a request. I think we can address this attack, however, by making sure that SNIPs have a published time in them, and that this time moves monotonically forward. * As I work, I'm identifying other issues in tor that stand in the way of a good efficient walking onion implementation that will require other follow-up work. This week I ran into a need for non-TAP-based v2 hidden services, and a need for a more efficient family encoding. I'm keeping track of these in my outline file. Fun fact: In number of bytes, the walking onions proposal is now the 9th-longest proposal in the Tor proposal repository. And it's still growing! Next week, I'm planning to specify ENDIVE reconstruction, circuit extension, and maybe start on a specification for voting. [CBOR] RFC 7049: "Concise Binary Object Representation (CBOR)" https://tools.ietf.org/html/rfc7049b [CDDL] RFC 8610: "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures" https://tools.ietf.org/html/rfc8610 [GITREPO] https://github.com/nmathewson/walking-onions-wip [SNIPFMT] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/02-endiv…

3 6

Tails vs the capacity of the Meek bridges
by anonym 24 Mar '20

24 Mar '20

Hi, tl;dr: if Tails makes it too easy to use Meek bridges, could it overload the current set of Meek bridges? First some background: during startup Tails can be told to start Tor Launcher so users can e.g. configure any bridges they want. So far we have not provided any pre-configured bridges, i.e. the only option has been to manually enter the bridge information you have obtained yourself. In Tails' threat model it is assumed that adversaries monitor the default bridges provided by the Tor Browser, and that our users want to avoid detection of that, so we are not interested in adding the default bridges to Tails, but we are interested in adding support for Meek [1] (at least because it's the only PT that works in China), since our understanding is that it adds enough plausible deniability to avoid the above problem. So, in summary, Tails would like to provide two options in its (patched) Tor Launcher, Meek or manually providing the bridge info. [2] However, we wonder if the combination of not providing the default bridges while making Meek available could overwhelm the Meek bridges; we expect some significant amount of Tails bridge users to select the Meek option over manual entry simply out of convenience. Let's do some back-of-the-envelope estimations to see what we can expect: (Assumption: Tor Browser users and Tails users are very similar, e.g. similar ratios want to use each PT, similar requirements for "convenience over security", and so on.) Looking at your metrics, there are 1000k daily Tor Browser users [3], overall bridge/PT usage is 50k [4], Meek usage is around 10k [5], so 5% of Tor Browser users use bridges/PTs, and 1% use Meek. On the Tails side, we measure around 30k daily users (from update pings). >From what I said above, I don't think we can expect only 1% of the Tails users to pick Meek; I would expect it to be closer to the 1% of Meek users plus the x% of default bridge users, but I couldn't find stats for that in your metrics. Still, we know that it cannot be more than the percentage of all bridge/PT users, so we know that 1% + x% <= 5%. So, let's just assume the worst case, that 5% of Tails users will use Meek, and we have that we expect 5% of 30k = 1.5k additional Meek users. Also, if we consider places where Meek is the only options, Tails probably has close to zero users there, but once Meek is supported it could grow. I guess China is the only such place (?), so with its 1250 Meek users in China [6] we can expect 1250 * 30k/1000k = 34 new Tails users using Meek. Basically a negligible amount. Even if we consider all Meek users, 10k * 30k/1000k = 300 doesn't change much. So, all-in-all, we can expect Tails to bring up to 1.5k + 300 = 1.8k new Meek users, but since those are upper-bound estimations it would probably be much lower. Looking on Meek usage over time, it seems to fluctuate way more than that, e.g. during the summer of 2019 it was up to over 25k, i.e. more than three times what it is now. So I guess we don't have to worry about shocking the Meek bridges? OTOH, a possible side-effect is that this change in Tails increases usage of Meek outside of China. Perhaps whoever pays the bills for the Meek instances don't want this? Please advice! Also, please let us know if there is something else we haven't thought of! Cheers! [1] https://redmine.tails.boum.org/code/issues/8243 [2] If you are really interested you can check out our PoC/WIP here: https://nightly.tails.boum.org/build_Tails_ISO_feature-8243-meek/builds/las… [3] https://metrics.torproject.org/webstats-tb.html [4] https://metrics.torproject.org/userstats-bridge-transport.html [5] https://metrics.torproject.org/userstats-bridge-transport.html?transport=!%… [6] https://metrics.torproject.org/userstats-bridge-combined.html?start=2019-12…

4 5

Re: [tor-dev] GSOC'20
by Sanjiban Sengupta 23 Mar '20

23 Mar '20

Dear Sir, This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I have practical and working knowledge of C, C++, Python and Java, for web, I am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel and know the technicalities to apply these to solve modern real life problems. On going through the project proposals, I found the projectImplementing Salmon as a bridge distribution mechanism, and Tor Weather interesting to work upon and contribute and thus will be thankful for your kind guidance. Thus I request the mentors to kindly guide me for the beginning processes. Regards, Sanjiban Sengupta On Sat, Mar 21, 2020 at 11:47 AM Sanjiban Sengupta <b518041(a)iiit-bh.ac.in> wrote: > Dear Sir, > > This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT > Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I > have practical and working knowledge of C, C++, Python and Java, for web, I > am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS > and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel > and know the technicalities to apply these to solve modern real life > problems. > > On going through the project proposals, I found the projectImplementing > Salmon as a bridge distribution mechanism, and Tor Weather interesting to > work upon and contribute and thus will be thankful for your kind guidance. > > Thus I request the mentors to kindly guide me for the beginning processes. > > Regards, > Sanjiban Sengupta > > > > On Sat, Mar 21, 2020 at 3:50 AM Sanjiban Sengupta <b518041(a)iiit-bh.ac.in> > wrote: > >> Dear Sir, >> >> This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT >> Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I >> have practical and working knowledge of C, C++, Python and Java, for web, I >> am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS >> and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel >> and know the technicalities to apply these to solve modern real life >> problems. >> >> On going through the project proposals, I found the projectImplementing >> Salmon as a bridge distribution mechanism, and Tor Weather interesting to >> work upon and contribute and thus will be thankful for your kind guidance. >> >> Thus I request the mentors to kindly guide me for the beginning processes. >> >> Regards, >> Sanjiban Sengupta >> >

2 1

Lets give every circuit its own exit IP?
by nusenu 18 Mar '20

18 Mar '20

The unbearable situation with Google's reCAPTCHA motivated this email (but it is not limited to this specific case). This idea came up when seeing a similar functionality in unbound (which has it for a different reason). Assumption: There are systems that block some tor exit IP addresses (most likely the bigger once), but they are not blocked due to the fact that they are tor exits. It just occurred that the IP got flagged because of "automated / malicious" requests and IP reputation systems. What if every circuit had its "own" IP address at the exit relay to avoid causing collateral damage to all users of the exit if one was bad? (until the exit runs out of IPs and starts to recycle previously used IPs again) The goal is to avoid accumulating a bad "reputation" for the single used exit IP address that affects all tor users of that exit. Instead of doing it on the circuit level you could do it based on time. Change the exit IP every 5 minutes (but do _not_ change the exit IPs for _existing_ circuits even if they live longer than 5 minutes). Yes, no one has that many IPv4 addresses but with the increasing availability of IPv6 at exits and destinations, this could be feasible to a certain extend, depending on how many IPv6 addresses the exit operator has. There are exit operators that have entire /48 IPv6 blocks. problems: - will not solve anything since reputation will shift to netblocks as well (How big of a netblock are you willing to block?) - you can tell two tor users easily apart from each other even if they use the same exit (or more generally: you can tell circuits apart). There might be all kinds of bad implications that I'm not thinking off right now. - check.tpo would no longer be feasible - how can do we still provide the list of exit IPs for easy blocking? Exits could signal their used netblock via their descriptor. What if they don't? (that in turn opens new kinds of attacks where an exit claims to be /0 and the target effectively blocks everything) - more state to track and store at the exit -... some random thoughts, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

4 6

Auto close old tor GitHub pull requests?
by teor 16 Mar '20

16 Mar '20

Hi all, I'd like to use a GitHub bot to automatically close old tor and torspec pull requests. The Network Team currently uses GitHub to review pull requests (but we use Trac for task tracking). Some GitHub users won't contribute to a project with lots of open pull requests, because they think it's been abandoned. (Or they think we're bad at merging PRs.) But lots of our PRs get left open because we've squashed and merged the PR, and GitHub doesn't recognise the squashed commits, so GitHub doesn't auto-close the PR. I'm proposing closing PRs with no activity for 60 days, but I think anywhere between 30 and 180 days would work. (Most of our backports happen within 60 days.) For more details, see: https://trac.torproject.org/projects/tor/ticket/33629 T -- teor ----------------------------------------------------------------------

1 0

Request to provide code for Padding in Tor
by DIVYA GUPTA 16 Mar '20

16 Mar '20

Hello, We are under-graduate students from Malaviya National Institute of Technology currently pursuing Computer Science & Engineering. We are working on developing a defense for Tor against Website Fingerprinting Attacks. While reading about tor we came across padding which is implemented in Tor. It would be great if you could make available the code for the padding. Regards, Divya.

2 1

Fwd: Upcoming Tor security releases to fix a denial-of-service issue
by Nick Mathewson 16 Mar '20

16 Mar '20

---------- Forwarded message --------- From: Nick Mathewson <nickm(a)torproject.org> Date: Mon, Mar 16, 2020 at 1:25 PM Subject: Upcoming Tor security releases to fix a denial-of-service issue To: <tor-talk(a)lists.torproject.org> Hello! Some time this week, we currently plan to put out a set of security updates for all supported versions of Tor. These releases will fix a pair of denial-of-service bugs: one that we are classifying at "low" severity, and one that we are classifying at "high" severity. Our recommendation will be for everybody, including relays and clients, to upgrade once packages are available for their platforms. Although these vulnerabilities are "only" denial-of-service issues, any denial-of-service attack against Tor could be leveraged by an attacker to aid in a traffic analysis attack. To the best of our knowledge, these vulnerabilities are not being exploited in the wild. Currently supported release series are 0.3.5, 0.4.1, 0.4.2, and 0.4.3 (alpha). If you have not yet upgraded to one of those, the time to do so is soon. For our policy and process for handing security issues, please see: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Securit… best wishes, -- Nick

1 0

SOOC (Same Origin Onion Certificates) discussion tomorrow
by Richard Pospesel 10 Mar '20

10 Mar '20

Hi Everyone, The current UX situation around HTTPS-enabled onionsites is not the best: Tor Browser shows security warning splash screens and icons in scenarios that do not warrant them. On top of that, the only way for your onion site to not show these warnings is by getting an EV cert which is only practical if you have the $$$. alecmuffet's SOOC proposal ( https://github.com/alecmuffett/onion-dv-certificate-proposal/blob/master/te… ) does seem to fix these UX problems for us. Though the proposal is still incomplete, we can make forward progress with a prototype implementation as the certificate specification is fully fleshed out. We'll be talking about this tomorrow during the Sponsor 27 meeting tomorrow on Tuesday 10th May @ 1500UTC in #tor-meeting. Please feel free to come by if you wish to discuss. best, -Richard

2 1