tor-dev

tor-dev@lists.torproject.org

1 participants
3590 discussions

Walking Onions status update: week 2 notes
by Nick Mathewson 29 Mar '20

29 Mar '20

Walking onions -- week 2 update Hi! On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send these out thee updates once a week, in case anybody is interested. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html You might like to have a look at that update, and its references, if this update doesn't make sense to you. === This week, I worked specifying the nitty-gritty of the SNIP and ENDIVE document formats. I used the CBOR meta-format [CBOR] to build them, and the CDDL specification language [CDDL] to specify what they should contain. As before, I've been working in a git repository at [GITHUB]; you can see the document I've been focusing on this week at [SNIPFMT]. (That's the thing to read if you want to send me patches for my grammar.) There were a few neat things to do here: * I had to define SNIPs so that clients and relays can be mostly agnostic about whether we're using a merkle tree or a bunch of signatures. * I had to define a binary diff format so that relays can keep on downloading diffs between ENDIVE documents. (Clients don't download ENDIVEs). I did a quick prototype of how to output this format, using python's difflib. * To make ENDIVE diffs as efficient as possible, it's important not to transmit data that changes in every ENDIVE. To this end, I've specified ENDIVEs so that the most volatile parts (Merkle trees and index ranges) are recomputed on the relay side. I still need to specify how these re-computations work, but I'm pretty sure I got the formats right. Doing this calculation should save relays a bunch of bandwidth each hour, but cost some implementation complexity. I'm going to have to come back to this choice going forward to see whether it's worth it. * Some object types are naturally extensible, some aren't. I've tried to err on the size of letting us expand important things in the future, and using maps (key->value mappings) for object that are particularly important. In CBOR, small integers are encoded with a little less space than small strings. To that end, I'm specifying the use of small integers for dictionary keys that need to be encoded briefly, and strings for non-tor and experimental extensions. * This is a fine opportunity to re-think how we handle document liveness. Right now, consensus directories have an official liveness interval on them, but parties that rely on consensuses tolerate larger variance than is specified in the consensus. Instead of that approach, the usable lifetime of each object is now specified in the object, and is ultimately controlled by the authorities. This gives the directory authorities more ability to work around network tolerance issues. Having large lifetime tolerances in the context of walking onions is a little risky: it opens us up to an attack where a hostile relay holds multiple ENDIVEs, and decides which one to use when responding to a request. I think we can address this attack, however, by making sure that SNIPs have a published time in them, and that this time moves monotonically forward. * As I work, I'm identifying other issues in tor that stand in the way of a good efficient walking onion implementation that will require other follow-up work. This week I ran into a need for non-TAP-based v2 hidden services, and a need for a more efficient family encoding. I'm keeping track of these in my outline file. Fun fact: In number of bytes, the walking onions proposal is now the 9th-longest proposal in the Tor proposal repository. And it's still growing! Next week, I'm planning to specify ENDIVE reconstruction, circuit extension, and maybe start on a specification for voting. [CBOR] RFC 7049: "Concise Binary Object Representation (CBOR)" https://tools.ietf.org/html/rfc7049b [CDDL] RFC 8610: "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures" https://tools.ietf.org/html/rfc8610 [GITREPO] https://github.com/nmathewson/walking-onions-wip [SNIPFMT] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/02-endiv…

3 6

Tails vs the capacity of the Meek bridges
by anonym 24 Mar '20

24 Mar '20

Hi, tl;dr: if Tails makes it too easy to use Meek bridges, could it overload the current set of Meek bridges? First some background: during startup Tails can be told to start Tor Launcher so users can e.g. configure any bridges they want. So far we have not provided any pre-configured bridges, i.e. the only option has been to manually enter the bridge information you have obtained yourself. In Tails' threat model it is assumed that adversaries monitor the default bridges provided by the Tor Browser, and that our users want to avoid detection of that, so we are not interested in adding the default bridges to Tails, but we are interested in adding support for Meek [1] (at least because it's the only PT that works in China), since our understanding is that it adds enough plausible deniability to avoid the above problem. So, in summary, Tails would like to provide two options in its (patched) Tor Launcher, Meek or manually providing the bridge info. [2] However, we wonder if the combination of not providing the default bridges while making Meek available could overwhelm the Meek bridges; we expect some significant amount of Tails bridge users to select the Meek option over manual entry simply out of convenience. Let's do some back-of-the-envelope estimations to see what we can expect: (Assumption: Tor Browser users and Tails users are very similar, e.g. similar ratios want to use each PT, similar requirements for "convenience over security", and so on.) Looking at your metrics, there are 1000k daily Tor Browser users [3], overall bridge/PT usage is 50k [4], Meek usage is around 10k [5], so 5% of Tor Browser users use bridges/PTs, and 1% use Meek. On the Tails side, we measure around 30k daily users (from update pings). >From what I said above, I don't think we can expect only 1% of the Tails users to pick Meek; I would expect it to be closer to the 1% of Meek users plus the x% of default bridge users, but I couldn't find stats for that in your metrics. Still, we know that it cannot be more than the percentage of all bridge/PT users, so we know that 1% + x% <= 5%. So, let's just assume the worst case, that 5% of Tails users will use Meek, and we have that we expect 5% of 30k = 1.5k additional Meek users. Also, if we consider places where Meek is the only options, Tails probably has close to zero users there, but once Meek is supported it could grow. I guess China is the only such place (?), so with its 1250 Meek users in China [6] we can expect 1250 * 30k/1000k = 34 new Tails users using Meek. Basically a negligible amount. Even if we consider all Meek users, 10k * 30k/1000k = 300 doesn't change much. So, all-in-all, we can expect Tails to bring up to 1.5k + 300 = 1.8k new Meek users, but since those are upper-bound estimations it would probably be much lower. Looking on Meek usage over time, it seems to fluctuate way more than that, e.g. during the summer of 2019 it was up to over 25k, i.e. more than three times what it is now. So I guess we don't have to worry about shocking the Meek bridges? OTOH, a possible side-effect is that this change in Tails increases usage of Meek outside of China. Perhaps whoever pays the bills for the Meek instances don't want this? Please advice! Also, please let us know if there is something else we haven't thought of! Cheers! [1] https://redmine.tails.boum.org/code/issues/8243 [2] If you are really interested you can check out our PoC/WIP here: https://nightly.tails.boum.org/build_Tails_ISO_feature-8243-meek/builds/las… [3] https://metrics.torproject.org/webstats-tb.html [4] https://metrics.torproject.org/userstats-bridge-transport.html [5] https://metrics.torproject.org/userstats-bridge-transport.html?transport=!%… [6] https://metrics.torproject.org/userstats-bridge-combined.html?start=2019-12…

4 5

Re: [tor-dev] GSOC'20
by Sanjiban Sengupta 23 Mar '20

23 Mar '20

Dear Sir, This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I have practical and working knowledge of C, C++, Python and Java, for web, I am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel and know the technicalities to apply these to solve modern real life problems. On going through the project proposals, I found the projectImplementing Salmon as a bridge distribution mechanism, and Tor Weather interesting to work upon and contribute and thus will be thankful for your kind guidance. Thus I request the mentors to kindly guide me for the beginning processes. Regards, Sanjiban Sengupta On Sat, Mar 21, 2020 at 11:47 AM Sanjiban Sengupta <b518041(a)iiit-bh.ac.in> wrote: > Dear Sir, > > This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT > Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I > have practical and working knowledge of C, C++, Python and Java, for web, I > am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS > and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel > and know the technicalities to apply these to solve modern real life > problems. > > On going through the project proposals, I found the projectImplementing > Salmon as a bridge distribution mechanism, and Tor Weather interesting to > work upon and contribute and thus will be thankful for your kind guidance. > > Thus I request the mentors to kindly guide me for the beginning processes. > > Regards, > Sanjiban Sengupta > > > > On Sat, Mar 21, 2020 at 3:50 AM Sanjiban Sengupta <b518041(a)iiit-bh.ac.in> > wrote: > >> Dear Sir, >> >> This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT >> Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I >> have practical and working knowledge of C, C++, Python and Java, for web, I >> am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS >> and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel >> and know the technicalities to apply these to solve modern real life >> problems. >> >> On going through the project proposals, I found the projectImplementing >> Salmon as a bridge distribution mechanism, and Tor Weather interesting to >> work upon and contribute and thus will be thankful for your kind guidance. >> >> Thus I request the mentors to kindly guide me for the beginning processes. >> >> Regards, >> Sanjiban Sengupta >> >

2 1

Lets give every circuit its own exit IP?
by nusenu 18 Mar '20

18 Mar '20

The unbearable situation with Google's reCAPTCHA motivated this email (but it is not limited to this specific case). This idea came up when seeing a similar functionality in unbound (which has it for a different reason). Assumption: There are systems that block some tor exit IP addresses (most likely the bigger once), but they are not blocked due to the fact that they are tor exits. It just occurred that the IP got flagged because of "automated / malicious" requests and IP reputation systems. What if every circuit had its "own" IP address at the exit relay to avoid causing collateral damage to all users of the exit if one was bad? (until the exit runs out of IPs and starts to recycle previously used IPs again) The goal is to avoid accumulating a bad "reputation" for the single used exit IP address that affects all tor users of that exit. Instead of doing it on the circuit level you could do it based on time. Change the exit IP every 5 minutes (but do _not_ change the exit IPs for _existing_ circuits even if they live longer than 5 minutes). Yes, no one has that many IPv4 addresses but with the increasing availability of IPv6 at exits and destinations, this could be feasible to a certain extend, depending on how many IPv6 addresses the exit operator has. There are exit operators that have entire /48 IPv6 blocks. problems: - will not solve anything since reputation will shift to netblocks as well (How big of a netblock are you willing to block?) - you can tell two tor users easily apart from each other even if they use the same exit (or more generally: you can tell circuits apart). There might be all kinds of bad implications that I'm not thinking off right now. - check.tpo would no longer be feasible - how can do we still provide the list of exit IPs for easy blocking? Exits could signal their used netblock via their descriptor. What if they don't? (that in turn opens new kinds of attacks where an exit claims to be /0 and the target effectively blocks everything) - more state to track and store at the exit -... some random thoughts, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

4 6

Auto close old tor GitHub pull requests?
by teor 16 Mar '20

16 Mar '20

Hi all, I'd like to use a GitHub bot to automatically close old tor and torspec pull requests. The Network Team currently uses GitHub to review pull requests (but we use Trac for task tracking). Some GitHub users won't contribute to a project with lots of open pull requests, because they think it's been abandoned. (Or they think we're bad at merging PRs.) But lots of our PRs get left open because we've squashed and merged the PR, and GitHub doesn't recognise the squashed commits, so GitHub doesn't auto-close the PR. I'm proposing closing PRs with no activity for 60 days, but I think anywhere between 30 and 180 days would work. (Most of our backports happen within 60 days.) For more details, see: https://trac.torproject.org/projects/tor/ticket/33629 T -- teor ----------------------------------------------------------------------

1 0

Request to provide code for Padding in Tor
by DIVYA GUPTA 16 Mar '20

16 Mar '20

Hello, We are under-graduate students from Malaviya National Institute of Technology currently pursuing Computer Science & Engineering. We are working on developing a defense for Tor against Website Fingerprinting Attacks. While reading about tor we came across padding which is implemented in Tor. It would be great if you could make available the code for the padding. Regards, Divya.

2 1

Fwd: Upcoming Tor security releases to fix a denial-of-service issue
by Nick Mathewson 16 Mar '20

16 Mar '20

---------- Forwarded message --------- From: Nick Mathewson <nickm(a)torproject.org> Date: Mon, Mar 16, 2020 at 1:25 PM Subject: Upcoming Tor security releases to fix a denial-of-service issue To: <tor-talk(a)lists.torproject.org> Hello! Some time this week, we currently plan to put out a set of security updates for all supported versions of Tor. These releases will fix a pair of denial-of-service bugs: one that we are classifying at "low" severity, and one that we are classifying at "high" severity. Our recommendation will be for everybody, including relays and clients, to upgrade once packages are available for their platforms. Although these vulnerabilities are "only" denial-of-service issues, any denial-of-service attack against Tor could be leveraged by an attacker to aid in a traffic analysis attack. To the best of our knowledge, these vulnerabilities are not being exploited in the wild. Currently supported release series are 0.3.5, 0.4.1, 0.4.2, and 0.4.3 (alpha). If you have not yet upgraded to one of those, the time to do so is soon. For our policy and process for handing security issues, please see: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Securit… best wishes, -- Nick

1 0

SOOC (Same Origin Onion Certificates) discussion tomorrow
by Richard Pospesel 10 Mar '20

10 Mar '20

Hi Everyone, The current UX situation around HTTPS-enabled onionsites is not the best: Tor Browser shows security warning splash screens and icons in scenarios that do not warrant them. On top of that, the only way for your onion site to not show these warnings is by getting an EV cert which is only practical if you have the $$$. alecmuffet's SOOC proposal ( https://github.com/alecmuffett/onion-dv-certificate-proposal/blob/master/te… ) does seem to fix these UX problems for us. Though the proposal is still incomplete, we can make forward progress with a prototype implementation as the certificate specification is fully fleshed out. We'll be talking about this tomorrow during the Sponsor 27 meeting tomorrow on Tuesday 10th May @ 1500UTC in #tor-meeting. Please feel free to come by if you wish to discuss. best, -Richard

2 1

Walking Onions status update: week 1 notes
by Nick Mathewson 08 Mar '20

08 Mar '20

Hi! On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. If you haven't read it, Walking Onions is a set of ideas meant to transition Tor away from its current dependency on a directory system, to improve scaling in the long term, and performance of low-bandwidth clients in the short term. I'm going assume in the rest of my email that you already have a decent idea of how Tor works, and of the Walking Onions proposal. If you need a Walking Onions refresher, see proposal 300 [PROP300] and the whitepaper that Chelsea Komlo, Ian Goldberg, and I have been working on [WHITEPAPER]. The current scope for this work is to try to solve all the open design questions for Walking Onions, and to write a set of specifications we could use to build the system. I'm aiming to have the initial versions of these specs wrapped up in April. Rather than dumping everything onto the mailing list at once, I'm going to try working in the open, sending weekly updates about my work. Design and specification are a lot of fun for me; I hope you'll have fun too reading along. This past week, I started by writing: - An outline of the set of specifications we'd need to have before we could implement walking onions. (OUTLINE0.txt) - A list of areas of uncertainty that we need to solve before or during writing the specs. (UNCERTAINTY.txt) - A list of related problems that we might wind up solving along with the walking onion specs (OPPORTUNITIES.txt) - A rough grouping of initial tasks, which should eventually turn into a schedule. (PLAN.txt) You can find all these documents in a working git repository [GITREPO] that I've been making for this purpose, if you like. I don't intend for this repository to be a permanent reference or to be necessarily useful for anybody but me: everything important about it should eventually wind up in specifications and on this mailing list. == The biggest area of uncertainty was to pick a meta-format for ENDIVEs and SNIPs. I looked at our existing directory meta-format, along with protobufs, messagepack, and a whole bunch of other systems. I also looked into what kind of efficiency we'd get by just using a hand-tooled non-extensible binary format, so that I could see what kind of overhead we were looking at. The issue here is that I think we don't want to be using a text-based metaformat for SNIPs: we need SNIPs to be as small as possible, and our current text-based directory metaformat only compresses well when we're using huge amounts of it at once. But if we're doing a binary format, we probably shouldn't hand-roll it. Trunnel lets us handle our existing formats safely, but those formats aren't so efficient in practice, and they aren't too standardized. If we use a standardized binary format, then we get encoding, decoding, debugging dumps, optimization, and lots of other stuff "for free". After looking at a pretty huge variety of formats, I think that our best bet is something called "CBOR", standardized as RFC 7049 [CBOR]. It is substantially based on MessagePack, but with a number of improvements. Here are some of CBOR's advantages: + It's optimized first for allowing encoding/decoding to be done with small, simple code -- and second, for having a small binary format. IMO it succeeds at both: the metaformat is simple, and there are decoder implementations in less than 1kb of binary size. + You can think of its data model as a "binary json"; it's very flexible. (yes I know about the other binary jsons) + It's standardized, and there are lots of implementations of it, in lots of different languages. + It has a data definition language [CDDL], which we don't actually need to implement, but which we can use to help write our specs and validate our implementation. + It has a textual encoding for debugging, and a one-way mapping to json. Here are some of its disadvantages: - It doesn't have a built-in object mapping like protobufs does. - It allows multiple encodings for a single piece of data. (The CBOR RFC explains how to make a canonical encoding.) - Our diff story gets slightly more complex when we have a binary format. (I'd be happy to talk about other possibilities, but please make sure you know about the bikeshed first. [BIKESHED]) == Doing diffs from one ENDIVE to the next will be helpful if we want to keep relay bandwidth on the small side. We'll need binary diffs for this. Fortunately, I think that shouldn't be too hard to do with our existing diff code: instead of doing line-by-line diffs, we'll just do CBOR-item-by-item diffs, and encode them in a byte-oriented way. I've got a prototype half-written to make sure this is right. The simplicity of CBOR makes this pretty easy. == There are a few ways to authenticate SNIPs. It's looking like right now the most efficient will be merkle trees, with the root of the tree signed by a threshold signature algorithm like BLS. One of the neat things about merkle trees is that if we are transmitting the leaves and the signature on the root, we don't need to actually transmit the intermediate nodes as part of the ENDIVE. So that's cool, and it will make ENDIVE diffs smaller. One of the not-so-neat things is that the merkle tree paths are a bit long. There are ways to make them shorter: you only need to transmit one digest for each layer, plus a bit to say which hash it is. (I learned about these tricks while working on [WHITEPAPER], thanks to Ian and Chelsea.) One risky thing I've been thinking about is whether we can use a shorter-than-256-bits hash for the merkle tree nodes, if we don't need to worry about collision resistance for the digest functions. The SNIPs are generated by authorities, but they do contain a certain amount of adversary-created material. Randomized hashes with an unpredictable key might help, particularly for intermediate nodes on the tree? [XMSS] has some ideas here. (Suggested by Jack Lloyd) == In the coming week, I'm hoping to finish up making my draft schedule, start doing research as needed to handle more open questions, and write an initial spec for the ENDIVE and SNIP formats. [BIKESHED] "Why Should I Care What Color the Bikeshed Is?" http://bikeshed.com/ [CBOR] RFC 7049: "Concise Binary Object Representation (CBOR)" https://tools.ietf.org/html/rfc7049 [CDDL] RFC 8610: "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures" https://tools.ietf.org/html/rfc8610 [GITREPO] https://github.com/nmathewson/walking-onions-wip [PROP300] "Walking Onions: Scaling and Saving Bandwidth" https://gitweb.torproject.org/torspec.git/plain/proposals/300-walking-onion… [WHITEPAPER] "Walking Onions: Scaling Anonymity Networks while Protecting Users" https://crysp.uwaterloo.ca/software/walkingonions/ [XMSS] RFC 8391: "XMSS: eXtended Merkle Signature Scheme". https://tools.ietf.org/html/rfc8391

1 0

Regarding GSoC 2020
by Aman Rastogi 05 Mar '20

05 Mar '20

Respected Developers, I am Aman, GSoC 2020 aspirant, and i would like to work for Torr. I am interested in "Privacy Friendly Web" and "OONI Explorer Advanced Search". Can I get some guidance, please? Thanking you, and sorry for such an informal mail. -- Aman

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev