- tor-dev - lists.torproject.org

Enhancement for Tor 0.3.4.x
by David Goulet 19 Feb '18

19 Feb '18

Hello everone! As an effort to better organize our 0.3.4.x release for which the merge window opens in 3 days (Feb 15th, 2018), we need to identify the enhancement(s) that we want so we can better prioritize the development during the merge window timeframe (3 months). Each feature should have its ticket marked for 0.3.4 milestone and with an Owner set so we know who is "leading" that effort. It doesn't have to be the person who code the whole thing but should be a good point of contact to start with (and it can change over time as well). It is possible that an enhancement can have more than one ticket so in this case, please make a "parent" ticket that explains the whole thing and child tickets assigned to it. The network team just had its weekly meeting and if I recall correctly, these enhancement should be planned for 0.3.4 (please the people who works on this, can you tell us the tickets and make sure they are up to date?) - Privcount (prop#280) - large CREATE cells (prop#249) If you plan to do a set of patches for a feature or enhancement, please do submit it on this thread and make sure a proper ticket exists with an Owner. Also, if you just want something in 0.3.4 as enhancement but might not work on it, it is also OK to propose it (ticket and all) but please lets try as much as possible to keep that thread focused on doable work with tickets and not just a "dump all my tor wishes" :). For this, open a ticket :). Big thanks everyone! David -- 1xYrq8XhE25CKCQqvcX/cqKg04v1HthMMM3PwaRqqdU=

5 6

Cross compiling Tor for Windows
by Alexander Færøy 19 Feb '18

19 Feb '18

Hello. I have recently been trying to wrap my head around some of the Windows/Unix I/O code of Tor to do an API experiment with the PT subsystem. While working on this I remembered that Nick did a cool trick during the development of the consensus diff feature where he managed to reproduce a Windows only test case failure with Wine on Linux using a cross compiled version of Tor that he build on Linux as well. I wanted to try out if I was able to make use of this "trick" as well and decided to write a short guide for setting up the cross compilation environment for Tor on Debian. The guide is published at https://github.com/ahf/tor-win32 The tor-win32 repository also contains a Makefile that will fetch and cross compile a minimal set of dependencies of Tor (OpenSSL, Libevent, and zlib) and Tor itself to the point where you should be able to run both tor.exe and the test suites in Wine. I hope this might be useful to someone else if they find themselves having a need to quickly try something out on "Windows" without actually having to install it -- remember though that this is not the same as running Tor and the test suites on an actual Windows machine, which is what many of our users are using! :-) Cheers, Alex. -- Alexander Færøy

1 0

monitoring significant drops of flags in dirauth votes
by nusenu 17 Feb '18

17 Feb '18

Hi Damian and Tom, Roger discovered that dannenberg did not include any exit flags in certain votes anymore [1]. It would be great if we would detect and notify about such events in the future. I see two places where this could be added: DocTor: a new check that alerts on events where a certain dir auth does either no longer include certain flags (guard, hsdir, exit, ..) at all or better: if the amount of relays with a certain flag significantly dropped by xx % from one vote to the next. consenus-health graphs: we have nice graphs per dirauth and bwauth, if we would have per-dirauthvote-per-flag (mainly guard, exit, hsdir - we have already running) graphs as well we could spot such events (and even trends) better. (btw: what caused there recent flat-line in graphs on 2018-02-03 - 2018-02-05) What do you think? thanks for considering it, nusenu [1] https://lists.torproject.org/pipermail/tor-relays/2018-February/014480.html -- https://mastodon.social/@nusenu twitter: @nusenu_

5 12

Atlas is not that friendly to Web Archive
by Leonid Evdokimov 13 Feb '18

13 Feb '18

Hello! I've recently found out that new Atlas re-design is not that friendly to web archive. http://archive.li/ can't properly detect "page loaded" event that leads to capturing "loading" page[%]. Moreover, https://web.archive.org/ can't capture #-based links at all, as far as I see. [%] https://archive.li/https://atlas.torproject.org/%23details/5C3B8FB35A13C508… Ability to archive atlas pages is kinda nice to be able to "cite" some relay status in some specific date as Atlas has no it's own time machine and information about relay is purged in a few days after relay going down. https://archive.li/RzGpJ is better than https://archive.li/JGQRW :-) I'm not a skilled frontend developer, but maybe trading some Time-to-DOM making JS loading and onionoo.tpo request synchronous should be enough to make website friendly for that sort of crawlers... But it's unclear to me if T2DOM is valuable KPI for Atlas or not :) What do you think? -- WBRBW, Leonid Evdokimov, xmpp:leon@darkk.net.ru http://darkk.net.ru tel:+79816800702 PGP: 6691 DE6B 4CCD C1C1 76A0 0D4A E1F2 A980 7F50 FAB2

2 1

Proposal: Authenticating sendme cells to mitigate bandwidth attacks
by Roger Dingledine 13 Feb '18

13 Feb '18

Filename: xxx-authenticated-sendmes.txt Title: Authenticating sendme cells to mitigate bandwidth attacks Author: Rob Jansen, Roger Dingledine Created: 2016-12-01 Status: open 1. Overview and Motivation In Rob's "Sniper attack", a malicious Tor client builds a circuit, fetches a large file from some website, and then refuses to read any of the cells from the entry guard, yet sends "sendme" (flow control acknowledgement) cells down the circuit to encourage the exit relay to keep sending more cells. Eventually enough cells queue at the entry guard that it runs out of memory and exits [0, 1]. We resolved the "runs out of memory and exits" part of the attack with our Out-Of-Memory (OOM) manager introduced in Tor 0.2.4.18-rc. But the earlier part remains unresolved: a malicious client can launch an asymmetric bandwidth attack by creating circuits and streams and sending a small number of sendme cells on each to cause the target relay to receive a large number of data cells. This attack could be used for general mischief in the network (e.g., consume Tor network bandwidth resources or prevent access to relays), and it could probably also be leveraged to harm anonymity a la the "congestion attack" designs [2, 3]. This proposal describes a way to verify that the client has seen all of the cells that its sendme cell is acknowledging, based on the authenticated sendmes design from [1]. 2. Sniper Attack Variations There are some variations on the attack involving the number and length of the circuits and the number of Tor clients used. We explain them here to help understand which of them this proposal attempts to defend against. We compare the efficiency of these attacks in terms of the number of cells transferred by the adversary and by the network, where receiving and sending a cell counts as two transfers of that cell. 2.1 Single Circuit, without Sendmes The simplest attack is where the adversary starts a single Tor client, creates one circuit and two streams to some website, and stops reading from the TCP connection to the entry guard. The adversary gets 1000 "attack" cells "for free" (until the stream and circuit windows close). The attack data cells are both received and sent by the exit and the middle, while being received and queued by the guard. Adversary: 6 transfers to create the circuit 2 to begin the two exit connections 2 to send the two GET requests --- 10 total Network: 18 transfers to create the circuit 22 to begin the two exit connections (assumes two for the exit TCP connect) 12 to send the two GET requests to the website 5000 for requested data (until the stream and circuit windows close) --- 5052 total 2.2 Single Circuit, with Sendmes A slightly more complex version of the attack in 2.1 is where the adversary continues to send sendme cells to the guard (toward the exit), and then gets another 100 attack data cells sent across the network for every three additional exitward sendme cells that it sends (two stream-level sendmes and one circuit-level sendme). The adversary also gets another three clientward sendme cells sent by the exit for every 100 exitward sendme cells it sends. If the adversary sends N sendmes, then we have: Adversary: 10 for circuit and stream setup N for circuit and stream sendmes --- 10+N Network: 5052 for circuit and stream setup and initial depletion of circuit windows N*100/3*5 for transferring additional data cells from the website N*3/100*4 for transferring sendmes from exit to client --- 5052 + N*166.79 It is important to note that once the adversary stops reading from the guard, it will no longer get feedback on the speed at which the data cells are able to be transferred through the circuit from the exit to the guard. It needs to approximate when it should send sendmes to the exit; if too many sendmes are sent such that the circuit window would open farther than 1000 cells (500 for streams), then the circuit may be closed by the exit. In practice, the adversary could take measurements during the circuit setup process and use them to estimate a conservative sendme sending rate. 2.3 Multiple Circuits The adversary could parallelize the above attacks using multiple circuits. Because the adversary needs to stop reading from the TCP connection to the guard, they would need to do a pre-attack setup phase during which they construct the attack circuits. Then, they would stop reading from the guard and send all of the GET requests across all of the circuits they created. The number of cells from 2.1 and 2.2 would then be multiplied by the number of circuits C that the adversary is able to build and sustain during the attack. 2.4 Multiple Guards The adversary could use the "UseEntryGuards 0" torrc option, or build custom circuits with stem to parallelize the attack across multiple guard nodes. This would slightly increase the bandwidth usage of the adversary, since it would be creating additional TCP connections to guard nodes. 2.5 Multiple Clients The adversary could run multiple attack clients, each of which would choose its own guard. This would slightly increase the bandwidth usage of the adversary, since it would be creating additional TCP connections to guard nodes and would also be downloading directory info, creating testing circuits, etc. 2.6 Short Two-hop Circuits If the adversary uses two-hop circuits, there is less overhead involved with the circuit setup process. Adversary: 4 transfers to create the circuit 2 to begin the two exit connections 2 to send the two GET requests --- 8 Network: 8 transfers to create the circuit 14 to begin the two exit connections (assumes two for the exit TCP connect) 8 to send the two GET requests to the website 5000 for requested data (until the stream and circuit windows close) --- 5030 2.7 Long >3-hop Circuits The adversary could use a circuit longer than three hops to cause more bandwidth usage across the network. Let's use an 8 hop circuit as an example. Adversary: 16 transfers to create the circuit 2 to begin the two exit connections 2 to send the two GET requests --- 20 Network: 128 transfers to create the circuit 62 to begin the two exit connections (assumes two for the exit TCP connect) 32 to send the two GET requests to the website 15000 for requested data (until the stream and circuit windows close) --- 15222 The adversary could also target a specific relay, and use it multiple times as part of the long circuit, e.g., as hop 1, 4, and 7. Target: 54 transfers to create the circuit 22 to begin the two exit connections (assumes two for the exit TCP connect) 12 to send the two GET requests to the website 5000 for requested data (until the stream and circuit windows close) --- 5088 3. Design This proposal aims to defend against the versions of the attack that utilize sendme cells without reading. It does not attempt to handle the case of multiple circuits per guard, or try to restrict the number of guards used by a client, or prevent a sybil attack across multiple client instances. The proposal involves three components: first, the client needs to add a token to the sendme payload, to prove that it knows the contents of the cells that it has received. Second, the exit relay needs to verify this token. Third, to resolve the case where the client already knows the contents of the file so it only pretends to read the cells, the exit relay needs to be able to add unexpected randomness to the circuit. (Note: this proposal talks about clients and exit relays, but since sendmes go in both directions, both sides of the circuit should do these changes.) 3.1. Changing the sendme payload to prove receipt of cells In short: clients put the latest received relay cell digest in the payload of their circuit-level sendme cells. Each relay cell header includes a 4-byte digest which represents the rolling hash of all bytes received on that circuit. So knowledge of that digest is an indication that you've seen the bytes that go into it. We pick circuit-level sendme cells, as opposed to stream-level sendme cells, because we think modifying just circuit-level sendmes is sufficient to accomplish the properties we need, and modifying just stream-level sendmes is not sufficient: a client could send a bunch of begin cells and fake their circuit-level sendmes, but never send any stream-level sendmes, attracting 500*n queued cells to the entry guard for the n streams that it opens. Which digest should the client put in the sendme payload? Right now circuit-level sendmes are sent whenever one window worth of relay cells (100) has arrived. So the client should use the digest from the cell that triggers the sendme. How shall we version the sendme payload so we can change the format of it later? Right now sendme payloads are empty. Here's a simple design: we use five bytes in the payload, where the first byte indicates the sendme payload version (0 in the original design, and 1 once we've implemented this proposal), and the rest of the payload is formatted based on the payload version number: in this case, it's simply the four bytes of digest. Is there a better way to version the payload, e.g. a way that is already standard in other parts of the design, so we aren't adding a new paint color to keep track of on the bike shed? 3.2. Verifying the sendme payload In the current Tor, the exit relay keeps no memory of the cells it has sent down the circuit, so it won't be in a position to verify the digest that it gets back. But fortunately, the exit relay can count also, so it knows which cell is going to trigger the sendme response. Each circuit can have at most 10 sendmes worth of data outstanding. So the exit relay will keep a per-circuit fifo queue of the digests from the appropriate cells, and when a new sendme arrives, it pulls off the next digest in line, and verifies that it matches. If a sendme payload has a payload version of 1 yet its digest doesn't match the expected digest, or if the sendme payload has an unexpected payload version (see below about deployment phases), the exit relay must tear down the circuit. (If we later find that we need to introduce a newer payload version in an incompatible way, we would do that by bumping the circuit protocol version.) 3.3. Making sure there are enough unpredictable bytes in the circuit So far, the design as described fails to a very simple attacker: the client fetches a file whose contents it already knows, and it uses that knowledge to calculate the correct digests and fake its sendmes just like in the original attack. The fix is that the exit relay needs to be able to add some randomness into its cells. It can add this randomness, in a way that's completely orthogonal to the rest of this design, simply by choosing one relay cell every so often and not using the entire relay cell payload for actual data (i.e. using a Length field of less than 498), and putting some random bytes in the remainder of the payload. How many random bytes should the exit relay use, and how often should it use them? There is a tradeoff between security when under attack, and efficiency when not under attack. We think 1 byte of randomness every 1000 cells is a good starting plan, and we can always improve it later without needing to change any of the rest of this design. (Note that the spec currently says "The remainder of the payload is padded with NUL bytes." We think "is" doesn't mean MUST, so we should just be sure to update that part of the spec to reflect our new plans here.) 4. Deployment Plan In phase one, both sides begin remembering their expected digests, and they learn how to parse sendme payloads. When they receive a sendme with payload version 1, they verify its digest and tear down the circuit if it's wrong. But they continue to send and accept payload version 0 sendmes. In phase two, we flip a switch in the consensus, and everybody starts sending payload version 1 sendmes. Payload version 0 sendmes are still accepted. In phase three, we flip a different switch in the consensus, and everybody starts refusing payload version 0 sendmes. (It has to be two separate switches, not one unified one, because otherwise we'd have a race where relays learn about the update before clients know to start the new behavior.) We'll want to do a bunch of testing in chutney before flipping the switches in the real network: I've long suspected we still have bugs in our sendme timing, and this proposal might expose some of them. Alas, this deployment plan leaves a pretty large window until relays are protected from attack. It's not all bad news though, since we could flip the switches earlier than intended if we encounter a network-wide attack. 5. Security Discussion Does our design enable any new adversarial capabilities? An adversarial middle relay could attempt to trick the exit into killing an otherwise valid circuit. An adversarial relay can already kill a circuit, but here it could make it appear that the circuit was killed for a legitimate reason (invalid or missing sendme), and make someone else (the exit) do the killing. There are two ways it might do this: by trying to make a valid sendme appear invalid; and by blocking the delivery of a valid sendme. Both of these depend on the ability for the adversary to guess which exitward cell is a sendme cell, which it could do by counting clientward cells. * Making a valid sendme appear invalid A malicious middle could stomp bits in the exitward sendme so that the exit sendme validation fails. However, bit stomping would be detected at the protocol layer orthogonal to this design, and unrecognized exitward cells would currently cause the circuit to be torn down. Therefore, this attack has the same end result as blocking the delivery of a valid sendme. (Note that, currently, clientward unrecognized cells are dropped but the circuit is not torn down.) * Blocking delivery of a valid sendme A malicious middle could simply drop a exitward sendme, so that the exit is unable to verify the digest in the sendme payload. The following exitward sendme cell would then be misaligned with the sendme that the exit is expecting to verify. The exit would kill the circuit because the client failed to prove it has read all of the clientward cells. The benefits of such an attack over just directly killing the circuit seem low, and we feel that the added benefits of the defense outweigh the risks. 6. Open problems With the proposed defenses in place, an adversary will be unable to successfully use the "continue sending sendmes" part of these attacks. But this proposal won't resolve the "build up many circuits over time, and then use them to attack all at once" issue, nor will it stop sybil attacks like if an attacker makes many parallel connections to a single target relay, or reaches out to many guards in parallel. We spent a while trying to figure out if we can enforce some upper bound on how many circuits a given connection is allowed to have open at once, to limit every connection's potential for launching a bandwidth attack. But there are plausible situations where well-behaving clients accumulate many circuits over time: Ricochet clients with many friends, popular onion services, or even Tor Browser users with a bunch of tabs open. Even though a per-conn circuit limit would produce many false positives, it might still be useful to have it deployed and available as a consensus parameter, as another tool for combatting a wide-scale attack on the network: a parameter to limit the total number of open circuits per conn (viewing each open circuit as a threat) would complement the current work in #24902 to rate limit circuit creates per client address. But we think the threat of parallel attacks might be best handled by teaching relays to react to actual attacks, like we've done in #24902: we should teach Tor relays to recognize when somebody is *doing* this attack on them, and to squeeze down or outright block the client IP addresses that have tried it recently. An alternative direction would be to await research ideas on how guards might coordinate to defend against attacks while still preserving user privacy. In summary, we think authenticating the sendme cells is a useful building block for these future solutions, and it can be (and should be) done orthogonally to whatever sybil defenses we pick later. 7. References [0] https://blog.torproject.org/blog/new-tor-denial-service-attacks-and-defenses [1] https://www.freehaven.net/anonbib/#sniper14 [2] https://www.freehaven.net/anonbib/#torta05 [3] https://www.freehaven.net/anonbib/#congestion-longpaths

3 2

Fwd: contribute to gsoc 2018 ideas
by Ankur Goyal 10 Feb '18

10 Feb '18

Hello, I am going to participate in gsoc 2018 and i want to contribute in your idea,can anyone provide me some more pointers about how to move forward. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

1 0

Re: [tor-dev] Starting with contributing to Anonymous Local Count Statistics.
by Aruna Maurya 09 Feb '18

09 Feb '18

Hey! What is the current status of the project, how much work has been done and where can I pick up from? On Fri, Feb 2, 2018 at 3:04 PM, Aruna Maurya <aruna.maurya12(a)gmail.com> wrote: > > ---------- Forwarded message ---------- > From: George Kadianakis <desnacked(a)riseup.net> > Date: Wed, Jan 31, 2018 at 6:32 PM > Subject: Re: [tor-dev] Starting with contributing to Anonymous Local Count > Statistics. > To: Aruna Maurya <aruna.maurya12(a)gmail.com>, tor-dev(a)lists.torproject.org > > > Aruna Maurya <aruna.maurya12(a)gmail.com> writes: > > > [ text/plain ] > > Hey! > > > > I was going through the Tor Volunteer page and came across the Anonymous > > local count statistics project. As a student it would be a great starting > > point and an even bigger opportunity to get a chance to collaborate and > > learn in the process. > > > > I would like to contribute to it, and would love to start as soon as > > possible. It would be great if someone could guide me through. > > > > Hello Aruna, > > thanks for reaching out. > > I also find this project interesting. I'd like to help you but my time > is quite limited lately. > > What would you like guidance with? > > With regards to design, I suggest you take a look at the last comments > of this trac ticket: https://trac.torproject.org/pr > ojects/tor/ticket/7532#comment:22 > Particularly it seems like the PCSA algorithm might be a reasonable way > forward. > > With regards to coding, I suggest you familiarize yourself with the Tor > codebase. Some specific places to look at would be the way that Tor > currently counts users. For example, see geoip_note_client_seen() and > its callers, for when bridges register new clients to their stats > subsystem. Also check geoip_format_bridge_stats() for when bridges > finally report those stats. > > Let us know if you have any specific questions! > > Cheers! > > > > -- > Regards, > Aruna Maurya, > CSE,B.tech, > Blog <https://themindreserves.wordpress.com/> | Medium > <https://medium.com/@arunamaurya> > > -- Regards, Aruna Maurya, CSE,B.tech, Blog <https://themindreserves.wordpress.com/> | Medium <https://medium.com/@arunamaurya>

4 6

Stem ORPort protocol support
by Damian Johnson 07 Feb '18

07 Feb '18

Hi all. Over the last few months Tim and I have been collaborating on Python support for the ORPort protocol. With it you can download descriptors without a DirPort, and possibly fancier things in the future like full circuit construction. Tim put together a wonderful proof of concept called Endosome... https://github.com/teor2345/endosome ... and I just finished integrating it into Stem... https://gitweb.torproject.org/stem.git/tree/stem/client/__init__.py With stem.client you can now download descriptors... import stem.client with stem.client.Relay.connect('127.0.0.1', 12345, [3]) as relay: circ = relay.create_circuit() circ.send('RELAY_BEGIN_DIR', stream_id = 1) desc = circ.send('RELAY_DATA', 'GET /tor/server/authority HTTP/1.0\r\n\r\n', stream_id = 1).data circ.close() print(desc) When run this looks like... % python demo.py HTTP/1.0 200 OK Date: Wed, 07 Feb 2018 18:42:41 GMT Content-Type: text/plain Content-Encoding: identity Expires: Fri, 09 Feb 2018 18:42:41 GMT router Unnamed 97.113.177.53 12345 0 23456 identity-ed25519 -----BEGIN ED25519 CERT----- AQQABm/qAazUltT1iUUbIMw8VNNhGb50FDHKJz6S94FLQNxL0LObAQAgBAAapbO9 iLFD0l9SEiEMFQWIT2VnbLyCZKvbrxTs5ULC1l1hQPoui6Y/lEd3yjrQhIs/vl6R 1S6FbwSFDmiXOzq47mFrse4C71ht3TpLOD0F3wiyjWtsqU1k7iPmmpejUgs= -----END ED25519 CERT----- master-key-ed25519 GqWzvYixQ9JfUhIhDBUFiE I'd like to emphasize this is still very alpha. The API isn't set in stone and there's no doubt quite a few rough edges. However, I wanted the list to be aware just in case anyone would care to build on it. I plan to draw a line at 'download descriptors through ORPorts' but I'd be delighted to help others if there's more ambitious directions they'd care to go (potentially all the way up to a Python Tor client, similar to Orchid). Now that we've reached this milestone I'm taking a break to focus on Stem support for v3 Onion Services for a bit. However, when I come back the next things on my dance card are... a. Support ORPort downloads in the stem.descriptor.remote module. b. More integ tests so Stem can be used as a tool for testing tor's ORPort. c. Give more thought to the API we'd like to vend. d. Brainstorm a GSoC project idea that expands these capabilities. Cheers! -Damian

3 4

[release] Onionoo 5.0-1.10.1
by iwakeh 07 Feb '18

07 Feb '18

Hi there! A fix for the most recent Oninonoo version is available at: https://dist.torproject.org/onionoo/5.0-1.10.1/ The changes [1] concern also the resolution of graph data: three months weights and bandwidth data are now provided in 24 hours detail. Deployment of the new version 5.0-1.10.1 finished successfully for all onionoo.torproject.org instances. The changes for bandwidth data might take another day to be visible. So, we need to have a little patience. Please direct comments and questions to the metrics-team mailing list [2]. Cheers, iwakeh [1] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md?id=a7418b7c4d1… [2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

[release] Onionoo 5.0-1.10.0
by iwakeh 07 Feb '18

07 Feb '18

Hi there! the new Oninonoo version is available at: https://dist.torproject.org/onionoo/5.0-1.10.0/ All changes [1] this time are supposed to make writing of bandwidth, clients, uptime, and weights documents independent of system time, which is a step toward making Onionoo instances serve the same data where possible. Other steps remain and will be addressed in future [2]. Deployment of version 5.0-1.10.0 will start at 13:00 UTC for all onionoo.torproject.org instances. It might cause a a few hours without updates as we need to make additional backups. For those interested, I will post a deployment finished message to metrics-team mailing list [3] approximately during UTC evening. Please direct comments and questions to the metrics-team mailing list [3]. Cheers, iwakeh [1] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md [2] https://trac.torproject.org/projects/tor/ticket/25002 [3] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0