- tor-dev - lists.torproject.org

Re: [tor-dev] [TOR+QUIC] Progress Update
by Xiaofan Li 05 Apr '16

05 Apr '16

Tim: Thank you for the detailed email. I've also included tor-dev on CC. Anyone is welcome to comment. To reply to your concerns: 1. All the documentation we have is basically in email correspondences as well as in the generic simple-quic codebase here: https://github.com/kku1993/simple-quic. The quicksock_t interface is in the above repo under /quicsock. There isn't much documentation because: a) our designs are still under test and large portions are not stable yet, b) we haven't gotten much time to properly document designs and changes to the Tor codebase. But the emails should be somewhat clear on some key design aspects. 2. About providing an alternative for QUIC connections: - Great idea about keeping both type of sockets in connection. I think this will give us a cleaner design. - Exit relay will speak normal TCP. This is part of our design from the start but we haven't gotten around to look into the exit connection code. - Connections to the directory authorities should be normal TCP. - While compatibility with the current system is very important, our main goal is to compare performances of the OR connections with TCP vs. QUIC. So we haven't spent much effort making things beautiful yet. After all, it is possible that QUIC is a bad idea for Tor.. 3. I was looking at chutney the other day but I couldn't find much documentation on how it works. I will play with it more and then I might ask for more help. If anyone else is expert on the SSL part of the codebase, please provide some insights about the security aspect of the design, specifically: "About SSL, we want to adopt the design where QUIC handles SSL by not using the ssl_buffer in TOR code. Any concerns at all?" (not using read_to_buf_tls() and other similar ssl functions) Another concern is the assumption on TOR_SOCKET_T in the Tor codebase. In particular, there are many call sites of fcntl, getsockname, setsockopt etc made directly on this object, assuming that it is a UNIX socket. For compatibility, after replacing UNIX socket with quicsock_t, we would use "qs_get_fd" for all those low-level calls. Note that quicsock_t holds a UDP socket internally. Without the testing system ready, we don't know the effect of these calls on a UDP socket instead of the normal TCP socket. Any other feedback also welcomed! Li. On Mon, Apr 4, 2016 at 10:43 PM, Tim Wilson-Brown - teor <teor2345(a)gmail.com > wrote: > Hi Li, > > If you're ok with emailing the tor-dev mailing list, you will likely get > better answers to some of these questions there. > All of these questions could benefit from the wider Tor developer > community. > When I know I can't fully answer a question, I'll say so below. > > Please feel free to forward my response to tor-dev. > > > On 5 Apr 2016, at 05:38, Xiaofan Li <xli2(a)andrew.cmu.edu> wrote: > > > > Hi Tim: > > I think maybe now is a good time to bother you again. In the following > email, we outlined some current progress on the TOR+QUIC project. This > weekend we booted up our first TOR node running QUIC (then of course it > failed to contact the directory authorities and hanged). We hope to move on > to testing our design while clean up some uncertain "loose-ends" in the > integration process, with your help. In particular: > > • We decided to implement a QUIC interface similar to the UNIX > socket interface. We have replaced TOR_SOCKET_T with our quicksock_t. Any > issues about this design? > > I'm not particularly familiar with this part of the tor codebase, you > might get better answers from tor-dev. > Also, without knowing the definition of quicksock_t, I can't say much more. > > Overall, I think your design has two major flaws: > * you need to provide QUIC as an alternative, not a replacement, because > Exits need to speak TCP, and so might Clients and Onion Services > * you've chosen to modify all tor sockets to speak QUIC, rather than > replacing SSL over TCP with QUIC > > Why not put your quicksock_t in connection_t (in or.h), and then choose > whether to use QUIC or a plain socket based on whether you're using an OR > connection (or_connection_t) or not? > You'd also need some logic in OR listener connections > (listener_connection_t) that creates a QUIC or plain socket from the > listener depending on whether it's an OR listener or not. > > See "Kinds of Connections" in > https://gitweb.torproject.org/user/nickm/torguts.git/tree/02-dataflow.md > And > https://gitweb.torproject.org/user/nickm/torguts.git/plain/diagrams/02/02-c… > And maybe > https://gitweb.torproject.org/user/nickm/torguts.git/plain/diagrams/02/02-d… > > Here are my concerns in detail: > > I think you need a design that enables QUIC over UDP as an alternative to > SSL over TCP sockets, like the way Tor's IPv6 support provides IPv6 as an > alternative to IPv4. (Of course, IPv6 is not the same level of the protocol > stack as QUIC.) > > If you completely replace TCP with QUIC, then Exits will also speak QUIC > to sites on the wider Internet. (As you describe below.) > Also, relays with speak QUIC to the directory authorities when downloading > descriptors, rather than using HTTP over TCP. > And you'll force the browser and Tor controllers to speak QUIC to the Tor > Client. > And Tor Onion Services to speak QUIC to the backend servers they connect > to. > (I think some parts of Tor's DNS might also be affected, but it depends on > the level of abstraction they're at.) > > This is unlikely to be what you want. And there could also be other > impacts on connections that should stay TCP. > > > • About SSL, we want to adopt the design where QUIC handles SSL by > not using the ssl_buffer in TOR code. Any concerns at all? > > You'll need to ask tor-dev for this one, I'm not familiar with this code > at all. > > > • During integration, I found many instances where TOR assumes > TOR_SOCKET_T is INT. Why can TOR make this assumption? Is the type > TOR_SOCKET_T only created to accommodate windows dev? > > I think the assumption was already there when Windows support was added. > It would appear that tor_socket_t exists for Windows compatibility. > > From the definition of tor_socket_t in compat.h: > > /* XXX Actually, this should arguably be SOCKET; we use intptr_t here so > that > * any inadvertent checks for the socket being <= 0 or > 0 will probably > * still work. */ > > > • For testing, what's the most correct (and easy enough to do in a > few weeks) way to simulate the TOR network? Right now I'm thinking using > the microdesc cache to avoid contacting any directory authorities. I will > create a cluster of about 5 nodes in server mode, all of which only know > the other nodes running QUIC. Do you think this will work? > > Tor has a TestingTorNetwork config option, which modifies a significant > number of settings to enable local testing. > If you define your own authorities, they can provide a consensus to your > clients. > This could be much more reliable than using the microdesc cache. > (Please read the TestingTorNetwork entry in the tor manual for more > details.) > > You may find this easiest to set up using chutney, which is how we test a > full tor network on a local machine. > Chutney will configure authorities, relays, and clients for you, then > launch the network. > The easiest way to set it up is to: > 1. Clone https://gitweb.torproject.org/chutney.git > 2. Clone your tor repository next to the chutney clone > 3. Run tor's ./configure > 4. Run tor's make test-network > > If this doesn't work, you might find it easiest to start with the chutney > "basic" template: > 5. Run src/test/test-network.sh --flavor basic > > If you start by running chutney on tor master, you'll get an idea of how > it works. > Then you can see what breaks when you try it with your QUIC-based tor. > > > • Anything else bad about the implementation? > > I haven't seen the implementation. > Have you sent me a link, or did you want me to give advice based on the > design issues you describe in the email you forwarded? > > As far as I can see, your major issues are getting a Tor test network > running (I strongly recommend chutney), and dealing with Exits needing > QUIC, TCP, and DNS/UDP. If you want interoperability with non-QUIC browsers > and servers, you'll need to work out similar issues with Clients and Onion > Services as well. > > Tim > > > Thank you! > > Li. > > > > > > > > > > ---------- Forwarded message ---------- > > From: Xiaofan Li <xli2(a)andrew.cmu.edu> > > Date: Mon, Apr 4, 2016 at 3:20 PM > > Subject: [TOR+QUIC] Progress Update > > To: Mashael AlSabah <malsabah(a)gmail.com> > > Cc: Srinivasan Seshan <srini(a)cs.cmu.edu>, Kevin Ku <kku(a)andrew.cmu.edu> > > > > > > Hi all, > > Before the project meetings, we want to debrief you on the recent > progress of the project so that our discussions can be more efficient. In > general, this week we made reasonable progress and now we are in the > debugging/testing phase. Particularly, we: > > • Designed and implemented the interaction between QUIC and the > libevent system inside TOR. > > • Integrating QUIC with TOR causes mismatches of > abstractions (see below). We worked hard to come up with a clean design > while not blocking development. > > • We have to pass the event_base to QUIC for it to > register an alarm callback that is required by QUIC. > > • libevent -> TOR-register callbacks -> QUIC interface -> > data -> QUIC processing -> only application data gets passed up to TOR > > • Minimum code change in TOR: we still use all the > callback mechanisms (timer + signal + async) in TOR but we modify those > callbacks. > > • Modified code paths inside TOR to accommodate QUIC. > > • We turned off Tor's SSL code path in the read/write > callbacks. We should probably also turn off the handshake inside TOR as > well. > > • TOR has a lot of implementation shortcut that make > assumptions about the TOR_SOCKET_T type. We cleaned this up. > > • Some remaining issues about direct low-level calls on > the socket (e.g. fcntl or setsockopt). The effects of these issues are > unclear right now. > > • Small hacks to make sure things ran. > > • We hardcoded the certificate and private key used by > QUIC. > > • QUIC is compiled into a shared library and it seems to > be working well. (Don't know about the SSL things right now.) > > • Tweaked the our QUIC interface to allow generality and smooth > integration with TOR. > > Some challenging problems we encountered: > > • Mismatched abstractions: This is obviously the most challenging > implementation problem. We hope our experience can perhaps suggestion some > designs for the ongoing QUIC interface design: > > • SSL/TLS: Fundamentally, QUIC is providing TLS security > on a lower level than what TOR is expecting and this is causing problems. > We should take away the TOR's SSL/TLS code to achieve the benefits of QUIC. > > • Other Implementations: Compared to the UNIX socket type, > our QUIC "socket" provides different interface and functionalities and it > is possible that there are assumptions made by TOR about the socket > interface. For example, the fact that QUIC requires an alarm system is > destroying the abstraction separation between TOR and QUIC. The alarms are > used for path discovery, handshakes, retransmission etc. We believe that > those should be internal to the lower level transport and thus hidden by > QUIC. We are unable to hide them right now because QUIC does not have the > necessary OS support to do so. The performance implication is unclear. > > • Testing: We are in the phase of testing. We are investigating > various testing methods. The key problem is that we have to basically scrap > all the systems that TOR already provides because we speak a different > protocol. Particularly, we need to: > > • Replace or disable the directory authorities. Disabling > is easier because we can hardcode the bootstrapping process and make the OP > contact our QUIC node, but this way we might not have the most accurate > (and comparable) data on the bootstrapping process. > > • We need to set up some number of circuits and > differentiate normal nodes from exit nodes. Right now the exit nodes will > also speak QUIC which is not good. > > • We will probably need 3-5 machines at least. (Some > Amazon AWS credit maybe??) > > Right now, our simple_quic interface is relatively stable and it is > located at: https://github.com/kku1993/simple-quic . It offers a purely > generic interface that does not specifically tie to TOR. Feel free to take > a look! We are thinking about giving it to the QUIC people at Google after > we polish and document it better. > > Kevin, feel free to add anything. > > > > Looking forward to the meetings! > > Li. > >

2 1

Re: [tor-dev] Yawning's CFC, web caching, and PETs
by leep＠sigaint.org 04 Apr '16

04 Apr '16

Check out CENO. Right now it runs Freenet as a back-end for distributed caching but it can gain Tahoe-LAFS support. https://censorship.no/

1 0

Proposal 259: New Guard Selection Behaviour
by George Kadianakis 04 Apr '16

04 Apr '16

Hello, proposal 259 has evolved plenty since the last time it was posted on [tor-dev]. All the technical discussion can be found in various threads, and here is a thread about the proposal itself as it is now. The current proposal is pretty much done, and it's currently being implemented for testing. We imagine that during testing we might have to slightly alter the algorithm and its parameters to improve performance and security. Here are some behaviors that are still unspecified in the current proposal based on our discussion yesterday: - What exactly should be done if a circuit has restrictred requirements? (e.g. it needs a Fast/Stable guard, but our top guard is not Fast/Stable) - How exactly should directory guards work? Should the guard lists be initialized only with directory guards? Or should we just initialize our guard lists from the set of all guards, and just skip non-V2Dir guards whenever we need to make a directory circuit? FWIW, there are currently 2076 guards, out of which 1659 support V2Dir (i.e. they are directory guards). - How should the algorithm work wrt ReachableAddresses? I wonder how the current algorithm work wrt ReachableAddresses and if it's behavior is good. --- Filename: 259-guard-selection.txt Title: New Guard Selection Behaviour Author: Isis Lovecruft, George Kadianakis, [Ola Bini] Created: 2015-10-28 Status: Draft §1. Overview Tor uses entry guards to prevent an attacker who controls some fraction of the network from observing a fraction of every user's traffic. If users chose their entries and exits uniformly at random from the list of servers every time they build a circuit, then an adversary who had (k/N) of the network would deanonymize F=(k/N)^2 of all circuits... and after a given user had built C circuits, the attacker would see them at least once with probability 1-(1-F)^C. With large C, the attacker would get a sample of every user's traffic with probability 1. To prevent this from happening, Tor clients choose a small number of guard nodes (currently 3). These guard nodes are the only nodes that the client will connect to directly. If they are not compromised, the user's paths are not compromised. But attacks remain. Consider an attacker who can run a firewall between a target user and the Tor network, and make many of the guards they don't control appear to be unreachable. Or consider an attacker who can identify a user's guards, and mount denial-of-service attacks on them until the user picks a guard that the attacker controls. In the presence of these attacks, we can't continue to connect to the Tor network unconditionally. Doing so would eventually result in the user choosing a hostile node as their guard, and losing anonymity. This proposal outlines a new entry guard selection algorithm, which addresses the following concerns: - Heuristics and algorithms for determining how and which guard(s) is(/are) chosen should be kept as simple and easy to understand as possible. - Clients in censored regions or who are behind a fascist firewall who connect to the Tor network should not experience any significant disadvantage in terms of reachability or usability. - Tor should make a best attempt at discovering the most appropriate behaviour, with as little user input and configuration as possible. §2. Design Alice, an OP attempting to connect to the Tor network, should undertake the following steps to determine information about the local network and to select (some) appropriate entry guards. In the following scenario, it is assumed that Alice has already obtained a recent, valid, and verifiable consensus document. The algorithm is divided into four components such that the full algorithm is implemented by first invoking START, then repeatedly calling NEXT while adviced it SHOULD_CONTINUE and finally calling END. For an example usage see §A. Appendix. Several components of NEXT can be invoked asynchronously. SHOULD_CONTINUE is used for the algorithm to be able to tell the caller whether we consider the work done or not - this can be used to retry primary guards when we finally are able to connect to a guard after a long network outage, for example. This algorithm keeps track of the unreachability status for guards in state global to the system, so that repeated runs will not have to rediscover unreachability over and over again. However, this state does not need to be persisted permanently - it is purely an optimization. The algorithm expects several arguments to guide its behavior. These will be defined in §2.1. The goal of this algorithm is to strongly prefer connecting to the same guards we have connected to before, while also trying to detect conditions such as a network outage or a network environment that blocks most ports. The way it does this is by keeping track of how many guards we have exposed ourselves to, and if we have connected to too many we will fall back to only retrying the ones we have already tried. The algorithm also decides on sample set that should be persisted - in order to minimize the risk of an attacker forcing enumeration of the whole network by triggering rebuilding of circuits. §2.1. The START algorithm In order to start choosing an entry guard, use the START algorithm. This takes four arguments that can be used to fine tune the workings: USED_GUARDS This is a list that contains all the guards that have been used before by this client. We will prioritize using guards from this list in order to minimize our exposure. The list is expected to be sorted based on priority, where the first entry will have the highest priority. SAMPLED_UTOPIC_GUARDS This is a set that contains all guards that should be considered for connection under utopic conditions. This set should be persisted between runs. It will be filled in by the algorithm if it's empty, or if it contains less than SAMPLE_SET_THRESHOLD guards after winnowing out older guards. It should be filled by using NEXT_BY_BANDWIDTH with UTOPIC_GUARDS as an argument. SAMPLED_DYSTOPIC_GUARDS This is a set that contains all guards that should be considered for connection under dystopic conditions. This set should be persisted between runs. It will be filled in by the algorithm if it's empty, or if it contains less than SAMPLE_SET_THRESHOLD guards after winnowing out older guards. It should be filled by using NEXT_BY_BANDWIDTH with DYSTOPIC_GUARDS as an argument. EXCLUDE_NODES A set of nodes that we should not consider using as a guard. N_PRIMARY_GUARDS The number of guards we should consider our primary guards. These guards will be retried more frequently and will take precedence in most situations. By default the primary guards will be the first N_PRIMARY_GUARDS guards from USED_GUARDS. DIR If this argument is set, we should only consider guards that can be directory guards. If not set, we will consider all guards. The primary work of START is to initialize the state machine depicted in §2.2. The initial state of the machine is defined by: GUARDS This is a set of all guards from the consensus, without EXCLUDE_NODES and potentially filtered if DIR is set. UTOPIC_GUARDS This is a set of all guards to use under utopic conditions. It will primarily be used to fill in SAMPLED_UTOPIC_GUARDS. This set will be initialized to be the same as GUARDS. DYSTOPIC_GUARDS This is a set of all guards to use under dystopic conditions (usually when we are subject to a firewall that restricts the ports we can connect to). It will primarily be used to fill in SAMPLED_DYSTOPIC_GUARDS. This set will be initialized to be the subset of GUARDS that listen to ports that are allowed by dystopic conditions. REMAINING_UTOPIC_GUARDS This is a running set of the utopic guards we have not yet tried to connect to. It should be initialized to be SAMPLED_UTOPIC_GUARDS without USED_GUARDS. REMAINING_DYSTOPIC_GUARDS This is a running set of the dystopic guards we have not yet tried to connect to. It should be initialized to be SAMPLED_DYSTOPIC_GUARDS without USED_GUARDS. STATE A variable that keeps track of which state in the state machine we are currently in. It should be initialized to STATE_PRIMARY_GUARDS. PRIMARY_GUARDS This list keeps track of our primary guards. These are guards that we will prioritize when trying to connect, and will also retry more often in case of failure with other guards. It should be initialized by calling algorithm NEXT_PRIMARY_GUARD repeatedly until PRIMARY_GUARDS contains N_PRIMARY_GUARDS elements. §2.2. The NEXT algorithm The NEXT algorithm is composed of several different possibly flows. The first one is a simple state machine that can transfer between four different states. Every time NEXT is invoked, it will resume at the state where it left off previously. In the course of selecting an entry guard, a new consensus can arrive. When that happens we need to update the data structures used, but nothing else should change. Before jumping in to the state machine, we should first check if it was at least PRIMARY_GUARDS_RETRY_INTERVAL minutes since we tried any of the PRIMARY_GUARDS. If this is the case, and we are not in STATE_PRIMARY_GUARDS, we should save the previous state and set the state to STATE_PRIMARY_GUARDS. §2.2.1. The STATE_PRIMARY_GUARDS state Return each entry in PRIMARY_GUARDS in turn. For each entry, if it was not possible to connect to it and mark the entry as unreachable [XXX defining "was not possible to connect" as "entry is not live" according to current definition of "live entry guard" in tor source code, seems to improve success rate on the flaky network scenario. See: https://github.com/twstrike/tor_guardsim/issues/1#issuecomment-187374942] If all entries have been tried, restore the previous state and go there. If there is no previous state, transition to STATE_TRY_UTOPIC. §2.2.2. The STATE_TRY_UTOPIC state Return each entry in USED_GUARDS that is not in PRIMARY_GUARDS in turn. For each entry, if it was not possible to connect to it and mark the entry as unreachable. Return each entry from REMAINING_UTOPIC_GUARDS using NEXT_BY_BANDWIDTH. For each entry, if it was not possible to connect to it, remove the entry from REMAINING_UTOPIC_GUARDS and mark it as unreachable. If no entries remain in REMAINING_UTOPIC_GUARDS, transition to STATE_TRY_DYSTOPIC. §2.2.3. The STATE_TRY_DYSTOPIC state Return each entry from REMAINING_DYSTOPIC_GUARDS using NEXT_BY_BANDWIDTH. For each entry, if it was not possible to connect to it, remove the entry from REMAINING_DYSTOPIC_GUARDS and mark it as unreachable. If no entries remain in REMAINING_DYSTOPIC_GUARDS, transition to STATE_PRIMARY_GUARDS. §2.2.5. ON_NEW_CONSENSUS First, ensure that all guard profiles are updated with information about whether they were in the newest consensus or not. If a guard is not included in the newest consensus, the guard is considered bad. If any PRIMARY_GUARDS have become bad, remove the guard from PRIMARY_GUARDS. Then ensure that PRIMARY_GUARDS contain N_PRIMARY_GUARDS entries by repeatedly calling NEXT_PRIMARY_GUARD. If any guards in USED_GUARDS have switched from being bad to being non-bad, add it back in the place it should have been in PRIMARY_GUARDS if it had been non-bad when populating PRIMARY_GUARDS. If this results in PRIMARY_GUARDS being larger than N_PRIMARY_GUARDS, truncate the list to be N_PRIMARY_GUARDS entries long. §2.3. The SHOULD_CONTINUE algorithm This algorithm takes as an argument a boolean indicating whether the circuit was successfully built or not. After the caller have tried to build a circuit with a returned guard, they should invoke SHOULD_CONTINUE to understand if the algorithm is finished or not. SHOULD_CONTINUE will always return true if the circuit failed. If the circuit succeeded, SHOULD_CONTINUE will always return false, unless the guard that succeeded was the first guard to succeed after INTERNET_LIKELY_DOWN_INTERVAL minutes - in that case it will set the state to STATE_PRIMARY_GUARDS and return true. §2.4. The END algorithm The goal of this algorithm is simply to make sure that we keep track of successful connections made. This algorithm should be invoked with the guard that was used to correctly set up a circuit. Once invoked, this algorithm will mark the guard as used, and make sure it is in USED_GUARDS, by adding it at the end if it was not there. §2.5. Helper algorithms These algorithms are used in the above algorithms, but have been separated out here in order to make the flow clearer. NEXT_PRIMARY_GUARD - Return the first entry from USED_GUARDS that is not in PRIMARY_GUARDS and that is in the most recent consensus. - If USED_GUARDS is empty, use NEXT_BY_BANDWIDTH with REMAINING_UTOPIC_GUARDS as the argument. NEXT_BY_BANDWIDTH - Takes G as an argument, which should be a set of guards to choose from. - Return a randomly select element from G, weighted by bandwidth. §3. Consensus Parameters, & Configurable Variables This proposal introduces several new parameters that ideally should be set in the consensus but that should also be possible to set or override in the client configuration file. Some of these have proposed values, but for others more simulation and trial needs to happen. PRIMARY_GUARDS_RETRY_INTERVAL In order to make it more likely we connect to a primary guard, we would like to retry the primary guards more often than other types of guards. This parameter controls how many minutes should pass before we consider retrying primary guards again. The proposed value is 3. SAMPLE_SET_THRESHOLD In order to allow us to recognize dystopic situations or a completely unreachable network, we would like to avoid connecting to too many guards before switching modes. We also want to avoid exposing ourselves to too many nodes in a potentially hostile situation. This parameter, expressed as a fraction, determines the number of guards we should keep as the sampled set of the only guards we will consider connecting to. It will be used as a fraction for both the utopic and the dystopic sampled set. If we assume there are 1900 utopic guards and of them there are 500 dystopic guards, a setting of 0.02 means we will have a sample set of 38 utopic guards and 10 dystopic guards. This limits our total exposure. Proposed value is 0.02. INTERNET_LIKELY_DOWN_INTERVAL The number of minutes since we started trying to find an entry guard before we should consider the network down and consider retrying primary guards before using a functioning guard found. Proposed value 5. §4. Security properties and behavior under various conditions Under normal conditions, this algorithm will allow us to quickly connect and use guards we have used before with high likelihood of working. Assuming the first primary guard is reachable and in the consensus, this algorithm will deterministically always return that guard. Under dystopic conditions (when a firewall is in place that blocks all ports except for potentially port 80 and 443), this algorithm will try to connect to 2% of all guards before switching modes to try dystopic guards. Currently, that means trying to connect to circa 40 guards before getting a successful connection. If we assume a connection try will take maximum 10 seconds, that means it will take up to 6 minutes to get a working connection. When the network is completely down, we will try to connect to 2% of all guards plus 2% of all dystopic guards before realizing we are down. This means circa 50 guards tried assuming there are 1900 guards in the network. In terms of exposure, we will connect to a maximum of 2% of all guards plus 2% of all dystopic guards, or 3% of all guards, whichever is lower. If N is the number of guards, and k is the number of guards an attacker controls, that means an attacker would have a probability of 1-(1-(k/N)^2)^(N * 0.03) to have one of their guards selected before we fall back. In real terms, this means an attacker would need to control over 10% of all guards in order to have a larger than 50% chance of controlling a guard for any given client. In addition, since the sampled set changes slowly (the suggestion here is that guards in it expire every month) it is not possible for an attacker to force a connection to an entry guard that isn't already in the users sampled set. §A. Appendix: An example usage In order to clarify how this algorithm is supposed to be used, this pseudo code illustrates the building of a circuit: OPEN_CIRCUIT: context = ALGO_CHOOSE_ENTRY_GUARD_START(used_guards, sampled_utopic_guards=[], sampled_dystopic_guards=[], exclude_nodes=[], n_primary_guards=3, dir=false, guards_in_consensus) while True: entryGuard = ALGO_CHOOSE_ENTRY_GUARD_NEXT(context) circuit = composeCircuitAndConnect(entryGuard) if not SHOULD_CONTINUE(isSuccessful(circuit)): ALGO_CHOOSE_ENTRY_GUARD_END(context, entryGuard) return circuit -*- coding: utf-8 -*-

5 12

Is it possible to leak huge load of data over onions?
by Ivan Markin 04 Apr '16

04 Apr '16

Recently someone leaked enormous amount of docs (2.6 TiB) to the journalists [1]. It's still hard to do such thing even over plain old Internet. Highly possible that these docs were transfered on a physical hard drive despite doing so is really *risky*. Anyways, in the framework of anonymous whistleblowing, i.e. SecureDrop and Tor specifically it's seems to be an interesting case. I'm wondering about the following aspects: o Even if we use exit mode/non-anonymous onions (RSOS) is such leaking reliable? The primary issue here is time of transmission. It's much longer than any time period we have in Tor. o What is going to happen with the connection after the HS republishes its descriptor? Long after? [This one is probably fine if we are not using IPs, but...] o Most importantly, is transferring data on >1 TiB scale (or just transferring data for days) safe at all? At least the source should not change their location/RP/circuits. Or need to pack all this stuff into chunks and send them separately. It's not obvious how it can be done properly. So at what point the source should stop the transmission (size/time/etc)/change location or the guard/ pick new RP? -- [1] http://panamapapers.sueddeutsche.de/articles/56febff0a1bb8d3c3495adf4/ -- Happy hacking, Ivan Markin

2 2

[::]/8 is marked as private network, why?
by Sebastian Hahn 04 Apr '16

04 Apr '16

Hey, I've been wondering about the private_nets const in src/or/policies. It was added in a96c0affcb4cda1a2e0d83d123993d10efc6e396 but Nick doesn't remember why, and I'm hoping someone has an idea (maybe teor, who I've CCed here, who documented this in a later commit?). If nobody knows why we do this I think we should remove it as likely incorrect. Cheers and thanks Sebastian

4 3

Yawning's CFC, web caching, and PETs
by Jeff Burdges 04 Apr '16

04 Apr '16

Should we try to organize some public chat about web caching at PETs or HotPETs this summer? By that I mean, a discussion with anonymity researchers on security and anonymity concerns around making tools like Yawning's CFC a long-term solution to the CloudFlare problem? Aside from our not knowing if CloudFlare will become more accommodating, a trustworthy web cache would enable more serious efforts towards alpha-mixing, either in Tor itself, or with mixnets on the side of Tor. And archival tools make the web better in numerous ways, like by making it harder to removing anything. There are interesting problems in this space like : Big scary adversary issues. Archiving TLS sessions along with HTML transformations so that subsequent clients can verify the original site's certificate. How best to one distribute the cache. Jeff

2 1

A few ideas about improved design/modularity in Tor
by Nick Mathewson 04 Apr '16

04 Apr '16

(Hi! Here's a document I've been poking at , on and off, for a while. There's a lot more to say here, but I think it's ready to share with tor-dev for comment. Thanks!) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ 0. PRELIMINARIES. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ============== WHY? If you've ever looked at even slightly serious software engineering methodology, you've seen some discussion of modularity: the idea that programs should be divided into a number of smaller modules; that complexity should be hidden within modules and not in the interfaces between them; that which modules can call which other modules should be limited; and other little rules like that. We'd like modules isolated from one another for these reasons: * Make them simpler to test * Minimize what is allowed to touch what * Make tor easier to understand * Make tor easier to hack on * Let us write parts of tor in other languages. * Make sandboxing more powerful This is actually several problems all in one! - OO/Isolation/Modularity design that doesn't actually split into multiple processes, but makes the callgraph much much simpler. - Multiprocess sandbox framework with RPC implementing the above abstraction - Finding the right stuff to carve out into separate sandboxes - Finding the right stuff to carve into separate modules. ============ Steps/phases: Over the past 6-8 months we've gone a lot of work to simplify Tor's callgraph in preparation for this work. We've got our callgraph simplified to a pretty extreme degree, for example. Here are some of our next steps, some of which can be done in parallel. 1. Figure out which modules should be able to call each others. Isolate them based on their ability to do so. Refactor until they actually work that way. - In some cases we will need to split - hide big piles of modules behind single entry points. - separate src/common and src/or into separate parts. - Add tools to enforce this separation - Add tools to separate Tor functions safely and not too error-prone. (See sections 1 and 3 below.) 2. Add backend abstractions as needed to minimize module coupling. These should be abstractions that are friendly to in- and multi-process implementations. We will need at least: - publish/subscribe{,/acknowledge}. (See https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern. The 'acknowledge' abstraction allows the publisher to wait for every subscriber to acknowledge receipt. More on this in section 4 below.) - handles (We need frequently need handles to objects where the handles need to be persistent while the objects themselves can go away. We use several patterns for this now. We should just have a handle pattern for this instead. This would need separate implementations for in-process and cross-process access. See section 2 below for more information.) - Need a way to mark functions as "internal to module". We're going to have a larger number of multi-file modules. Right now, our only restriction for making functions less visible is "static", and the fact that we don't allow src/common to call src/or. Both of these should be cleaned up. 3. Write abstractions for splitting code into multiple processes that talk over IPC. (See section 5 below) ========================= Metrics We can measure our success in a number of ways: * How large are the strongly connected components among modules calling another? Right now the largest SCC is 52 C files. * How many layer/module violations can we find? This needs to be an automated process. * How many exported functions from each module can we remove? * How much can we remove from the main linux sandbox code? How many functions can we isolate to purpose-built modules? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ 0. THE MODULES ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ========================= Modules in src/or LOWEST LEVEL These should only get called from higher-level components. They shouldn't call anything of a higher level. Types and data structures. - part of configuration. - parts of statefile - routerset - policy (some) - torcert Various backends and utilities. - confparse - geoip - fp_pair? - reasons - replaycache - buffers Nodelist data structures, excluding routerinfo. - nodelist - microdesc - networkstatus - routerparse.c (parts) SLIGHTLY HIGHER These should only get called from higher-level components, and only call lower-level components. When they currently call a higher-level piece, they should use a callback or a publish/subscribe API or something. Nothing in here should really be tor-specific. mainloop: - connections (common, genericparts) - main (nonspecific parts) - config (maybe) - workqueue - scheduler - periodic - cpuworker (parts) - hibernate ? (Parts?) HIGHER THAN THAT These modules are where we start making Tor do Tor. They should only call lower-level modules onionrouting: - channels - onion* - cpuworker (parts) - connection_or - connection_edge - circuitlist - command - relay - parts of router.c - transports HIGHER STILL These modules can call downwards, though they should really be made as independent as possible from each other and from one another. We should enumerate which of these can call which others; mostly, they should leave one another alone. hidden services: - rend* controller: - control.c auxiliary: - rephist authority: - dirvote - dirserver (parts) - dirauth - dircollate - keypin relay: - ext_orport - parts of router.c - routerkeys.c cache: - parts of directory - dirserver (parts) - routerlist (parts) - routerparse.c (parts) exit: - dns client: - dnsserv - most circuitbuild, circuituse stuff - circpathbias - circuitstats directory: - directory - dirserver - HIGHEST LEVEL Here's the parts of the application that know all the other parts, and call down. Nothing else calls up to these modules. application (highest level) - Main (parts) - parts of config - parts of statefile - ntmain - status ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ 2. INFRASTRUCTURE: HANDLES. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ======================== Handles in C * We need a light-weight handle mechanism. (A mechanism where we can have a reference to X that can still live even when X dies, where lookups of X through the reference are safe even when X is freed, and where lookups of X through the reference are FAST.) * It's okay if the things being handled need to be marked as 'handleable'. * When the handle is in a separate process, we probably need a table-based implementation: * Unique handle (64/96 bit int) for each thingie; hashtable to look up by handle. * When the reference is local, can get as fast as pointer lookup. * Threadsafety can be important but is not always required. For an example implementation, see #18362. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ 3. INFRASTRUCTURE: ENFORCING ISOLATION. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ === Making isolation happen. It'll do us no good to declare that modules are isolated if they actually aren't. Once we have defined the allowed call patterns, we need to make sure that calls that fall outside of those patterns aren't permitted. We can build something pretty simple out of 'nm', I believe. See ticket #18617. This is a work in progress. === Moving functions around. I'm working on a tool to move functions from one file to another in a deterministic way based on annotations in the original file. This lets us version our movement, and have the movement happen deterministically so that the patches are easier to verify. If we integrate it with our callgraph-examination tools above, we can make sure that our movement plans actually make the code more conformant with our modularity plans. See ticket #18618. This is a work in progress. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ 4. INFRASTRUCTURE: PUBLISH/SUBSCRIBE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Many of our modularity violations happen because when some property changes, or some event occurs, we need to inform a wide variety of other modules. The publish/subscribe{,/acknowledge} pattern, and the similar observer pattern, were made for this. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ 5. ADVANCED INFRASTRUCTURE: MULTIPROCESS SANDBOXING ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ======================== Multiprocess abstraction / design We'll consider multiprocess for these cases: * Easier/stronger/more-portable sandboxing. * Writing subcomponents in languages whose runtime/link rules don't play so nicely with C. * Enforcing modularity when modularity delivers security. For sandboxing, here are some high-risk privileges we should make sure that other code doesn't have. * Filesystem calls -- anything that can open a file. * Exec -- anything that can launch a process. * Access to identity or signing keys. * Invoking getaddrinfo() [because it uses a fairly huge segment of the underlying operating system] And here is some higher-risk code that we could most safely isolate from the main modules. * Anything that parses an object. * Consensus diff processing. Our basic abstractions are those described above (pub/sub, handles, hooks), with the addition of: * Blocking RPC * Capabilities???? * message queues. Controversial: * We should NOT require that most of this be blindingly fast. Correct is perfectly adequate. Fast is only needed in the critical path. Here's the general architecture I have in mind: * At the highest level, have a launcher/supervisor process whose only job is to start the other processes, monitor them, and notice if they go down. * Additionally, have a communications-hub process whose role is to pass messages between other processes, performing access control on the messages to make sure that each message is only sent by a process that is allowed to send it, to a process that it is allowed to send to. Processes may only communicate with the hub. * Messages should use a simple RPC format. I vote for protobufs or capnproto. They sure have a speed/simplicity tradeoff though. * pipes or socketpairs should get used on Unix. Windows can use windows Pipes, which aren't quite FDs. Windows will need to have its pipes in a different thread from the socket-based libevent loop (if there is one.) * We need a way to pass fds or handles back and forth. Both Windows and Unix can do this. (via DuplicateHandle() on Windows and sendmsg() on PF_UNIX sockets on unix.) The protocol needs to handle this explicitly. Here's the development plan I would suggest: * In parallel, work on improving Tor's modularity as the earlier sections of this document suggest. Unless we get more modular, we won't be able to make anything actually separate besides the privilege-restriction sandboxing parts above (like isolating keys and FS access). * Research what exactly windows and osx allow. * Write the top-level supervisor process and hub process. * Ensure that the APIs we expose can be written in C in a fairly mechanism-agnostic way, so that we can migrate to this architecture with minimal additional effort. * Move privilege-restriction sandboxing parts into subprocesses. And here is an alternative plan: * Investigate the chromium sandbox code; see if it can be extracted, or how much of it can be extracted. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ A. APPENDIX. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ === progress so far I've gotten these done so far: * The starts of a build-time callgraph enforcement mechanism using nm and readelf. * A handle-based-reference module. (See ticket #18362, but cpunks doesn't like it.) * A draft refactoring tool that lets you annotate code with what should move where, so that you can deterministicly try the code movement and see what happens, and so you can see what effects it will have on the module-level callgraph.

9 8

Re: [tor-dev] Is it possible to leak huge load of data over onions?
by Ryan Carboni 04 Apr '16

04 Apr '16

I've never seen anything download faster than ten megabits per second on Tor. Presumably the inverse is true if you have upload.

1 0

Re: [tor-dev] Advice regarding Cloudflare
by Ryan Carboni 03 Apr '16

03 Apr '16

> > (as opposed to the people that seem to think that Exits > should actively combat abuse by having the capability for censorship). > > Well, a large number of exit nodes already have the capability for a man-in-the-middle attack. This capability could very well be a default option. b) In your magic world, how would accessing any site that uses > multiple hosts for content to work? > > Yes, yes. It is you who is being imposed upon, not Cloudflare, not the businesses that serve content. In my magic world, people produce things for free! This might seem patronizing, but you seem genuinely ignorant. Cloudflare runs a business. They also get paid for it. That business is to protect email addresses from scrapes, and so forth. If they tell their customers that malicious actors can do those things, but only through Tor, because Tor does good work, their customers will take their business elsewhere. In a libertarian world, people can bar entry to their property from people who seem suspicious. You do not believe that Cloudflare should be allowed to bar entry out of some egocentric concern. By any reasonable definition of ethics, one must find a middle ground, and essentially, Cloudflare has all the negotiating power, unless you plan on personally battering down the doors of Cloudflare. A good step would be to ask Cloudflare for statistics on Tor misuse. Perhaps a maximum of 63 domain names (forgot Cloudflare only has a dozen IPs) per Tor circuit could be done.

3 2

Advice regarding Cloudflare
by Ryan Carboni 03 Apr '16

03 Apr '16

I could see why cloudflare is annoyed with you, you are annoying activists from their perspective, although you folks aren't chaining yourselves to coal power plants . But I also use Tor from time to time, so I'll offer some advice. On the Tor side, a way to minimize abuse is for exit nodes not to allow multiple IP addresses from a single circuit. This would make web crawling with Tor expensive, although it will disable the ability for a single tab to use a single exit node. Another method is to request a sort of proof of work on the Cloudflare side. Given current protocols, the simplest would be to require ECDHE with secp521r1 with each Tor connection and disallow unencrypted connections. It will be more bearable than a captcha.

2 1

Re: [tor-dev] Quantum-safe Hybrid handshake for Tor
by Ryan Carboni 03 Apr '16

03 Apr '16

I just want to note you only need an algorithm that protects against 2^80 quantum operations for short-term keys. Regardless, I doubt anyone is going to be spending a billion dollars to crack data sent over a single Tor connection.

1 0

(no subject)
by Nick Mathewson 28 Mar '16

28 Mar '16

Hi, folks! The Tor git master branch is now 0.2.9.0-alpha-dev. Development on 0.2.8.x continues in the branch "maint-0.2.8". I expect that 0.2.8.2-alpha will be the last 0.2.8.x-alpha, and then we can move on to rc and stable. We now return you to your semiregularly semischeduled Tor development. yrs, -- Nick

1 0

Re: [tor-dev] Should we disable the collection of some stats published in extra-infos?
by Karsten Loesing 28 Mar '16

28 Mar '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 15/01/16 23:00, Rob Jansen wrote: > Hello, Hi Rob, I'm moving this discussion from metrics-team@ to tor-dev@, because I think it's relevant for little-t-tor devs who are not subscribed to metrics-team@. Hope you don't mind. > I was recently reviewing the statistics that Tor allows relays to > collect and report to the dir servers [1], which then get published > in extra-info documents [2]. Most of this can be enabled by simply > setting a torrc option. There are quite a few statistics that I > feel should not be collected. I'm wondering if the original purpose > for collecting many of these statistics still exists, and if we > still feel that the privacy compromises that were made when the > collection was implemented are still valid in most cases. > > Here are the stats I am most worried about, and why: > > [unique ips per country code] *-ips (there are many of these, e.g. > "entry-ips") Usually this involves storing individual user IP > addresses in memory (in order to track uniqueness) over some period > of time (usually 24 hours), sometimes for longer than the user > would have otherwise been known to Tor (if a user's session is 1 > hour, Tor could remember the IP for at most 23 additional hours). > This is reported, e.g., per entry; there are many cases in the data > where it is very likely that only one user is connecting to a guard > from a given country (because it is rounded up to 8). Users in > small countries have the greatest risk (intersection attacks become > really easy). I agree that might just lose these statistics. We used them in the past as first approximation to counting users, but obviously that only works as long as clients only connect to a single relay. The only place where we're still using them is in a workaround for estimating bridge users. See #15469 for more details and #8786 for something we'd have to implement before taking these statistics out. > [exit statistics by port number] exit-kibibytes-written > exit-kibibytes-read exit-streams-opened Tor is classifying its > traffic into ports, which could uniquely identify the application > being used by the client. They also track bandwidth usage per port > (and per exit); again, this is bad for those using a random or > unique looking ports (that a given exit does not see very often) > because it could be used to create a fingerprint. Intersection > attacks become easier with this information. Agreed, I can see us dropping these statistics, too. We're currently not using them. But also see my suggestion below. > The less problematic stats: > > [circuit-based cell statistics] cell-processed-cells > cell-queued-cells cell-time-in-queue cell-circuits-per-decile This > provides queue timings and number of cells being processed at a > relay. The number of cells can be used to compute bandwidth of > circuits. It may be possible to launch some attacks that create > several circuits with the intent of moving which decile buckets > some legitimate circuits get placed into, but this is less > worrisome of an attack than the others. I'm less worried about this one. But, suggestion below. > Should Tor still be collecting these things? Should Tor disable the > collection of these statistics until we have a more > privacy-preserving way to collect and aggregate them? > > The good news is that privacy-preserving techniques exist that can > reduce information leakage. I'm developing a tool based on the > secret-sharing variant of PrivEx [3] to collect some of these types > of statistics while providing privacy guarantees. We are currently > using it to collect only those stats that are useful for producing > Tor traffic models. A great advantage of this tool is that the > various counters that we store during the collection phase get > noise added and are randomized during initialization; only the > aggregates are ever known and revealed by the aggregation server, > limiting the information that is lost if a relay is compromised. > This is a large improvement over the current collection method, > which only adds noise before publication and reveals statistics on > a per-relay basis. Suggestion: How about we evaluate these statistics published by relays in the past years to see if there are other benefits or risks we didn't think of, and then we decide whether to leave them in, modify them, or take them out? The reason is that I'd want to avoid removing this code only to realize shortly after that we overlooked a good reason for keeping it. These statistics are being collected for years now, and it might take another year or so for relays to upgrade to stop collecting them. So what's another month. Thanks for (re-)starting this discussion! > All the best, Rob All the best, Karsten > [1] https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt [2] > https://collector.torproject.org/recent/relay-descriptors/extra-infos/ > > [3] www.cypherpunks.ca/~iang/pubs/privex-ccs14.pdf > > > > _______________________________________________ metrics-team > mailing list metrics-team(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team > -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWnffFAAoJEJD5dJfVqbCrSREH/1G3nZDwC7a8FGFEySgQX3MZ n1dlMcylDU6kypgGMEYs2KmMRwhNYmCVFn6fRJgrFN/KgJ0oZJG1pRcglnIFwNwH 5VUWDZp3+frhY5jBbJ3JXqA4Exhi/0mUYCTRwfcFQ/JOyzjlcFbLRW/rkqFHn62H wLidaVeagiJ+TI/T8zgwtqQjTDSZrZPpmlxvlO57D1bGhW1ZPJVbOeUzuudhNpRS WVu//MH51juCzcML32MeiMV6wWYYOm1irKZ8lZHVCPbsL98qoh3ewtO6fb62THIa sWcQZlwMQdcC0+509PmMeYwkRn+40MUwkk84/glyP57dfpgtOwzZCohL3yQRQ1s= =F6F+ -----END PGP SIGNATURE-----

2 4

guide to start developing tor
by RAJAT MODI 28 Mar '16

28 Mar '16

hi i am new to tor and want to contribute to tis development .i have already browsed the volunterr page on your website .However i am unable to decide from where should i start ,Kindly guide me for the same . -- RAJAT MODI

1 0

Remember: Network Team/tor core dev meeting Monday at new time!
by Nick Mathewson 28 Mar '16

28 Mar '16

Hello! Tomorrow on Monday, we will be having our meeting on the new time: 1700 UTC (1pm eastern, 10am Pacific). The patch party will be Tuesday at 2300 UTC. If you like hacking Tor, set an alarm and stop on by! Peace, -- Nick

1 0

Sibyls: introducing AS-level sign-up rate limits, per relay caps, family-level caps
by nusenu 26 Mar '16

26 Mar '16

Hi, there is a strict limit on the number of relays that one can run on a single IP address to make Sybil attacks harder, but there are no limits on sign-up rates. Such sign-up rate limits would prevent many of the known and simple Sibyls of the past months and probably reduce the manual actions required by dir auth ops to mitigate them (but I've little inside on that part). In the past 8 months the proposed AS sign-up rate limit thresholds (see below) triggered on ~27 events [some-examples]. I consider all of them to be true-positives, but 15 (vs total: >750) likely unrelated relays signed up on the same ASes during these events. AS-level sign-up rate limit thresholds --------------------------------------- max new relays per 24 hours per AS per OS family: 9 (default); value for OVH/ONLINE/Digital Ocean ASes: 14 I deduced to these threshold values by watching the past 8 months of OrNetRadar emails while keeping false-positives at a minimum (at the price of more false-negatives). How does a well behaving operator (aka Brian) add more than 9 relays per AS per day? All relays in an effective family should be treated as "1 new relay". (In my opinion a reason to keep the family [0] functionality and implement prop242). denial of service risk To prevent trivial dos attacks where an attacker with a single IP generates several new relay fingerprints until the entire AS is blocked from adding new relays for a few hours these relays should come from distinct IP addresses. Pros: + automatically enforced + less manual action required by dir auth ops (maybe some dir auths can commend on that.) + address more (and smaller) Sibyls (I assume that many small Sibyls are currently ignored due to risk/effort trade-offs) Cons: - maxmind AS-IP mapping db becomes crucial and needs to be in sync across dir auths - false-positives - non-dir auths will not notice if an AS run into this limit unless they publish some kind of transparency reports - dir auths must keep more state (not sure if they know all existing FPs of the past already) Feasibility Initially the idea was to provide immediate feedback to the relay that tries to publish its descriptor to the dir auth via the HTTP response (an important point), but since descriptors are not uploaded to a single central dir auth that does not work I guess. So before thinking on how one would actually achieve that or something similar I wanted to gather the general opinion about something like this. single relay caps ------------------ 250000 consensus weight (to prevent [1]) family-level caps ------------------ max cw fraction: 2.5% [2] max. exit probability: 10% [3] guard probability: 5% [4] AS-level caps: --------------- max cw fraction/guard/exit probability: 15% [5] AS12876 (ONLINE SAS) and AS16276 (OVH) are already past that value, so if you do not want to change the game for them you would have to set it to ~20%. [0] https://trac.torproject.org/projects/tor/ticket/6676 [1] https://lists.torproject.org/pipermail/tor-talk/2015-December/039696.html https://trac.torproject.org/projects/tor/ticket/17810#comment:8 [2] https://raw.githubusercontent.com/ornetstats/stats/master/o/main_operators_… [3] https://raw.githubusercontent.com/ornetstats/stats/master/o/main_exit_opera… [4] https://raw.githubusercontent.com/ornetstats/stats/master/o/main_guard_oper… [5] https://compass.torproject.org [some-examples] http://article.gmane.org/gmane.network.onion-routing.ornetradar/1135 http://article.gmane.org/gmane.network.onion-routing.ornetradar/1132 http://article.gmane.org/gmane.network.onion-routing.ornetradar/1100 http://article.gmane.org/gmane.network.onion-routing.ornetradar/1030 http://article.gmane.org/gmane.network.onion-routing.ornetradar/939 http://article.gmane.org/gmane.network.onion-routing.ornetradar/943 http://article.gmane.org/gmane.network.onion-routing.ornetradar/898 http://article.gmane.org/gmane.network.onion-routing.ornetradar/856 http://article.gmane.org/gmane.network.onion-routing.ornetradar/813

1 1

[GSoC 2016] Roster - Gamification for Tor Operators (Sean Saito)
by Sean Saito 26 Mar '16

26 Mar '16

Hi! I have finished and submitted my final proposal for GSoC yesterday. This summer I would like to complete Roster, a Tor Project initiative I’ve been working on since last year. It is basically a social incentivization scheme to gamify operating a Tor relay. All the details are included in the Google doc, and, while the deadline for edits has passed, I would appreciate any comments and feedback. https://docs.google.com/document/d/1wOBUhOPAAWTLfQIt4ChKixLMgIrZlO3YBQ-eixh… Thanks a lot! Best, Sean

1 0

Tor 0.2.8 and Fallback Directory Mirrors
by Tim Wilson-Brown - teor 26 Mar '16

26 Mar '16

Dear Tor App Developers, Just a heads' up that tor 0.2.8 includes a fallback directory mirrors feature, where tor clients bootstrap from a set of hard-coded long-lived directory mirrors. This makes bootstrap more reliable, and makes it harder to block tor. The wiki has an entry with more details. [0] We're currently trying to finalise a list of around 100 fallback directory mirrors by the time tor 0.2.8-stable is released. [1] If we have this many mirrors, I am confident we won't overwhelm any of the mirrors when an app like Orbot, Tor Browser, or Tails switches to tor 0.2.8. (Are there any other tor apps with millions of users that I've missed?) If this feature does cause problems, or if your app needs to bootstrap only from the authorities (Tails time syncing? [2]), the fallback directory mirrors feature can be disabled using the tor config option "UseDefaultFallbackDirs 0". Tim [0]: https://trac.torproject.org/projects/tor/wiki/doc/FallbackDirectoryMirrors <https://trac.torproject.org/projects/tor/wiki/doc/FallbackDirectoryMirrors> [1]: https://trac.torproject.org/projects/tor/ticket/17158 <https://trac.torproject.org/projects/tor/ticket/17158> [2]: https://tails.boum.org/contribute/design/Time_syncing/ <https://tails.boum.org/contribute/design/Time_syncing/> Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

2 2

Much-revised draft, RFC: removing current obsolete clients from the network
by Nick Mathewson 26 Mar '16

26 Mar '16

[This is a significantly revised version of the last version of this proposal draft, sent here for comment. The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html . This is more or less a complete rewrite, after I convinced myself that the original ideas wouldn't work. Part of the functionality of that old draft has been subsumed in proposal 264.] 1. Introduction Frequently, we find that very old versions of Tor should no longer be supported on the network. To remove relays is easy enough: we simply update the directory authorities to stop listing relays that advertise versions that are too old. But to disable clients is harder. In another proposal I describe a system for letting future clients go gracefully obsolete. This proposal explains how we can safely disable the obsolete clients we have today (and all other client versions of Tor to date, assuming that they will someday become obsolete). 1.1. Why disable clients? * Security. Anybody who hasn't updated their Tor client in 5 years is probably vulnerable to who-knows-what attacks. They aren't likely to get much anonymity either. * Withstand zombie installations. Some Tors out there were once configured to start-on-boot systems that are now unmaintained. (See 1.4 below.) They put needless load on the network, and help nobody. * Be able to remove backward-compatibility code. Currently, Tor supports some truly ancient protocols in order to avoid breaking ancient versions or Tor. This code needs to be maintained and tested. Some of it depends on undocumented or deprecated or non-portable OpenSSL features, and makes it hard to produce a conforming Tor server implementation. * Make it easier to write a conforming Tor relay. If a Tor relay needs to support every Tor client back through the beginning of time, that makes it harder to develop and test compatible implementations. 1.2. Is this dangerous? I don't think so. This proposal describes a way to make older clients gracefully disconnect from the network only when a majority of authorities agree that they should. A majority of authorities already have the ability to inflict arbitrary degrees of sabotage on the consensus document. 1.3. History The earliest versions of Tor checked the recommended-versions field in the directory to see whether they should keep running. If they saw that their version wasn't recommended, they'd shut down. There was an "IgnoreVersion" option that let you keep running anyway. Later, around 2004, the rule changed to "shut down if the version is _obsolete_", where obsolete was defined as "not recommended, and older than a version that is recommended." In 0.1.1.7-alpha, we made obsolete versions only produce a warning, and removed IgnoreVersion. (See 3ac34ae3293ceb0f2b8c49.) We have still disabled old tor versions. With Tor 0.2.0.5-alpha, we disabled Tor versions before 0.1.1.6-alpha by having the v1 authorities begin publishing empty directories only. In version 0.2.5.2-alpha, we completely removed support for the v2 directory protocol used before Tor 0.2.0; there are no longer any v2 authorities on the network. Tor versions before 0.2.1 will currently not progress past fetching an initial directory, because they believe in a number of directory authority identity keys that no longer sign the directory. Tor versions before 0.2.4 are (lightly) throttled in multihop circuit creation, because we prioritize ntor CREATE cells over TAP ones when under load. 1.4. The big problem: slow zombies and fast zombies It would be easy enough to 'disable' old clients by simply removing server support for the obsolete protocols that they use. But there's a problem with that approach: what will the clients do when they fail to make connections, or to extend circuits, or whatever else they are no longer able to do? * Ideally, I'd like such clients to stop functioning _quietly_. If they stop contacting the network, that would be best. * Next best would be if these clients contacted the network only occasionally and at different times. I'll call these clients "slow zombies". * Worse would be if the clients contact the network frequently, over and over. I'll call these clients "fast zombies". They would be at their worst when they focus on authorities, or when they act in synchrony to all strike at once. One goal of this proposal is to ensure that future clients to not become zombies at all; and that ancient clients become slow zombies at worst. 2. Some ideas that don't work. 2.1. Dropping connections based on link protocols. Tor versions before before 0.2.3.6-alpha use a renegotiation-based handshake instead of our current handshake. We could detect these handshakes and close the connection at the relay side if the client attempts to renegotiate. I've tested these changes on versions maint-0.2.0 through maint-0.2.2. They result in zombies with the following behavior: The client contact each authority it knows about, attempting to make a one-hop directory connection. It fails, detects a failure, then reconnects more and more slowly ... but one hour later, it resets its connection schedule and starts again. In the steady state this appears to result in about two connections per client per authority per hour. That is probably too many. (Most authorities would be affected: of the authorities that existed in 0.2.2, gabelmoo has moved and turtles has shut down. The authorities Faravahar and longclaw are new. The authorities moria1, tor26, dizum, dannenberg, urras, maatuska and maatuska would all get hit here.) (We could simply remove the renegotiation-detection code entirely, and reply to all connections with an immediate VERSIONS cell. The behavior would probably be the same, though.) If we throttled connections rather than closing them, we'd only get one connnection per authority per hour, but authorities would have to keep open a potentially huge number of sockets. 2.2. Blocking circuit creation under certain circumstances In tor 0.2.5.1-alpha, we began ignoring the UseNTorHandshake option, and always preferring the ntor handshake where available. Unfortunately, we can't simply drop all TAP handshakes, since clients and relays can still use them in the hidden service protocol. But we could detect these versions by: Looking for use of a TAP handshake from an IP not associated with with any known relay, or on a connection where the client did not authenticate. (This could be from a bridge, but clients don't build circuits that go to an IntroPoint or RendPoint directly after a bridge.) This would still result in clients not having directories, however, and retrying once an hours. 3. Ideas that might work 3.1. Move all authorities to new ports We could have each authority known to older clients start listening for connections at a new port P. We'd forward the old port to the new port. Once sufficiently many clients were using the new ports, we could disable the forwarding. This would result in the old clients turning into zombies as above, but they would only be scrabbling at nonexistent ports, causing less load on the authorities. [This proposal would probably be easiest to implement.] 3.2. Start disabling old link protocols on relays We could have new relays start dropping support for the old link protocols, while maintaining support on the authorities and older relays. The result here would be a degradation of older client performance over time. They'd still behave zombieishly if the authorities dropped support, however. 3.3. Changing the consensus format. We could allow 'f' (short for "flag") as a synonym for 's' in consensus documents. Later, if we want to disable all Tor versions before today, we can change the consensus algorithm so that the consensus (or perhaps only the microdesc consensus) is spelled with 'f' lines instead of 'f' lines. This will create a consensus which older clients and relays parse as having all nodes down, which will make them not connect to the network at all. We could similarly replace "r" with "n", or replace Running with Online, or so on. In doing this, we could also rename fresh-until and valid-until, so that new clients would have the real expiration date, and old clients would see "this consensus never expires". This would prevent them from downloading new consensuses. [This proposal would result in the quietest shutdown.]

7 8

[GSoC '16] Exitmap project - Final Proposal
by Mridul Malpotra 25 Mar '16

25 Mar '16

Hey everyone! I have completed my proposal on the project Exitmap improvements. Sorry for submitting it late. Link for suggestions: https://docs.google.com/document/d/1AqjH3AsBbfesJg36SZg1frPQsdpAnJmzH8k1-G6… Some points that have changed from the previous discussion: 1. Telnet/SNMP sniffing attacks would not be appropriate for Exitmap as it is an active attack scanner compared to say HoneyConnector etc. hence it has not been included. 2. I have added most of the points that I had suggested above with tweaks as advised by Dr. Winter. I will be submitting the proposal in an hour. Thank you for your time. Sincerely, -- Mridul Malpotra Undergrad @ IIIT-Delhi <http://www.iiitd.ac.in/> mridulmalpotra.github.io ============================= PGP keyID: 0xb716e33ab6d0a653 <http://pgp.mit.edu/pks/lookup?op=get&search=0xB716E33AB6D0A653>

1 1

Re: [tor-dev] GSoC proposal: Panopticlick
by Akito Ono 25 Mar '16

25 Mar '16

Hi Georg, >Thanks. What does "check fingerprint's uniqueness" entail? Is it just a >"Unique? - Yes/No"-thing? If not, what are your ideas in this regard? I think it is not a "Yes/No"-thing. My idea is that the uniqueness of fingerprint is evaluated in each element relatively in recorded data and total uniqueness of a fingerprint is calculated by those ratios. I added my ideas to my proposal. Best regards, Akito

1 0

Notes from the prop259 proposal reading group
by George Kadianakis 25 Mar '16

25 Mar '16

Hello, we had a meeting about proposal 259 "New Guard Selection Behaviour". You can see the logs here: http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-03-23-15.01.log.txt Some notes: - The latest version of the proposal can be found here: https://lists.torproject.org/pipermail/tor-dev/2016-March/010625.html There is also some discussion with Tim that has been happening in that thread since yesterday. It's likely that the proposal will be slightly changed (read: improved) as implementation and testing proceeds. The implementation of the proposal is currently happening here: https://github.com/twstrike/tor_for_patching/tree/prop259 - We discussed whether the guard algorithm should care about circuit restrictions like requiring that the guard of a circuit needs to be Stable (needs_uptime) or Fast (needs_capacity). We decided that for now the algorithm should be able to handle these restrictions, by skipping to the next guard that satisfies the conditions in case the top guard does not. This is also how the current guard algorithm works. In the future we should make it so that all Guards are both Stable and Fast, so that this stupid check does not need to happen [TODO: I should open a ticket for this if it doesn't already exist]. - We discussed how ReachableAddresses should work with regards to guardlists. A suggestion by Tim came from here: https://lists.torproject.org/pipermail/tor-dev/2016-March/010630.html who says: "I suggest that we compose the set of UTOPIC guards based on addresses that are reachable and preferred (or, if there are no guards with preferred addresses, those guards that are reachable). I suggest that we use the same mechanism with DYSTOPIC guards, but add a port restriction to 80 & 443 to all the other restrictions. (This may result in the empty set.)" I think this suggstion makes sense for now. - We also talked about directory guards and how they should work but we didn't come to a conclusion. My current intuition is that if a directory circuit appears that requires a directory guard, we treat it as a constraint the same way we treat needs_capacity and needs_uptime. So if our top guard cannot be a directory guard (is not a V2Dir), we skip it and go down our list till we find a guard that does. In the future (almost) all guards will be directory guards so this should become less of an issue (see #12538). - Finally, a topic that came up after the meeting in: https://lists.torproject.org/pipermail/tor-dev/2016-March/010635.html had to do with how we treat bad guards in SAMPLED_UTOPIC_GUARDS and SAMPLED_DYSTOPIC_GUARDS. So if suddenly 90% of the guards in SAMPLED_UTOPIC_GUARDS drop out of the consensus, what do we do? We mark them as bad, and then what? Do we remove them from the sampled list? Or do we keep them in there in case they come back? (This latter behavior is what tor currently does). But then if we keep them, don't we also need to add some more guards in there to make up for the bad ones? How exactly should this work? Some thinking needs to be done here. - We also discussed the need for good debug logging in the prop259 tor branch, so that we can test it ourselves. Ideally, the logging should be nice and intuitive, so that we can monitor the logs every now and then and check if the guard picking works properly. For example, if the algorithm keeps on switching guards all the time, there is a problem. If the algo is using the 5th guard while the 1st guard is up, it's a problem. Good logging should be able to reveal all these problems without having to spend 20 minutes investigating every new log message.

2 1

[GSoC 16] Proposal to work on the Ahmia.fi search engine
by Ismael R 25 Mar '16

25 Mar '16

Hi everyone, I'm a french student and I would like to share my GSoC application with all of you. I plan to work on the Ahmia hidden service search engine. Because of my experience with python, django and web in general, I feel like I can make a substantial contribution to this project during summer. I'm also very interested about the Tor ecosystem, and privacy in general. You can read my proposal on Google doc [1] or download a pdf [2]. You are welcome to comment it, I'm open to suggestions. Best regards, Ismael R. [1] : https://goo.gl/AT37hA [2] : http://dl.free.fr/f6EMq8vOu

1 0

Review of GSoC proposal - Ahmia.fi
by Srikrishna 25 Mar '16

25 Mar '16

Hi people, Sorry for the late approach. Here is my proposal for the project, Improving Ahmia.fi, search engine for hidden services. Any suggestions are most welcome. Thanks to Juha ( juha.nurmi(a)ahmia.fi ) for helping me out with the proposal. I'll be available on irc as 'skrish13' or you can comment on the doc itself or reply here :) Link for my proposal -> https://docs.google.com/document/d/1GOQMi7qkR7Ppuh4sF5EoSv4Io5D2Dwpz8OyzNoi… -- Thanks and Regards Srikrishna <Attached my PGP signature> <Using PGP/MIME, notify me for Inline PGP>

1 0

[GSoC] CONIKS implementation for Tor Messenger
by Huy Vu Quoc 25 Mar '16

25 Mar '16

Dear devs, My name is Huy, and I am a computer science student at Hanoi University of Science and Technology, Vietnam. GSoC is a great opportunity for me to contribute to Tor. I have been watching the development of Tor Messenger since its release. As my major is cryptography, I find the idea “implement and integrate CONIKS for Tor Messenger” very interesting. I am studying the CONIKS paper and my confusions were mostly answered in Elias's thread. I'm thinking about how to integrate CONIKS module to current implementation of Tor Messenger. I have some questions to ask: - A standard for data formats & protocol: I know the standard is essential for the project, so can I define a standard when developing (with your review) or I have to wait for your work in future? Another question about GSoC: - What should be deliverable by mid-term of GSoC? Btw, I also created a pull-request for ctypes-otr project on Github. Best, Huy Vu

3 9