- tor-dev - lists.torproject.org

Propsal 263 Quantum-safe Hybrid handshake for Tor, updated feature request v1.2
by Zhenfei Zhang 08 Feb '16

08 Feb '16

Hello list, We had a very constructive discussion on this proposal last Thursday. I have updated the feature request based on the feedback from the discussion. We also like to thank Roger for grammar fixes on the proposal. The diff file is available at: https://github.com/zhenfeizhang/ntru-tor/commit/dc48c2065d1772c5497cd6ad900… Major tech updates are: 1. Use SHA3 and SHAKE instead of HMAC_SHA3 2. In previous proposal, when server sends back the message, it sends a ciphertext C = ENCRYPT( P | B, QSPK), and its one-time ECC public key Y = e^y; where P is the secret that server chose, B is servers long term public key. In this new version, we also include servers short term public key in the plaintext, i.e., C = ENCRYPT( P | B | Y, QSPK) and Y is no longer send to the client separately. The client will need to decrypt C first and retrieve Y from the message. Y is not revealed to any one other than the client. This is proposed by Nick. It was also suggested to us by Aniket Kate. We thank Nick and Aniket for the suggestion. This modification will save 32 bytes of traffic as Y is no longer transmitted. Also since Y is now encrypted it will be a little harder for the attacker to attack the 25519 keys. 3. Choosing algorithms and parameters for the protocol. We have identified NTRUEncrypt and R-LWE as two candidates for this protocol. We have agreed to use NTRUEncrypt for now, in the meantime, keep the option open for "newesthope" version of R-LWE type key exchange when it becomes available. Also, for NTRUEncrypt we will be using parameters sets with SHA3, to align with Tor specs. I have modified the feature request to reflect this change. Also in the discussion we were talking about the possibility of using non-product form polynomial version of NTRUEncrypt, as this version will become patent free by Aug 2017, while the patent for product form will last for another 4 years. The main concern is that Debian will not allow patented software in the package. However, through our discussion, it turns out that we may be able to include this proposal in the next one of two release of Tor. From this point of view, both patent (basic NTRU patent, till 2017 and product form patent, till 2021) are going to be an issue if Debian does not agree with SI's patent statement. So does it make sense to keep the product form polynomials as they enables roughly 3 times faster operations on both client side and server side? Thanks for your time, and please let us know if you have any further comments/suggestions. Best regards, Zhenfei

2 1

Re: [tor-dev] Detailed algorithm
by George Kadianakis 08 Feb '16

08 Feb '16

> Hey, > > So - thinking through your comments and thoughts, I realized that the > better way to proceed was to describe the algorithm, and then put it > into a proposal format. The following is updated with most of your > comments and should be a bit clearer. I decided to remove #241 again, > since it's unclear how it actually fits together with #259. The idea > is that the algo is divided into three pieces. If failure is reported > from the algorithm, the caller can wait for a while or just restart it > from scratch immediately: > I feel that this algorithm is a step in the right direction! It's interface logic is quite similar to how Tor picks its guards, and it's more specific than the previous proposals. It's a good first step so I'll CC tor-dev so that Nick, Isis and other people can take a look if they want. After we pin this down, we also need to collect all its security properties in one place. For example, we should state clearly what security properties the thresholds GUARDS_TRY_THRESHOLD_TIME and GUARDS_FAILOVER_THRESHOLD provide. I inline various questions below. I can take a deeper look tomorrow as well. > - Start of algorithm > - Ensure USED_GUARDS is populated (from state file etc) > - Create a list of PRIMARY_GUARDS that contain N_PRIMARY_GUARDS that are not bad by: > - Taking the next entry from USED_GUARDS > - If USED_GUARDS is empty: > - randomly select an entry from UTOPIC_GUARDS, weighted by bandwidth Hmm. PRIMARY_GUARDS stays like this for ever? It probably needs to be updated when a primary guard get removed from the consensus (become bad), and put the next entry of USED_GUARDS in its place. Maybe we need another event (and algorithm) for "receive new consensus"? Or is this considered out of scope? > - Set TRIED_GUARDS to be an empty set Hm. What is the point of TRIED_GUARDS? It's like a list of guards that we found unreachable at some point in time. It's never cleaned, so it will eventually accumulate many nodes. IIUC, we only use that list to enforce our thresholds by looking at timestamps. Can we do without this list, just by checking the timestamps of unreachable USED_GUARDS? Or the logic will be much more complicated? > - Set TRIED_DYSTOPIC_GUARDS to be an empty set > - Set state = STATE_PRIMARY_GUARDS > > - Each iteration of algorithm > - If it was at least 3 minutes since we tried the primary guards and we are not in STATE_PRIMARY_GUARDS: > - save previous state > - set state = STATE_PRIMARY_GUARDS > > - STATE_PRIMARY_GUARDS: > - return each entry in PRIMARY_GUARDS in turn > - mark each entry as "unreachable" if algorithm doesn't terminate What is the "algorithm" here? Is it the guard selection algorithm, or the circuit construction algorithm? I guess the latter? If it's indeed the circuit construction algorithm, then the steps here are not asynchronous anymore, which diverges from how Tor networking is designed. Maybe this could be fixed by introducing some new states for when we are waiting for a circuit to connect and we are now learning whether it was succesful or not. Basically what entry_guard_register_connect_status() does in Tor right now. For example we could have STATE_WAITING_FOR_PRIMARY_GUARD_CIRCUIT for when we are waiting for a primary guard circuit to connect. If it fails, then we need to mark the entry as unreachable. Maybe we also need STATE_WAITING_FOR_UTOPIC_GUARD_CIRCUIT etc. > - restore previous state (or STATE_TRY_UTOPIC if no previous) > > > - STATE_TRY_UTOPIC: > - for each entry in TRIED_GUARDS that was marked as unreachable more than 20 minutes ago > - add to the front of remaining UTOPIC_GUARDS Isn't it already in UTOPIC_GUARDS? I don't see us ever removing stuff from UTOPIC_GUARDS. BTW, what is this step supposed to do? Setup previously TRIED_GUARDS to be retried? > - return each remaining entry from USED_GUARDS in turn > - for each entry, if algorithm doesn't terminate > - mark entry as "unreachable" > - add entry to TRIED_GUARDS Add it there even if it it's already there (TRIED_GUARDS is never cleaned)? Or maybe update a timestamp if it's already there? How to specify this nicely? > - if the number of entries in TRIED_GUARDS that were tried within GUARDS_TRY_THRESHOLD_TIME > is larger than GUARDS_TRY_THRESHOLD, return failure from the algorithm > - if the number of entries in TRIED_GUARDS is larger than a GUARDS_FAILOVER_THRESHOLD > proportion of UTOPIC_GUARDS, set state = STATE_TRY_DYSTOPIC > - return each remaining entry from UTOPIC_GUARDS randomly selected, weighted by bandwidth > - for each entry, if algorithm doesn't terminate > - mark entry as "unreachable" > - add entry to TRIED_GUARDS > - if the number of entries in TRIED_GUARDS that were tried within GUARDS_TRY_THRESHOLD_TIME > is larger than GUARDS_TRY_THRESHOLD, return failure from the algorithm > - if the number of entries in TRIED_GUARDS is larger than a GUARDS_FAILOVER_THRESHOLD > proportion of UTOPIC_GUARDS, set state = STATE_TRY_DYSTOPIC > TRIED_GUARDS is never cleaned, so this will be almost always true after a while. > - STATE_TRY_DYSTOPIC: > - for each entry in TRIED_DYSTOPIC_GUARDS that was marked as unreachable more than 20 minutes ago > - add to the front of remaining UTOPIC_GUARDS Does it matter if we add it to the front or the back of UTOPIC_GUARDS? >From what I understand, we always sample from UTOPIC_GUARDS based on relay bandwidth so the list order should not matter, right? [This comment and also the comments below also apply for the STATE_TRY_UTOPIC case above] > - return each remaining DYSTOPIC entry from USED_GUARDS in turn Do we even try previously-found unreachable guards here? > - for each entry, if algorithm doesn't terminate > - mark entry as "unreachable" > - add entry to TRIED_DYSTOPIC_GUARDS Order matters here right? We probably want FIFO. > - if the number of entries in TRIED_GUARDS+TRIED_DYSTOPIC_GUARDS that were tried within GUARDS_TRY_THRESHOLD_TIME > is larger than GUARDS_TRY_THRESHOLD, return failure from the algorithm > - if the number of entries in TRIED_DYSTOPIC_GUARDS is larger than a GUARDS_FAILOVER_THRESHOLD > proportion of DYSTOPIC_GUARDS: > - mark all guards in PRIMARY_GUARDS, TRIED_GUARDS and TRIED_DYSTOPIC_GUARDS as not "unreachable" > - return failure from the algorithm > - return each remaining entry from DYSTOPIC_GUARDS randomly selected, weighted by bandwidth > - for each entry, if algorithm doesn't terminate > - mark entry as "unreachable" > - add entry to TRIED_DYSTOPIC_GUARDS > - if the number of entries in TRIED_GUARDS+TRIED_DYSTOPIC_GUARDS that were tried within GUARDS_TRY_THRESHOLD_TIME > is larger than GUARDS_TRY_THRESHOLD, return failure from the algorithm > - if the number of entries in TRIED_DYSTOPIC_GUARDS is larger than a GUARDS_FAILOVER_THRESHOLD > proportion of DYSTOPIC_GUARDS: > - mark all guards in PRIMARY_GUARDS, TRIED_GUARDS and TRIED_DYSTOPIC_GUARDS as not "unreachable" Hm, not sure what this is supposed to do. Is it trying to setup the algorithm to retry all guards from the beginning? Shouldn't we edit USED_+GUARDS in that case? > - return failure from the algorithm > Instead of failing in this case, shouldn't we just retry from the top without adding any new guards if we hit these thresholds? How do we do this? Or is this logic considered out of scope? :x > - End of algorithm > - If circuit is set up correctly, let algorithm know > - Algorithm marks the guard chosen as used and makes sure it is in USED_GUARDS > - Otherwise do another run of the algorithm > When does this "End of algorithm" happen?

1 0

Configuration of tor relay using setup files (use of API via Tor Expert Bundle)
by Nathan Bliss 07 Feb '16

07 Feb '16

Hi, Is there a way to configure a bridge in tor (e.g. meek) via the config files from the command line without having to use the GUI in the Tor browser? I've been searching for documentation on this, so if I've missed it I would be grateful for a pointer to where this is in the docs... I found something about the torrc but this was an option for configuring a tor server node as a bridge whereas I want to configure the bridge as a client (in client mode). I need to do this in order to have automation scripts working with search engines such as google. Google always seems to be able to detect http connection via a script or via the tor interface running in standard mode. Only a client connection via google-meek bridge seems to work and I'm looking for a way to do this via the tor stem API or via the config files. If this feature isn't currently exposed as an API hook, I'd be happy to contribute this feature to the source code if someone can give me a few pointers as I'm new to the project and haven't yet seen any of the source [&#X1f60a] Many thanks, Nathan

2 1

Re: [tor-dev] Hashring understanding
by George Kadianakis 07 Feb '16

07 Feb '16

Ola Bini <obini(a)thoughtworks.com> writes: > Hi, > >> I think it's fine to use randomness to generate it for now; that's what tor >> also currently does [0]. > Cool - we have a local clone of the specs and updating it with this understanding. > Great! For better or worse I think making design decisions (or at least simulating them) as well as improving and finalizing the spec is a big part of this task :) >> The real lifetime of guards is currently a random value between 2 to 3 months >> (see chosen_on_date) This is regardless of whether the guard is your primary >> guard or not. We might increase the lifetime in the future, but probably the >> precise value should not matter too much since it's a constant >> change. > Does this mean the choice of guard is saved in the state file or > config file? Indeed. It is saved in the state file. Check the end of: https://lists.torproject.org/pipermail/tor-dev/2016-February/010355.html also see here for another explanation of how parts of it work: https://lists.torproject.org/pipermail/tor-dev/2014-June/007042.html > Should the PRIMARY_GUARDS chosen by #259 here be saved as well? Hmm. Not sure how this will work exactly in practice. The way I imagine it, is that the first N_PRIMARY_GUARDS active guards on your guard list are considered to be your primary guards. It's more of a label, than a static set. That's because the set of primary guards changes over time, as nodes go inactive. Like prop259 says, "by 'active' we mean that we only consider guards that are present in the latest consensus as primary". > I suspect that might not work very well because of the difference between > dystopic and utopic networks, and when people move around between > networks. Or maybe we should have two sets, UTOPIC and DYSTOPIC > PRIMARY_GUARDS that are more long lived? Hmm... Not sure how this would work if utopic and dystopic guard lists are separated. That is if there should be both dystopic primary guards _and_ entropic primary guards. I'd personally prefer the approach where we don't have such strictly separated guard lists. I tried to explain parts of it in the top of this email: https://lists.torproject.org/pipermail/tor-dev/2016-February/010355.html but it probably requires more design work. The way I'm thinking of it, we have a main guardlist GUARDS, where we put every guard we have sampled and connected or planning to connect to. We would sample both normal and 80/443 guards for this guardlist. This guardlist should work for most clients (except from 80/443 firewalled clients). To handle these 80/443 firewalled clients, if we try more than GUARDLIST_FAILOVER_THRESHOLD nodes from GUARDS and we can't connect, then we consider it to be a firewall issue, and go into "dystopic firewall mode". During this mode we only sample dystopic guards (potentially from a separate DYSTOPIC_GUARDS guardlist). If we try more than GUARDLIST_FAILOVER_THRESHOLD of guards from the dystopic guardlist, then we complain about network issues. (There are various optimizations and heuristics that can be applied to the above system, but let's not get to them now.) [The reason I don't like having separate disjoint utopic and dystopic guardlists and first trying the former and then the latter, is because it means that normal people will use the utopic guards, and the dystopic guards will only be used by people that have 80/443 firewalled. We don't really know how big these user sets are right now so it's hard to make load-balancing assumptions, and from a security PoV we should probably not isolate those user sets if we can help it.] Bleh sorry for the mess... It's been a while since I've thought of this stuff.

2 3

Re: [tor-dev] Guards selection - current algorithm simulation
by George Kadianakis 05 Feb '16

05 Feb '16

Iván Pazmiño <ipazmino(a)thoughtworks.com> writes: > Hello, thanks for your reply. > > On 02/04/2016 04:28 PM, George Kadianakis wrote: >> We don't actually and it's a pity because it would have been very useful for >> this particular case indeed. > We will write one then. > Exciting! >> >> However, the entry guard code in Tor is not particularly clean or well >> abstracted. So finding the underlying algorithm will not be a trivial task. >> That's also another reason we are trying to move to a more formalized >> algorithm, so that it's more easily studied. > We've started looking at the code and so far we believe the relevant > scenarios would be those when you invoke choose_random_entry_impl with a > NULL state. Would that be correct? > I _think_ that also the non-NULL state scenario is relevant. choose_random_entry_impl() is called with an initialized state in onion_extend_cpath(). That's the codepath when "we are building a circuit and we need an entry guard for it". choose_random_entry_impl() is called with a NULL state in add_an_entry_guard(). That's the codepath for "shit we are out of entry guards. We need a new one." I think both of these codepaths are relevant. Maybe to make sure, add some logging calls in choose_random_entry_impl() and start up your Tor, to see if it's called with a NULL state or not. > Cheers, > Iván

1 0

Re: [tor-dev] Hashring understanding
by George Kadianakis 04 Feb '16

04 Feb '16

Ola Bini <obini(a)thoughtworks.com> writes: > Hi, > > Sorry for the string of emails! > > Hopefully a simple question: > The current proposal contains logic for keeping track of network > up/down and setting timeouts for exponential backoff to test the > network again. But if I understand correctly, this proposal is > basically about replacing the algorithm used for > choose_good_entry_server() - correct? So it seems like keeping track > of network status doesn't really belong inside this algorithm at > all. Wouldn't it make sense to return a specific failure to the caller > and let the caller be in charge of when to retry? > Hmm. It's quite hard (if not impossible) for a user-land application like Tor to actually keep track of whether the network is up or down in a multiplatform, secure and scalable manner (e.g. without using the dirauths as an oracle). So instead all we do is connect to Tor nodes and check whether we could reach them or not. I think currently if a relay fails to answer a CREATE cell, we treat the relay as unreachable and we mark it as such in our guardlist (see entry_guard_register_connect_status()). This logic is considered suboptimal and it sometimes ends up marking good relays as unreachable. The retry logic in prop259 tries to compensate for this, by eventually retrying nodes that might have been marked as offline by a shaky network. I agree it would be nice to decouple the above behavior from the guard picking behavior. However, the two behaviors are certainly linked with each other, since the only nodes that a Tor client connects to are its guard nodes. I'm not sure how separating these two behaviors would look like, but if you guys think it would simplify things I'd definitely be interested in hearing about it :) WRT current code, if you see choose_random_entry_impl() (which is called by choose_good_entry_server()) you will notice that the status of relays is very central to this function since populate_live_entry_guards() will remove any guards that are inactive or previously found unreachable from the guardlist. choose_random_entry_impl() does not contain logic about the network itself being up and down, because currently Tor does not have such logic. Instead, Tor will endlessly keep on cycling through guards till the network comes up. Hopefully I answered the right question here. BTW, if you guys want we can have another meeting next week Tuesday same time iff that will be helpful to you. cheers!

1 0

Re: [tor-dev] Guards selection - current algorithm simulation
by George Kadianakis 04 Feb '16

04 Feb '16

Iván Pazmiño <ipazmino(a)thoughtworks.com> writes: > Hey guys, > > Just wondering if by any chance you have a simulator for the current > algorithm to select guards. It would be very useful to compare against > new algorithm's numbers. > > Thanks, Hello, We don't actually and it's a pity because it would have been very useful for this particular case indeed. However, the entry guard code in Tor is not particularly clean or well abstracted. So finding the underlying algorithm will not be a trivial task. That's also another reason we are trying to move to a more formalized algorithm, so that it's more easily studied.

1 0

Re: [tor-dev] Hashring understanding
by George Kadianakis 04 Feb '16

04 Feb '16

Ola Bini <obini(a)thoughtworks.com> writes: > Hi, > > Thanks for the confirmation of our understanding! Very helpful. > > If I understand things correctly, we are supposed to take > GUARDLIST_FAILOVER_THRESHOLD guards from the list of all guards and > generate the GUARDLIST from that. Is that process supposed to be > deterministic for a specific client on a specific network, or is it > fine to use randomness to generate it? > Hello, I think it's fine to use randomness to generate it for now; that's what tor also currently does [0]. I think coming up with the right way to do deterministic sampling based on specific networks can be the subject of a separate proposal :). I imagine that there will be engineering and security issues that will complicate any easy approaches. So you can ignore this part of the problem for now, and just use fresh local randomness everytime you sample a guard :] > And how long lived are the PRIMARY_GUARDS supposed to be? The real lifetime of guards is currently a random value between 2 to 3 months (see chosen_on_date) This is regardless of whether the guard is your primary guard or not. We might increase the lifetime in the future, but probably the precise value should not matter too much since it's a constant change. [0]: see choose_good_entry_server() -> router_choose_random_node() -> smartlist_choose_node_by_bandwidth_weights()

1 0

Re: [tor-dev] Hashring understanding
by George Kadianakis 04 Feb '16

04 Feb '16

Ola Bini <obini(a)thoughtworks.com> writes: > Hi, > > Sorry for this - had some questions about the hashring component of > #259 that I haven't been able to figure out myself. I'm sure it's just > me being unused to the Tor code base and how you write your proposals, > but it would be super helpful if I can get a quick answer to a few > questions. > > First, my simplified understanding of the data structures is this: > GUARDS are all the guards in the consensus > > UTOPIC_GUARDS is all utopic guards from GUARDS > DYSTOPIC_GUARDS is all dystopic guards from GUARDS > > UTOPIC_GUARDLIST is a subset of UTOPIC_GUARDS > DYSTOPIC_GUARDLIST is a subset of DYSTOPIC_GUARDS > > We will only ever choose guards to use from UTOPIC/DYSTOPIC_GUARDLIST > > The idea of the hashring is to be a structure to make it possible for us to > go from UTOPIC/DYSTOPIC_GUARDS to UTOPIC/DYSTOPIC_GUARDLIST. > The reason we want to avoid shifting is because we want the hashring to be a > long lived data structure that can be cheaply updated over p> several consensuses. > > Here are my questions: > - Is the above understanding correct? Yes, that's also my understanding. I think what prop259 tries to do with that hashring construction is to provide a deterministic way to sample guards, with minimum shifting as time passes. The aim here is to make it harder to track and fingerprint clients based on their guard lists. The idea is that when you are in WiFi "home" you always have one set of guards, and when you move to the WiFi "cafe" you have a different set of guards. See #10969 for some background and check http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-01-19-16.03.log.html starting from 16:21:58 for some recent discussion. Personally, I think that the hashring is kind of a red herring in the sense that it makes prop259 more complex than it needs to be, and it also tries to tackle a problem that we don't really know how to solve. I mean even if we have the hashring construction, how do we use it? Do we just blend in the MAC of the network gateway when we sample guards? What happens if an evil gateway changes its MAC constantly, till we sample the guards it wants? I think solving the guard fingerprinting issue should be the subject of a separate proposal. Isis suggested this during the meeting but I was too dump at that point. Until we have a proper solution here I'd suggest we stick to the KISS thing, which probably is to sample guards from a simple list weighted by their bandwidth like Tor is currently doing (see node_sl_choose_by_bandwidth()). Of course, if you guys are interested in this and want to simulate the hashring idea (or any othe similar idea) that's fine as well. That's the good part of a simulation; that we don't need to commit to any particular construction. > - At what point do we add/update/change guards in UTOPIC/DYSTOPIC_GUARDLIST? I think prop259 suggests that we populate the guardlists the first time Tor attempts to connect to a network. This is different to how Tor currently does it, where it starts with a minimal guardlist (one guard), and adds new guards to it on demand (when the first guard fails). Both approaches have their merits. I'm not sure which one is better. We add more guards to guardlists if all the previous guards don't work anymore. This can be seen in step 4 of §2. We also need to mark (update) guards as unreachable when they get removed from the consensus, or when we fail to connect to them. Indeed the proposal should be updated to better specify when these actions happen and how they work. > - Since each key in the hashring structure depends acutely on bw_weight_total > and the BW of the guard, how will it be possible to update it > efficiently? My assumption is that the bw_weight_total will change on every > consensus, and at that point all the old keys are invalid?

1 0

(no subject)
by Christian Sturm 04 Feb '16

04 Feb '16

Hello, a while ago I set up a BSD buildbot for tor[1]. It's actually not just for BSDs, but also for other systems, such as Solaris. I wanted to make it easier to notice when something breaks on non-popular systems, not actively used by the developers. There also is reporting through an IRC bot and I wanted to ask if it would be interesting to have it report there. Currently it just lurks in there and lacks the permission to report build failures. So it would be great if the user 'bsdbuildbot' could get those permissions. I was wondering whether this is something that developers are interested in and if so what kind of reporting they'd like to have. Currently (if it would be allowed to) it would report individual build failures and success, but of course if that's too noisy could also change to just notify when this changes. Thanks to weasel there also is a FreeBSD builder for Jenkins. However that's only FreeBSD for now. The buildbot already has some other operating systems as build slaves and it's rather simple to set it up. Chris [1] http://buildbot.pixelminers.net/

1 1

Next (and subsequent) proposal discussion meetings
by Nick Mathewson 04 Feb '16

04 Feb '16

Hi, friends! This is a followup to the previous thread on the subject of proposal reviews. If you didn't read it, you can find it here: https://lists.torproject.org/pipermail/tor-dev/2016-January/010219.html [If you don't know how proposals work, see Lunar's explanation here: https://lists.torproject.org/pipermail/tor-dev/2016-January/010246.html ] First, the next meeting is scheduled on #tor-dev this Thursday, at 9:00 am EST (=1400 UTC). We'll be discussing these proposals: * Proposal 263: Postquantum circuit extension * Proposal 249: Large create cells Second, I'm trying to use the wiki to plan and schedule these meetings going forward. I've asked people who need or want to attend them to comment on a set of proposed times, and tried to list the people who really need to be at each one. The wiki page is at: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Meeting… I'll try to start outlining meeting times roughly a month in advance there, and finalizing them at least a week in advance. (No promises; it will take a while to ramp up.) best wishes, -- Nick

5 7

Re: [tor-dev] Getting people started on guard selection algorithms [prop259]
by George Kadianakis 03 Feb '16

03 Feb '16

> On Tue, Feb 02, 2016 at 02:29:22PM -0500, Ola Bini wrote: >> Hi, >> >> We have now started looking at the proposal and the existing code. Our >> current plan is to first of all code more of the simulations referred >> to in stuff-to-test.txt. We also noticed that the hashring >> implementation for choosing the [DYSTOPIC/UTOPIC]_GUARDLIST hasn't >> been implemented in the simulation code - in fact, in the code it >> seems the [DYSTOPIC/UTOPIC]_GUARDS is used as the guardlist. We are >> planning on implementing the two different varieties of the guardlist >> selection algorithm as well. >> Sounds good. Another thing to note is that during our last meeting, we decided to not have the utopic/dystopic guardlists be disjoint (mainly for load balancing reasons): https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html Check the meeting logs if you care for more info. Unfortunately, I think prop259 has not been updated to specify this new behavior. This might be too researchy for what you are trying to do, but finding a nice behavior here would be very helpful. Here are is an example idea of doing the 80/443 fascist firewall detection heuristic without two disjoint guard pools: Alice initializes a single guard list with all the guard nodes. Then she does steps 1 to 4 from §2 of prop259. Then in step 5, if Alice has tried more than GUARDLIST_FAILOVER_THRESHOLD guards from her guard list, she goes into "dystopic firewall" mode. During this mode, Alice only picks 80/443 nodes as guards (maybe from a separate dystopic guardlist). If those don't work either (she tries GUARDLIST_FAILOVER_THRESHOLD of them), then Alice "should make a note to herself that the network has potentially gone down" as suggested by step 5.b. So in the above idea, there are two guardlists. Guardlist GUARDS has all the guards in the network, and then there is a DYSTOPIC_GUARDS guard list (which is a subset of GUARDS) which is only used during "dystopic firewall" mode. I think this has better load balancing and anonymity properties. But there might be even better behaviors. Feel free to come up with your own and test them! >> After that, we are going to start looking at where it fits in the main >> Tor code base. >> Sounds good. You might enjoy choose_random_entry_impl() as a starting point. Feel free to ask us any questions you have about the Tor code base. Either here or on IRC. >> The only thing I'm a bit unclear about from the specification is the >> idea of primary guards, and what the procedure is when no primary >> guards are possible - 259.§2.3 talks about "all available and fitting >> entry guards" - is this from the list of primary guards or the >> guardlist? >> IIUC it's from the whole guardlist, but this should only happen if the primary guards are unreachable. The idea with primary guards is that in an ideal world, a Tor client would always only connect to the top guard of its guardlist. To expose itself minimally to the network. Unfortunately, the network is fiddly so this is not possible because the top guard will eventually go down. The concept of primary guards tries to compensate for that, by going to extra lengths to ensure that at least you always connect to one of your N=3 top guards in your guard list. It does this by periodically checking the reachability of those top N=3 guards, and marking them online if they are (see step 2 of §2). So, even if you or your guards have reachability issues and you drift on your 12th guard or something, you will eventually come back to one of your primary guards when they are found online again. Also, for the above heuristic to work, nodes that are not listed in the latest consensus should not be considered primary guards. >> 259.§2.4 says "adds a new entry guard" - is that adding it to the list >> of primary guards or something else? >> Good question. I'm not sure what the proposal means there. Maybe isis can clarify this further? Here is an attempt to help. Hope I don't confuse you further. The currently implemented Tor guard algorithm keeps a list USED_GUARDS of the guards it has already connected to (it's also saved on disk and is the guard list you see in your Tor state file). Everytime Tor tries a new guard node, it adds it to USED_GUARDS. The top N guards of USED_GUARDS are the primary guards. If you have exhausted all of the guards in USED_GUARDS and you still can't connect, then you need to add a new node to USED_GUARDS and attempt to connect to it. I imagine that the "list of all available and fitting entry guards" referenced in step 3 is something like USED_GUARDS. (It's actually not called USED_GUARDS in the code. I just named it like this for this email.)

1 0

Git users, enable fsck by default!
by Nick Mathewson 02 Feb '16

02 Feb '16

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=813157 for more information why. The tl;dr here is: * By default Git doesn't verify the sha1 checksums it receives by default. * It doesn't look like we've got any inconsistencies in our repositories I use, though. That's good! * To turn on verification, I think you run: git config --add transfer.fsckobjects true git config --add fetch.fsckobjects true git config --add receive.fsckobjects true peace, -- Nick

4 3

Re: [tor-dev] Monitoring Service
by Karsten Loesing 02 Feb '16

02 Feb '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Tom, On 27/01/16 20:02, Tom Ritter wrote: > [feel free to reply adding tor-project or whomever] Sure, let me copy tor-dev@. > Remember a while ago I lamented that I wished there was some > monitoring service that could tell me when my metrics service; > relay; or bwauth went down? I finally built one. I'm still kicking > the tires on it, and I intend to improve it more over the next week > or two - but I think it's here to stay. > > https://github.com/tomrittervg/checker > > Right now I have it running several monitoring jobs, with a second > instance running with no jobs but serving as a peer. I have it > checking a number of TCP ports (to see if my relays are still up), > and I have custom jobs for metrics and the bwauth. They're in the > samplejobs folder. They're very simplistic and bare-bones. My > hope is that they can be fleshed out over time to account for more > imaginative ways things could fail. > > I'm already discovering that my bwauth file sometimes gets more > than two hours behind.... > > But I think the most useful thing here is that now I have a > minimal framework for writing simplistic python jobs and having it > monitor things for me. Maybe it would be useful for more people? Yes! Well, I can't speak for other people, but having a monitoring system for Metrics-related services would be very useful for me. In fact, it's been on my list for a long time now. This seems like a great opportunity to spend more thoughts on it. I'm not sure if I mentioned this before, but we're using Nagios to monitor Onionoo. The Nagios script we're using makes a tiny request to Onionoo to see whether the contained timestamps are still recent. That's an indirect way to notice problems with the data back-end, and it has helped with detecting numerous problems in the past. More details here: https://gitweb.torproject.org/admin/tor-nagios.git/tree/tor-nagios-checks/c… So, I don't know Nagios enough to say how it compares to your system. But I could imagine that we write a similar check for CollecTor that runs on your system and notifies us of problems. And maybe it's possible to write that script in a way that it can also be deployed on Nagios. Here's what I could imagine that the script would do: every, say, 10 minutes it would fetch CollecTor's index.json which is specified here: https://collector.torproject.org/#index-json The script would then run a series of checks and report one of statuses OK, WARNING, CRITICAL, or UNKNOWN: - host is unreachable or index.json cannot be found for at least 30 minutes (CRITICAL) - index.json contains invalid JSON for all checks in the last 30 minutes (CRITICAL) - the contained "index_created" timestamp is older than 30 minutes (WARNING) or older than 3 hours (CRITICAL) - when concatenating "path" fields of nested objects, most recent "last_modified" timestamp by path prefix is more than X behind "index_created" (CRITICAL): - /archive/bridge-descriptors/: 5 days - /archive/exit-lists/: 5 days - /archive/relay-descriptors/certs.tar.xz: 5 days - /archive/relay-descriptors/consensuses/: 5 days - /archive/relay-descriptors/extra-infos/: 5 days - /archive/relay-descriptors/microdescs/: 5 days - /archive/relay-descriptors/server-descriptors/: 5 days - /archive/relay-descriptors/votes/: 5 days - /archive/torperf/: 5 days - /recent/torperf/: 12 hours - /recent/bridge-descriptors/extra-infos/: 3 hours - /recent/bridge-descriptors/server-descriptors/: 3 hours - /recent/bridge-descriptors/statuses/: 3 hours - /recent/exit-lists/: 3 hours - /recent/relay-descriptors/consensuses/: 1.5 hours - /recent/relay-descriptors/extra-infos/: 1.5 hours - /recent/relay-descriptors/microdescs/consensus-microdesc/: 1.5 hours - /recent/relay-descriptors/microdescs/micro/: 1.5 hours - /recent/relay-descriptors/server-descriptors/: 1.5 hours - /recent/relay-descriptors/votes/: 1.5 hours In the detailed checks above, the script would not warn if index.json does not contain any files with a given prefix (so that you can run the script on your CollecTor instance that doesn't collect all the things). And ideally, the script would include all warnings in its output, not just the first. That's one check, and it would probably catch most problems where things get stale. I would like to add more checks, but those would need more access to the CollecTor host than its index.json which is publicly available. (That could mean that we export more status in a debug.json or in another public place.) Some examples as follows, and I'm only listing them for later: - Is the host soon going to run out of disk space or inodes? (This can easily be done with Nagios, I think.) - Did the relay-descriptor part of CollecTor fail to parse a descriptor using metrics-lib and hence did not store it to disk? (I'm receiving hourly cron mails in this case, but I'd prefer something better.) - Are we missing more than a certain threshold of relay descriptors referenced from other relay descriptors? For example, are we missing more than 0.5% of server descriptors referenced from the consensus? (I'm also receiving hourly cron mails, but the checks there are somewhat broken and can create a lot of noise.) - Same as above but for sanitized bridge descriptors. (I don't have such checks in place right now.) - Does the bridge sanitizer have trouble sanitizing bridge IP addresses in particular storing secret keys for the keyed hash function? (Cron would send mail, but that never happened so far.) - Did the bridge sanitizer run into an unknown line and hence refrain from storing the sanitized version of a bridge descriptor? (Cron sends mail which happens every now and then.) - Do the exit lists downloaded from TorDNSEL contain recent publication times but no recent scan times? - Does one or more of the Torperf hosts not report recent measurements? There, that's a long list. And it's just the list for CollecTor. Next would be more detailed checks for Onionoo that look into its internal operation or into problems with its web-facing part; Metrics including the various data-aggregating scripts running in the background; and ExoneraTor with its database importer to make sure it always has fresh and complete descriptors. Hope this was not too overwhelming. If you're still reading this and still want to help, let's talk more about writing that first script for CollecTor. Thanks! All the best, Karsten -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWsIYbAAoJEJD5dJfVqbCrY3EH/00mw1INpPniTRXnd981nJEl EAZdySgKNcuA6NIVrGMrRdRgH3p3W2UhVJYlyHpHquZOx0nLmEDH8XDyBcxvICEQ UviwPGZGiQCw+TYqFYlUPvM2d0DwZmjyQbMeoANo4r8BczqFKmwi9GC1xI/zc3/3 UT91OXyZeKIbhopaeyIWW868p/u8Cs9sGDSJuWZHmDqQjPL2sE0lM8bQtdK1wJWj UNTCYPT35FfSHYeuWXASDgqt0e8qrppWKhEzQgmWggAU6sPx1MMSz323fXymdPCX OyHbXYjzFsABE6+CAO5iR1c6AI3TFHxOtVutA8gfXC52GjgDApVETd+kywrNSZM= =kRdX -----END PGP SIGNATURE-----

1 0

Want to help in community team
by Carolin Zöbelein 02 Feb '16

02 Feb '16

--------------------------------------- Want to help in community team => Question: Is this the right mailing list for this? --------------------------------------- Hello, for several weeks I have already watched the goings-on at Tor (irc, blog, wiki, ...). Now I decided to say hello. I'm a very creative and autistic artist, satirist (writing), photographer, physicist, ... . Caused by some talent I work as mathematical adviser and now I'm also studying math to pretend more professionalism. Also I do what I call "creative (crazy) Nerd-consultant" after I didn't found an already existing job title which describes what I do. People say: "You are extremely strange, crazy and creative" and I thought I should use this. Sadly I'm not the best programmer although I earned my money with it in the past. Or better I'm able to code but I prefer to spend my time with math or other creative work. It just makes more fun. I think I'm best suited into the community team with my skills. But I'm also open for other tasks of course ;-). > > Monday, 18.01.2016, Georg Koppen <gk(a)torproject.org>: > Hi, > > I realized recently that there seem to be no community team meetings > anymore (is that the case or am I just missing something?). At least not > in #tor-outreach. Nor are there status reports sent to tor-reports. > > Is there any other way to get to know what the community team is > currently discussing/planning/concerned with? > > Georg > > > _______________________________________________ > tor-project mailing list > tor-project(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project > That's a good question. In particular for new people like me. What's the typical way to get involved in the community team? I hope I'm not too crazy for Tor people and don't know if and how I can help. So that's enough for a public mail. Bye, Carolin -- ________________________________________________________________________ URL: http://www.carolin-zoebelein.de E-mail: contact(a)carolin-zoebelein.de PGP: D4A7 35E8 D47F 801F 2CF6 2BA7 927A FD3C DE47 E13B

2 1

Applications Team Meeting Next Tues (Feb 9) at 19:00 UTC
by Mike Perry 01 Feb '16

01 Feb '16

Due to conflicts with other meetings, Georg and I both will be busy tomorrow during the regularly scheduled applications meeting (first Tuesday of each month at 19:00 UTC). So instead, we will have this month's Applications Team meeting next Tuesday, Feb 9, at 19:00 UTC. As usual, we will discuss usability, configuration, and any recent user study progress. Anyone interested in these topics for any user-facing Tor application is welcome and encouraged to participate! -- Mike Perry

1 0

Proposal: Load Balancing with Overhead Parameters
by Mike Perry 31 Jan '16

31 Jan '16

This proposal aims to allow us to load balance properly between Guard, Middle, and Exit nodes with the addition of padding traffic to the network. Canonical proposal current lives in my load_balancing-squashed branch: https://gitweb.torproject.org/user/mikeperry/torspec.git/tree/proposals/xxx… Here is the text in-line, for ease of commenting on-list: ============================================================== Filename: xxx-load-balancing-with-overhead.txt Title: Load Balancing with Overhead Parameters Authors: Mike Perry Created: 01 January 2016 Status: Draft 0. Motivation In order to properly load balance in the presence of padding and non-negligible amounts of directory and hidden service traffic, the load balancing equations in Section 3.8.3 of dir-spec.txt are in need of some modifications. In addition to supporting the idea of overhead, the load balancing equations can also be simplified by treating Guard+Exit nodes as Exit nodes in all cases. This causes the 9 sub-cases of the current load balancing equations to consolidate into a single solution, which also will greatly simplify the consensus process, and eliminate edge cases such as #16255[1]. 1. Overview For padding overhead due to Proposals 251 and 254, and changes to hidden service path selection in Proposal 247, it will be useful to be able to specify a pair of parameters that represents the additional traffic present on Guard and Middle nodes due to these changes. The current load balancing equations unfortunately make this excessively complicated. With overhead factors included, each of the 9 subcases goes from being a short solution to over a page of calculations for each subcase. Moreover, out of 8751 hourly consensus documents produced in 2015[2], only 78 of them had a non-zero weight for using Guard+Exit nodes in the Guard position (weight Wgd), and most of those were well under 1%. The highest weight for using Guard+Exits in the Guard position recorded in 2015 was 2.62% (on December 10th, 2015). This means clients that chose a Guard node during that particular hour used only 2.62% of Guard+Exit flagged nodes' bandwidth when performing a bandwidth-weighted Guard selection. All clients that chose a Guard node during any other hour did not consider Guard+Exit nodes at all as potential candidates for their Guards. This indicates that we can greatly simplify these load balancing equations with little to no change in diversity to the network. 2. Simplified Load Balancing Equations Recall that the point of the load balancing equations in section 3.8.3 of dir-spec.txt is to ensure that an equal amount of client traffic is distributed between Guards, Middles, Exits, and Guard+Exits, where each flag type can occupy one or more positions in a path. This allocation is accomplished by solving a system of equations for weights for flag position selection to ensure equal allocation of client traffic for each position in a circuit. If we ignore overhead for the moment and treat Guard+Exit nodes as Exit nodes, then this allows the simplified system of equations to become: Wgg*G == M + Wme*E + Wmg*G # Guard position == middle position Wgg*G == Wee*E # Guard position == equals exit position Wmg*G + Wgg*G == G # Guard allocation weights sum to 1 Wme*E + Wee*E == E # Exit allocation weights sum to 1 This system has four equations and four unknowns, and by transitivity we ensure that allocated capacity for guard, middle, and exit positions are all equal. Unlike the equations in 3.8.3 of dir-spec.txt, there are no special cases to the solutions of these equations because there is no shortage of constraints and no decision points for allocation based on scarcity. Thus, there is only one solution. Using SymPy's symbolic equation solver (see attached script) we obtain: E + G + M E + G + M 2*E - G - M 2*G - E - M Wee: ---------, Wgg: ---------, Wme: -----------, Wmg: ------------ 3*E 3*G 3*E 3*G For the rest of the flags weights, we will do the following: Dual-flagged (Guard+Exit) nodes should be treated as Exits: Wgd = 0, Wmd = Wme, Wed = Wee Directory requests use middle weights: Wbd=Wmd, Wbg=Wmg, Wbe=Wme, Wbm=Wmm Handle bridges and strange exit policies: Wgm=Wgg, Wem=Wee, Weg=Wed 2.1. Checking for underflow and overflow In the old load balancing equations, we required a case-by-case proof to guard against overflow and underflow, and to decide what to do in the event of various overflow and underflow conditions[3]. Even still, the system proved fragile to changes, such as the implementation of Guard uptime fractions[1]. Here, with the simplified equations, we can plainly see that the only time that a negative weight can arise is in Wme and Wmg, when 2*E < G+M or when 2*G < E+M. In other words, only when Exits or Guards are scarce. Similarly, the only time that a weight exceeding 1.0 can arise is in Wee and Wgg, which also happens when 2*E < G+M or 2*G < E+M. This means that parameters will always overflow in pairs (Wee and Wme, and/or Wgg and Wmg). In both these cases, simply clipping the parameters at 1 and 0 provides as close of a balancing condition as is possible, given the scarcity. 3. Load balancing with Overhead Parameters Intuitively, overhead due to padding and path selection changes can be represented as missing capacity in the relevant position. This means that in the presence of a Guard overhead fraction of G_o and a Middle overhead fraction of M_o, the total fraction of actual client traffic carried in those positions is (1-G_o) and (1-M_o), respectively. Then, to achieve a balanced allocation of traffic, we consider only the actual client capacity carried in each position: # Guard position minus overhead matches middle position minus overhead: (1-G_o)*(Wgg*G) == (1-M_o)*(M + Wme*E + Wmg*G) # Guard position minus overhead matches exit position: (1-G_o)*(Wgg*G) == 1*(Wee*E) # Guard weights still sum to 1: Wmg*G + Wgg*G == G # Exit weights still sum to 1: Wme*E + Wee*E == E Solving this system with SymPy unfortunately yields some unintuitively simplified results. For each weight, we first show the SymPy solution, and then factor that solution into a form analogous to Section 2: -(G_o - 1)*(M_o - 1)*(E + G + M) Wee: --------------------------------------- E*(G_o + M_o - (G_o - 1)*(M_o - 1) - 2) (1 - G_o)*(1 - M_o)*(E + G + M) Wee: --------------------------------------- E*(2 - G_o - M_o + (1 - G_o)*(1 - M_o)) (M_o - 1)*(E + G + M) Wgg: --------------------------------------- G*(G_o + M_o - (G_o - 1)*(M_o - 1) - 2) (1 - M_o)*(E + G + M) Wgg: --------------------------------------- G*(2 - G_o - M_o + (1 - G_o)*(1- M_o)) -E*(M_o - 1) + G*(G_o - 1)*(-M_o + 2) - M*(M_o - 1) Wmg: --------------------------------------------------- G*(G_o + M_o - (G_o - 1)*(M_o - 1) - 2) (2 - M_o)*G*(1 - G_o) - M*(1 - M_o) - E*(1 - M_o) Wmg: --------------------------------------------------- G*(2 - G_o - M_o + (1 - G_o )*(1 - M_o)) E*(G_o + M_o - 2) + G*(G_o - 1)*(M_o - 1) + M*(G_o - 1)*(M_o - 1) Wme: ----------------------------------------------------------------- E*(G_o + M_o - (G_o - 1)*(M_o - 1) - 2) (2 - G_o - M_o)*E - G*(1 - G_o)*(1 - M_o) - M*(1 - G_o)*(1 - M_o) Wme: ----------------------------------------------------------------- E*(2 - G_o - M_o + (1 - G_o)*(1 - M_o)) A simple spot check with G_o = M_o = 0 shows us that with zero overhead, these solutions become identical to the solutions in Section 2 of this proposal. The final condition that we need to ensure is that these weight values never become negative or greater than 1.0[3]. 3.1. Ensuring against underflow and overflow Note that if M_o = G_o = 0, then the solutions and the overflow conditions are the same as in Section 2. Unfortunately, SimPy is unable to solve multivariate inequalities, which prevents us from directly deriving overflow conditions for each variable independently (at least easily and without mistakes). Wolfram Alpha is able to derive closed form solutions to some degree for this, but they are more complicated than checking the weights for underflow and overflow directly. However, for all overflow and underflow cases, simply warning in the event of overflow or underflow in the weight variable solutions above is equivalent anyway. Optimal load balancing given this scarcity should still result if we clip the resulting solutions to [0, 1.0]. It will be wise in the implementation to test the overflow conditions with M_o = G_o = 0, and with their actual values. This will allow us to know if the overflow is a result of inherent unbalancing, or due to input overhead values that are too large (and need to be reduced by, for example, reducing padding). 4. Consensus integration 4.1. Making use of the Overhead Factors In order to keep the consensus process simple on the Directory Authorities, the overhead parameters represent the combined overhead from many factors. The G_o variable is meant to account for sum of directory overhead, netflow padding overhead, future two-hop padding overhead, and future hidden service overhead (for cases where Guard->Middle->Exit circuits are not used). The M_o variable is meant to account for multi-hop padding overhead, hidden service overhead, as well as an overhead for any future two-hop directory connections (so that we can consolidate Guards and Directory guard functionality into a single Guard node). There is no need for an E_o variable, because even if there were Exit-specific overhead, it could be represented by an equivalent reductions in both G_o and M_o instead. Since all relevant padding and directory overhead information is included in the extra-info documents for each relay, the M_o and G_o variables could be computed automatically from these extra-info documents during the consensus process. However, it is probably wiser to keep humans in the loop and set them manually as consensus parameters instead, especially since we have not previously had to deal with serious adversarial consequences from malicious extra-info reporting. For clarity, though, it may be a good idea to separate all of the components of M_o and G_o into separate consensus parameters, and combine them (via addition) in the final equations. That way it will be easier to pinpoint the source of any potential overflow issues. This separation will also enable us to potentially govern padding's contribution to the overhead via a single tunable value. 4.2 Integration with Guardfraction The GuardFraction changes in Proposal 236 and #16255 should continue to work with these new equations, so long as the total T, G, and M values are counted after the GuardFraction multiplier has been applied. 4.3. Guard flag assignment Ideally, the Guard flag assignment process would also not count Exit-flagged nodes when determining the Guard flag uptime and bandwidth cutoffs, since we will not be using Guard+Exit flagged nodes as Guard nodes at all when this change is applied. This will result in more accurate thresholds for Guard node status, as well as better control over the true total amount of Guard bandwidth in the consensus. 1. https://trac.torproject.org/projects/tor/ticket/16255 2. https://collector.torproject.org/archive/relay-descriptors/consensuses/ 3. http://tor-dev.torproject.narkive.com/17H9FewJ/correctness-proof-for-new-ba… Appendix A: SymPy Script for Balancing Equation Solutions #!/usr/bin/python from sympy.solvers import solve from sympy import simplify, Symbol, init_printing, pprint # Sympy variable declarations (G,M,E,D) = (Symbol('G'),Symbol('M'),Symbol('E'),Symbol('D')) (Wgd,Wmd,Wed,Wme,Wmg,Wgg,Wee) = (Symbol('Wgd'),Symbol('Wmd'),Symbol('Wed'), Symbol('Wme'),Symbol('Wmg'),Symbol('Wgg'), Symbol('Wee')) (G_o, M_o) = (Symbol('G_o'),Symbol('M_o')) print "Current Load Balancing Equation Solutions, Case 1:" pprint(solve( [Wgg*G + Wgd*D - (M + Wmd*D + Wme*E + Wmg*G), Wgg*G + Wgd*D - (Wee*E + Wed*D), Wed*D + Wmd*D + Wgd*D - D, Wmg*G + Wgg*G - G, Wme*E + Wee*E - E, Wmg - Wmd, 3*Wed - 1], Wgd, Wmd, Wed, Wme, Wmg, Wgg, Wee)) print print "Case 1 with guard and middle overhead: " pprint(solve( [(1-G_o)*(Wgg*G + Wgd*D) - (1-M_o)*(M + Wmd*D + Wme*E + Wmg*G), (1-G_o)*(Wgg*G + Wgd*D) - (Wee*E + Wed*D), Wed*D + Wmd*D + Wgd*D - D, Wmg*G + Wgg*G - G, Wme*E + Wee*E - E, Wmg - Wmd, 3*Wed - 1], Wgd, Wmd, Wed, Wme, Wmg, Wgg, Wee)) print "\n\n" print "Elimination of combined Guard+Exit flags (no overhead): " pprint(solve( [(Wgg*G) - (M + Wme*E + Wmg*G), (Wgg*G) - 1*(Wee*E), Wmg*G + Wgg*G - G, Wme*E + Wee*E - E], Wme, Wmg, Wgg, Wee)) print print "Elimination of combined Guard+Exit flags (Guard+middle overhead): " combined = solve( [(1-G_o)*(Wgg*G) - (1-M_o)*(M + Wme*E + Wmg*G), (1-G_o)*(Wgg*G) - 1*(Wee*E), Wmg*G + Wgg*G - G, Wme*E + Wee*E - E], Wme, Wmg, Wgg, Wee) pprint(combined)

4 7

Fwd: Need to know if this is a bug
by Siddhartha De 29 Jan '16

29 Jan '16

Dear all , I just found out that the tor.exe included in the Tor Browser ( ver 5.5 ) for Windows bundle ( Tor ver 0.2.7.6 ) does not support TransPort and TransListenAddress ... The error message given was :- Jan 29 23:16:24.129 [warn] Failed to parse/validate config: TransPort and TransListenAddress are disabled in this build. Jan 29 23:16:24.129 [err] Reading config failed--see warnings above. Is this a deliberate design consideration or a bug ... If we keep the tor.exe of both the browser bundle and expert bundles synced I think it would be easier for users who would like to use Tor both as a browser as well as a Transparent Proxy . Thanking you , Yours sincerely , Siddhartha De

2 1

Re: [tor-dev] Fwd: [guardian-dev] RC-7 is out - try to repro? Re: towards reproducible Orbot
by Nathan Freitas 29 Jan '16

29 Jan '16

On Thu, Jan 28, 2016, at 02:27 PM, Ximin Luo wrote: > Hey, thanks for this! > > I'd like to ask please upload this (or 15.1) to F-Droid soon. The version > currently in F-Droid is RC-4 and is broken with orwall - > orwall-allowed/redirected apps see no internet, but tor is connected and > orfox can access it. RC-7 fixes this issue, I just tested it out. Yes, will do. Thanks for testing.

1 0

Proposals should have reviews. Let's make sure that happens. Here's a schedule.
by Nick Mathewson 29 Jan '16

29 Jan '16

So, one of the longstanding problems with Tor's (change) proposal system has been that proposals sit around for a long time without sufficient discussion or approval. When they're about to be implemented, we do an informal "hey did anybody look at that?" check, but that's not really good enough. Here's what I've been thinking of doing, as discussed at the tor dev IRC meeting yesterday. 1. Everybody who will review proposals* should nominate one or two proposals currently in state "Open" or "Draft". People should nominate proposals that they themselves didn't write. 2. I'll schedule review & discussion sessions, about one per week, trying to hit times where interested people can make it. Everybody who will review proposals* should try to show up if they can, having read the proposal or proposals under discussion for that week. Then we talk about the proposal(s)! 3. We'll try to move nominated proposals towards "Accepted" or "Rejected", improving them as necessary. * I'm not going to be too elitist here, but: If you show up and contribute usefully to discussions, I will prioritize your nominations more than I will prioritize nominations from people who don't contribute so much. This is a draft process, so we'll try to see after a month or two (possibly at the next face-to-face dev meeting?) how it's going. Here's the schedule I have in mind for the first meeting, based on proposals already nominated at the IRC dev meeting. I'm going to go a bit aggressively for the first couple of weeks, so that we can get a sense of how this process works out. Everything is subject to change. We'll be putting notes online from these meetings. Tuesday January 19: 11am Eastern (1600 UTC) Guard node proposals! We'll be talking about these proposals: Prop#259: New Guard Selection Behaviour [DRAFT] Prop#241: Resisting guard-turnover attacks [DRAFT] Prop#247: Defending Against Guard Discovery Attacks using Vanguards [DRAFT] Friday January 22: 11 am Eastern (1600 UTC) Prop#251: Padding for netflow record resolution reduction [DRAFT] Monday January 25: 8pm Eastern (0100 UTC on Tuesday Jan 26) Prop#250: Random Number Generation During Tor Voting [DRAFT] Friday January 29: 8:30 am eastern (1330 UTC) Prop#252: Single Onion Services [DRAFT] Prop#260: Rendezvous Single Onion Services [DRAFT] Prop#246: Merging Hidden Service Directories and Introduction Points [OPEN] Also nominated are: Prop#264: Putting version numbers on the Tor subprotocols [OPEN] Prop#257: Refactoring authorities and taking parts offline [DRAFT] Prop#258: Denial-of-service resistance for directory authorities [OPEN] -- Nick

8 15

Fwd: [guardian-dev] RC-7 is out - try to repro? Re: towards reproducible Orbot
by Nathan Freitas 28 Jan '16

28 Jan '16

With this release and updates to our build process, we are close to having Orbot be reproducible. Otherwise, this should be the last RC for v15.1, and it now includes built-in Obfs4, Obfs3, Scramblesuit bridges from TBB, plus a few new ones from Nima. This will make it much easier for users in places who need bridges to be able to use them without taking an extra step. ----- Original message ----- From: Nathan of Guardian <nathan(a)guardianproject.info> To: guardian-dev(a)lists.mayfirst.org Subject: [guardian-dev] RC-7 is out - try to repro? Re: towards reproducible Orbot Date: Thu, 28 Jan 2016 10:12:59 -0500 On Wed, Jan 27, 2016, at 10:10 AM, Hans-Christoph Steiner wrote: > >>>>> Then anyone will be able to run this to verify the build: > >>>>> > >>>>> git checkout 15.1.0-RC-4 > >>>>> ./make-release-build > >>>>> ./compare-to-official-release ~/Downloads/Orbot-v15.1.0-RC-4.apk \ > >>>>> bin/Orbot-v15.1.0-RC-4.apk I've just posted RC7 built from make-release-build. I haven't attempted to run the compare script yet, so if someone out there wants to try, that would be great. My bet is that I didn't do something right, and it won't match, but I would love to be proven wrong! APK: https://guardianproject.info/releases/Orbot-v15.1.0-RC-7.apk Sig: https://guardianproject.info/releases/Orbot-v15.1.0-RC-7.apk.asc /** 15.1.0-RC-7 / 27-January-2016 / 91225ab053d0ffec4a414be461ee41e6465446bd **/ * fd45fa3 enable TransProxy and DNSPort by default without root - some users run their own iptables transproxy scripts with AFWall and n /** 15.1.0-RC-6 / 27-January-2016 / a8dbdacbcb2412bb08c4a665145371c3ac4abef1 **/ Fixes to enable/attempt reproducible builds for this release * 226d92e make-release-build: env vars need to be first on the command line * 20c16ae ignore more files generated by the build * 9883a89 faketime is only needed when building, not when cleaning * eaa2dde explicitly indicate we are building armeabi * 1b76c36 add make calls for both armeabi and x86 with clean * 75eb36e remove lib binaries * 3be93a0 only build for x86 and armeabi ...and one small UI bug: 1839b8f add-in missing "break" for Meek Google -- Nathan of Guardian nathan(a)guardianproject.info _______________________________________________ List info: https://lists.mayfirst.org/mailman/listinfo/guardian-dev To unsubscribe, email: guardian-dev-unsubscribe(a)lists.mayfirst.org

2 1

Getting people started on guard selection algorithms [prop259]
by George Kadianakis 28 Jan '16

28 Jan '16

Hello Nick, isis and Mike! It seems like some Thoughtworks folks (from here on referred to as FRITO [0]) are interested in helping us with Tor hidden services development for 4 weeks or so. They start early next week! :) They were looking for a coding project that is tractable within a month given a team of four people. Also, it should be fun, interesting and be related to hidden services. One of the projects we discussed and found potentially fitting was proposal 259: https://gitweb.torproject.org/torspec.git/tree/proposals/259-guard-selectio… It is my undertanding that the next steps here involve implementing the various proposed algorithms and evaluating how they cope in some simulated scenarios. Nick and Isis started working on this in Python here: https://github.com/isislovecruft/guardsim You can find the scenarios that need to be tested here: https://github.com/isislovecruft/guardsim/blob/master/doc/stuff-to-test.txt This project seems quite suitable for the FRITO crew :) The idea here is that after we decide which one is the best algorithm to use, we would implement it for little-t-tor in C. The best algorithm is the one that achieves the greatest reachability and security tradeoff. The algorithm should also be designed in such a way that fits into the tor asynchronous networking API. It would be amazing if FRITO could help us here as well, but maybe we can start up with the Python simulation and see if they like the project! Isis and Nick, are you currently working on the guardsim code? Would you be OK if it was taken over by FRITO? Any advice for them? Also, is there a ticket we should be using for this project? If not, I will make one ASAP. Cheers! [0]: FRITO is derived from their names: Fan, Reinaldo, Ivan, Tania and Ola. They are all CCed. PS: We recently did a reading group about prop259 and a few other proposals that you might find interesting. The minutes can be found here: https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html and the raw logs here: http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-01-19-16.03.log.html

2 1

Tor not affected by recent openssl security advisories
by Nick Mathewson 28 Jan '16

28 Jan '16

Somebody always asks whether Tor is affected by each OpenSSL advisory, so I'm sending this mail in order to get a URL to send people to. :) Here are today's advisories: https://mta.openssl.org/pipermail/openssl-announce/2016-January/000061.html With respect to the first ( "DH small subgroups (CVE-2016-0701)" ), Tor is not affected because we set the SSL_OP_SINGLE_DH_USE() option. We started setting this option back in Tor 0.1.1.9-alpha, back in 2005. With respect to the second ( "SSLv2 doesn't block disabled ciphers (CVE-2015-3197)" ), Tor is not affected because we disable SSLv2 by setting SSL_OP_NO_SSLv2. We started setting this option in Tor 0.0.2pre8, back in 2003. Of course, other applications that you use over Tor may be affected, even though Tor is not; please remember to upgrade them as patches become available. best wishes, -- Nick

3 3

Proposal 247: Alternate Path Lengths
by Mike Perry 27 Jan '16

27 Jan '16

While discussing proposal 247 with George yesterday, we realized that we still get security benefit from additional ephemeral hops beyond the vanguards themselves. Recall the high-level 247 path design is: C - L - M - S -- S - M - L - H Where: C = Client L = Long lived Layer1 guard M = Medium lifespan Layer2 guard S = Short lifespan Layer3 guard H = Hidden Service (or hsdir) E = Ephemeral Hop (used below) I = Intro Point (used below) R = Rend Point (used below) The benefit to just one more hop is easiest to see in the Introduction Point case, where George and I reasoned that it probably is a good idea to pick an intro point that is not the same as the Layer3 (S) set, otherwise the hidden service is effectively publishing its Layer3 guards in its descriptor, and using those same nodes to connect to rendezvous points. Clients probably also do not want the multi-visit linkability of using their layer3 vanguards to directly connect to an HS intro point. This means the intro circuit becomes: C - L - M - S - E -- I - S - M - L - H Similarly, in the rend case, hidden services probably do not want to expose their Layer3 (S) guards quite so easily to a client's chosen Rendezvous Point, and again, the client probably does not want to use their Layer3 (S) guards as its Rendezvous point, to avoid visit linkability. This means we again have 8 hops for rends: C - L - M - S - R -- E - S - M - L - H Unfortunately, this is starting to get ridiculous. While there are clear security benefits here, I think 8 hops is definitely at the point where we can forget about voice and other interactive traffic behaving reasonably. So what could we cut, if we wanted to? Well, going back to the Prop247 threat model, we want the adversary to perform at least two attacks: a Sybil and one or mode node compromise attacks. So maybe we can (in some cases, or optionally?) eliminate the M nodes from the path. Since the linkability risks may be acceptable for some applications, maybe we can also optionally allow clients or servers to omit the ephemeral hop. This basically gives us three options for path lengths. Let's consider each path type: Hsdir post/fetch: 1. C - L - M - S - E - H 2. C - L - S - E - H 3. C - L - S - H Intro: 1. C - L - M - S - E -- I - S - M - L - H 2. C - L - S - E -- I - S - L - H *3. C - L - S -- I&S - L - H (* IP Intersection attack!) Rend: 1. C - L - M - S - R -- E - S - M - L - H 2. C - L - S - R -- E - S - L - H 3. C - L - R&S -- S - L - H Looking at these, we can see that we sacrifice the middle guards in the second option, which will come at the cost of one less compromise attack (but still the need to compromise the long-lived guard). We also lose the unlinkability in the third option, and this actually bites us in Intro 3: the hidden service L guard can perform a long-term intersection attack, watching for published intro points and matching that to the circuits that H makes to them. So that path length probably should not be used. But what about the others? Especially that Rend case? I really like the security properties of the full 8 hop paths, but it seems to me that for highly-interactive applications, we can provide the option for users to give up some of the unlinkability in exchange for that 4 hop circuit, which might actually allow for e2e hidden service voice and video to have a shot at working. Are there any risks with paths this short for that case? Does it make sense to provide users with these different path length and latency options? I'm thinking that the service could list its preferred path length in its hsdesc, and the client could override that as it chooses (either for more or less security). Is that dangerous? We were already considering letting users choose their guard set sizes. Why not path lengths also (or instead)? -- Mike Perry

3 8

Notes from 3rd Tor proposal reading group [prop250]
by George Kadianakis 27 Jan '16

27 Jan '16

Hello there, yesterday we held another reading group on #tor-dev to discuss proposal 250 (Random Number Generation During Tor Voting). You can find the logs here: http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-01-26-15.58.log.html Here are some code and spec changes that we need to do after discussion with Nick: Code & spec changes: - Remove ed25519 keys from code and spec. - SHA256 -> Keccak - "SUGGESTION: On 3.1.1, wouldn't it just be easier to sort the R_b in ascending order and use them in HASHED_REVEALS?" - "Why does REVEAL put the TS first and COMMIT puts it last? (Still on 4.1.1)" Spec changes: - Specify TIMESTAMP better in spec - Fix base32 commits in proposal - Specify what happens during SRV generation when there is no previous_SRV - Remove the XXX in 3.1.1 since it's taken care of - Document the phase determination logic better. - Remove MUST paragraph from 3.1.2

1 0