- tor-dev - lists.torproject.org

Hello, I can't use Google Search Engine Service through Tor proxy.
by Li Xiaodong 18 Oct '15

18 Oct '15

*Hello, I can't use Google Search Engine Service through Tor proxy. Google asked me to enter a Verification Code. After I entered the Verification Code, Google asked me enter a new Verification Code. Could you please communicate this problem to Google Corporation? Could you please suggest to Google Corporation that it should allow Tor users to normally use Google Search Engine Service? Thank you very much for your help. I really appreciate it.*

2 1

Re: [tor-dev] Crux: Privacy-preserving statistics for Tor
by str4d 17 Oct '15

17 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 str4d wrote: > Vasilios Mavroudis wrote: >> Hello, > >> I would like to introduce our project "Crux", which enables the >> computation of privacy preserving statistics on sensitive data. >> The project was developed at University College London (UCL) by >> me, in close collaboration with George Danezis. > > This is fantastic work. Haven't read all of your thesis yet, but > it looks similar to the stats collection system that we ran for a > while inside I2P, but with the privacy protections that I wanted it > to have. Well done! > > >> "Crux" was designed with Tor hidden services in mind (in >> accordance with this >> <https://research.torproject.org/techreports/hidden-service-stats-201 5 > >> - -04-28.pdf> > > > Tor >> Tech Report), but it can be applied on various kinds of data >> (given an appropriate parser). > > Excellent. I want to use this in I2P :) > > >> It supports three statistics (i.e. mean, median, variance) and >> its threat model is compatible Tor's (see technical document >> below). > > I will look into this, but I doubt there will be any > insurmountable incompatibilities with I2P's threat model. > >> Currently, we don't have a parser specific for the data >> collected by the Tor relays, but we plan to keep adding >> functionality. > >> The source code can be found here: >> https://github.com/mavroudisv/Crux The theoretical background, >> threat model, security argument and technical details can be >> found here: link <http://mavroudisv.eu/files/thesis.pdf> > > >> Ideas, comments, feedback much appreciated! > > Firstly: needs more docs. I shouldn't have to read your entire > thesis in order to figure out how to run it :P The README is a good > start, but the command parameters need clarification/definitions, > and it is not at all obvious how the relays work. An XLS file is > also mentioned without explanation. I have now tried running the code, but the "unit test" run by authority.py fails because bsbdb.btopen() returns an empty dict. I don't know if this is because the table files are broken or because of a local issue with my setup (I am using a virtualenv, but I get the same empty dict when I try to open the tables manually outside the virtualenv). I am not going to try and fix the latter on my own, because bsddb is deprecated since Python 2.6, so you shouldn't really be using it :) (But it must be something else, because I tried using bsddb3 instead and that also returns an empty dict). str4d > > Secondly, you could probably simplify configuration by managing > hidden service connections yourself. I believe the Stem library can > do this? I2P has a python library that can be used to listen on and > connect to I2P Destinations, with a socket-like interface. > > It's not clear to me from your thesis whether you plan to merge > the relay component into the Tor relay code (and have the Tor > relays talk directly to the query processors), or have your relays > running separately and interfacing with the Tor relays to fetch > stats. This will potentially alter how it would be used with other > networks. In I2P's case I'd imagine it would be easiest (in terms > of usability by those setting up relays) to write an I2P plugin in > Java that performs the relay role, but I wouldn't want to be > duplicating a lot of logic that subsequently needs to be kept > in-sync with upstream :) > > str4d > > >> Vasilis > > > >> _______________________________________________ tor-dev mailing >> list tor-dev(a)lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > _______________________________________________ tor-dev mailing > list tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJWItIgAAoJEBO17ljAn7PglsQP/3tOrL9EZfYMb5fxxVI6/wJm qDvA8RpbEJDhoMjOc9sLNyMVgvGGPxEdma8I/R8H37yIPOp2GJg82MoXfUk+AGFe 5ZIIXwf2EzHH0RFLRI5xaQ4DW3Ru1yIzH+E3vhuGclL87YRKnAM0xkMQcueR5xo5 /ETe9TI7oPCdETI4J6iFoM+Ob9qsRiEVeuT5HEFFKwQhBvl0NSpfMHV9ftuxQHuu I2B9jomnBdzF9OXnSQrYXg3ho2rxX4hO1TZfQHg5DgHvkt8TQYQM3cwdGE7/rsqC d4+kBW2RuAz68wq+fLPeeZ2U2ff2QqmDLqVyUJvpP5RqeUcv66J/iYm7md886tTc palwVGodKiTJ6DBM0z1K97x6Sl1jJXEAkZQAr2yQ8jrC/BJqQRPu2vHeZILqbMEQ UU3JLjIk/Jc2RGK6zitvj4kTrLdfACEQMYy3brQUawZhaM42QNQgb3LQq1bJXzUo vISZckX/CVfF+R63oO085LYOO7oTLyY3EhA1PCXiV09Usl/To66Oh4AVYtz/lPLH 6uoWR/ic5X9gFIXt1t+617+i/L/ILMfqkqN7GzoSbQ8QfyBJrjIikuj575M7AH3o RuhRYHUTmaCRL+9niDFG/mSVGBxBqiSfW1eS5pGE3hIAcLQd0eFT/38HRdonkfLP Ug5ihMFSKfKmCIljQ6mm =jDmS -----END PGP SIGNATURE-----

1 0

Tor Environment Variables
by Dewayne Hicks 16 Oct '15

16 Oct '15

Hi, Is there a specific page (wiki/docs) that details all of the Tor environment variables for Linux? Ideally this would also map to TBB preferences access via about:config. The topic has been mentioned occasionally on the issue tracker [1] but the actual variables are spread out over a number of pages and it's not clear if the lists are complete. Thanks [1]: https://trac.torproject.org/projects/tor/ticket/13005

1 0

Trac priorities and severities
by David Goulet 15 Oct '15

15 Oct '15

Hello tor-dev! While 028 bug triaging, we realized that we *really* need priorities to not be a banana field deprive of useful meaning. If you are unaware or don't remember, the priority field in a trac ticket can be: blocker critical major normal minor trivial Those are "severities" *NOT* priorities. Now I propose the following: 1) I take those above and copy them to the already existing but empty severity field. They seem reasonable and we are used to them. 2) Rename all priorities to something more meaninful: blocker --> Immediate criticial --> Very High major --> High normal --> Medium minor --> Low trivial --> Very Low 3) World trac domination! This could get confusing for some tickets after that but at least we can transition towards using the Severity field from now on. Anyone object or think it's a bad bad idea? This is very little work for lots of benefits so I don't want to get into a massive reorg. or bike-shedding the naming convention. I would be open though to drop the "Very [High|Low]" field but that could make some tickets with less semantic. I'll let that sync for some days before doing anything. Let's not wait anymore and take back our Trac system! Freeeeeeeeeedom!!!!! Cheers! David

6 8

Proposal: Padding Negotiation
by Mike Perry 14 Oct '15

14 Oct '15

Here's a proposal describing some padding negotiation cell primitives that should be useful to defend against website traffic fingerprinting, hidden service circuit fingerprinting, and probably just about any other traffic analysis attack under the sun. The proposal is in my git remote at: https://gitweb.torproject.org/user/mikeperry/torspec.git/tree/proposals/xxx… ============================== Filename: xxx-padding-negotiation.txt Title: Padding Negotiation Authors: Mike Perry Created: 01 September 2015 Status: Draft 0. Overview This proposal aims to describe mechanisms for requesting various types of padding from relays. These padding primitives are general enough to use to defend against both website traffic fingerprinting as well as hidden service circuit setup fingerprinting. 1. Motivation Tor already supports both link-level padding via (CELL_PADDING cell types), as well as circuit-level padding (via RELAY_COMMAND_DROP relay cells). Unfortunately, there is no way for clients to request padding from relays, or request that relays not send them padding to conserve bandwidth. This proposal aims to create a mechanism for clients to do both of these. It also establishes consensus parameters to limit the amount of padding that relays will send, to prevent custom wingnut clients from requesting too much. 2. Link-level padding Padding is most useful if it can defend against a malicious or compromised guard node. However, link-level padding is still useful to defend against an adversary that can merely observe a Guard node externally, such as for low-resolution netflow-based attacks (see Proposal 251[1]). In that scenario, the primary negotiation mechanism we need is a way for mobile clients to tell their Guards to stop padding, or to pad less often. The following Trunnel payloads should cover the needed parameters: const CELL_PADDING_COMMAND_STOP = 1; const CELL_PADDING_COMMAND_START = 2; /* This command tells the relay to stop sending any periodic CELL_PADDING cells. */ struct cell_padding_stop { u8 command IN [CELL_PADDING_COMMAND_STOP]; }; /* This command tells the relay to alter its min and max netflow timeout range values, and send padding at that rate (resuming if stopped). */ struct cell_padding_start { u8 command IN [CELL_PADDING_COMMAND_START]; /* Min must not be lower than the current consensus parameter nf_ito_low. */ u16 ito_low_ms; /* Max must not be lower than ito_low_ms */ u16 ito_high_ms; }; More complicated forms of link-level padding can still be specified using the primitives in Section 3, by using "leaky pipe" topology to send the RELAY commands to the Guard node instead of to later nodes in the circuit. 3. End-to-end circuit padding For circuit-level padding, we need two types of additional features: the ability to schedule additional incoming cells at one or more fixed points in the future, and the ability to schedule a statistical distribution of arbitrary padding to overlay on top of non-padding traffic (aka "Adaptive Padding"). In both cases, these messages will be sent from clients to middle nodes using the "leaky pipe" property of the 'recognized' field of RELAY cells, allowing padding to originate from middle nodes on a circuit in a way that is not detectable from the Guard node. This same mechanism can also be used to request padding from the Guard node itself, to achieve link-level padding without the additional overhead requirements on middle nodes. 3.1. Fixed-schedule padding message (RELAY_COMMAND_PADDING_SCHEDULE) The fixed schedule padding will be encoded in a RELAY_COMMAND_PADDING_SCHEDULE cell. It specifies a set of up to 80 fixed time points in the future to send cells. XXX: 80 timers is a lot to allow every client to create. We may want to have something that checks this structure to ensure it actually schedules no more than N in practice, until we figure out how to optimize either libevent or timer scheduling/packet delivery. See also Section 4.3. The RELAY_COMMAND_PADDING_SCHEDULE body is specified in Trunnel as follows: struct relay_padding_schedule { /* Number of microseconds before sending cells (cumulative) */ u32 when_send[80]; /* Number of cells to send at time point sum(when_send[0..i]) */ u16 num_cells[80]; /* Adaptivity: If 1, and server-originating cells arrive before the next when_send time, then decrement the next non-zero when_send index, so we don't send a padding cell then, too */ u8 adaptive IN [0,1]; }; To allow both high-resolution time values, and the ability to specify timeout values far in the future, the time values are cumulative. In other words, sending a cell with when_send = [MAX_INT, MAX_INT, MAX_INT, 0...] and num_cells = [0, 0, 100, 0...] would cause the relay to reply with 100 cells in 3*MAX_INT microseconds from the receipt of this cell. 3.2. Adaptive Padding message (RELAY_COMMAND_PADDING_ADAPTIVE) The following message is a generalization of the Adaptive Padding defense specified in "Timing Attacks and Defenses"[2]. The message encodes either one or two state machines, each of which can contain one or two histograms ("Burst" and "Gap") governing their behavior. The "Burst" histogram specifies the delay probabilities for sending a padding packet after the arrival of a non-padding data packet. The "Gap" histogram specifies the delay probabilities for sending another padding packet after a padding packet was just sent from this node. This self-triggering property of the "Gap" histogram allows the construction of multi-packet padding trains using a simple statistical distribution. Both "Gap" and "Burst" histograms each have a special "Infinity" bin, which means "We have decided not to send a packet". Each histogram is combined with state transition information, which allows a client to specify the types of incoming packets that cause the state machine to decide to schedule padding cells (and/or when to cease scheduling them). The client also maintains its own local histogram state machine(s), for reacting to traffic on its end. Note that our generalization of the Adaptive Padding state machine also gives clients full control over the state transition events, even allowing them to specify a single-state Burst-only state machine if desired. See Sections 3.2.1 and 3.2.2 for details. The histograms and the associated state machine packet layout is specified in Trunnel as follows: /* These constants form a bitfield to specify the types of events * that can cause transitions between state machine states. * * Note that SENT and RECV are relative to this endpoint. For * relays, SENT means packets destined towards the client and * RECV means packets destined towards the relay. On the client, * SENT means packets destined towards the relay, where as RECV * means packets destined towards the client. */ const RELAY_PADDING_TRANSITION_EVENT_NONPADDING_RECV = 1; const RELAY_PADDING_TRANSITION_EVENT_NONPADDING_SENT = 2; const RELAY_PADDING_TRANSITION_EVENT_PADDING_SENT = 4; const RELAY_PADDING_TRANSITION_EVENT_PADDING_RECV = 8; /* This payload encodes a histogram delay distribution representing * the probability of sending a single RELAY_DROP cell after a * given delay in response to a non-padding cell. */ struct burst_state { u8 histogram_len IN [2..51]; u16 histogram[histogram_len]; u32 start_usec; u16 max_sec; /* This is a bitfield that specifies which direction and types * of traffic that cause us to abort our scheduled packet and * return to waiting for another event from transition_burst_events. */ u8 transition_start_events; /* This is a bitfield that specifies which direction and types * of traffic that cause us to remain in the burst state: Cancel the * pending padding packet (if any), and schedule another padding * packet from our histogram. */ u8 transition_reschedule_events; /* This is a bitfield that specifies which direction and types * of traffic that cause us to transition to the Gap state. */ u8 transition_gap_events; /* If true, remove tokens from the histogram upon padding and * non-padding activity. */ u8 remove_toks IN [0,1]; }; /* This histogram encodes a delay distribution representing the * probability of sending a single additional padding packet after * sending a padding packet that originated at this hop. */ struct gap_state { u8 histogram_len IN [2..51]; u16 histogram[histogram_len]; u32 start_usec; u16 max_sec; /* This is a bitfield which specifies which direction and types * of traffic should cause us to transition back to the start * state (ie: abort scheduling packets completely). */ u8 transition_start_events; /* This is a bitfield which specifies which direction and types * of traffic should cause us to transition back to the burst * state (and schedule a packet from the burst histogram). */ u8 transition_burst_events; /* This is a bitfield that specifies which direction and types * of traffic that cause us to remain in the gap state: Cancel the * pending padding packet (if any), and schedule another padding * packet from our histogram. */ u8 transition_reschedule_events; /* If true, remove tokens from the histogram upon padding and non-padding activity. */ u8 remove_toks IN [0,1]; }; struct adaptive_padding_machine { /* This is a bitfield which specifies which direction and types * of traffic should cause us to transition to the burst * state (and schedule a packet from the burst histogram). */ u8 transition_burst_events; struct burst_state burst; struct gap_state gap; }; /* This is the full payload of a RELAY_COMMAND_PADDING_ADAPTIVE * cell. */ struct relay_command_padding_adaptive { u8 num_machines IN [1,2]; struct adaptive_padding_machine machines[num_machines]; }; 3.2.1. Histogram state machine operation Each of pair of histograms ("Burst" and "Gap") together form a state machine whose transitions are governed by incoming traffic and/or locally generated padding traffic. Each state machine has a Start state S, a Burst state B, and a Gap state G. The state machine starts idle (state S) until it receives a packet of a type that matches the bitmask in machines[i].transition_burst_events. This causes it to enter burst mode (state B), in which a delay t is sampled from the Burst histogram, and a timer is scheduled to count down until either another matching packet arrives, or t expires. If the "Infinity" time is sampled from this histogram, the machine returns to the Start state. If a packet that matches machines[i].burst.transition_start_events arrives before t expires, the machine transitions back to the Start state. If a packet that matches machines[i].burst.transition_reschedule_events arrives before t expires, a new delay is sampled and the process is repeated again, i.e. it remains in burst mode. Otherwise, if t expires, a padding message is sent to the other end. If a packet that matches machines[i].burst.transition_gap_events arrives (or is sent), the machine transitions to the Gap state G. In state G, the machine samples from the Gap histogram and sends padding messages when the time it samples expires. If an infinite delay is sampled while being in state G we jump back to state B. If a packet arrives that matches gap.transition_start_events, the machine transitions back to the Start state. If a packet arrives that matches gap.transition_burst_events, the machine transitions back to the Burst state. If a packet arrives that matches machines[i].gap.transition_reschedule_events, the machine remains in G but schedules a new padding time from its Gap histogram. In the event that a malicious or buggy client specifies conflicting state transition rules with the same bits in multiple transition bitmasks, the transition rules of a state that specify transition to earlier states take priority. So burst.transition_start_events takes priority over burst.transition_reschedule_events, and both of these take priority over burst.transition_gap_events. Similarly, gap.transition_start_events takes priority over gap.transition_burst_events, and gap.transition_burst_events takes priority over gap.transition_reschedule_events. In our generalization of Adaptive Padding, either histogram may actually be self-scheduling (by setting the bit RELAY_PADDING_TRANSITION_EVENT_PADDING_SENT in their transition_reschedule_events). This allows the client to create a single-state machine if desired. Clients are expected to maintain their own local version of the state machines, for reacting to their own locally generated traffic, in addition to sending one or more state machines to the middle relay. The histograms that the client uses locally will differ from the ones it sends to the upstream relay. On the client, the "SENT" direction means packets destined towards the upstream, where as "RECV" means packets destined towards the client. However, on the relay, the "SENT" direction means packets destined towards the client, where as "RECV" means packets destined towards the relay. 3.2.2. The original Adaptive Padding algorithm As we have noted, the state machines above represent a generalization of the original Adaptive Padding algorithm. To implement the original behavior, the following flags should be set in both the client and the relay state machines: num_machines = 1; machines[0].transition_burst_events = RELAY_PADDING_TRANSITION_EVENT_NONPADDING_SENT; machines[0].burst.transition_reschedule_events = RELAY_PADDING_TRANSITION_EVENT_NONPADDING_SENT; machines[0].burst.transition_gap_events = RELAY_PADDING_TRANSITION_EVENT_PADDING_SENT; machines[0].gap.transition_reschedule_events = RELAY_PADDING_TRANSITION_EVENT_PADDING_SENT; machines[0].gap.transition_burst_events = RELAY_PADDING_TRANSITION_EVENT_NONPADDING_SENT; The rest of the transition fields would be 0. Adding additional transition flags will either increase or decrease the amount of padding sent, depending on their placement. The second machine slot is provided in the event that it proves useful to have separate state machines reacting to both sent and received traffic. 3.2.3. Histogram decoding/representation Each of the histograms' fields represent a probability distribution that is expanded into bins representing time periods a[i]..b[i] as follows: start_usec,max_sec,histogram_len initialized from appropriate histogram body. n = histogram_len-1 INFINITY_BIN = n a[0] = start_usec; b[0] = start_usec + max_sec*USEC_PER_SEC/2^(n); for(i=1; i < n; i++) { a[i] = start_usec + max_sec*USEC_PER_SEC/2^(n-i) b[i] = start_usec + max_sec*USEC_PER_SEC/2^(n-i-1) } To sample the delay time to send a padding packet, perform the following: i = 0; curr_weight = histogram[0]; tot_weight = sum(histogram); bin_choice = crypto_rand_int(tot_weight); while (curr_weight < bin_choice) { curr_weight += histogram[i]; i++; } if (i == INFINITY_BIN) return; // Don't send a padding packet // Sample uniformly between a[i] and b[i] send_padding_packet_at = a[i] + crypto_rand_int(b[i] - a[i]); In this way, the bin widths are exponentially increasing in width, where the width is set at max_sec/2^(n-i) seconds. This exponentially increasing bin width allows the histograms to most accurately represent small interpacket delay (where accuracy is needed), and devote less accuracy to larger timescales (where accuracy is not as important). 3.2.4. Token removal and refill If the remove_tok field is set to true for a given state's histogram, then whenever a padding packet is sent, the corresponding histogram bin's token count is decremented by one. If a packet matching the current state's transition_reschedule_events bitmask arrives from the server before the chosen padding timer expires, then a token is removed from the nearest non-empty bin corresponding to the delay since the last packet was sent, and the padding packet timer is re-sampled from the histogram. If the entire histogram becomes empty, it is then refilled to the original values. 3.2.5. Constructing the histograms Care must be taken when constructing the histograms themselves, since their non-uniform widths means that the actual underlying probability distribution needs to be both normalized for total number of tokens, as well as the non-uniform histogram bin widths. Care should also be taken with interaction with the token removal rules from Section 3.2.4. Obviously using a large number of tokens will cause token removal to have much less of an impact upon the adaptive nature of the padding in the face of existing traffic. Actual optimal histogram and state transition construction for different traffic types is expected to be a topic for further research. Intuitively, the burst state is used to detect when the line is idle (and should therefore have few or no tokens in low histogram bins). The lack of tokens in the low histogram bins causes the system to remain in the burst state until the actual application traffic either slows, stalls, or has a gap. The gap state is used to fill in otherwise idle periods with artificial payloads from the server (and should have many tokens in low bins, and possibly some also at higher bins). It should be noted that due to our generalization of these states and their transition possibilities, more complicated interactions are also possible. 4. Security considerations and mitigations The risks from this proposal are primarily DoS/resource exhaustion, and side channels. 4.1. Rate limiting Fully client-requested padding introduces a vector for resource amplification attacks and general network overload due to overly-aggressive client implementations requesting too much padding. Current research indicates that this form of statistical padding should be effective at overhead rates of 50-60%. This suggests that clients that use more padding than this are likely to be overly aggressive in their behavior. We recommend that three consensus parameters be used in the event that the network is being overloaded from padding to such a degree that padding requests should be ignored: * CircuitPaddingMaxRatio - The maximum ratio of padding traffic to non-padding traffic (expressed as a percent) to allow on a circuit before ceasing to pad. Ex: 75 means 75 padding packets for every 100 non-padding packets. - Default: 100 * CircuitPaddingLimitCount - The number of padding cells that must be transmitted before the ratio limit is applied. - Default: 500 * CircuitPaddingLimitTime - The time period in seconds over which to count padding cells for application of the ratio limit. - Default: 60 XXX: Should we cap padding at these rates, or fully disable it once they're crossed? Proposal 251 introduced extra-info accounting at relays to enable us to measure the total overhead of both link and circuit-level padding at various relays. 4.2. Malicious state machines The state machine capabilities of RELAY_COMMAND_PADDING_ADAPTIVE are very flexible, and as a result may specify conflicting or non-deterministic state transitions. We believe that the rules in Section 3.2.1 for prioritizing transitions towards lower states remove any possibility of non-deterministic transitions. However, because of self-triggering property that allows the state machines to schedule more padding packets after sending their own locally generated padding packets, care must be taken with the interaction with the rate limiting rules in Section 4.1. If the limits in section 4.1 are exceeded, the state machines should stop, rather than continually poll themselves trying to transmit packets and being blocked by the rate limiter at another layer. 4.3. Libevent timer exhaustion As mentioned in section 3.1, scheduled padding may create an excessive number of libevent timers. Care should be taken in the implementation to devise a way to prevent clients from sending padding requests specifically designed to impact the ability of relays to function by causing too many timers to be scheduled at once. XXX: Can we suggest any specifics here? I can imagine a few ways of lazily scheduling timers only when they are close to their expiry time, and other ways of minimizing the number of pending timer callbacks at a given time, but I am not sure which would be best for libevent. 4.4. Side channels In order to prevent relays from introducing side channels by requesting padding from clients, all of these commands should only be valid in the outgoing (from the client/OP) direction. Clients should perform accounting on the amount of padding that they receive, and if it exceeds the amount that they have requested, they alert the user of a potentially misbehaving node, and/or close the circuit. Similarly, if RELAY_DROP cells arrive from the last hop of a circuit, rather than from the expected interior node, clients should alert the user of the possibility of that circuit endpoint introducing a side-channel attack, and/or close the circuit. ------------------- 1. https://gitweb.torproject.org/torspec.git/tree/proposals/251-netflow-paddin… 2. http://freehaven.net/anonbib/cache/ShWa-Timing06.pdf -- Mike Perry

3 3

[Measurement Team] *NO* IRC meeting tomorrow, but on Thursday, Oct 22, 14:00 UTC in #tor-dev
by Karsten Loesing 14 Oct '15

14 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be *NO* IRC meeting of the Measurement Team tomorrow, but on Thursday, October 22, 2015, 14:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20151022T14 The reason is that we decided in Berlin to switch to meeting every other week. I hope that we'll soon have our own (public) team mailing list to satisfy our communication needs between IRC meetings. Talk to you in eight days. I'll send out another reminder next week. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWHfsEAAoJEJD5dJfVqbCr/N4H/2ms2VQ68PVHjX4dLQl4GHaP dEXKBfgwd0VAPjpk+ldPo5snbHf6ciO6m9QKraej5LWrw3tYbQDbA4vn2HAnSNPK N22v7HLtcdF3QF81SiH1B5tAK5TNEJJLI8qA8/R8qt6O5VXw9DXWNef5s0ja+dLC R/sGrt0jmpBFvTbMyhbPhKUH0mbBodY+Ph+kxrtNv447FMgv6q5+eLFuS+ay3WOx zOWqggl/xiUyqUJODKeKMQst9NWK7q8shuC2YAtEGybvOm0nhNjT5oElj4Go7cXV HUAQT1yCyLRTO3BrXdbOLT5vGEJLUek5cjltWDg0IVWGSrrT5bVkIZYISpdR090= =H7Nt -----END PGP SIGNATURE-----

1 0

Getting meek covered by a CDN for free
by Virgil Griffith 12 Oct '15

12 Oct '15

I met with some CDNs today and they have expressed interest in doing meek for us. Is there someone at Tor Project I can forward the CDNs to who are more serious about hosting meek? -V

2 1

Dynamically updating tor
by Sebastian Hahn 10 Oct '15

10 Oct '15

Hi list, I recently played with the Kitsune[0] framework to figure out whether we might be able to use it to dynamically update (that is, perform an update without restarting the process) Tor relays on Linux. This post is just to make you aware such a thing has been attempted. I deem it currently infeasible due to the high overhead (in developer time) and the lack of confidence in the resulting update binaries. Nevertheless, people might want to take a look. I have not cleaned this up at all and it is quite dependent on the software currently available in Debian jessie, including the versions of OpenSSL and libevent. In short, THIS WILL NOT WORK FOR YOU UNLESS YOU ARE VERY LUCKY AND ARE PREPARED TO SPEND SIGNIFICANT TIME WITH IT. THIS IS NOT RELEASE QUALITY CODE BY A LONG SHOT. The Tor code is in a branch[1] in my tor repo and there's a tarball[2] of the modifications I had to make to Kitsune to enable building it with a newer version of one of its dependencies. This is a generally exciting topic imo, but I currently have no intentions to keep pursuing this angle more because of other commitments and due to the fact that I think it is unrealistic to expect the Tor developers to learn this tool. If there is serious interest both in improving Kitsune as well as dynamic updatability support for Tor, I might reconsider in the future. Cheers and have fun Sebastian [0]: http://kitsune-dsu.com / https://github.com/kitsune-dsu/kitsune-core [1]: https://gitweb.torproject.org/sebastian/tor.git/log/?h=dsu [2]: https://people.torproject.org/~sebastian/kitsune-core.tar.gz

1 0

[Applications Team] New meeting schedule
by Georg Koppen 09 Oct '15

09 Oct '15

Hi, at the recent dev meeting we decided to slightly change the applications team meeting procedure in order to focus more on cross-application issues. Instead of a scrum-style meeting we also think it should be agenda-driven. The current plan is to meet once a month (on the first Tuesday) given that cross-application issues might not arise on a day-by-day basis. Let's get this started on the first Tuesday in November (Nov 3) 18:00 UTC in #tor-project on irc.oftc.net. The proposed topic is talking about UX/internationalization issues. If there are other things worth discussing we can easily amend the agenda at the beginning of the meeting. The meetings on Monday 18:00 UTC in #tor-dev will again be Tor Browser meetings from now on. Thanks! Georg

1 0

[measurement team] analytics server proposal
by thomas lörtsch 09 Oct '15

09 Oct '15

analytics server proposal ========================= tl;dr ----- measurement team is setting up a server for you to play with raw metrics data. please check below if it meets your requirements and if you have any suggestions for the setup. Motivation + Setup ------------------ An idea came up during the Berlin dev meeting: it would be good to have all measurement data on one server, loaded into a database, with a suite of analytics software readily set up, available to everybody who needs to do aggregation, analytics, research etc on the data. Right now everybody who wants to analyze Tor metrics data first has to downlod raw data from the metrics website, load it into a local database, massage it, setup a query environment etc. That can be cumbersome and use siginificant computing resources, making your notebook unusable for hours or eating up hundreds of gigabytes of disk space if you don’t optimize properly. That makes on the fly quick and dirty prototyping and analysis quite unfeasable and prohibitively tedious. We plan to change this by setting up an analytics server free for everybody to use: * containing as much raw metrics data as possible * in a HBase database on Hadoop/HDFS * with analytic tools like Spark and Drill on top * supporting diverse languages - SQL, R, Python, Java, Scala, Clojure - * with gateways to desktop-ish tools like Tableau and MongoDB Please see the detailed description below, tell us if you have any suggestions concerning the setup and request a login (to be available soon)! Data ---- The data will be available as raw as possible, like it is served by CollecTor, preloaded into a database and maybe also in some pre-aggregated sets that concentrate on “popular” aspects of the data. The complete dataset is about 50 GB compressed, or 5 TB uncompressed. For a start we may need to concentrate on the last 2 or 3 years. Analytics ———— The amount of data practically rules out the usual RDBMS suspects (PostgreSQL etc). We also want to have easy access to the analytical tools that are available in the big data ecosystem. Hadoop [0] is a popular candidate for Big Data processing, but has a reputation of being slightly outdated, not so well engineered [1] and is not available in Debian stable (which likely won’t change [2]). It seems like choosing Hadoop means risking to run into nasty little problems down the road. Most solutions outlined below are or can be based on Hadoop File System (HDFS) but that is available independently of the Hadoop MapReduce system. Disco [3] is another MapReduce engine. Seems solid, heavyly uses Python, but relies on it’s own file system [4] which would make it harder or impossible to use with other tools like Spark, that require HDFS (or compatible). Would need more research/googling to be really sure but right now it doesn't seems worth the effort. Spark [5] is faster than Hadoop MapReduce when working in memory and can work equally well from disk. Storage is based on Hadoop and "works with any Hadoop compatible data source including HDFS, HBase, Cassandra, etc.” [6]. It can work with Amazons Elastic MapReduce EMR instances (which in turn allows running Spark on EMR). EMR pricing is not prohibitive [7]. Ooni uses this setup [8]. Spark supports R, Scala, Java, Python and Clojure. It provides an interactive shell for Python and Scala and plugins for SQL-like queries, MLib (machine learning lib) and some graph processing. Its performance but also the SQL plugin and the wide range of supported programming languages make Spark quite attractive. Hive [9] is another analytics engine on top of Hadoop that has similar features to Spark [10] (its feature set was lagging behind Spark for some time but has catched up recently). Since Ooni uses Spark and the two are not very different with respect to feature set we favor Spark (and hope for synergies). MongoDB [11], which is available in Debian stable and supports Map Reduce [12], may not be performant enough and is in some ways rather shoddy. Redis [13] has a good reputation and while optimized for in-memory operations it can work from disk too. It is available as Debian stable [14]. Map reduce operations are available as add ons [15] but not in Debian stable. Like MongoDB there is not much interesting analytics software available besides MapReduce. Drill [16] is a nice analytic tool: a 'schema-free SQL query engine for Big Data’ which accepts a variety of data sources including HDFS, MongoDB, Amazon S3, with impressive performance and a wide range of possible applications. Both Spark and Drill support Tableau, a nice Desktop data visualization tool that allows to work with data in a rather intuitive way. In summary it seems like Spark and Drill are the aggregation and analytic softwares that we want to have. Together they support a nice array of languages (SQL, R, Java, Python, Scala, Clojure) and desktop-ish tools (MongoDB, Tableau) which should make them readily useable for a lot of people. Storage ------- Amazons EMR is an option but for now we’ll install our own box. We could go with CSV files in the Hadoop Distributed File System HDFS to keep complexity down but given the amount of data a proper DBMS seems justified. Two databases are supported by both Spark and Drill: Apache Cassandra [17][18] and Apache HBase [19][20][21]. Both are column-oriented key-value-store. HBase follows the same design principles as Cassandra but comes out of the Hadoop community and is better integrated wiht other offerings. Also while Cassandra is more optimized for easy scaling and write performance HBase has the emphasis on fast read [22]. It also seems like getting Drill to work with Cassandra is not as easy as advertized [23]. So HBase seems like the best fit. Hopefully Drill and Spark will work from the same data in HBase and not want to import it into their own store (thereby duplicating it) but we’re not sure about that right now. If they do import the data into their own stores anyway then HBase might be overkill and CSV files on HDFS sufficient. Or Parquet [24][25]? We plan to figure that out during the next days. Any help and remarks appreciated! Redis and MongoDB, the 2 systems that are available as Debian stable packages, are not well enough supported by analytic softwares to be useful. Conclusion ---------- Measurement team will set up a hosted server with 32 GB RAM and 5 TB storage (partly as SSD), install Apache HBase as database and Apache Spark and Apache Drill as analytics softwares and import a lot of raw measurement data into HBase. We then give login to the machine on request and hope that you have fun with map reducing, data sciencing, visualizing... This is a test: we plan to keep the server running for at least the next 3 months or so. If it doesn’t prove useful we will probably shut it down afterwards, but we hope otherwise. --- [0] http://hadoop.apache.org/ [1] https://news.ycombinator.com/item?id=4298284 - see the 4. comment thread for a nice rundown on the cons of Hadoop. [2] https://wiki.debian.org/Hadoop [3] http://discoproject.org/ [4] http://disco.readthedocs.org/en/develop/howto/ddfs.html?highlight=hdfs [5] http://spark.apache.org/ [6] http://www.infoq.com/articles/apache-spark-introduction [7] https://aws.amazon.com/elasticmapreduce/pricing/ [8] https://github.com/TheTorProject/ooni-pipeline [9] http://hive.apache.org/ [10] http://www.zdnet.com/article/sql-and-hadoop-its-complicated/ [11] https://www.mongodb.org/ [12] https://docs.mongodb.org/manual/core/map-reduce/ [13] https://redis.io [14] https://packages.debian.org/search?suite=jessie&arch=any&searchon=names&key… [15] http://heynemann.github.io/r3/ - maybe also others although Google didn’t really help [16] https://drill.apache.org/, https://drill.apache.org/faq/ [17] http://cassandra.apache.org/ [18] http://www.planetcassandra.org/getting-started-with-apache-spark-and-cassan… [19] http://hbase.apache.org/ [20] http://www.vidyasource.com/blog/Programming/Scala/Java/Data/Hadoop/Analytic… [21] http://www.slideshare.net/cloudera/5-h-base-schemahbasecon2012 [22] http://www.infoworld.com/article/2848722/nosql/mongodb-cassandra-hbase-thre… [23] http://stackoverflow.com/questions/31017755 [24] https://parquet.apache.org/documentation/latest/ [25] http://arnon.me/2015/08/spark-parquet-s3/

3 6

Concerns regarding a multilingual version of TBB,
by Yet Another Tor User 09 Oct '15

09 Oct '15

I saw an email on the Wiki mailing list concerning a multilingual version of Tor. If that was made possible your language tag be used to track you. This could break your anonymity. Your thoughts?

3 2

Summary of meek's costs, September 2015
by David Fifield 09 Oct '15

09 Oct '15

Here's the summary of meek's CDN fees for September 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 September 2015 $617.08 + $710.75 + $490.58 = $1818.41 -- total by CDN $6771.57 + $7876.60 + $645.47 = $15293.64 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… Last month was pretty stable at about 5,000 simultaneous users all month. This month is the first time we have actual usage numbers from Azure. They told us what the cost would have been, but did not actually charge us. This month (September) is the last month on the free grant we have for Azure. After that, we will probably have to throttle Azure just like the other backends. If you want to help make domain fronting more efficient and future-proof, a good place to direct your attention is to this ticket, which aims to start using domain fronting as a cheap, low-volume interface to BridgeDB, instead of as a bulk data transfer channel: https://bugs.torproject.org/16650 If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2871 GB $344.53 5451 instance hours $272.55 Compared to the previous month: 2920 GB $350.38 6128 instance hours $306.38 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 102M requests $122.83 909 GB $120.81 Asia Pacific (Sydney) 2M requests $2.53 6 GB $0.88 Asia Pacific (Tokyo) 9M requests $11.60 57 GB $7.54 EU (Ireland) 188M requests $226.56 1131 GB $89.42 South America (Sao Paulo) 6M requests $14.37 40 GB $9.45 US East (Northern Virginia) 70M requests $70.04 441 GB $34.73 -- total 379M requests $447.93 2586 GB $262.83 https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… == Azure a.k.a. meek-azure == This is the first month where we have actual (not estimated) usage numbers, as a result of the new kind of account we're using. Unfortunately (also as a result of the account transition), meek-azure was broken until August 14, so these number really only represent half a month. Zone 1 3726 GB $324.20 Zone 2 1206 GB $166.39 -- total 4932 GB $490.58 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html https://lists.torproject.org/pipermail/tor-dev/2015-August/009213.html https://lists.torproject.org/pipermail/tor-dev/2015-September/009533.html

1 0

UniformDH question
by Jeff Burdges 08 Oct '15

08 Oct '15

What is the advantage of using X or p-X in UniformDH in obfsproxy? https://gitweb.torproject.org/pluggable-transports/obfsproxy.git/tree/d oc/obfs3/obfs3-protocol-spec.txt#n65 Isn't just X itself dense pretty quickly anyways? Jeff

2 1

Re: [tor-dev] Trac priorities and severities
by Hugo Maxwell Connery 08 Oct '15

08 Oct '15

Hi, Yes, renaming to a more meaningful thing is great. Also, if only some group can grade entries, then it is wise to have an 'uncategorised' category which appears in a grading list towards, but not at the bottom. Field experience in these systems tells me that people get annoyed at being auto-classified at the bottom. I advise putting 'uncategorised' at 5 of 7 or something similar. e.g Critical, High Import, Import, Medium, Uncategorised, Small, Trivial. This also helps to raise the visibility of Uncategorised items; i.e Categorise them :) Regards, Hugo

1 0

Re: [tor-dev] Crux: Privacy-preserving statistics for Tor
by str4d 07 Oct '15

07 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Vasilios Mavroudis wrote: > Hello, > > I would like to introduce our project "Crux", which enables the > computation of privacy preserving statistics on sensitive data. The > project was developed at University College London (UCL) by me, in > close collaboration with George Danezis. This is fantastic work. Haven't read all of your thesis yet, but it looks similar to the stats collection system that we ran for a while inside I2P, but with the privacy protections that I wanted it to have. Well done! > > "Crux" was designed with Tor hidden services in mind (in accordance > with this > <https://research.torproject.org/techreports/hidden-service-stats-2015 - -04-28.pdf> > > Tor > Tech Report), but it can be applied on various kinds of data (given > an appropriate parser). Excellent. I want to use this in I2P :) > > It supports three statistics (i.e. mean, median, variance) and its > threat model is compatible Tor's (see technical document below). I will look into this, but I doubt there will be any insurmountable incompatibilities with I2P's threat model. > Currently, we don't have a parser specific for the data collected > by the Tor relays, but we plan to keep adding functionality. > > The source code can be found here: > https://github.com/mavroudisv/Crux The theoretical background, > threat model, security argument and technical details can be found > here: link <http://mavroudisv.eu/files/thesis.pdf> > > > Ideas, comments, feedback much appreciated! Firstly: needs more docs. I shouldn't have to read your entire thesis in order to figure out how to run it :P The README is a good start, but the command parameters need clarification/definitions, and it is not at all obvious how the relays work. An XLS file is also mentioned without explanation. Secondly, you could probably simplify configuration by managing hidden service connections yourself. I believe the Stem library can do this? I2P has a python library that can be used to listen on and connect to I2P Destinations, with a socket-like interface. It's not clear to me from your thesis whether you plan to merge the relay component into the Tor relay code (and have the Tor relays talk directly to the query processors), or have your relays running separately and interfacing with the Tor relays to fetch stats. This will potentially alter how it would be used with other networks. In I2P's case I'd imagine it would be easiest (in terms of usability by those setting up relays) to write an I2P plugin in Java that performs the relay role, but I wouldn't want to be duplicating a lot of logic that subsequently needs to be kept in-sync with upstream :) str4d > > Vasilis > > > > _______________________________________________ tor-dev mailing > list tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJWFP9HAAoJEBO17ljAn7PgAq0P/jrjLnPxg2+rfOX1H46xClpG wfeeNuDltlrvTtNwtDO+lqSzpz+LY/PkUbHow2B7wprRgGvav2kCtGh3xga4BVEX SoYiNcWhRKFmrw6jNjAGPh060hcC7WzBbAjxQ28kotl7NiXO4ZbeMdQ5vqDVE5+K 2f0lSFnBYmj4SYxaVWIxGir3FRn+VdACWjQPL1JV4LDbAPXmNlg17nBwvj5g9lhS n9InsXUqoyhJ5zbVqwHoNl8N2IMui+i2ppF3ZiuHQCHw88+6qJxVW5u11zP2TccJ ycFNE5jcxxNtK9+1jwdfpen5yAnAlaaeKHkHZ+oNN1nb156QbLhcIwqt8IeRFcZv SzaCQk6NIAZFbOVPNzTJ6akXjajd7AbkdVnGBeziD8z8yD1Qa+OH0vBoT/jTP5Ec OdU9f0a3Vc3zftoSaRsCHaQX7+1OEKjDkyk6Gs6lt2PUWVvnNu2+ztXBttI2v8Ee UjEbAC7OR8NpyBAn0p+ja/353zCZU0TXzjccEFb7YtjCCy0hACQGuxwjEECcpbQH MKRZa0JmbSVbDsZoRuMNEwPzg4/sXTVaS8mGmOurotXR1iPgs80DxiAym0kLOe0d xllTDtf561gekxed/+Zm5hjH5FAR7uMI4cpywRUMqyCzJGSKZKr/2uxjnv832j4H ecujGrMEByFdgfG3co5P =etz3 -----END PGP SIGNATURE-----

2 1

#9623 [Tor Browser]: Referers being sent from hidden service websites
by Tim Wilson-Brown - teor 07 Oct '15

07 Oct '15

Hi All, Currently there’s an information leak in Tor Browser: it sends referrer headers containing .onion site addresses when the user clicks on a link on the .onion site. There’s a fix in the works, but we were wondering: Does anyone’s hidden service depend on the referrer header? The currently favoured fix is to stop sending referrers cross-origin (between different .onion sites, and between .onion sites and sites on the internet). But this may break sites that are set up with multiple .onion addresses and use referrers to check that requests are coming from the parent site. (People sometimes set up different .onion sites to serve different types of content, such as images.) In general, I would discourage people from using referrers in this way, because they aren’t secure and can be faked. But does anyone have a compelling use case for cross-origin referrers, or is using them at the moment? We could include a preference if removing them would break too many sites. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

4 3

[Measurement Team] Next IRC meeting happens on Thursday, Oct 8, 14:00 UTC in #tor-dev
by Karsten Loesing 07 Oct '15

07 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Thursday, October 8, 2015, 14:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20151008T14 This will be the first IRC meeting since the Berlin dev meeting, and we're going to talk about the outcomes of that, among other things. For example, we talked about meeting every two weeks on IRC, instead of every week, and moving more communication to email. Be sure to be around tomorrow if you want to discuss things like that. Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWFKklAAoJEJD5dJfVqbCrgV4H/3DoPjbDOk+mMmbCnc5ui0rM sihlWNx7xgKSAKJ5fK6BfCBnSeBHdWC3AzzpZGre21JpZp7FgQqkVOq65z70tANG WaGDHuHmNvtmu45nIDgSDZrgt3teHG2T5EALTcBy7bo3JdBKIVICkheBhO99tvCE Do4/MzHUF4kQ6EMD9bo2n7ASnIi7Tr/RPwA/mNDvHYGlGztJOkH9um+HK99+PLDf hx0c7G3jL2/jiItH+NnBFAoXAahC7F6x3Pga5EhhQ5gy1C4mT//ZWECMAEo5eMDg m9t2y+30ecpJ0x0HIXvx6v5XH0rjOHWu6ASaMim63goQJ1NYy8GAL5dvQZjCR+w= =JnLB -----END PGP SIGNATURE-----

1 0

Networks Blocking Tor's SSL Connections
by Tim Wilson-Brown - teor 07 Oct '15

07 Oct '15

Hi All, This morning I observed a “free wifi” network blocking tor’s SSL connections. While other SSL connections from my machine went through, I observed multiple network traces of tor completing a TCP 3-way handshake, and then getting no reply to the first SSL packet it sent. I think they may have been blocking unknown or untested certificates, but I can’t be sure. Still, I was able to use meek(-google) to access tor. Has anyone else seen this kind of blocking behaviour? (Is this the right list?) Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

2 1

Crux: Privacy-preserving statistics for Tor
by Vasilios Mavroudis 07 Oct '15

07 Oct '15

Hello, I would like to introduce our project "Crux", which enables the computation of privacy preserving statistics on sensitive data. The project was developed at University College London (UCL) by me, in close collaboration with George Danezis. "Crux" was designed with Tor hidden services in mind (in accordance with this <https://research.torproject.org/techreports/hidden-service-stats-2015-04-28…> Tor Tech Report), but it can be applied on various kinds of data (given an appropriate parser). It supports three statistics (i.e. mean, median, variance) and its threat model is compatible Tor's (see technical document below). Currently, we don't have a parser specific for the data collected by the Tor relays, but we plan to keep adding functionality. The source code can be found here: https://github.com/mavroudisv/Crux The theoretical background, threat model, security argument and technical details can be found here: link <http://mavroudisv.eu/files/thesis.pdf> Ideas, comments, feedback much appreciated! Vasilis -- Vasilios Mavroudis

1 0

Re: [tor-dev] Networks Blocking Tor's SSL Connections
by Tim Wilson-Brown - teor 07 Oct '15

07 Oct '15

> On 7 Oct 2015, at 10:52, Brandon Wiley <brandon(a)blanu.net> wrote: > > Hello! I am a researcher focusing on blockability of Tor connections. If you have pcap files that you would like to share showing Tor SSL connections that have been blocked and non-Tor SSL connections that have not been blocked, this would be helpful for my research. I have this pcap analysis framework: https://github.com/blanu/AdversaryLab-offline <https://github.com/blanu/AdversaryLab-offline> as well as this transport for Tor that uses the analysis to find an optimal encoding so that Tor connections are not blocked: https://github.com/blanu/Dust <https://github.com/blanu/Dust> So essentially given two sets of pcaps, one for blocked connections and one for unblocked connections, I can automatically unblock Tor connections. I’m sorry, I only have a pcap file for blocked Tor connections. > If you don't have pcap files or you don't want to share them, if you have access to a network that blocks Tor in a reproducible way, perhaps we could run some tests I’m sorry, I no longer have access to that network. > Let me know if you're interested! > > On Tue, Oct 6, 2015 at 6:06 PM, Tim Wilson-Brown - teor <teor2345(a)gmail.com <mailto:teor2345@gmail.com>> wrote: > Hi All, > > This morning I observed a “free wifi” network blocking tor’s SSL connections. While other SSL connections from my machine went through, I observed multiple network traces of tor completing a TCP 3-way handshake, and then getting no reply to the first SSL packet it sent. > > I think they may have been blocking unknown or untested certificates, but I can’t be sure. > > Still, I was able to use meek(-google) to access tor. > > Has anyone else seen this kind of blocking behaviour? > (Is this the right list?) > > Tim > > Tim Wilson-Brown (teor) > > teor2345 at gmail dot com > PGP 968F094B > > teor at blah dot im > OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F > > > _______________________________________________ > tor-dev mailing list > tor-dev(a)lists.torproject.org <mailto:tor-dev@lists.torproject.org> > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev> > > Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

1 0

Special-use-TLD support
by Jeff Burdges 06 Oct '15

06 Oct '15

This is the first of two torspec proposals to help Tor work with Sepcial-Use TLDs, like the GNU Name system or NameCoin. The second part will be an anycast facility. - Jeff Filename: xxx-special-use-tld-support.txt Title: Special-Use TLD Support Author: Jeffrey Burdges Created: 20 Sept 2015 Status: Draft Implemented-In: ? Abstract Suppose Special-Use TLDs in Tor via external Domain Name System (DNS) suppliers, such as the GNU Name System and NameCoin. Background Special-use TLD supplier software integrates with the host operating system's DNS layer so that other software resolves the special-use TLD identically to standard DNS TLDs. On Linux for example, a Special-Use TLD package could create a plugin for the Name Service Switch (NSS) subsystem of the GNU C Library. Tor cannot safely use the local system's own DNS for name resolution, as doing so risks deanonmizing a user through their DNS queries. Instead Tor does DNS resolution at a circut's exit relay. It follows that Tor users cannot currently use special-use TLDs packages in a safe manor. In addition, there are projects to add public key material to DNS, like TLSA records and DNSSEC, that necessarily go beyond NSS. Design We denote by N an abstract name service supplier package. There are two steps required to integrate N safely with Tor : Of course, N must be modified so as to (a) employ Tor for it's own traffic and (b) to use Tor in a safe way. We deem this step outside the scope of the present document since it concerns modifications to N that depend upon N's design. We caution however that peer-to-peer technologies are famous for sharing unwanted information and producing excessively distinctive traffic profiles, making (b) problematic. Another proposal seeks to provide rudementary tools to asist with (a). We shall instead focus on modifying Tor to route some-but-not-all DNS queries to N. For this, we propose a NameService configuration option that tells Tor where to obtain the DNS record lying under some specific TLD. Anytime Tor resolves a DNS name ending in an Special-Use TLD appearing in an NameService configuration line then Tor makes an RPC request for the name record using given UNIX domain socket or address and port. We should allow CNAME records to refer to .onion domains, and to regular DNS names, but care must be taken in handling CNAME records that refer to Special-Use TLDs handled by NameSerice lines. Tor should reject CNAME records that refer to the .exit domains. Configuration We propose two Tor configuration options : NameSubstitution [.]source_dnspath [.]target_dnspath NameService [.]dnspath socketspec [noncannonical] [timeout=num] [-- service specific options] We require that socketspec be either the path to a UNIX domain socket or an address of the form IP:port. We also require that that each *dnspath be a string conforming to RFC 952 and RFC 1123 sec. 2.1. In other words, a dnsspec consists of a series of labels separated by periods . with each label of up to 63 characters consisting of the letters a-z in a case insensitive mannor, the digits 0-9, and the hyphen -, but hyphens may not appear at the beginning or end of labels. NameSubstitution rules are applied only to DNS query strings provided by the user, not CNAME results. If a trailing substring of a query matches source_dnspath then it is replaced by target_dnspath. NameService rules route matching query to to appropriate name service supplier software. If a trailing substring of a query matches dnspath, then a query is sent to the socketspec using the RPC protcol descrived below. Of course, NameService rules are applied only after all the NameSubstitution rules. There is no way to know in advance if N handles cahcing itself, much less if it handles caching in a way suitable for Tor. Ideally, we should demands that N return an approporaite expiration time, which Tor can respect without harming safety or performance. If this proves problematic, then configuration options could be added to adjust Tor's caching behavior. Seconds is the unit for the timeout option, which defaults to 60 and applies only to the name service supplier lookup. Tor DNS queries, or attempts to contact .onion addresses, that result from CNAME records should be given the full timeout alloted to standard Tor DNS queries, .onion lookups, etc. Any text following -- is passed verbatim to the name service suppllier as service specific options, according to the RPC protocol described below. Control Protocol An equivalent of NameService and NameSubstitution should be added to the Tor control protocol, so that multiple users using the same Tor daemon can have different name resolution rules. RPC protocol We require an RPC format that communicates two values, first any service specific options give on the NameService line, and second the query name itself of course. We might however discuss if there are any standardized flags, distinct from these options, and whether they should be communicated separately. In principle, Tor could make due with simply receiving a strong in return. We recommend however that Tor expect a return format as or more powerful than full DNS queries. In particular, we should endever to return TLSA records at the same time as the underlying DNS record, so that Tor Browser can utilize that key material. The GNS Record format used by the GNU Name System addresses this and other issues, so it should be taken as a candidate. See : https://github.com/GNUnet/gnunet/blob/gnunet/src/include/gnunet_gnsrecord_l… Sepcial-use tLD suppliers should internally process CNAME records that fall into their own domains, but they should return CNAME records to Tor that refer to .onion or .exit domains, or to normal DNS names. Initially, Tor should issue an error if it recieves a CNAME record that matches an NameService line. If however that NameService line contains the noncannonical option, then CNAME records should instead bypass it, and use Tor's DNS system. At present, alternative DNS packages should not pass CNAME records between themselves, despite speaking the same RPC protocol, as this creates unknown risks. As such forwarding can be done most safely by Tor itself, the Tor Project reserves the right to forward CNAME records between NameService lines in the future. Applications should therefore not depend upon the above error being returned. Variations Tor would conceivably benefit from externalizing its own DNS handling as a separate process. This might however require that Tor have the ability to start name service suppliers. A fuller consideration of this might alter our design of the NameService configuration option. Acknowledgments Based on extensive discussions with Christian Grothoff and George Kadianakis.

8 20

Proposal: HTTP header distinguish TBB users
by Virgil Griffith 04 Oct '15

04 Oct '15

Yesterday Lief compellingly argued that if a TBB user accidentally clicks on a link to my tor2web proxy (onion.link), that they should be redirected to the .onion address. It hadn't occurred before that a Tor user might accidentally click a onion.link URL, but yes I completely concur and I told Lief I would prioritize this and would owe him a Bitcoin if I didn't get this implemented within a week. Now the trouble starts. If the TBB user gets to the tor2web backend I check if they're coming from an Exit relay and redirect them---all good. But a CDN (Fastly.com) sits in front of my backends and right now it's unclear how to detect TBB at the CDN level. Going over my CDN's documentation. They do have the standard MaxMind database for geo-IP. So that's good. But plugging in an exit-node IP# merely reports as an "A1" for "Anonymous Proxy". Unfortunately there are many anonymous proxies other than Tor so that won't do. There are two ways to solve this. (1) For an given IP#, MaxMind reports numerous entries aside from the "A1" for country code. We could ask MaxMind to specify whatever else it knows about the Anonymous Proxy in the other fields such as the "Location" or "Organization" field. So when plugging in a Tor exit relay it would return something like: { "countrycode": "A1", "location": "Tor", "domain": "torproject.org" } or some such. This seems a reasonable request. Do we know someone at MaxMind to forward this request to? (2) If we (Tor Project) is going to ask MaxMind to do something special to distinguish TBB users, it seems reasonable we should make the same effort. I know in the past it's been proposed for TBB to include a special HTTP header, e.g., Tor-Browser-Bundle: true to distinguish TBB users. If this header existed, I could detect it at the CDN-level and do the appropriate redirect. Alternatively, We could do something equivalent with the "Via": HTTP header, but that seems overkill. Between these two options, I personally opt for (2) because it seems inappropriate to request MaxMind to help us do X when we have not done what we can do to achieve X. Q: Does anyone (especially Mike Perry) have any objections to (2)? If not, I will write the proposal. -V P.S. Lief... even if we go at maximum speed, it looks like I'm going to owe you that Bitcoin. Email me your BTC address? How embarrassing.

6 10

Notes+Action Items from Hidden Service Fingerprinting session
by Mike Perry 03 Oct '15

03 Oct '15

Our session on Friday about Hidden Service Fingerprinting wandered a lot. Ultimately, we concluded that it will be hard to fully defend against this attack without a more formally specified state machine that describes hidden service circuit usage fully. However, we may still be able to extend the Netflow Padding defense to pad more often during circuit setup, which should at least improve the situation against an adversary that is externally monitoring the Guard node. Here's the specific action items from the session yesterday: * Review Prop 224 and other proposals for distinguishes for client traffic * Specify a protocol state machine as part of the design * If it is more complicated than the TCP state machine, we should run in fear * Otherwise, researchers should be able to make use of this state machine to determine optimal padding * Netflow padding should have a state to cover circuit setup * Based on Circuit Build Timeout * This allows only 2-10 seconds or so of more frequent padding * The "HS's sometimes need to use a second Guard" distinguisher * The RP should never be a Guard node * The third level node should never be a Guard node * TBB Update pings over HS * Increases the Base Rate, since more clients are building hidden service circuits * This is far easier to do for the Torbutton RecommendedTBBVersions fetch since the updater has special cert pinning that would need to be altered for hidden services * Future Research: * Conflux + Guard Sets * Provides Redundancy against active delay and DoS attacks * May help against fingerprinting and correlation * However, still won't help against a datacenter adversary :/ -- Mike Perry

1 0

Hello, the obfs4 bridge which I got on October 1st is blocked in China today.
by Li Xiaodong 03 Oct '15

03 Oct '15

*Hello, the obfs4 bridge which I got on October 1st is blocked in China today. I think that the number of Tor bridges is limited. If China's firewall were to block all of Tor bridges, how Tor could work in China? Are there any better ways to connect to Tor network than bridges? Thank you very much for your help. I really appreciate it. All the best with your work. Good luck in all that you will achieve. Thank you so much for your help again.*

1 0

Hello, are there better ways to connect to the Tor network than bridges?
by Li Xiaodong 02 Oct '15

02 Oct '15

*Hello, are there any better ways to connect to the Tor network than bridges?*

2 1