- tor-dev - lists.torproject.org

Video of my presentation on domain fronting (PETS 2015)
by David Fifield 26 Aug '15

26 Aug '15

My presentation on domain fronting (the meek pluggable transport in Tor), which I gave at PETS 2015 on June 30, 2015, is online. This page has the video and a copy of the slides. It is about 17 minutes long. https://www.bamsoftware.com/talks/fronting-pets2015/ Here is the paper on which the talk is based, by Chang Lan, Rod Hynes, Percy Wegmann, Vern Paxson, and me: https://www.bamsoftware.com/papers/fronting/

1 0

[Measurement Team] Next IRC meeting happens on Wednesday, Aug 26, 14:00 UTC in #tor-project
by Karsten Loesing 25 Aug '15

25 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Wednesday, August 26, 2015, 14:00 UTC in #tor-project https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150826T14 Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV3IStAAoJEJD5dJfVqbCr0NYIAK00hHH1HjxkjNGyjnJuAfyV 6XpSzI4zGLPNuuG5+rap3yGnYo/LgsvE4mfGfMGIfk4FWJ6mEC//rh0b/F6BgDsm lKfIG4FBaPkkjWHn/jTPfbSdHSc/6P3tXjM4M2ZJ1Kj28axmD3nIwAP7miRRQK5C fEq5WuKpmcW++rtJfuGml10pqOtMkbjgiTK41myVSqbWd4rdgix9A6siUcWY9Mkd ZDltyth1hDXuhDpmtBz3RuN6s4901KU6cc3ZrOk3mVUG/Eos6lfpThVkcIpRIUxz XrWPtQxUxBqOe2GuID/0pvZK4/n7R/nDldrOJ+QKd3ST5rgAVu2Q8CYo1a9YYoY= =qYpH -----END PGP SIGNATURE-----

1 0

[PATCH] Log malformed hostnames in socks5 request respecting SafeLogging
by Andreas Stieger 25 Aug '15

25 Aug '15

Hello, I found a warning-level message in socks5 code relating to malformed hostnames that did not respect the SafeLogging setting, breaking the rule of least surprise. Please review the attached simple patch. Andreas -- Andreas Stieger <astieger(a)suse.com> Project Manager Security SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

2 3

Updated aggregated data served by Metrics
by Karsten Loesing 25 Aug '15

25 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi devs, if you're fetching .csv files from Metrics or if you're interested in discussing future .csv files served by Metrics you should read on. (And if you're surprised that Metrics serves .csv files at all and want to learn more, tick the "Advanced" box under "Level" and take a look at all the pages of type "Data".) I just deployed a new data-aggregating module on Metrics that produces the .csv file here: https://metrics.torproject.org/connbidirect2-data.html That .csv file replaces the following .csv file which will go away in the future: https://metrics.torproject.org/connbidirect-data.html The main difference between the two files is that the new file contains daily aggregates whereas the old file contained raw statistics as reported by relays, which is why I added the aggregation step. So, that's news number one: if you're currently using that file for any reason, please let me know, and I'll try to help you generate the same data from CollecTor's data yourself. But the main reason for changing the format is that I wanted to separate the code to extract and aggregate statistics into a new Metrics module for better maintenance. And here's news number two: I'm planning to do the same with the following three .csv files: https://metrics.torproject.org/servers-data.html https://metrics.torproject.org/bandwidth-data.html https://metrics.torproject.org/torperf-data.html If you have any ideas for improving those data formats or adding closely related data while doing so, please let me know. Also, if you have ideas for adding new .csv files and new graphs based on them, I'd want to learn about that. Note that I might not be able to include all suggestions. But now's a much better time to make them than when I'm done replacing those three .csv files. (I'll also look through open Trac tickets before writing any new code.) Thanks! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV3DoyAAoJEJD5dJfVqbCroRgIAKL8lV6MkY+NQZcE7PkuVxzl lN/GadzzQVgN0bXzRw/qu10ds8yq7R3ir3z85OH2ojHxrSzrqK+7oZPpdhSELEvf jTE5Elr60G1YOGq1bw84v1eUoOnu4BfQOanPaIVG8Q8jzPCiIbGRhVOaWwR32gSf MFVphGwAjB+quz4I+63aPC/EPo0420QwnGSAEKpQiU4Cg2FJmj4NWGO/gkoQ0Xor 5hjJQGHKiwtzSUfiV+kGBaz65XPTLZv0od/IP6tCk6FAU8T1IphhANx8ZmWmlJas WlpoUzW+rRUQRY9bjkfDxRW5dRxt2EXTvoLLMlElUtnyCferX1g27TJQhwWjXNI= =n1mW -----END PGP SIGNATURE-----

1 0

Proposal: Padding for netflow record resolution reduction
by Mike Perry 25 Aug '15

25 Aug '15

Filename: xxx-netflow-padding.txt Title: Padding for netflow record resolution reduction Authors: Mike Perry Created: 20 August 2015 Status: Draft 0. Motivation It is common practice by many ISPs to record data about the activity of endpoints that use their uplink, if nothing else for billing purposes, but sometimes also for monitoring for attacks and general failure. Unfortunately, Tor node operators typically have no control over the data recorded and retained by their ISP. They are often not even informed about their ISP's retention policy, or the associated data sharing policy of those records (which tends to be "give them to whoever asks" in practice[1]). It is also likely that defenses for this problem will prove useful against proposed data retention plans in the EU and elsewhere, since these schemes will likely rely on the same technology. 0.1. Background At the ISP level, this data typically takes the form of Netflow, jFlow, Netstream, or IPFIX flow records. These records are emitted by gateway routers in a raw form and then exported (often over plaintext) to a "collector" that either records them verbatim, or reduces their granularity further[2]. Netflow records and the associated data collection and retention tools are very configurable, and have many modes of operation, especially when configured to handle high throughput. However, at ISP scale, per-flow records are very likely to be employed, since they are the default, and also provide very high resolution in terms of endpoint activity, second only to full packet and/or header capture. Per-flow records record the endpoint connection 5-tuple, as well as the total number of bytes sent and received by that 5-tuple during a particular time period. They can store additional fields as well, but it is primarily timing and bytecount information that concern us. When configured to provide per-flow data, routers emit these raw flow records periodically for all active connections passing through them based on two parameters: the "active flow timeout" and the "inactive flow timeout". The "active flow timeout" causes the router to emit a new record periodically for every active TCP session that continuously sends data. The default active flow timeout for most routers is 30 minutes, meaning that a new record is created for every TCP session at least every 30 minutes, no matter what. This value can be configured to be from 1 minute to 60 minutes on major routers. The "inactive flow timeout" is used by routers to create a new record if a TCP session is inactive for some number of seconds. It allows routers to avoid the need to track a large number of idle connections in memory, and instead emit a separate record only when there is activity. This value ranges from 10 seconds to 600 seconds on common routers. It appears as though no routers support a value lower than 10 seconds. 0.2. Default timeout values of major routers For reference, here are default values and ranges (in parenthesis when known) for common routers, along with citations to their manuals. Some routers speak other collection protocols than Netflow, and in the case of Juniper, use different timeouts for these protocols. Where this is known to happen, it has been noted. Inactive Timeout Active Timeout Cisco IOS[3] 15s (10-600s) 30min (1-60min) Cisco Catalyst[4] 5min 32min Juniper (jFlow)[5] 15s (10-600s) 30min (1-60min) Juniper (Netflow)[6,7] 60s (10-600s) 30min (1-30min) H3C (Netstream)[8] 60s (60-600s) 30min (1-60min) Fortinet[9] 15s 30min MicroTik[10] 15s 30min 1. Proposal Overview The combination of the active and inactive netflow record timeouts allow us to devise a low-cost padding defense that causes what would otherwise be split records to "collapse" at the router even before they are exported to the collector for storage. So long as a connection transmits data before the "inactive flow timeout" expires, then the router will continue to count the total bytes on that flow before finally emitting a record at the "active flow timeout". This means that for a minimal amount of padding that prevents the "inactive flow timeout" from expiring, it is possible to reduce the resolution of raw per-flow netflow data to the total amount of bytes send and received in a 30 minute window. This is a vast reduction in resolution for HTTP, IRC, XMPP, SSH, and other intermittent interactive traffic, especially when all user traffic in that time period is multiplexed over a single connection (as it is with Tor). 2. Implementation Tor clients currently maintain one TLS connection to their Guard node to carry actual application traffic, and make up to 3 additional connections to other nodes to retrieve directory information. We propose to pad only the client's connection to the Guard node, and not any other connection. We propose to treat Bridge node connections to the Tor network as client connections, and pad them, but otherwise not pad between normal relays. Both clients and Guards will maintain a timer for all application (ie: non-directory) TLS connections. Every time a non-padding packet is sent or received by either end, that endpoint will sample a timeout value from between 4 seconds and 14 seconds. If the connection becomes active for any reason before this timer expires, the timer is reset to a new random value between 4 and 14 seconds. If the connection remains inactive until the timer expires, a single CELL_PADDING cell will be sent on that connection. In this way, the connection will only be padded in the event that it is idle. 2.1. Tunable parameters We propose that the defense be controlled by the following consensus parameters: * nf_ito_low - The low end of the range to send padding when inactive, in ms. - Default: 4000 * nf_ito_high - The high end of the range to send padding, in ms. - Default: 14000 * nf_pad_relays - If set to 1, we also pad inactive relay-to-relay connections - Default: 0 * conn_timeout_low - The low end of the range to decide when we should close an idle connection (not counting padding). - Default: 900 seconds after last circuit closes * conn_timeout_high - The high end of the range to decide when we should close an idle connection. - Default: 1800 seconds after last circuit close If nf_ito_low == nf_ito_high == 0, padding will be disabled. 2.2. Maximum overhead bounds With the default parameters, we expect a padded connection to send one padding cell every 8 seconds (see Appendix A for the statistical analysis of expected padding packet rate on an idle link). This averages to 63 bytes per second, assuming a 512 byte cell and 55 bytes of TLS+TCP+IP headers. For a connection that remains idle for a full 30 minutes of inactivity, this is about 112KB of overhead. With 2.5M completely idle clients connected simultaneously, 63 bytes per second still amounts to only 157MB/second network-wide, which is roughly 1.5X the current amount of Tor directory traffic[11]. Of course, our 2.5M daily users will neither be connected simultaneously, nor entirely idle, so we expect the actual overhead to be much lower than this. If we choose to set the range such it is no longer even possible to configure an inactive timeout low enough such that a new record would be generated, (for example: 6.5 to 9.5 seconds for an average of 8 seconds), then the maximum possible overhead is 70 bytes/second, 125KB per connection, and 175MB/sec network-wide. 2.3. Measuring actual overhead To measure the actual padding overhead in practice, we propose to export the following statistics in extra-info descriptors for the previous (fixed, non-rolling) 24 hour period: * Total cells read (padding and non-padding) * Total cells written (padding and non-padding) * Total CELL_PADDING cells read * Total CELL_PADDING cells written * Total RELAY_COMMAND_DROP cells read * Total RELAY_COMMAND_DROP cells written These values will be rounded to 100 cells each, and no values are reported if the relay has read or written less than 10000 cells in the previous period. RELAY_COMMAND_DROP cells are circuit-level padding not used by this defense, but we may as well start recording statistics about them now, too, to aid in the development of future defenses. 2.4. Load balancing considerations Eventually, we will likely want to update the consensus weights to properly load balance the selection of Guard nodes that must carry this overhead. We propose that we use the extra-info documents to get a more accurate value for the total average Guard and Guard+Exit node overhead of this defense in practice, and then use that value to fractionally reduce the consensus selection weights for Guard nodes and Guard+Exit nodes, to reflect their reduced capacity relative to middle nodes. 3. Threat model and adversarial considerations This defense does not assume fully adversarial behavior on the part of the upstream network administrator, as that administrator typically has no specific interest in trying to deanonymize Tor, but only in monitoring their own network for signs of overusage, attack, or failure. Therefore, in a manner closer to the "honest but curious" threat model, we assume that the netflow collector will be using standard equipment not specifically tuned to capturing Tor traffic. We want to reduce the resolution of logs that are collected incidentally, so that if they happen to fall into the wrong hands, we can be more certain will not be useful. We feel that this assumption is a fair one because correlation attacks (and statistical attacks in general) will tend to accumulate false positives very quickly if the adversary loses resolution at any observation points. It is especially unlikely for the the attacker to benefit from only a few high-resolution collection points while the remainder of the Tor network is only subject to connection-level netflow configuration data retention, or even less data retention than that. Nonetheless, it is still worthwhile to consider what the adversary is capable of, especially in light of looming data retention regulation. Because no major router appears to have the ability to set the inactive flow timeout below 10 seconds, it would seem as though the adversary is left with three main options: reduce the active record timeout to the minimum (1 minute), begin logging full packet and/or header data, or develop a custom solution. It is an open question to what degree these approaches would help the adversary, especially if only some of its observation points implemented these changes. 3.1 What about sampled data? At scale, it is known that some Internet backbone routers at AS boundaries and exchanges perform sampled packet header collection and/or produce netflow records based on a subset of the packets that pass through their infrastructure. The effects of this against Tor were studied before against the (much smaller) Tor network as it was in 2007[12]. At sampling rate of 1 out of every 2000 packets, the attack did not achieve high accuracy until over 100MB of data were transmitted, even when correlating only 500 flows in a closed-world lab setting. We suspect that this type of attack is unlikely to be effective at scale on the Tor network today, but we make no claims that this defense will make any impact upon sampled correlation, primarily because the amount of padding that this defense introduces is comparatively low relative to the amount of transmitted traffic that sampled correlation attacks require to attain any accuracy. 3.2. What about long-term statistical disclosure? This defense similarly does not claim to defeat long-term correlation attacks involving many observations over large amounts of time. However, we do believe it will significantly increase the amount of traffic and the number of independent observations required to attain the same accuracy if the adversary uses default per-flow netflow records. 3.3. What about prior information/confirmation? In truth, the most dangerous aspect of these netflow logs is not actually correlation at all, but confirmation. If the adversary has prior information about the location of a target, and/or when and how that target is expected to be using Tor, then the effectiveness of this defense will be very situation-dependent (on factors such as the number of other tor users in the area at that time, etc). In any case, the odds that there is other concurrent activity (to create a false positive) within a single 30 minute record are much higher than the odds that there is concurrent activity that aligns with a subset of a series of smaller, more frequent inactive timeout records. 4. Synergistic effects with future padding and other changes Because this defense only sends padding when the OR connection is completely idle, it should still operate optimally when combined with other forms of padding (such as padding for website traffic fingerprinting and hidden service circuit fingerprinting). If those future defenses choose to send padding for any reason at any layer of Tor, then this defense automatically will not. In addition to interoperating optimally with any future padding defenses, simple changes to Tor network usage can serve to further reduce the usefulness of any data retention, as well as reduce the overhead from this defense. For example, if all directory traffic were also tunneled through the main Guard node instead of independent directory guards, then the adversary would lose additional resolution in terms of the ability to differentiate directory traffic from normal usage, especially when it is occurs within the same netflow record. As written and specified, the defense will pad such tunneled directory traffic optimally. Similarly, if bridge guards[13] are implemented such that bridges use their own guard node to route all of their connecting client traffic through, then users who run bridges bridges will also benefit from the concurrent traffic of their connected clients, which will also be optimally padded when it is concurrent with their own. Appendix A: Padding Cell Timeout Distribution Statistics It turns out that because the padding is bidirectional, and because both endpoints are maintaining timers, this creates the situation where the time before sending a padding packet in either direction is actually min(client_timeout, server_timeout). If client_timeout and server_timeout are uniformly sampled, then the distribution of min(client_timeout,server_timeout) is no longer uniform, and the resulting average timeout (Exp[min(X,X)]) is much lower than the midpoint of the timeout range. To compensate for this, instead of sampling each endpoint timeout uniformly, we instead sample it from max(X,X), where X is uniformly distributed. If X is a random variable uniform from 0..R-1 (where R=high-low), then the random variable Y = max(X,X) has Prob(Y == i) = (2.0*i + 1)/(R*R). Then, when both sides apply timeouts sampled from Y, the resulting bidirectional padding packet rate is now third random variable: Z = min(Y,Y). The distribution of Z is slightly bell-shaped, but mostly flat around the mean. It also turns out that Exp[Z] ~= Exp[X]. Here's a table of average values for each random variable: R Exp[X] Exp[Z] Exp[min(X,X)] Exp[Y=max(X,X)] 5000 2499.5 2662 1666.2 3332.8 6000 2999.5 3195 1999.5 3999.5 10000 4999.5 5328 3332.8 6666.2 15000 7499.5 7995 4999.5 9999.5 20000 9900.5 10661 6666.2 13332.8 In this way, we maintain the property that the midpoint of the timeout range is the expected mean time before a padding packet is sent in either direction. 1. https://lists.torproject.org/pipermail/tor-relays/2015-August/007575.html 2. https://en.wikipedia.org/wiki/NetFlow 3. http://www.cisco.com/en/US/docs/ios/12_3t/netflow/command/reference/nfl_a1g… 4. http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-swi… 5. https://www.juniper.net/techpubs/software/erx/junose60/swconfig-routing-vol… 6. http://www.jnpr.net/techpubs/en_US/junos15.1/topics/reference/configuration… 7. http://www.jnpr.net/techpubs/en_US/junos15.1/topics/reference/configuration… 8. http://www.h3c.com/portal/Technical_Support___Documents/Technical_Documents… 9. http://docs-legacy.fortinet.com/fgt/handbook/cli52_html/FortiOS%205.2%20CLI… 10. http://wiki.mikrotik.com/wiki/Manual:IP/Traffic_Flow 11. https://metrics.torproject.org/dirbytes.html 12. http://freehaven.net/anonbib/cache/murdoch-pet2007.pdf 13. https://gitweb.torproject.org/torspec.git/tree/proposals/188-bridge-guards.… -- Mike Perry

3 2

Feature freeze plans for Tor 0.2.7: Please read if you hack Tor!
by Nick Mathewson 25 Aug '15

25 Aug '15

Hi, all! Here's the schedule we worked out for the Tor 0.2.7 feature freeze. (These are defaults, not promises. We can make exceptions, but please remember that delaying a freeze will delay release, and every day we delay a release will delay all the _other_ features getting out into a stable Tor. We've been aiming to speed up our release cycle, and this represents the latest installment in that effort.) Sep 1: Last day to merge (most) feature patches into 0.2.7. I'll make an exception for small features that get permission in advance and don't seem likely to endanger stability. If you want to nominate a ticket as such a feature, **and you are going to implement it**, please add the PostFreeze027 tag to it ASAP. Make sure it's in before next week's dev meeting at the earliest. Also please don't drop a huge branch on me on August 27 and expect me to have it merged by Sep 1. :) Sep 15: Last day to merge non-critical patches into 0.2.7. After this date, we won't accept any more features for 0.2.7. We also won't accept most bugfixes: bugfixes will only be accepted for regressions, security bugs, stability bugs, documentation, and typos. When it's ready: we put out a stable 0.2.7. peace, -- Nick

3 2

request for advice regarding bug 15901
by starlight.2015q3＠binnacle.cx 23 Aug '15

23 Aug '15

Could someone familiar with consensus document downloading, manipulation and validation by the tor relay examine the most recent post to bug #15901 and comment with advice regarding an idea I have for isolating the problem? Tor #15901: apparent memory corruption -- very difficult to isolate https://trac.torproject.org/projects/tor/ticket/15901#comment:20 Thank You

1 0

"Seeing through Network-Protocol Obfuscation"
by Philipp Winter 22 Aug '15

22 Aug '15

<https://kpdyer.com/publications/ccs2015-measurement.pdf> They claim that they are able to detect obfs3, obfs4, FTE, and meek using entropy analysis and machine learning. I wonder if their dataset allows for such a conclusion. They use a (admittedly, large) set of flow traces gathered at a college campus. One of the traces is from 2010. The Internet was a different place back then. I would also expect college traces to be very different from country-level traces. For example, the latter should contain significantly more file sharing, and other traffic that is considered inappropriate in a college setting. Many countries also have popular web sites and applications that might be completely missing in their data sets. Considering the rate difference between normal and obfuscated traffic, the false positive rate in the analysis is significant. Trained classifiers also seem to do badly when classifying traces they weren't trained for. The authors suggest active probing to reduce false positives, but don't mention that this doesn't work against obfs4 and meek. Cheers, Philipp

3 7

Re: [tor-dev] Number of directory connections
by tordev123＠Safe-mail.net 21 Aug '15

21 Aug '15

-------- Original Message -------- From: Yawning Angel <yawning(a)schwanenlied.me> Subject: Re: [tor-dev] Number of directory connections Date: Fri, 21 Aug 2015 16:49:18 +0000 >> It looks like when the consensus is older than 5 days, a directory authority is used (and the >> UseEntryGuardsAsDirGuards setting basically ignored). >> >> I can't think of a good reason for that behavior. Is there a valid >> security issue using an old consensus to fetch the current one from >> the entry guard? > > Your entry guard being rejected as malicious by the DirAuths in the > intervening time period. But the new consensus would need to be signed by the directory authorities. Tor won't relay anything before it has the new valid consensus. So I don't see any issue with using a potentially malicious entry guard for obtaining the current consensus.

1 0

Re: [tor-dev] Number of directory connections
by tordev123＠Safe-mail.net 21 Aug '15

21 Aug '15

-------- Original Message -------- From: "l.m" <ter.one.leeboi(a)hush.com> Subject: Re: [tor-dev] Number of directory connections Date: Fri, 21 Aug 2015 09:31:25 -0400 > Oh I see, so they happened before. I wasn't sure about that. In that case the last consensus stored locally must have been many days old. If that's the case you would bootstrap from dirauths then use your guard for tunneling later directory request. I did some experiments. It looks like when the consensus is older than 5 days, a directory authority is used (and the UseEntryGuardsAsDirGuards setting basically ignored). I can't think of a good reason for that behavior. Is there a valid security issue using an old consensus to fetch the current one from the entry guard?

2 1

Re: [tor-dev] tor callgrinds
by coderman 21 Aug '15

21 Aug '15

On 2/16/07, Christopher Layne <clayne(a)anodized.com> wrote: > Thought you guys might find this interesting. I did a couple of callgrind > runs ... yes, really back to 2007 to find a callgrind on a relay. does anyone have a more recent profile of a busy relay, by chance? if none exist, what do i need to bribe you with? :) thank you! best regards,

1 0

Re: [tor-dev] Number of directory connections
by tordev123＠Safe-mail.net 21 Aug '15

21 Aug '15

-------- Original Message -------- From: George Kadianakis <desnacked(a)riseup.net> Subject: Re: [tor-dev] Number of directory connections Date: Fri, 21 Aug 2015 15:02:02 +0300 > There are many unlucky people whose guard is not a directory cache, so they have > to reach out to additional guards to get directory documents :x > > I think most powerful guards are directory caches, so most people are lucky in > the end. All people will be lucky when we finally implement and deploy #12538. Per the current consensus: Number of Guards: 1610 Number of V2Dir Guards: 1237 (77%) Total Guard Bandwidth: 27387966 Total V2Dir Guard Bandwidth: 22603019 (83%)

1 0

Re: [tor-dev] Number of directory connections
by tordev123＠Safe-mail.net 21 Aug '15

21 Aug '15

-------- Original Message -------- From: "l.m" <ter.one.leeboi(a)hush.com> Subject: Re: [tor-dev] Number of directory connections Date: Fri, 21 Aug 2015 08:02:22 -0400 > UseEntryGuardsAsDirGuards defaults to 1 in torrc. > > So if you did not change this default you will use entry guards for tunneling directory connections. Then there must be some bug given the 4 directory connections. The directory connections all happened before the entry node connection. torrc: ------------------------------------------------------------ DataDirectory [censored] GeoIPFile [censored] GeoIPv6File [censored] Socks5Proxy [censored] EntryNodes [censored],[censored],[censored] ------------------------------------------------------------

2 1

Number of directory connections
by tordev123＠Safe-mail.net 21 Aug '15

21 Aug '15

-------- Original Message -------- From: Mike Perry <mikeperry(a)torproject.org> Subject: [tor-dev] Proposal: Padding for netflow record resolution reduction Date: Thu, 20 Aug 2015 21:12:54 -0700 > Tor clients currently maintain one TLS connection to their Guard node to > carry actual application traffic, and make up to 3 additional connections to > other nodes to retrieve directory information. That doesn't seem to be the case. Tor from TBB 5.0 startup (directory info was a couple of days old): [GUARD] xxx.xxx.xxx.xxx:443 received: 17967 sent: 12865 [dizum] 194.109.206.212:443 received: 564549 sent: 19078 [Faravahar] 154.35.175.225:443 received: 253909 sent: 55616 [maatuska] 171.25.193.9:80 received: 132954 sent: 28519 [tor26] 86.59.21.38:443 received: 370630 sent: 80132 There were no other connection attempts.

3 2

Re: [tor-dev] Hash Visualizations to Protect Against Onion Phishing
by Jesse V 21 Aug '15

21 Aug '15

> Jacek Wielemborek <d33tah(a)gmail.com> writes: > >> George Kadianakis pisze: >>> Some real UX research needs to be done here, before we decide something terrible. >> >> Just curious, has anybody seen any cognitive studies on the SSH >> randomart visualisation? I always found them impossible to remember. >> Perhaps adding a bit more color could help... > Hm. Indeed. > > I can remember the general shape and edges of my SSH server's key, but not any > details. > > I doubt I would remember the randomart of like 10 onion websites, especially if > I didn't visit them regularly. But maybe I would remember the randomart of my > webmail better than my SSH server's. > The main issue is that there's always going to be a tradeoff between memorability and security. It's difficult to cram sufficient entropy into a visualization and expect people to remember it. I agree that coloring schemes or perhaps faces stand the best chance of memorability, but they are difficult to deploy. I wonder if a simpler scheme is sufficient. I suggest using a word-bank to generate a series of words. First, take the .onion address or the hidden service public identity key (basically the same thing for 224) and run it through scrypt or similar algorithm. Then, based on the output, select a series of words from the dictionary. Present the series of words in George's mockup (https://people.torproject.org/~asn/tbb_randomart/randomart_mockup.png) in lieu of the art. It's not a new idea to use a word-list for this purpose: I recall reading a paper suggesting to use a word-list to encode .onion addresses rather than base32. The scheme has also been deployed by websites; Gfycat, for example, shows a series of words in its URL to provide an identifier to user images. It's also successful in practice: everyone in the /r/globaloffensive subreddit recognizes the DelayedAutisticGuppy reference before they even open the gif. People remember random information best if it's grouped, but usually the maximum group size is about 4, which is why phone numbers are split with delimiters. To make it simple, I suggest showing one row of four words. Each character in a .onion address has 32 possible combinations. If you used a word-bank of 1,024 words, you cover two characters per word. If your dictionary consists of 32,768 words, you can capture three at a time. Assuming this latter case, if four words are used, you can cover 12 characters, or 60 bits. If you combine that with the number of characters in the address that users intrinsically check subconsciously, it's extremely difficult for an attacker to match all of them. The Shallot README indicates that it would take nearly 10 millenia to find a 12-character match, and this of course does not take into account a round of scrypt before the word-list is used. Everyone remembers "correct horse battery staple", right?

4 3

6th status report for OnioNS
by Jesse V 20 Aug '15

20 Aug '15

This status report covers three main points of progress for the Onion Name System (OnioNS) project. 1) Since my last report, I opened up this project for beta testing. The server logs show that a number of people opened and ran servers for some time, and a number of individuals claimed names for their hidden services. S7r was extremely helpful in beta testing, we had a very productive private conversation on IRC. He helped me identify bugs and several areas of improvement. He also reproduced a few show-stopping issues that I had considered closed, so I'll be taking care of those as soon as I can. It was a thorough test and I very much appreciate the help. 2) I spent a significant amount of time over the last few weeks rewriting my design paper. I applied two suggestions that I heard during PETS and the Arlington HS meeting: a) to switch from using the consensus for a global source of entropy to the new commit-and-reveal proposal, and b) to select members of the Quorum (the authoritative set of OnioNS nodes) with selection weight proportional to their consensus weight, rather than with even probability. Both of these turned out to be excellent ideas. The dirauth commitment protocol is far more resistant to attacks than my original scheme and the new Quorum selection technique is highly resistant to Sybil attacks. In fact, even if all Fast and Stable routers in the lowest 70 percentile by consensus weight are malicious, it's still extremely unlikely that a random Quorum selection will result in a situation where the attacker wins. 3) There's an upstream change that I forgot to mention in my last report: the fact that Tor 0.2.7 now creates and manages Ed25519 keys brings OnioNS one step closer towards integration with Tor: my design documents call for OnioNS servers to manage Ed25519 signatures. It's a stronger and faster cryptosystem than RSA and has much smaller signatures, which reduces the overhead of my data structures. Down the road, I should be able to use the online signing key, so I won't need to touch the offline identity key. As we come closer to the end of the Summer of Privacy timeline, I'm going to concentrate on addressing the bugs that s7r identified, finish implementing some security protocol, and try to move to a fully decentralized setup. Jesse V.

1 0

bwauths: Are you publishing your output somewhere?
by nusenu 19 Aug '15

19 Aug '15

Hi, since a request to make bwauth output available has been closed as wontfix previously [1] I'm not going to ask whether this data would be added to CollecTor, but maybe bwauth ops are publishing it 'inofficially' (like Tom does [2]) for the dir auth to fetch already and would be willing to disclose the URI to the files? thanks! (one can also use the votes [3] to find out what the dir auth submitted but the raw bwauth files would include the timestamps/frequency) [1] https://trac.torproject.org/projects/tor/ticket/2532 [2] https://bwauth.ritter.vg/bwauth/ [3] https://collector.torproject.org/recent/relay-descriptors/votes/

1 0

[Measurement Team] Next IRC meeting happens on Wednesday, Aug 19, 14:00 UTC in #tor-project
by Karsten Loesing 18 Aug '15

18 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Wednesday, August 19, 2015, 14:00 UTC in #tor-project https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150819T14 Suggested agenda is quite similar to last week which worked pretty well: - 1-1-1 task exchange: you get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do for you within 1 week (review a document, write some analysis code, fix a small bug, etc.; better come prepared to get the most out of this; give 1, take 1) - Misc/discussion: whatever comes up during the meeting If you think this can be fun, please join us at the meeting! Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV0zycAAoJEJD5dJfVqbCrwS4H/2YWIf9LxzQJMgmfD//Stqr2 6Set5ZYzMnzNgSgiOY1edaza0tepG+oDPOCaaM2ly7+2HAH9HmjoGb6qBRl6Y8Of XdhXp5jgGLrHeee1oznNWsrgY8K/I1Oh26Q6IlPsTyNEVbxL3IvHMX/nz0q7QXUT iL1UkzwldU6wVVnn8E1Y6HgQlRnipsC1ii0VXEnSp+O4JQ0fXU0l88ZsEywff6fo iSyYzLFT1YUcoFfHRALyCwLKrIDCWjWDlP3Zi7C3BZD2uWyJTqL1FExnoAXlBkTq ruCt+nJWNFsF4QFxhQqxOfeGLGPja4Lap1BTqYRjXe8pMZ4ZPZoah+0ht7okCr4= =/tK9 -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] Get Stem and zoossh to talk to each other
by tordev123＠Safe-mail.net 17 Aug '15

17 Aug '15

> > zoossh's test framework says that it takes 36364357 nanoseconds to > > lazily parse a consensus that is cached in memory (to eliminate the I/O > > bottleneck). That amounts to approximately 27 consensuses a second. > > > > I used the following simple Python script to get a similar number for > > [...] > > This script manages to parse 24 consensus files in ~13 seconds, which > > amounts to 1.8 consensuses a second. Let me know if there's a more > > efficient way to do this in Stem. > > Interesting! First thought is 'wonder if zoossh is even reading the > file content'. Couple quick things to try are... > > with open(file_name) as consensus_file: > consensus_file.read() > > ... to see how much time is disk IO verses parsing. Second is to try > doing something practical (say, count the number of relays with the > exit flag). Stem does some bytes => unicode normalization which might > account for some difference but other than that I'm at a loss for what > would be taking the time. Why are you surprised? Python is a very, very slow language. My very simple unoptimized C++ parsing code is more than 10x faster than Stem (parsing from memory). Go has some nice string parsing features, so zoossh should have an easy time to get fast and simple parsing code.

2 1

Get Stem and zoossh to talk to each other
by Philipp Winter 17 Aug '15

17 Aug '15

Hi Damian, I'm interested in building a lightweight, internal domain-specific language to explore archived Tor data. The goal is to make it easy to answer questions like the one that recently came up on tor-relays, "how many guards shift location significantly across the Internet, and how often?" Combining Stem and zoossh seems like a good solution. Ideally, zoossh should do the heavy lifting as it's implemented in a compiled language. For data exploration, however, having a Stem-enabled Python shell with a set of analysis methods sounds better. Now the question is how to pass potentially large amounts of readily-parsed consensuses and descriptors from zoossh to Stem? In a perfect world, we would have bindings to use zoossh in Python. The gopy [0] folks are working on that, but it's a young project; interfaces are not yet supported. Two workarounds come to mind until gopy catches up, both requiring some glue code: 1. Let zoossh do the data filtering and then return a list of files that are then parsed again by Stem. That's easy to implement, but can be quite inefficient if the filtering step still returns plenty of data. 2. Have some IPC mechanism that passes objects from zoossh to Stem. Objects could be serialised in some way to minimise unnecessary parsing. While that might be the most efficient option for now, it probably requires too much work. 3. ...something else I didn't consider? Please let me know if you have any thoughts. [0] <https://github.com/go-python/gopy> Cheers, Philipp

3 10

Alpha/Beta/Release cycle
by tordev123＠Safe-mail.net 14 Aug '15

14 Aug '15

TBB 5.0 Bug 16771 which seems to be causing a lot of crashes was apparently introduced with commit 77dcb2dfde4bdc1a2f6c399a1173f65908dccd5c on 2015-07-02 (new request isolation feature). The first release including that commit was apparently TBB 5.0a4 announced 2015-08-03. So there was only 1 week before the stable TBB 5 release and the commit got very little testing. IMHO, it would make sense to make more alpha releases and/or increase the time before the stable release.

1 0

Future Onion Addresses and Human Factors
by Alec Muffett 13 Aug '15

13 Aug '15

Hi All, Having Beer with Donncha, Yan and others in Berlin a few days ago, discussion moved to Onion-Address Human Factors. Summary points: 1) it’s all very well to go an mine something like “facebookcorewwwi” as an onion address, but 16 characters probably already exceeds human ability for easy string comparison. 2) example of the above: there are already “in the field” a bunch of onion addresses “passing themselves off” as other onion addresses by means of common prefixes. 3) next generation onion addresses will only make this worse 4) from Proposal 244, the next generation addresses will probably be about this long: a1uik0w1gmfq3i5ievxdm9ceu27e88g6o7pe0rffdw9jmntwkdsd.onion 5) taking a cue from World War Two cryptography, breaking this into banks of five characters which provide the eyeball a point upon which to rest, might help: a1uik-0w1gm-fq3i5-ievxd-m9ceu-27e88-g6o7p-e0rff-dw9jm-ntwkd-sd.onion 6) using underscores would be a pain (tendency to have to use SHIFT to type) 7) using dots would pollute subdomains, and colons would cause parser confusion with port numbers in URIs 8) being inconsistent (meaning: “we extract the second label and expunge anything which is not a base32 character”, ie: that with-hyphens and without-hyphens) may help or hinder, we’re not really sure; it would permit mining addresses like: agdjd-recognisable-word-kjhsdhkjdshhlsdblahblah.onion # illustration purposes only …which *looks* great, but might encourage people to skimp on comparing [large chunks of] the whole thing and thereby enable point (2) style passing-off. 9) appending a credit-card-like “you typed this properly” extra few characters checksum over the length might be helpful (10..15 bits?) - ideally this might help round-up the count of characters to a full field, eg: XXX in this? a1uik-0w1gm-fq3i5-ievxd-m9ceu-27e88-g6o7p-e0rff-dw9jm-ntwkd-sdXXX.onion 10) it might be good to discuss this now, rather than later? Hence this email, in the hope of kicking off a discussion between people who care about human factors. :-) - alec — Alec Muffett Security Infrastructure Facebook Engineering London

11 27

tor's definition of 'median'
by nusenu 13 Aug '15

13 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2028 > If 3 or more authorities provide a Measured= keyword for a router, > the authorities produce a consensus containing a "w" Bandwidth= > keyword equal to the median of the Measured= votes. a random sample from recent votes: grep 37.59.38.117 -A 3 *|grep Measured w Bandwidth=6869 Measured=7570 w Bandwidth=6869 Measured=15500 w Bandwidth=6869 Measured=18100 w Bandwidth=6869 Measured=30500 Tor says the median value is 15500 2015-08-10-16-00-00-consensus: w Bandwidth=15500 but the median of these 4 values is actually: (18100+15500)/2 = 16800 no? Has tor a different definition of 'median' and simply takes always the second ordered measurement vote out of 4 votes or is there a bug in the spec or implementation? -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVyNtYAAoJEFv7XvVCELh0YU8P/0hWhCLfafvFDPdAfUwUJFPA A1ZA946JGKg1Vjy+ch3tffjDHRosXt7K5U33N9rKMUlW9ul2BQ+uNgzK4eTbghHz yCMn6D+uLk1xruYTsIUZ+Pk/ZywaUKj/FngohVvQnaJIgJCHCEnCIqqBNEK0PjUh EBzI5GdyWpEA2fh55PSRuSNCVzbiVhGwYSKgrVwFrFcKr9iPLmdZsa9SD1mb1PD1 IZOkU6TnSnFGbPaN+pHYdNr5/QJA1/08yYBpG7qAJaTCAMvMif7bKMJ7ElYo7opc oXcECIcBBPnATgKvbO47dbSnX3s+vPMsrRhhdTa1BMr1MIzVG3RWGhNSJwXXQTJh jyaPGj10JRWNxDsY0ro001MX0HymmXeLLCkY4nWsUqPXDiZcQe4oRzLQas5bOI94 ct4tCavj9pRNp2XYCWe631gqcsQ3xV7y37dWUCvpdXqt2NC1B7j2w/Y/UiuNArSj QBtS4Ap5wqnU5JySjndi+lIPOlaPk9uitmzpKLxNF9fnpI6ZECP3T50vHVeZiiQ9 7o1ZyBsPLk3bi2hRy8ZJ0wyO+5WF8bdrlsKXSR40UjrwQZ5kCQf6xmWiSg9PqZFU rIL14yCOsQkR3P4/IgMlL8dtgFk3Asmkkx039144fRKveEhffFjo54CUMhg/jLra EF5cUqz4QdgcpRvKa/5h =lA/6 -----END PGP SIGNATURE-----

8 12

Re: [tor-dev] [Twisted-Python] please can haz code review -> Tor-ified Twisted Python HTTP proxy server - txtorhttpproxy
by David Stainton 13 Aug '15

13 Aug '15

Dear Aaon Gallagher, Thanks for the helpful observation. i'm going to make the correction soon and then we'll be underscore free. On Wed, Aug 12, 2015 at 7:06 AM, Aaron Gallagher <_(a)habnab.it> wrote: > David Stainton <dstainton415(a)gmail.com> writes: > >> If I get rid of this last terrible import: >> from twisted.web._newclient import ResponseDone >> >> then I'm not sure how to detect when to call the finish() method here: >> https://github.com/david415/txtorhttpproxy/blob/master/txtorhttpproxy/proxy… > > Fortunately, ResponseDone is re-exported by twisted.web.client: > > >>> import twisted.web.client > >>> twisted.web.client.ResponseDone is twisted.web._newclient.ResponseDone > True > >>> twisted.__version__ > '11.0.0' > > I couldn't get 9.0 to compile in the five seconds I tried, but 11.0 is > still pretty old. I think it always was re-exported like this.

1 0

[important] sponsors deliverables discussion point for Wed's meeting
by Isabela 12 Aug '15

12 Aug '15

Hello Core Tor folks! Note: if you are not a contractor or full time you can ignore this email, it is about paid deliverables. Me and Nick have been working on documenting the current status of our deliverables for the year to Sponsor S [project year ends on October] and Sponsor U [project year ends on November]. The result gave us a clear picture of where we need your help with. Please take a look at: https://docs.google.com/spreadsheets/d/1dTva10mu-FcX8KrxRjgkFvHSyNy7aBpD9xe… For Wednesday we want to figure out how can people jump in and help on these tasks. Right now we have a lot assigned to Nick and is a big red flag that we won't make it. Not to mention that Nick also has to carry on other work and to do both is just impossible. Please take a look, ask questions and lets figure out how we can conclude all this tasks. Priorities are: * Rows 5 & 6 * And any help on the other tasks that are all about writing docs (rows 12, 13, 15, 16, 17) * Or maybe help out with the final push for Sponsor S tasks We will be adding the related tickets (column H) later today so you can take a look before the meeting on Wednesday. thanks! isabela -- PM at TorProject.org gpg fingerprint = 8F2A F9B6 D4A1 4D03 FDF1 B298 3224 4994 1506 4C7B @isa

1 1