- tor-dev - lists.torproject.org

Onionoo as_number / as_name data
by Gareth Llewellyn 19 Sep '15

19 Sep '15

Hello, How often does Onionoo update its ASN information? Relays 518FF8708698E1DA09C823C36D35DF89A2CAD956, F92B3CB9BBE0CB22409843FB1AE4DBCD5EFAC835 & 707C1B61AC72227B34487B56D04BAA3BA1179CE8 are relatively new and whilst their IPs recently received updated GeoIP information (from Unassigned to UK) the ASN (AS28715) and ISP name aren't showing up. Gareth

2 1

Summary of meek's costs, August 2015
by David Fifield 18 Sep '15

18 Sep '15

Here's the summary of meek's CDN fees for August 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 -- total by CDN $6154.49 + $7165.85 + $154.89 = $13475.23 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… This month there was a big rebound as a temporary outage (not related to censorship) with meek-azure was resolved. The user count is now almost as high as it has ever been, at about 5000 simultaneous users. This month is the first time we have actual usage numbers from Azure. They told us what the cost would have been, but did not actually charge us. This month (September) is the last month on the free grant we have for Azure. After that, we will probably have to throttle Azure just like the other backends. If you want to help make domain fronting more efficient and future-proof, a good place to direct your attention is to this ticket, which aims to start using domain fronting as a cheap, low-volume interface to BridgeDB, instead of as a bulk data transfer channel: https://bugs.torproject.org/16650 If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2920 GB $350.38 6128 instance hours $306.38 Compared to the previous month: 2934 GB $352.11 7598 instance hours $379.90 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 150M requests $180.30 1191 GB $157.13 Asia Pacific (Sydney) 1M requests $2.26 7 GB $0.98 Asia Pacific (Tokyo) 28M requests $34.17 177 GB $22.80 EU (Ireland) 174M requests $209.36 905 GB $70.40 South America (Sao Paulo) 7M requests $16.87 49 GB $11.68 US East (Northern Virginia) 75M requests $75.07 490 GB $38.60 -- total 437M requests $518.03 2821 GB $301.59 https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… == Azure a.k.a. meek-azure == This is the first month where we have actual (not estimated) usage numbers, as a result of the new kind of account we're using. Unfortunately (also as a result of the account transition), meek-azure was broken until August 14, so these number really only represent half a month. Zone 1 1062 GB $92.40 Zone 2 453 GB $62.49 -- total 1515 GB $154.89 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html https://lists.torproject.org/pipermail/tor-dev/2015-August/009213.html

1 0

Re: [tor-dev] Proposal: Single onion services
by tordev123＠Safe-mail.net 18 Sep '15

18 Sep '15

Doesn't your proposal imply that you are turning all relays into exit-nodes lite? The last relay in the path will know what service you are connecting to (at least if that service is hosted with a unique relay), right? Have you considered all the implications?

7 7

Re: [tor-dev] Towards a new version of the PT spec...
by Adam Pritchard 18 Sep '15

18 Sep '15

At Psiphon we often discuss (and get asked about) using Tor's pluggable transports directly. The cost/benefit balance hasn't yet been in favour of doing this, but if there's discussion of a big PT revamp... maybe Psiphon should indicate how the cost side of the balance could come down for us. We're obviously not a priority for what Tor does with PTs, but there's surely no harm in us mentioning our wishlist. So... What would be best for us is if PTs were written in Go and implemented the net.Conn[1] interface. We have had good results with the composability of net.Conn implementations: an obfuscated SSH net.Conn on top of a meek net.Conn[2] on top of a upstream proxy net.Conn[3] on top of a TCPConn net.Conn[4]. Layering in a PT net.Conn would be sane and clean and reasonably easy. Conversely: Python is difficult-to-impossible due to runtime distribution. Separate executables are difficult-to-impossible due to Android PIE requirements and distribution size bloat. Anyway, if this is of any interest we can discuss it further. (Note: Probably Lantern people are reading this too. And probably they would benefit from the same things we would, since their architecture is similar to ours.) [1]: https://golang.org/pkg/net/#Conn [2]: https://github.com/Psiphon-Labs/psiphon-tunnel-core/blob/master/psiphon/mee… [3]: https://github.com/Psiphon-Labs/psiphon-tunnel-core/tree/master/psiphon/ups… [4]: https://golang.org/pkg/net/#TCPConn Adam Pritchard Psiphon Inc. > Date: Mon, 7 Sep 2015 22:45:52 +0000 > From: Yawning Angel <yawning(a)schwanenlied.me> > To: tor-dev(a)lists.torproject.org > Subject: [tor-dev] Towards a new version of the PT spec... > Message-ID: <20150907224552.6959bcbc(a)schwanenlied.me> > Content-Type: text/plain; charset="us-ascii" > > So, we currently have a Pluggable Transport (PT) spec, and it kind-of > sort-of works (The documentation is a mess that I'm working on > cleaning up, but it's an orthogonal issue for how well it works). > > There are a number of problems with the current PT spec that require > breaking backward compatibility to fix, so eventually I would like to > do so. > > I'm soliciting input on what people would also like to see in a > (currently hypothetical) PT spec 2.0 beyond what I already have in mind: > > MUST haves: > * Support dual stack Bridges correctly (Multiple server endpoints per > transport) > * Increase the argument space beyond 510 bytes (Prop. #227). > * Mandatory ExtORPort support (currently optional, but metrics are > good). > * Centralized logging by the calling process (Probably via stderr). > * AF_UNIX support where sensible for better sandboxing. > > MIGHT haves: > * Rename the env vars to not start with "TOR_PT". Some people claim > that this is a good idea (I think it is stupid and cosmetic). > * Ability to force at least clients to stop network activity without > tearing the PT down. > * Deprecate SOCKS4a, and make SOCKS5 mandatory for clients. > > UNLIKELY: > * Specify an interface for where fork()/exec() isn't possible (iOS). > I don't think this is makes sense because it is probably too > platform/caller specific. > * Allow operating both as a client and a server simultaneously. I > don't see a problem with running 2 copies of something for this > use case. > > I probably missed some things. If people have strong opinions about > this, do reply, otherwise I *will* design something that I like, which > will not include what other people want. > > Regards, > > -- > Yawning Angel -- Adam Pritchard Psiphon Inc.

3 2

How long does it take for 25/50/75% of relays to update to a new Tor version?
by Karsten Loesing 18 Sep '15

18 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Sebastian and list, Sebastian again suggested a fun task for the latest measurement team 1-1-1 task exchange round [0] that I picked up. He asked the following question: When a new Tor version is released, how long does it take for 25, 50, 75% of all relays to update? How often do relays update in general? Is there an effect when for example Debian updates its packages? It would help me to get an idea what I need to answer these questions :-) "update" means you're running, then you go down, then within a few minutes you're back up again with a changed version Here's what I came up with: I suggest we use two data sets as input for this task. The first data set consists of archived consensuses, available at: https://collector.torproject.org/archive/relay-descriptors/consensuses/ We'll need "valid-after" and "server-versions" from the header and "v" and "w" lines from all contained entries. Example: valid-after 2015-09-18 08:00:00 server-versions 0.2.4.23,0.2.4.24,0.2.4.25,0.2.4.26,0.2.4.27,0.2.5.8-rc,0.2.5.9-rc,0.2.5.10,0.2.5.11,0.2.5.12,0.2.6.5-rc,0.2.6.6,0.2.6.7,0.2.6.8,0.2.6.9,0.2.6.10,0.2.7.1-alpha,0.2.7.2-alpha v Tor 0.2.7.2-alpha-dev w Bandwidth=450 v Tor 0.2.6.10 w Bandwidth=23 [...] We can use those lines to *count* the number of running relays with a given version string, and we can sum up the *fraction* of consensus weights of relays running a given version string. Both can be useful, though I suspect the latter to be more meaningful. We should aggregate these consensus parts into daily statistics to make them easier to handle. We can do that by keeping counters per (date, version) tuple and per (date) for a) number of relays and b) consensus weight, and then we divide the (date, version) numbers by the (date) numbers to obtain averages. We'd also determine whether a version is recommended, whether it's the latest recommended version in its series, and whether it's the latest recommended stable version. The result would be a .csv file like the following (example data!): date,version,recommended,series,stable,count,fraction 2015-09-18,0.2.6.10,TRUE,TRUE,TRUE,0.387,0.451 2015-09-18,0.2.6.9,TRUE,FALSE,FALSE,0.063,0.0102 2015-09-18,0.2.6.8,TRUE,FALSE,FALSE,0.021,0.001 2015-09-18,0.2.6.7,TRUE,FALSE,FALSE,0.016,0.009 Read this as: "On 2015-09-18, version 0.2.6.10, which was both recommended and the latest in its series, was run by 38.7% of relays by count or 45.1% by consensus weight." (again, example data!) You could then draw graphs similar to https://metrics.torproject.org/versions.html, but with a percent scale on y. Though I'm not sure how readable those graphs would be with a few dozen lines in them. Still, they might be useful to explore the data set. The second data set that would be useful here is events related to versions. The following events come to mind: - tagged as version in Git (use a command similar to [1]), - tagged as alpha, beta, rc, or stable (look at version string), - first/last recommended by directory authorities for relays (parse from consensuses), - last recommended as newest version in a series or newest stable version (also parse from consensuses), or - first released in Debian stable (unclear how to obtain those dates). A possible data format would be: date,version,event 2015-07-12,0.2.6.10,git_tag 2015-06-10,0.2.6.9,git_tag 2015-05-19,0.2.6.8,git_tag 2015-04-06,0.2.6.7,git_tag For exploratory purposes, you could add these as colored vertical lines to the graphs above. Of course, adding more elements to a probably already overloaded graph doesn't exactly make it easier to understand. But if you limit the x axis on just a month or two, it might be useful. So, regarding your original question: "How long does it take for 25/50/75% of relays to update to a new Tor version?" I'm not sure whether that's really the best question to ask/answer here. Some versions will never be deployed on 75%, 50%, or even 25% of relays, because a new version was released shortly after. But that doesn't mean that the previous version was bad. This question also weights all versions the same, though in reality some releases are more important than others. If you get one data point for each version and then aggregate them by taking the average, what exactly does that tell you? I think that question could be rephrased to: "How long does it take for 25/50/75% of relays to update to a new Tor version *series*?" And I think that's something you could answer with the first data set above. But there are other interesting questions you could answer with the two data sets. For example, - What fraction of relays (by number or consensus weight) is running (un-)recommended versions? - What fraction of relays (by number of consensus weight) is (not) running the latest version in a series or latest stable version? Those could again be answered by a graph that uses the first data set, dates on x and fractions on y, and possibly events from the second data set as vertical lines. Explore, explore. There, the hour is over ("It took an hour to write, I thought it would take an hour to read.", Fry, Futurama). Hope this is useful in any way. And maybe others have good/better ideas for you as well? If you come up with some interesting answers, please post them here. All the best, Karsten [0] 1-1-1 task exchange: you get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do for you within 1 week (review a document, write some analysis code, fix a small bug, etc.; better come prepared to get the most out of this; give 1, take 1) [1] `git log --tags --simplify-by-decoration --pretty="format:%ci %d" | grep "tag: tor" | less` -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV++DHAAoJEJD5dJfVqbCr+sYH/1nViWWJgie+BD6Mo+9pC+g9 qEfMQ+NBvYWr3/SFcn/zWjydgZf+YPVXcVsOA1B1YcLX2s8OY1cWM5t5cDCstvfh /vOYmsDw1nddBh8VQxggYPkTLfEWRRSB0IeMiFWWUdVVKkcepOxgCfXMWJlBEAgG 8EtarcTkxSl1YMrurH3iuL8PrHGRiGf+mEKOZOMFtpmkSii/bWpO7BOAEyOXC7Ib EX2OouhCvae5X3Lyp2CnlQZLSrRGtAFV8sQUw5gcQb5/nouk38M7/ayrkBW/H3Ng qM9k0Y9EHAs1I1rn1ZY5lfGMKZ0WR+1NNCQ3mdsBSF5Qah01i1By7uk2FiH0e4U= =H6/Z -----END PGP SIGNATURE-----

1 0

List Administrivia [...encrypted onion services...]
by grarpamp 17 Sep '15

17 Sep '15

fabio, sky, et al... Please stop top quoting massive amounts of useless text that everyone's already seen before. It's seriously annoying to peoples workflow trying to figure out what youre replying to and if youve inlined other stuff below. And it needlessly bloats peoples mailboxes and thus search results. Trim the original to relavent bits and write your reply below that in context. Thanks, users everywhere.

1 0

Proposal: End-to-end encrypted onion services for non-Tor clients
by Donncha O'Cearbhaill 17 Sep '15

17 Sep '15

Hello all, I have been thinking about ideas to make Tor hidden services more available and secure for non-Tor users. Inline I've included a draft proposal which describes an end-to-end encrypted Tor2Web-like system. I'm really interested in hearing any suggestions, comments or criticism about this proposal. In particular I'd like to know if the trust requirements for the entry proxies and resolvers seem reasonable? Does this proposal make sense and is it something worth implementing? Thank you to David Stainton and Leif Ryge for your feedback on this proposal. Kind Regards, Donncha --- Filename: xxx-encrypted-onion-services-for-non-tor-clients.txt Title: End-to-end encrypted onion services for non-Tor clients Author: Donncha Ó Cearbhaill Created: 22-Aug-2015 Status: Draft 1. Overview: This proposal describes a system to allow non-Tor users to securely access location-hidden services via a domain name chosen by the hidden service operator. 2. Motivation: Tor hidden services have typically been used to provide anonymity to both services and clients. There are however use cases where a publisher or content provider requires anonymity but their users do not. The publishers typically would like their content to be available securely and to the widest possible audience. Tor2Web currently allows non-Tor users to access hidden services but it has a number of limitations. Tor2Web deployments are limited due to requirement to entrust Tor2Web service providers with users location information and the content of their requests. The onion subdomain based addressing in Tor2Web poses a usability issue and can be unwieldy for users without providing any self-authenticating properties. Tor2Web can be configured with a custom domain for an individual hidden service but this requires manual configuration and additional infrastructure. Ideally non-Tor users should be able to access location-hidden services under a standard domain name with end-to-end encryption from the client's browser to the hidden service web server. The system should be decentralized and it should not require the hidden service operator to run any clearnet systems. 3. Proposal: At a high level this proposal specifies a system of TCP entry proxies which transparently proxy the TLS connections of non-Tor clients to a hidden service. A client who requests a domain name such as https://myblog.com should get connected to a hidden service endpoint which has a valid SSL certificate for myblog.com. 3.1. Hidden Service configuration A hidden service operator obtains a public domain name and a corresponding CA signed SSL certificate. The operator configures the hidden service web server with the SSL certificate for the external domain. The domain name is pointed at an authoritative nameserver provider. The operator should also set a TXT DNS record containing the onion address of their hidden service. 3.2. Entry Proxy Modern web browsers send a Server Name Indication (SNI) field in the initial ClientHello of the TLS handshake. The entry proxy reads this field to determine the clients destination domain. The proxy performs a DNS lookup at that domain for a TXT record which specifies the corresponding onion address for the provided domain. The TLS entry proxy connects to the onion service via a Tor2Web-type circuit and then begins transparently proxying TLS traffic between the clients browser and the destination onion service. 3.3. Resolving Nameserver The key component of this system is a DNS name server which can direct clients to an online entry proxy via a round-robin type system. In a basic solution the onion service operator could manually specify one or more entry proxies in the A records for their domain on their existing authoritative DNS provider. In practice entry proxy churn would result in changing set of entry proxies. The nameserver will need regularly update its set of online entry proxies an remove proxies which are malfunctioning, malicious or otherwise unusable. The resolver may run a scanner to check its known proxies or load a list from an external service. Additionally the resolver should detect if an entry proxy blacklists a domain for which it is responsible and avoid routing clients to that entry proxy. It is expected that independent service providers will run their own domain->onion resolving nameserver in diverse jurisdictions as free or paid services. 4. Implementation: The entry proxy component could be implemented as part of the current Tor relay code base. Integration directly within Tor would allow use of the existing network consensus and bandwidth measurement systems to be used to discover available entry proxies. It would also allow for malicious entry proxies to be blacklisted. Alternatively the entry proxy could be implemented in the existing Tor2Web software or as a standalone software package. Implementing outside of Tor would be faster and it would avoid the risk of losing Tor relay capacity as a result of legal threats to the entry proxies. The resolving nameserver is the most complicated component of this system. The component will eventually require a DNS server, a management interface, and a set of network monitoring tools. 5. Security and resiliency implications: 5.1. Availability Attacks: Adversaries can attack the availability of a publicly-proxied hidden service at a number of levels: * Censorship or shutdown of entry proxy: Attacks on individual entry proxies are mitigated by performing DNS based round-robin between many online entry proxies. The Resolver system should be able to quickly remove entry proxies which misbehave or which go offline. * Censorship or seizure of the hidden service public domain: DNS based blocks are widely deployed for censorship and may be difficult to avoid. Domain registrars can also be forced to suspend domain names. Service operators should considering running their service under a TLD which is less vulnerable to these type of coercive threats. * Takedown of a nameserver provider: Multiple resolving nameservers can be configure for each forwarded domain. Using nameservers maintained by different providers can provide resilience to attacks against a single nameserver provider. 5.2. Security Attacks: Entry proxies and exit relays have a similar ability to monitor and interfere with client traffic. This is greater risk of targeted interference from entry proxies as they can also determine the client's network location. * Man-in-the-middle HTTP connections: Entry proxies have the ability to man-in-the-middle HTTP connections. Service operators should send HSTS header to force clients to automatically use TLS for all future connections. * TLS man-in-the-middle with CA-signed certificate Some commercial CA cert providers allow for domain ownership to be validated by providing a file over HTTP at the domain. A malicious entry proxy could successfully obtain a CA-signed certificate from one of these certificate authorities. Service operators can minimize their exposure to this type of attack by using HPKP headers to limit the set of valid certificate authorities for their domain.

5 5

multiple relay identities on a single IP:port, bug or feature?
by nusenu 16 Sep '15

16 Sep '15

Hi, in the last two days someone started generating a steady amount of new relay fingerprints on a single IP:port (2 per hour, actually a lot more than that but only to make it into the consensus) [1]. I'm surprised that actually both of them end up in the consensus. example: https://collector.torproject.org/recent/relay-descriptors/consensuses/2015-… search for "67.173.119.40 51256" - you will find it twice. Does that make sense or is this a bug? btw: Why do dir auths include non-running relays in their votes (flag line is "s" only), instead of not voting for them at all? thanks! [1] http://article.gmane.org/gmane.network.onion-routing.ornetradar/113

2 1

Draft Proposal: Random Number Generation During Tor Voting
by George Kadianakis 16 Sep '15

16 Sep '15

Hello there, we are glad to release a first draft of our proposal on distributed random generation using the Tor voting process. It specifies how Tor dirauths can generate a fresh random value every day using a commit-and-reveal protocol. The protocol piggybacks on top of the regular Tor voting procedure which happens every hour. Our proposal spends lots of effort resolving the various edge cases and engineering issues that come up when you are trying to fit such a protocol into the Tor voting system. It also introduces a new consensus flavor document which is used to host shared randomness information, so that the regular consensus does not get affected if this feature is flawed. We are especially interested in feedback from people who are familiar with the Tor voting protocol and can tell us whether we have messed something up, or whether there are attacks that we did not consider. We hope the document (and the illustration) is helpful for understanding the protocol. Let us know if you have any questions, or suggestions for improving the proposal. We believe this proposal is fundamental for the security of hidden services and for developing proposal 224, hence we invite everyone to provide feedback and improvements. Thanks! ----------------------------------------------------------------------------------- Filename: xxx-commit-reveal-consensus.txt Title: Random Number Generation During Tor Voting Authors: David Goulet, George Kadianakis Created: 2015-08-03 Status: Draft 1. Introduction 1.1. Motivation For the next generation hidden services project, we need the Tor network to produce a fresh random value every day in such a way that it cannot be predicted in advance or influenced by an attacker. Currently we need this random value to make the HSDir hash ring unpredictable (#8244), which should resolve a wide class of hidden service DoS attacks and should make it harder for people to gauge the popularity and activity of target hidden services. Furthermore, this random value can be used by other Tor-related protocols (or even non-Tor-related) like OnioNS to introduce unpredictability to the protocol. 1.2. Previous work Proposal 225 specifies a commit-and-reveal protocol that can be run as an external script and have the results be fed to the directory authorities. However, directory authority operators feel unsafe running a third-party script that opens TCP ports and accepts connections from the Internet. Hence, this proposal aims to embed the commit-and-reveal idea in the Tor voting process which should makes it smoother to deploy and maintain. Another idea proposed specifically for Tor is Nick Hopper's "A threshold signature-based proposal for a shared RNG" which was never turned into an actual Tor proposal. 2. Overview This proposal alters the Tor consensus protocol such that a random number is generated by the directory authorities during the regular voting process. The distributed random generator scheme is based on a commit-and-reveal technique. The proposal also specifies how the final shared random value is embedded in consensus documents so that clients who need it can get it. 2.1. Ten thousand feet view Our commit-and-reveal protocol aims to produce a fresh shared random value everyday at 12:00UTC. The final fresh random value is embedded in the microdescriptor consensus document at that time. Our protocol uses a *new* consensus flavor document called "shared randomness document" (SR doc). We use a new consensus document as a way to keep ground truth state and also as a way to apply the majority (see section [MAJORITY]) rule on commit and reveal values. It also allows rebooting authorities to rejoin the protocol in some cases. Our protocol has two phases and uses the hourly voting procedure of Tor. Each phase lasts 12 hours, which means that 12 voting rounds happen in between. In short, the protocol works as follows: Commit phase: Starting at 12:00UTC and for a period of 12 hours, authorities every hour send their commitments in their votes. They also include any received commitments from other authorities, if available. From those votes, a shared random document consensus is computed containing the commitments decided by the majority. Reveal phase: At 00:00UTC, the reveal phase starts and lasts till the end of the protocol at 12:00UTC. In this stage, authorities must reveal the value they committed to in the previous phase. The commitment and revealed values from other authorities, when available, are also added to the vote. Then a shared random document consensus is computed containing the commitments and the revealed values agreed on. Shared Randomness Calculation: At 12:00UTC, the shared random value is computed from the agreed revealed values located in the shared random document and finally added to the microdescriptor consensus. This concludes the commit-and-reveal procedure at 12:00UTC everyday. 2.2. Commit & Reveal Our commit-and-reveal protocol aims to produce a fresh shared random value everyday at 12:00UTC. In the beginning of that time period, each authority generates a new random value and keeps it for the whole day. The authority cryptographically hashes the random value and calls the output its "commitment" value. The original random value is called the "reveal value". Given a reveal value you can verify that it corresponds to a given commitment value. However given a commitment value you cannot derive the underlying reveal value. 2.3. Microdescriptor Consensus [MDCONS] Every hour, the microdescriptor consensus documents need to include the shared random value of the day, as well as the shared random value of the previous day. That's because either of these values might be needed at a given time for a Tor client to access a hidden service according to section [TIME-OVERLAP] of proposal 224. These means that these two values also need to be included in votes and in SR documents as well. Microdescriptor consensuses include: (a) The shared random value of the current time period. This is derived from the reveal values sent by the authorities during the voting session. (b) The shared random value of the previous time period. This is the same shared random value that was included in the votes. 2.4. Shared Random Document [SRDOC] The Shared Random document is a consensus flavor that contains the current state of our commit & reveal protocol. Since it uses the consensus mechanism of Tor, we use it as a way to enforce majority voting on the commitments and reveals without messing with the actual network status consensus. See section [REBOOT] for detail on how this document is handled when an authority reboots. During the commitment phase, the SR doc is populated with the commitments of all authorities. Then during the reveal phase, it's also used to store the reveal values as well. As discussed previously, the shared random values from the current and previous time period must be present in the document at all times if they are available. A shared random document requires 50% + 1 authority signatures to be considered valid. As this proposal is being written, there are 9 authorities thus we would need 5. 2.5. Protocol Illustration We have prepared an illustration to help you understand the protocol. You can find it here: https://people.torproject.org/~asn/hs_notes/shared_rand.jpg For every hour, it shows the authority votes and the resulting SR doc and microdescriptor consensus. The chain 'A_1 -> c_1 -> r_1' denotes that the authority committed to the value c_1 which corresponds to the reveal value r_1. The illustration depicts the first 25 hours of running the protocol. It starts with the very first commit round, then moves on to the second commit round, and then skips directly to the last commit round. Then the reveal phase starts, where we again show the first, second and last rounds. After the reveal phase is done, we generate the shared randomness (SR_1) and we start the new commit phase. The illustration finishes with the second round of this new commit phase. We advice you to revisit this after you have read the whole document. 3. Protocol In this section we give a detailed specification of the protocol. We describe the protocol participants' logic and the messages they send. The encoding of the messages is specified in the next section ([SPEC]). Now we go through the phases of the protocol: 3.1 Commitment Phase [COMMITMENTPHASE] The commit phase lasts from 12:00UTC to 00:00UTC. The goal is that at the end of this phase, the shared random document MUST contain a single commitment value from each authority (or none, if that authority did not participate in this phase). 3.1.1. Voting During Commitment Phase During the commit phase, each authority includes in its votes: - A commitment value for this consensus period. - Any commitments received from other authorities. - The two previous shared random values produced by the protocol (if any). After all votes have been received or pulled in, the authorities collectively generate the shared random document containing the commitments. 3.1.2. Shared Random Document During Commitment Phase [SRDOCCOMMIT] During the commitment phase, the shared random document contains: - The commitments received by the majority of authorities - The two previous shared random values produced by the protocol (if any). A commitment should only be transcribed to the shared random document if and only if the majority of the voting authorities agreed that a particular commitment was sent by a particular authority. Appendix section [COMMITEXAMPLE] contains an example of this procedure. The commit phase lasts for 12 hours, so authorities have multiple chances to commit their values. An authority can commit a second value during a subsequent round of the commit phase, but only the last value should be transcribed to the shared random document and only if it has been seen by the majority. Also, an authority should not be able to register a commitment value for a different authority. Hence, an authority X should only vote and place in the SR doc commitments by authority Y, iff authority Y included that commitment in its vote. 3.1.3. First & Last Round Of Commitment Phase [FIRSTLASTROUND] It's worth mentioning that during the very first round of the commitment phase at 12:00UTC, each authority votes its own commitment and is unaware of the commitments of the other authorities. For this reason, it's unlikely that a majority opinion of commitments will be created at 12:00UTC. Instead authorities are expected to form a majority opinion and transcribe commitments to the SR doc during the voting period of 13:00UTC or at least until the reveal phase. Similarly, an authority will not be able to commit to a new value during the last round of the commitment phase. That's because there won't be enough time for the other authorities to form a majority opinion about this value before the reveal phase. Hence, Tor authorities SHOULD NOT commit new values during the last round of the commitment phase at 23:00UTC. 3.2 Reveal Phase The reveal phase lasts from 00:00UTC to 12:00UTC. Now that the commitments have been agreed on, it's time for authorities to reveal their random values. 3.2.1. Voting During Reveal Phase During the reveal phase, each authority includes in its votes: - Its reveal value that was previously committed in the commit phase. - All the commitments and reveals received from other authorities. - The two previous shared random values produced by the protocol (if any). The set of commitments have been established during the commitment phase and must remain the same. If an authority tries to change its commitment during the reveal phase or introduce a new commitment, the entire vote MUST be ignored for the purposes of this system. To do so, authorities during the first reveal round MUST check that received votes contain the same commitments as the last SR document of the commitment phase. In subsequent reveal rounds, authorities check the previous hour SR document for commitment validation. After all votes have been received, authorities generate the shared random document along with the consensus. 3.2.2. Shared Random Document During Reveal Phase [SRDOCREVEAL] During the reveal phase, the shared random document contains: - The commitments agreed on during the commitment phase. - The corresponding reveal values from the majority of authorities. - The two previous shared random values produced by this system (if any). Similar to the commitment phase, authorities transcribe reveal values to the shared random document if and only if the majority of the voting authorities have voted on that particular reveal value. An example of this can be seen in section [REVEALEXAMPLE]. Section [FIRSTLASTROUND] also applies for the reveal phase. This means that Tor authorities SHOULD NOT reveal new values during the last round of the reveal phase at 11:00UTC. 3.3. Shared Random Value Calculation At 12:00UTC Finally, at 12:00UTC every day, authorities compute a fresh shared random value and this value must be added to the microdescriptor consensus so clients can use it. Authorities calculate the shared random value using the reveal values in the latest shared random document as specified in subsection [SRCALC]. If the shared random value contains reveal contributions by less than 3 directory authorities, it MUST NOT be created. Instead, the old shared random value should be used as specified in section [SRDISASTER]. Authorities at 12:00UTC start including this new shared random value in their votes, replacing the one from two protocol runs ago. Authorities also start including this new shared random value in the SR document and in the microdescriptor consensus as well. Apart from that, authorities proceed voting normally as they would in the first round of the commitment phase (section [COMMITMENTPHASE]). 3.3.1. Shared Randomness Calculation [SRCALC] An authority that wants to derive the shared random value V, should use the appropriate reveal values for that time period and calculate V as follows: V = H(ID_a | R_a | ID_b | R_b | ...) where the ID_k value is the identity fingerprint of directory authority k and R_k is its corresponding reveal value of that authority for the current period. H is sha256 for protocol version 1. XXX Should the hashing here include more elements? Like the previous random value for chaining? Or the current date? See how the NIST beacon does it in case we can steal some additional RNG security properties: http://hackaday.com/2014/12/19/nist-randomness-beacon/ 3.4. Bootstrapping Procedure As described in [MDCONS], two shared random values are required for the HSDir overlay periods to work properly as specified in proposal 224. Hence clients MUST NOT use the randomness of this system till it has bootstrapped completely; that is, until two shared random values are included in a consensus. This should happen after three 12:00UTC consensuses have been produced, which takes 48 hours. 3.5. Rebooting Directory Authorities [REBOOT] The shared randomness protocol must be able to support directory authorities who leave or join in the middle of the protocol execution. An authority that commits in the Commitment Phase and then leaves SHOULD store its reveal value on disk so that it continues participating in the protocol if it returns before or during the Reveal Phase. The reveal value MUST be stored timestamped to avoid sending it on wrong protocol runs. For this reason, other authorities should carry the commitment values of absent authorities in the shared randomness document until the end of the protocol. The shared randomness document can be used to verify that the commitment values are carried properly. An authority that misses the Commitment Phase cannot commit anymore, so it's unable to participate in the protocol for that run. Same thing for an authority that misses the Reveal phase. Authorities who do not participate in the protocol SHOULD still carry commits and reveals of others in their vote. 3.6. How we define majority [MAJORITY] The shared randomness protocol must be able to support directory authorities who participate in the consensus protocol but not in the shared randomness protocol. It must also be able to tolerate authorities who drop or join in the middle of the protocol. The security of this proposal strongly relies on forming majority opinion so it's important for the number of participants to always be well defined: In the voting session right before creating the SR doc, we define the number of active participants to be the number of directory authorities that included commit/reveal values in their votes. As specified in sections [SRDOCCOMMIT] and [SRDOCREVEAL], a commit/reveal value should be transcribed to the SR doc if and only if the majority voted for it. So for example, if there are 6 active participants, a commit value will only be transcribed if 4 or more participants agreed on it. Furthermore, as specified in section [SRDOC], the shared random document is considered valid only if it is signed by 50% + 1 authorities. XXX The number of active participants is dynamic as authorities leave and join the protocol. Since the number of active participants is dynamic , an attacker could trick some authorities believing there are N participants and some others believing there are N-1 participants, by sending different votes to different auths. Should we worry? [asn] A way to avoid a dynamic number of participants could be to set the number of participants to be the number of auths who committed during the very first commitment phase round. 3.7. Shared Randomness Disaster Recovery [SRDISASTER] If the consensus at 12:00UTC fails to be created, then there will be no new shared random value for the day. Directory authorities should keep including the previous shared random values in the consensus till the next 12:00UTC commit-and-reveal session. The time period needs to be updated to reflect the current time period even if the random value stays the same. Clients should keep on using this shared random values. 4. Specification [SPEC] 4.1 Voting This section describes how commitments, reveals and SR values are encoded in votes. We describe how to encode both the authority's own commits/reveals and also the commits/reveals received from the other authorities. Commits and reveals share the same line, but reveals are optional. 4.1.1 Encoding the authority's own commit/reveal value An authority that wants to commit (or reveal) a value during a vote, should generate a random 256-bit value REVEAL, and include its commitment COMMIT in its 12:00UTC vote as follows: "shared-rand-commitment" SP algname SP COMMIT [SP REVEAL] NL During the Reveal Phase, an authority can also optionally reveal the value REVEAL. The "algname" is the hash algorithm that should be used to compute COMMIT and REVEAL if any. It should be "sha256" for this version. The commitment value COMMIT is constructed as follows: C = base64-encode( SHA256(REVEAL) ) 4.1.2 Encoding commit/reveal values received by other authorities [COMMITOTHER] An authority puts in its vote the commitments and reveals it has seen from the other authorities. To do so, it includes the following in its votes: "shared-rand-received-commitment" SP identity SP algname SP COMMIT [SP REVEAL] NL when "identity" is the hex-encoded commitment's authority fingerprint and COMMIT is the received commitment value. Authorities can also optionally include the reveal value REVEAL. There MUST be only one line per authority else the vote is considered invalid. Finally, the "algname" is the hash algorithm that should be used to compute COMMIT and REVEAL if any. 4.1.3. Shared Random value Authorities include a shared random value in their votes using the following encoding for the previous and current value respectively: "shared-rand-previous-value" SP value NL "shared-rand-current-value" SP value NL where "value" is the actual shared random value. It's computed as specified in the section [SRCALC]. To maintain consistent ordering, the shared random values of the previous period should be listed before the values of the current period. 4.2. Shared Random Document As a way to keep ground truth state in this protocol, we introduce a new consensus flavor document. We call it the "Shared Random Document". This document is only used by directory authorities. This new consensus flavor should be signed with the sha256 signature format as documented in proposal 162. 4.2.1 Format [SRFORMAT] This document has a very strict format because authorities need to generate the exact same document. It contains a preamble, a commitment and reveal section, a list of shared random values and finally a footer. The preamble (or header) contains the following items. They MUST occur in the order given here: "shared-random-version" SP version SP flavor NL [At start, exactly once.] A document format version. For this specification, version is "1". The flavor is always "shared-random". "created" SP YYYY-MM-DD SP HH:MM:SS NL [Exactly once] The creation time of this document. "valid-until" SP YYYY-MM-DD SP HH:MM:SS NL [Exactly once] After this time, this document is invalid and shouldn't be used nor trusted. The validity time period is 3 hours. "protocol-phase" SP phase NL [Exactly once] The current protocol phase when this document is generated. The accepted values are: "commitment" and "reveal". The following details the commitment and reveal section. "shared-rand-commitment" SP algname SP identity SP commitment-value [SP revealed-value] NL [Exactly once per authority] This is the commitment or/and reveal value agreed upon by the majority from one authority. The algname is always "sha256" in version 1. The "identity" is the authority hex-encoded digest of the authority identity key of the signing authority from which the values are from. Finally, "{commitment|revealed}-value" is the value as specified in section [SPEC]. Next is the shared random value section. "shared-rand-previous-value" SP value NL [At most once] This is the previous shared random value agreed on at the previous period. The "value" is defined in section [SRCALC]. "shared-rand-current-value" SP value NL [At most once] This is the latest shared random value. The "value" is defined in section [SRCALC]. Finally, the footer of the document: "shared-random-footer" NL [Exactly once] It contains one subsection, the authority signatures. "authority-signature" SP algname SP identity SP signing-key-digest NL Signature NL [Exactly once per authority] The "identity" is the hex-encoded digest of the authority identity key and the "signing-key-digest" is the hex-encoded digest of the current authority signing key of the signing authority. The "algname" item is the algorithm used to compute the hash of the document before signing it. As proposal 162 proposed, "sha256" should be used. The authority-signature entry MUST be ignored if "algname" is unrecognized. See dir-spec.txt for a specification for the Signature item. 4.3. Shared Random Value in Consensus [SRCONSENSUS] Authorities insert the two shared random values in the consensus following the same encoding format as in [SRFORMAT]. 5. Security Analysis 5.1. Security of commit-and-reveal and future directions The security of commit-and-reveal protocols is well understood, and has certain flaws. Basically, the protocol is insecure to the extent that an adversary who controls b of the authorities gets to choose among 2^b outcomes for the result of the protocol. However, an attacker who is not a dirauth should not be able to influence the outcome at all. We believe that this system offers sufficient security especially compared to the current situation. More secure solutions require much more advanced crypto and more complex protocols so this seems like an acceptable solution for now. 5.2. Is there a need for a final agreement phase? Commit-and-reveal protocols usually also end with an agreement phase, during which participants agree on which reveal values should be used to make the shared random value. An agreement phase is needed, because if the protocol ended with the reveal phase, an evil authority could wait until the last reveal round, and reveal its value to half of the authorities. That would partition the authorities into two sets: the ones who think that the shared random value should contain this new reveal, and the rest who don't know about it. This would result in a tie and two different SR docs. However, we believe that an agreement phase is not necessary in our protocol since reveal values are transcribed in the SR document if only if the majority agrees. Hence, a tie is not enough to confuse the authorities since it's not majority and the offending value would just be discarded. That said, an attack that could still work here would be if an authority can make half of the authorities believe that the value should be discarded, and make the other half of the authorities believe that the value should be included. That could be achieved if the attacker could force honest authorities to send different votes to different authorities. We believe this should not be the case currently, but we should look more into this. XXX Needs feedback by a person who knows the voting protocol well!!! 5.3. Predicting the shared random value during reveal phase The reveal phase lasts 12 hours, and most authorities will send their reveal value on the first round of the reveal phase. This means that an attacker can predict the final shared random value about 12 hours before it's generated. This does not pose a problem for the HSDir hash ring, since we impose an uptime restriction on HSDir nodes, so 12 hours predictability is not an issue. Any other protocols using the shared random value from this system should be aware of this property. 6. Discussion 6.1. Why the added complexity from proposal 225? The complexity difference between this proposal and prop225 is in part because prop225 doesn't specify how the shared random value gets to the clients. This proposal spends lots of effort specifying how the two shared random values can always be readily accessible to clients. 6.2. Why do you do a commit-and-reveal protocol in 24 rounds? The reader might be wondering why we span the protocol over the course of a whole day (24 hours), when only 3 rounds would be sufficient to generate a shared random value. We decided to do it this way, because we piggyback on the Tor voting protocol which also happens every hour. We could instead only do the shared randomness protocol from 21:00 to 00:00 every day. Or to do it multiple times a day. However, we decided that since the shared random value needs to be in every consensus anyway, carrying the commitments/reveals as well will not be a big problem. Also, this way we give more chances for a failing dirauth to recover and rejoin the protocol. 6.3. Why can't we recover if we fail to do a consensus at 12:00UTC? Section [SRDISASTER] specifies that if the 12:00UTC consensus or SR doc fails to be created, we fall back to the random value of the previous day meaning authorities will carry the last valid SR values from the previous microdescriptor consensus to the new one. Theoretically, we could recover by calculating the shared randomness of the day at 13:00UTC instead. However, adding such fallback logic would complicate the protocol even further, so we have not yet considered it. 7. Appendix 7.1. Example commitment majority [COMMITEXAMPLE] Here is an example of voting during the commitment phase. The table below represents the votes of 6 individual authorities A_i (one vote per column). Since it's the commitment phase, votes include the authorities commitments and all commitments received. For example, below all authorities believe that A_1 has registered the value 7 as its commitment. +------------+------------+-------------+-------------+-------------+-----------+ | A_1 vote | A_2 vote | A_3 vote | A_4 vote | A_5 vote | A_6 vote | +------------+------------+-------------+-------------+-------------+-----------+ | A_1 -> 7 | A_1 -> 7 | A_1 -> 7 | A_1 -> 7 | A_1 -> 7 | A_1 -> 7 | | A_2 -> 66 | A_2 -> 66 | A_2 -> 42 | A_2 -> 42 | A_2 -> 42 | A_2 -> 42 | | A_3 -> 16 | A_3 -> 16 | A_3 -> 16 | A_3 -> 16 | A_3 -> 16 | A_3 -> 16 | | A_4 -> 22 | A_4 -> 22 | A_4 -> 22 | BLANK | A_4 -> 22 | BLANK | | A_5 -> 9 | A_5 -> 9 | A_5 -> 9 | A_5 -> 9 | A_5 -> 9 | A_5 -> 9 | | A_6 -> 33 | A_6 -> 33 | A_6 -> 33 | A_6 -> 33 | A_6 -> 33 | BLANK | +------------+------------+-------------+-------------+-------------+-----------+ In this case, following the majority rule, the final SR doc will contain: +-------------+ | SR Document | +-------------+ | A_1 -> 7 | | A_2 -> 42 | | A_3 -> 16 | | A_4 -> 22 | | A_5 -> 9 | | A_6 -> 33 | +-------------+ 7.2. Example reveal phase [REVEALEXAMPLE] Here is an example of voting during the reveal phase. The table below represents 6 votes by 6 different authorities A_i (one vote per column). Since it's the reveal phase, votes include all reveals received (commitments have been hidden for simplicity). For example, below all authorities believe that A_1 has revealed the value 444. Let's say that a malicious dirauth is trying to partition the group into two sets, by sending different votes to different auths. The attacker has splitted the group into two sets, the auths who think that A_6 has revealed the value 123, and the rest who have not seen a reveal from A_6. +------------+------------+-------------+-------------+-------------+------------+ | A_1 vote | A_2 vote | A_3 vote | A_4 vote | A_5 vote | A_6 vote | +------------+------------+-------------+-------------+-------------+------------+ | A_1 -> 444 | A_1 -> 444 | A_1 -> 444 | A_1 -> 444 | A_1 -> 444 | A_1 -> 444 | | A_2 -> 110 | A_2 -> 110 | A_2 -> 110 | A_2 -> 110 | A_2 -> 110 | A_2 -> 110 | | A_3 -> 420 | A_3 -> 420 | A_3 -> 420 | A_3 -> 420 | A_3 -> 420 | A_3 -> 420 | | BLANK | BLANK | A_4 -> 980 | BLANK | A_4 -> 980 | BLANK | | A_5 -> 666 | A_5 -> 555 | A_5 -> 555 | A_5 -> 555 | A_5 -> 555 | A_5 -> 555 | | A_6 -> 123 | A_6 -> 123 | A_6 -> 123 | BLANK | BLANK | BLANK | +------------+------------+-------------+-------------+-------------+------------+ Following the rules of the reveal phase, the reveal of A_4 should be ignored since it was not voted by > 3 authorities. The reveal from A_6 should also be ignored since it was only seen by half of the auths (3/6) which is not majority (it would require at least 4/6 votes). Hence, the final shared random document should contain: +-------------+ | SR Document | +-------------+ | A_1 -> 444 | | A_2 -> 110 | | A_3 -> 420 | | BLANK | | A_5 -> 555 | | BLANK | +-------------+

10 26

Towards a new version of the PT spec...
by Yawning Angel 15 Sep '15

15 Sep '15

So, we currently have a Pluggable Transport (PT) spec, and it kind-of sort-of works (The documentation is a mess that I'm working on cleaning up, but it's an orthogonal issue for how well it works). There are a number of problems with the current PT spec that require breaking backward compatibility to fix, so eventually I would like to do so. I'm soliciting input on what people would also like to see in a (currently hypothetical) PT spec 2.0 beyond what I already have in mind: MUST haves: * Support dual stack Bridges correctly (Multiple server endpoints per transport) * Increase the argument space beyond 510 bytes (Prop. #227). * Mandatory ExtORPort support (currently optional, but metrics are good). * Centralized logging by the calling process (Probably via stderr). * AF_UNIX support where sensible for better sandboxing. MIGHT haves: * Rename the env vars to not start with "TOR_PT". Some people claim that this is a good idea (I think it is stupid and cosmetic). * Ability to force at least clients to stop network activity without tearing the PT down. * Deprecate SOCKS4a, and make SOCKS5 mandatory for clients. UNLIKELY: * Specify an interface for where fork()/exec() isn't possible (iOS). I don't think this is makes sense because it is probably too platform/caller specific. * Allow operating both as a client and a server simultaneously. I don't see a problem with running 2 copies of something for this use case. I probably missed some things. If people have strong opinions about this, do reply, otherwise I *will* design something that I like, which will not include what other people want. Regards, -- Yawning Angel

5 6

[Measurement Team] Next IRC meeting happens on THURSDAY, Sep 17, 14:00 UTC in #TOR-DEV
by Karsten Loesing 15 Sep '15

15 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on THURSDAY, September 17, 2015, 14:00 UTC in #TOR-DEV https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150917T14 Note that the meeting day changed from Wednesday to Thursday and the meeting room from #tor-project to #tor-dev. The goal of this change is to avoid the overlap with the little-t tor development meeting. Talk to you on Thursday! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV96DGAAoJEJD5dJfVqbCr2MEH/2GdYX+/S5s7e4Xhd9TU/GwU 8HtJCXRrF42pRM5wFoHeq50/FokAJpAN8RrYsARdj0JNPERZWninNmwrurUdawe1 nBirn4zaHA4dRJM6eVzYpoFjNv9FO0U8H3gM8mdhqIIYUmb+ghtqo8+M+0aoujSX vSh28j4Dcb/aHpHEYsEumqMsBU6Lj7GQdCu5jGm7ttlUynQ0nCXnQ1h6dhlF6GzC APQ6coBc9J733fEwM5Xcn7B3D+2sdAq7oLINaAZ0YnMMVdKehJ4+5qnfTlLsjGe0 Wz2OPm3VSj8vyTREKmPxywvS04QeAf3hI3kHIY0tqW5q8TtWh8o+0KzTChdGWY4= =yhJ6 -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] tor-roster's geo diversity badge and self-ref relays
by Sean Saito 14 Sep '15

14 Sep '15

>> Can this be downgraded to an informational message? (or eliminated entirely?) >> >> Penalties can be quite discouraging, particularly for minor configuration variants. >> >> Tim > >I agree, and this one in particular is important to some operators: by allowing a relay to specify itself in the family, one can just have a single >configuration file for all relays in a family. I was not aware of that. I shall take down the penalty then. Sean

1 0

Re: [tor-dev] Let's identify which measurement-related tools need work when relays switch from RSA identities to ed25519 identities
by Damian Johnson 13 Sep '15

13 Sep '15

>> Hi Karsten. Quick question: with the switchover are relay fingerprints >> going away? That is to say, server descriptors no longer have a... >> >> fingerprint D203 4DDF 1275 A234 4F66 9935 A3EF B908 FFC7 AE9A >> >> ... line, and router status entries don't have it on their dir-source? If not, what in particular are being dropped? >> >> Cheers! -Damian > > yes, I think that's the plan. Please see proposal 220, Sections 8 and > 9 for some more details. Thanks Karsten. Now that I know this isn't a completely foolish question adding tor-dev@ back in. If we're dropping the fingerprint that's a lot more invasive. Fingerprints are relay's canonical identity, and simply dropping them will break... a lot. I'd suggest instead relays should continue to have a fingerprint but that it's the 40 character hex hash of the ed25519 identity. On a side note it would be nice to have a spec patch before changing things this time. Sections 8 and 9 are a fine summary, but it's not clear to me precisely how the descriptors are changing from it. They certainly don't say to me "we're dropping the fingerprint field". If that change went out without a spec patch first I'd borrow Mr. Potato Head's angry eyes to stare at its author. Cheers! -Damian

2 2

tor-roster's geo diversity badge and self-ref relays
by nusenu 12 Sep '15

12 Sep '15

Hi, tor-roster has a badge for in-family geo diversity: "Geo Diversity in Relays (Number of countries / Number of relays >= 0.5)" Do you consider in-family diversity so important - even though all of them are run by a single entity anyway? I'd consider the tor network's overall diversity far more important than in-family diversity because clients won't use more than one relay of a given family anyway. How about having a badge for tor network wide diversity? Something like: More than 4/5 of the family's CW is located in countries with a cw lower than 2%* (currently means non-top 7 country) and ASes lower than 1.5%* (currently means non-top 8 AS)? That implies some degree of in-family diversity since a big family would have to spread across multiple countries/ASes. potential problem: "growing" ASes/countries might cross the threshold in that case you would either have to accept the fact that someone else can take away that badge by adding relays to your AS/CC ;) or consider the diversity at relay signup time (less fun) *) these are arbitrary thresholds "No Self-Referencing Relays" I'm not sure what exactly you mean by that but I assume it is a MyFamily config where a relay includes his own fingerprint. Why does that hurt? The unnecessary descriptor space/bw? thanks

1 0

CollecTor data of the current month older than 72 hours
by nusenu 12 Sep '15

12 Sep '15

Hi, data in the /recent folder goes back 72 hours, /archive is updated "every few days" [1]. The latest file in the archive folder [2] is currently from 2015-09-07: votes-2015-08.tar.xz 07-Sep-2015 04:11 159M but does not contain any data from September 2015. is there currently any data for the time period between 2015-09-01 - 2015-09-07 9:00 on Collector or should I wait a few days? thanks! [1] https://collector.torproject.org/ [2] https://collector.torproject.org/archive/relay-descriptors/votes/

2 2

Reproducibility of Pluggable Transports python.msi
by Jeremy Rand 11 Sep '15

11 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I was looking at the Gitian descriptor for the pluggable transports at https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitia n/descriptors/windows/gitian-pluggable-transports.yml , and I noticed that it has an input file called "python.msi". Furthermore, I noticed the following line in https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitia n/versions : PYTHON_MSI_URL=https://www.python.org/ftp/python/${PYTHON_VER}/${PYTHON_ MSI_PACKAGE} - From this, I conclude that Python is not being built in Gitian, and the download from www.python.org is assumed to be safe / not backdoored. Is this correct? If I'm correct, is there a reason that Python is not being built in Gitian? Was it attempted and found that Python cannot easily be built for Windows in Gitian? Or was it not attempted and just still on the to-do list? I don't see any relevant ticket on Trac. Thanks, - -Jeremy Rand -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJV7Mt1AAoJEAHN/EbZ1y06nL8QAM4qhFMupoippBIvI4JwlLZ6 Vf4wNWA2/IY+62DQ4hpkPRPX/vT48lJIJnPXUxQ427ruX/txYs+T8Yw/RyiW6eq8 AWrkj3DZTYE4RtgEnTazA7NEpiv0YFOuRdyKoyjZyR1MAZTWpZ12TS/hkXaksWsx mZX7EJMv9GBNthMLGoJ5cpTdXymhbGgvOGcF1esxZlzEQnusbaMjCNHv+GzLXGBr xEFUzjuIeuvzZxHgmTqTujjMev9kMYqpRVoyg2JbRhZz/LwbfuQssDMlTRVOmpCu FwCw9RbyZyaBzrnoZLbJ9hs6JQz/cKYcv0kOmEncTprc6IVdZDnMj5guT0rwUycn r9x0ZB0Uz7FRPnqIgmeHLn9JmFPuF9IWc6Kl3fILNsYMCF+AyOBAB7QVFgSftXMQ 2+ifWG6OdAZfM0jau1L6phlILJtvc79ClHhp86wIeSQ7k0mEKn9JOzEg+YRXLS4L ZAoTFKSfNrukMtmdUhgb97yl7MA+wsdWCpMybZRpSJQYMiNlL33njG8kYMdbuhX/ uDTFmc1b3IGWhfRdAtaIK9sZzkTYTzFg30Ypr8uYte7McU+QOMHbWZGZ46KL79k7 uZxPt2bcKSzEP9KP28Pkz0Hc0v0qGYD0BrAzeYkYVB9FhLQF5vs1jDfn2YI7VQbI HSs0NTTkxWHkro3tnzeU =jg/n -----END PGP SIGNATURE-----

5 12

Re: [tor-dev] [tor-reports] Sebastian's August 2015
by ilv 11 Sep '15

11 Sep '15

On 06/09/15 13:01, Sebastian Hahn wrote: > Hi there, > Hi Sebastian, > Next up is more of the same, especially focusing on website tickets > and preparing the community team's dev meeting contributions. Maybe we could have a session at the dev meeting to talk about the website (content, structure, translations, etc.). What do you think? (cc'ing @tor-dev in case more people are interested) Saludos. --ilv

2 1

[PATCH] Update prop#188 to match implementation from #7144.
by Isis Lovecruft 10 Sep '15

10 Sep '15

* ADD a new section detailing loose-source routed circuits, including: - How the circuit should appear to the OP, the bridge, and the bridge guard, - How the hop(s) in the path is/are chosen, - How the first hop is handled and how circuit extension is handled, in a generalised sense (i.e. not specific to a bridge using bridge guards), - Which cell types are used and how they are chosen, - When the original create cell is answered (in relation to when cells are sent to the other additional hops), - What should be done when relay cells are received when the additional hops in the loose-source routed circuit are not fully constructed, - How the circuit crypto and cell digests for incoming/outgoing relay cells works (again, in a generalised sense, i.e. not specific to bridges using bridge guards), - How and whether a given relay cell is treated as "recognized", - Under what circumstances (based on whether a stream is attached and which relay command was sent) should cause a node which uses loose-source routed circuits to drop a relay cell or mark a circuit for close, and, - When and what statistics should be gathered for loose-source routed circuits. * UPDATE and clarify the example loose-source routed circuit construction (the original one from the proposal which was specific to a client using a bridge that uses bridge guards). * CHANGE the "Make it harder to tell clients from bridges" section which described the tradeoffs between using CREATE versus CREATE_FAST cells for the first additional hop (i.e. the bridge guard). Those concerns is no longer entirely relevant, since, in the meantime, we've introduced the NTor handshake and it's extremely likely that both the client and the bridge will use CREATE2 for all their hops. * ADD a new section on enumerating bridges via the behaviours inherent to bridge reachability testing. * REMOVE open question (from the very end of the proposal) regarding whether the standard guard selection algorithms are adequate. They are. (Even if they are convoluted and overly-complicated.) --- proposals/188-bridge-guards.txt | 458 ++++++++++++++++++++++++++++++++-------- 1 file changed, 369 insertions(+), 89 deletions(-) diff --git a/proposals/188-bridge-guards.txt b/proposals/188-bridge-guards.txt index 5a5a005..c4d354f 100644 --- a/proposals/188-bridge-guards.txt +++ b/proposals/188-bridge-guards.txt @@ -1,7 +1,8 @@ Filename: 188-bridge-guards.txt Title: Bridge Guards and other anti-enumeration defenses -Author: Nick Mathewson +Author: Nick Mathewson, Isis Lovecruft Created: 14 Oct 2011 +Modified: 10 Sep 2015 Status: Open 1. Overview @@ -76,111 +77,317 @@ Status: Open remove, and remove a layer of encryption on incoming cells on that circuit corresponding to the encryption that the guard will add. -3.1. An example +3.1. Loose-Source Routed Circuit Construction - This example doesn't add anything to the design above, but has some - interesting inline notes. + Alice, an OP, is using a bridge, Bob, and she has chosen the + following path through the network: - - Alice has connected to her bridge Bob, and built a circuit - through Bob, with the negotiated forward and reverse keys KB_f - and KB_r. + Alice -> Bob -> Charlie -> Deidra - - Alice then wants to extend the circuit to node Charlie. She - makes a hybrid-encrypted onionskin, encrypted to Charlie's - public key, containing her chosen g^x value. She puts this in - an extend cell: "Extend (Charlie's address) (Charlie's OR - Port) (Onionskin) (Charlie's ID)". She encrypts this with - KB_f and sends it as a RELAY_EARLY cell to Bob. + However, Bob has decided to take advantage of the loose-source + routing circuit characteristic (for example, in order to use a bridge + guard), and Bob has chosen N additional loose-source routed hop(s), + through which he will transparently relays cells. - - Bob receives the RELAY_EARLY cell, and decrypts it with KB_f. - He then sees that it's an extend cell for him. + NOTE: For the purposes of bridge guards, N is always 1. However, for + completion's sake, the following details of the circuit construction + are generalized to include N > 1. Additionally, the following steps + should hold for a hop at any position in Alice's circuit that has + decided to take advantage of the loose-source routing feature, not + only for bridge ORs. - So far, this is exactly the same as the current procedure that - Alice and Bob would follow. Now we diverge: + From Alice's perspective, her circuit path matches the one diagrammed + above. However, the overall path of the circuit is: - - Instead of connecting to Charlie directly, Bob makes sure that - he is connected to his guard, Guillaume. Bob uses a - CREATE_FAST cell (or a CREATE cell, but see 4.1 below) to open a - circuit to Guillaume. Now Bob and Guillaume share keys KG_f - and KG_b. + Alice -> Bob -> Guillaume -> Charlie -> Deidra - - Now Bob encrypts the Extend cell body with KG_f and sends it - as a RELAY_EARLY cell to Guillaume. + From Bob's perspective, the circuit's path is: - - Guillaume receives it, decrypts it with KG_f, and sees: - "Extend (Charlie's address) (Charlie's OR Port) (Onionskin) - (Charlie's ID)". Guillaume acts accordingly: creating a - connection to Charlie if he doesn't have one, ensuring that - the ID is as expected, and then sending the onionskin in a - create cell on that connection. + Alice -> Bob -> Guillaume -> Charlie -> UNKNOWN - Note that Guillaume is behaving exactly as a regular node - would upon receiving an Extend cell. + Interestingly, because Bob's behaviour towards Guillaume and choices + of cell types is that of a normal OP, Guillaume's perspective of the + circuit's path is: - - Now the handshake finishes. Charlie receives the onionskin - and sends Guillaume "CREATED g^y,KH". Guillaume sends Bob - "E(KG_r, EXTENDED g^y KH)". (Charlie and Guillaume are still - running as regular Tor nodes do today). + Bob -> Guillaume -> Charlie -> UNKNOWN - - With this extend cell, and with all future relay cells - received on this circuit, Bob first decrypts the cell with - KG_r, then re-encrypts it with KB_r, then passes it to Alice. - When Alice receives the cell, it will be just as she would - have received if Bob had extended to Charlie directly. + That is, to Guillaume, Bob appears (for the most part) to be a + normally connecting client. (See §4.1 for more detailed analysis.) - - With all future outgoing cells that he receives from Alice, - Bob first decrypts the cell with KA_f, and if the cell does - not have Bob as its destination, Bob encrypts it with KG_f - before passing it to Guillaume. +3.1.1. Detailed Steps of Loose-Source Routed Circuit Construction - Note that this design does not require that our stream cipher - operations be commutative, even though they are. + 1. Connection from OP - Note also that this design requires no change in behavior from any - node other than Bob the bridge. + Alice has connected to Bob, and she has sent to Bob either a + CREATE/CREATE_FAST or CREATE2 cell. - Finally, observe that even though the circuit is one hop longer - than it would be otherwise, no relay's count of permissible - RELAY_EARLY cells falls lower than it otherwise would. This is - because the extra hop that Bob adds is done with a CREATE_FAST - cell, and so he does not need to send any RELAY_EARLY cells not - originated by Alice. + 2. Loose-Source Path Selection -4. Other ideas and alternative designs + In anticipation of Alice's first RELAY_EARLY cell (which will + contain an EXTEND cell to Alice's next hop), Bob begins + constructing a loose-source routed circuit. To do so, Bob chooses + N additional hop(s): - In addition to the design above, there are more ways to try to - prevent enumeration. + 2.a. For the first additional hop, H_1, Bob chooses a suitable + entry guard node, Guillaume, using the same algorithm as OPs. + See "§5 Guard nodes" of path-spec.txt for additional + information on the selection algorithm. -4.1. Make it harder to tell clients from bridges + 2.b. Each additional hop, [H_2, ..., H_N], is chosen at random + from a list of suitable, non-excluded ORs. - Right now, there are multiple ways for the node after a bridge to - distinguish a circuit extended through the bridge from one - originating at the bridge. (This lets the node after the bridge - tell that a bridge is talking to it.) + 3. Loose-Source Routed Circuit Extension and Cell Types - One of the giveaways here is that the first hop in a circuit is - created with CREATE_FAST cells, but all subsequent hops are created - with CREATE cells. In the above design, it's no longer quite so - simple to tell, since all of the circuits that extend through a - bridge now reach its guards through CREATE_FAST cells, whether the - bridge originated them or not. + Bob now follows the same procedure as OPs use to complete the key + exchanges with his chosen additional hop(s). - (If we adopt a faster circuit extension algorithm -- for example, - Goldberg, Stebila, and Ustaoglu's design instantiated over - curve25519 -- we could also solve this issue by eliminating - CREATE_FAST/CREATED_FAST entirely, which would also help our - security margin a little.) + While undergoing these following substeps, Bob SHOULD continue to + proceed with Step 4, below, in parallel, as an optimization for + speeding up circuit construction. - The CREATE/CREATE_FAST distinction is not the only way for a - bridge's guard to tell bridges from orginary clients, however. - Most importantly, a busy bridge will open far more circuits than a - client would. More subtly, the timing on response from the client - will be higher and more highly variable that it would be with an - ordinary client. I don't think we can make bridges behave wholly - indistinguishably from clients: that's why we should go with guard - nodes for bridges. + 3.a. Create Cells + + Bob sends the appropriate type of create cell to Guillaume. + For ORs new enough to support the NTor handshake (nearly all + of them at this point), Bob sends a CREATE2 cell. Otherwise, + for ORs which only support the older TAP handshake, Bob sends + either a CREATE or CREATE_FAST cell, using the same + decision-making logic as OPs. + + See §4.1 for more information the distinguishability of + bridges based upon whether they use CREATE versus + CREATE_FAST. Also note that the CREATE2 cell has since + become ubiquitous after this proposal was originally drafted. + Thus, because we prefer ORs which use loose-source routing to + behave (as much as possible) like OPs, we now prefer to use + CREATE2. + + 3.b. Created Cells + + Later, when Bob receives a corresponding CREATED/CREATED_FAST + or CREATED2 cell from Guillaume, Bob extracts key material + for the shared forward and reverse keys, KG_f and KG_b, + respectively. + + 3.c. Extend Cells + + When N > 1, for each additional hop, H_i, in [H_2, ..., H_N], + Bob chooses the appropriate type of extend cell for H_i, and + sends this extend cell to H_i-1, who transforms it into a + create cell in order to perform the extension. To choose + which type of extend cell to send, Bob uses the same + algorithm as an OP to determine whether to use EXTEND or + EXTEND2. Similar to the CREATE* cells above, for most modern + ORs, this will very likely mean an EXTEND2 cell. + + 3.d. Extended Cells + + When a corresponding EXTENDED/EXTENDED2 cell is received for + an additional hop, H_i, Bob extracts the shared forward and + reverse keys, Ki_f and Ki_b, respectively. + + 4. Responding to the OP + + Now that the additional hops in Bob's loose-source routed circuit + are chosen, and construction of the loose-source routed circuit + has begun, Bob answers Alice's original CREATE/CREATE_FAST or + CREATE2 cell (from Step 1) by sending the corresponding created + cell type. + + Alice has now built a circuit through Bob, and the two share the + negotiated forward and reverse keys, KB_n and KB_p, respectively. + + Note that Bob SHOULD do this step in tandem with the loose-source + routed circuit construction procedure outlined in Step 3, above. + + 5. OP Circuit Extension + + Alice then wants to extend the circuit to node Charlie. She makes + a hybrid-encrypted onionskin, encrypted to Charlie's public key, + containing her chosen g^x value. She puts this in an extend cell: + "Extend (Charlie's address) (Charlie's OR Port) (Onionskin) + (Charlie's ID)". She encrypts this with KB_n and sends it as a + RELAY_EARLY cell to Bob. + + Bob's behaviour is now dependent on whether the loose-source + routed circuit construction steps (as outlined in Step 3, above) + have already completed. + + 5.a. The Loose-Source Routed Circuit Construction is Incomplete + + If Bob has not yet finished the loose-source routed circuit + construction, then Bob MUST store the first outgoing + (i.e. exitward) RELAY_EARLY cell received from Alice until + the loose-source routed circuit construction has been + completed. + + If any incoming (i.e. toward the OP) RELAY* cell is received + while the loose-source routed circuit is not fully + constructed, Bob MUST drop the cell. + + If Bob has already stored Alice's first RELAY_EARLY cell, and + Alice sends any additional RELAY* cell, then Bob SHOULD mark + the entire circuit for close with END_CIRC_REASON_TORPROTOCOL. + + 5.b. The Loose-Source Routed Circuit Construction is Completed + + Later, when the loose-source routed circuit is fully + constructed, Bob MUST send any stored cells from Alice + outward by following the procedure described in Step 6.a. + + 6. Relay Cells + + When receiving a RELAY* cell in either direction, Bob MAY keep + statistics on the number of relay cells encountered, as well as + the number of relay cells relayed. + + 6.a. Outgoing Relay Cells + + Bob decrypts the RELAY* cell with KB_n. If the cell becomes + recognized, Bob should now follow the relay command checks + described in Step 6.c. + + Bob MUST encrypt the relay cell's underlying payload to each + additional hop in the loose-source routed circuit, in + reverse: for each additional hop, H_i, in [H_N, ..., H_1], + Bob encrypts the relay cell payload to Ki_f, the shared + forward key for the hop H_i. + + Bob MUST update the forward digest, DG_f, of the relay cell, + regardless of whether or not the cell is recognized. See + 6.c. for additional information on recognized cells. + + Bob now sends the cell outwards through the additional hops. + At each hop, H_i, the hop removes a layer of the onionskin by + decrypting the cell with Ki_f, and then hop H_i forwards the + cell to the next addition additional hop H_i+1. When the + final additional hop, H_N, received the cell, the OP's cell + command and payload should be processed by H_N in the normal + manner for an OR. + + 6.b. Incoming Relay Cells + + Bob MUST decrypt the relay cell's underlying payload from + each additional hop in the loose-source routed circuit (in + forward order, this time): For each additional hop, H_i, in + [H_1, ..., H_N], Bob decrypts the relay cell payload with + Ki_b, the shared backward key for the hop H_i. + + If the cell has becomes recognized after all decryptions, Bob + should now follow the relay command checks described in Step + 6.c. + + Bob MUST update the backward digest, DG_b, of the relay cell, + regardless of whether or not the cell is recognized. See + 6.c. for additional information on recognized cells. + + Bob encrypts the cell towards the OP with KB_p, and sends the + cell inwards. + + 6.c. Recognized Cells + + If a relay cell, either incoming or outgoing, becomes + recognized (i.e. Bob sees that the cell was intended for him) + after decryption, and there is no stream attached to the + circuit, then Bob SHOULD mark the circuit for close if the + relay command contained within the cell is any of the + following types: + + - RELAY_BEGIN + - RELAY_CONNECTED + - RELAY_END + - RELAY_RESOLVE + - RELAY_RESOLVED + - RELAY_BEGIN_DIR + + Apart from the above checks, Bob SHOULD essentially treat + every cell as "unrecognized" by following the en-/de-cryption + procedures in Steps 6.a. and 6.b. regardless of whether the + cell is actually recognized or not. That is, since this is a + loose-source routed circuit, Bob SHOULD relay cells not + intended for him *and* cells intended for him through the + leaky pipe, no matter what the cell's underlying payload and + command are. + +3.1.2. Example Loose-Source Circuit Construction + + For example, given the following circuit path chosen by Alice: + + Alice -> Bob -> Charlie -> Deidra + + when Alice wishes to extend to node Charlie, and Bob the bridge is + using only one additional loose-source routed hop, Guillaume, as his + bridge guard, the following steps are taken: + + - Alice packages the extend into a RELAY_EARLY cell and encrypts + the RELAY_EARLY cell with KB_f to Bob. + + - Bob receives the RELAY_EARLY cell from Alice, and he follows + the procedure (outlined in §3.1.1. Step 6.a.) by: + + * Decrypting the cell with KB_f, + * Encrypting the cell to the forward key, KG_f, which Bob + shares with his guard node, Guillaume, + * Updating the cell forward digest, DG_f, and + * Sending the cell as a RELAY_EARLY cell to Guillaume. + + - When Guillaume receives the cell from Bob, he processes it by: + + * Decrypting the cell with KG_f. Guillaume now sees that it + is a RELAY_EARLY cell containing an extend cell "intended" + for him, containing: "Extend (Charlie's address) (Charlie's + OR Port) (Onionskin) (Charlie's ID)". + * Performing the circuit extension to the specified node, + Charlie, by acting accordingly: creating a connection to + Charlie if he doesn't have one, ensuring that the ID is as + expected, and then sending the onionskin in a create cell + on that connection. Note that Guillaume is behaving + exactly as a regular node would upon receiving an Extend + cell. + * Now the handshake finishes. Charlie receives the onionskin + and sends Guillaume "CREATED g^y,KH". + * Making an extended cell for Bob which contains + "E(KG_b, EXTENDED g^y KH)", and + * Sending the extended cell to Bob. Note that Charlie and + Guillaume are both still behaving in a manner identical to + regular ORs. + + - Bob receives the extended cell from Guillaume, and he follows + the procedure (outlined in §3.1.1. Step 6.b.) by: + + * Decrypting the cell with KG_b, + * Encrypting the cell to Alice with KB_b, + * Updating the cell backward digest, DG_b, and + * Sending the cell to Alice. + + - Alice receives the cell, and she decrypts it with KB_b, just + as she would have if Bob had extended to Charlie directly. + She then processes the extended cell contained within to + extract shared keys with Charlie. Note that Alice's behaviour + is identical to regular OPs. + +3.2. Additional Notes on the Construction -4.2. Client-enforced bridge guards + Note that this design does not require that our stream cipher + operations be commutative, even though they are. + + Note also that this design requires no change in behavior from any + node other than Bob, and as we can see in the above example in §3.1.2 + for Alice's circuit extension, Alice, Guillaume, and Charlie behave + identical to a normal OP and normal ORs. + + Finally, observe that even though the circuit N hops longer than it + would be otherwise, no relay's count of permissible RELAY_EARLY cells + falls lower than it otherwise would. This is because the extra hop + that Bob adds is done with RELAY_EARLY cells, then he continues to + relay Alice's cells as RELAY_EARLY, until the appropriate maximum + number of RELAY_EARLY cells is reached. Afterwards, further + RELAY_EARLY cells from Alice are repackaged by Bob as normal RELAY + cells. + +4. Alternative designs + +4.1. Client-enforced bridge guards What if Tor didn't have loose source routing? We could have bridges tell clients what guards to use by advertising those guard @@ -191,7 +398,7 @@ Status: Open Fortunately, we don't need to go down this path. So let's not! -4.3. Separate bridge-guards and client-guards +4.2. Separate bridge-guards and client-guards In the design above, I specify that bridges should use the same guard nodes for extending client circuits as they use for their own @@ -209,11 +416,84 @@ Status: Open through the bridge to a node it controls, and finding out where the extend request arrives from). -5. Other considerations +5. Additional bridge enumeration methods and protections + + In addition to the design above, there are more ways to try to + prevent enumeration. + + Right now, there are multiple ways for the node after a bridge to + distinguish a circuit extended through the bridge from one + originating at the bridge. (This lets the node after the bridge + tell that a bridge is talking to it.) + +5.1. Make it harder to tell clients from bridges + + When using the older TAP circuit handshake protocol, one of the + giveaways is that the first hop in a circuit is created with + CREATE_FAST cells, but all subsequent hops are created with CREATE + cells. + + However, because nearly everything in the network now uses the newer + NTor circuit handshake protocol, clients send CREATE2 cells to all + hops, regardless of position. Therefore, in the above design, it's + no longer quite so simple to distinguish an OP connecting through + bridge from an actual OP, since all of the circuits that extend + through a bridge now reach its guards through CREATE2 cells (whether + the bridge originated them or not), and only as a fallback (e.g. if + an additional node in the loose-source routed path does not support + NTor) will the bridge ever use CREATE/CREATE_FAST. (Additionally, + when using the fallback mathod, the behaviour for choosing either + CREATE or CREATE_FAST is identical to normal OP behaviour.) + + The CREATE/CREATE_FAST distinction is not the only way for a + bridge's guard to tell bridges from orginary clients, however. + Most importantly, a busy bridge will open far more circuits than a + client would. More subtly, the timing on response from the client + will be higher and more highly variable that it would be with an + ordinary client. I don't think we can make bridges behave wholly + indistinguishably from clients: that's why we should go with guard + nodes for bridges. + + [XXX For further research: we should study the methods by which a + bridge guard can determine that they are acting as a guard for a + bridge, rather than for a normal OP, and which methods are likely to + be more accurate or efficient than others. -IL] + +5.2. Bridge Reachability Testing + + Currently, a bridge's reachability is tested both by the bridge + itself (called "self-testing") and by the BridgeAuthority. + +5.2.1. Bridge Reachability Self-Testing + + Before a bridge uploads its descriptors to the BridgeAuthority, it + creates a special type of testing circuit which ends at itself: + + Bob -> Guillaume -> Charlie -> Bob + + Thus, going to all this trouble to later use loose-source routing in + order to relay Alice's traffic through Guillaume (rather than + connecting directly to Charlie, as Alice intended) is diminished by + the fact that Charlie can still passively enumerate bridges by + waiting to be asked to connect to a node which is not contained + within the consensus. + + We could get around this option by disabling self-testing for bridges + entirely, by automatically setting "AssumeReachable 1" for all bridge + relays… although I am not sure if this is wise. + +5.2.2. Bridge Reachability Testing by the BridgeAuthority + + After receiving Bob's descriptors, the BridgeAuthority attempts to + connect to Bob's ORPort by making a direct TLS connection to the + bridge's advertised ORPort. + + Should we change this behaviour? One the one hand, at least this + does not enable any random OR in the entire network to enumerate + bridges. On the other hand, any adversary who can observe packets + from the BridgeAuthority is capable of enumeration. + +6. Other considerations What fraction of our traffic is bridge traffic? Will this alter our circuit selection weights? - - Are the current guard selection/evaluation/replacement mechanisms - adequate for bridge guards, or do bridges need to get more - sophisticated? -- 2.1.4

1 0

Re: [tor-dev] How to introduce new RELAY_COMMAND_* types and newcell fields?
by tordev123＠Safe-mail.net 10 Sep '15

10 Sep '15

>> 3. Additional RELAY_COMMAND_* types for clients to request out-of-band >> HMAC request cells for Proposal 253. Do you need to request that data? How about always sending it from middle nodes? (Less leakage about the client.) >> 4. Additional RELAY_COMMAND_* opcodes for clients to request padding >> from relays (for an upcoming padding negotiation proposal). >> >> However, for items #3 and #4, if I introduce a new RELAY_COMMAND type >> and send it to a relay that doesn't support it, then that relay will >> emit a warning log message from connection_edge_process_relay_cell() in >> relay.c. How should I detect support? Based on advertised relay version >> in the consensus? What about non-standard relay implementations that >> don't use Tor's versioning? > > I don't think you can use the consensus for this: for HS connections you > wont have any relay info. Wouldn't matter for 4, right? > How about introducing very basic version info into > created2/extended/rendezvous cells? You'd have to be really careful to > not leak too much version info, of course. It seems much better to avoid that, if there is a workaround.

2 1

How to introduce new RELAY_COMMAND_* types and new cell fields?
by Mike Perry 10 Sep '15

10 Sep '15

I'm writing a handful of proposals that need to introduce either new cell sub-payloads or new command types. Specifically, I want to add: 1. Sub-fields to CELL_PADDING to allow clients to tell relays about the amount of link padding they want for Proposal 251. Mobile clients will probably want less or no padding, and need some way to say so. Adding this information to CELL_PADDING itself seemed to be an easy choice. 2. Sub-fields to RELAY_COMMAND_TRUNCATED and CELL_DESTROY to include HMAC data for Proposal 253. 3. Additional RELAY_COMMAND_* types for clients to request out-of-band HMAC request cells for Proposal 253. 4. Additional RELAY_COMMAND_* opcodes for clients to request padding from relays (for an upcoming padding negotiation proposal). I *think* that fields for items #1 and #2 can be simply added to the existing cell definitions, since we specify that cells should already be 0-filled, and I can rely on 0 fields to mean that the additional fields are absent. It should be OK if relays simply ignore/omit these fields until those proposals are implemented. Is this OK? However, for items #3 and #4, if I introduce a new RELAY_COMMAND type and send it to a relay that doesn't support it, then that relay will emit a warning log message from connection_edge_process_relay_cell() in relay.c. How should I detect support? Based on advertised relay version in the consensus? What about non-standard relay implementations that don't use Tor's versioning? -- Mike Perry

2 1

Fwd: Reproducibility of Pluggable Transports python.msi
by Joseph Bisch 09 Sep '15

09 Sep '15

I'm resending this, because tor-dev didn't accept my message originally. Sorry David and Jeremy for the double message. ____________________________________________________ Hello David and Jeremy, I am the "reproducible build specialist" that Jeremy was referring to, though I don't know if I would call myself that. As Jeremy said, Armory isn't yet reproducible on Windows, but hopefully some of this information will be helpful. First of all, none of the following has been wrapped in Gitian and checked for reproducibility yet, because I was primarily concerned with just getting cross-compiling of Python working first. But it is at least possible to cross-compile Python 3.4.3. I have a PR[0] open for Armory that covers the whole exe creation process using pyqtdeploy, but a lot of that is probably not relevant to Tor if something other than pyqtdeploy is used. So I will go over the parts specifically having to do with building Python from source. Also, the PR isn't fully working yet. I found that Python 2 is much more difficult to get cross-compiling than Python 3. So I decided to go with Python 3. I used the patches[1] that the Arch Linux Mingw Python 3 package uses. Before compiling Python, you will want to compile OpenSSL. I have instructions for building Python 3 from source. Since steps two and three are probably only relevant in Armory's case (we are building Python statically and then "freezing" it with pyqtdeploy, so all modules need to be embedded), you just have to download the Python 3.4.3 source and the Arch patches and run the build_py3.sh script[3]. That is the latest script, so don't use the one in the PR. The script is still a little fragile (everything seems to work with the build I did on my desktop, but importing json with the build I did on my laptop gives a PyInit_Thread error and importing hashlib gives a page fault). Also, I'm not sure if every flag/variable is necessary for the build to work. Also, my latest comment[4] on the PR explains some things I had to do to the OpenSSL and Python source trees to get the build working. I should probably make patches out of them and they may be appropriate for integration into OpenSSL and Python upstream. Though I don't know how my changes affect other system configurations, such as when someone is compiling on Windows. Hope that helps. I'd also appreciate any feedback anyone might have. Also, let me know if there is a better channel to talk about this, because getting Python to cross-compile isn't strictly tor-dev related. [0] - https://github.com/etotheipi/BitcoinArmory/pull/303 [1] - https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-python3 [2] - https://github.com/josephbisch/BitcoinArmory/blob/autotools-windows/mingwbu… [3] - https://gist.github.com/josephbisch/cf2d32a798f063f0bf71 [4] - https://github.com/etotheipi/BitcoinArmory/pull/303#issuecomment-138762224 Cheers, Joseph

1 0

fteproxy depends on obfsproxy...
by Kevin P Dyer 09 Sep '15

09 Sep '15

...and it shouldn't. Fortunately, the dependency is isolated to a single file. See [1]. My understanding is that pyptlib [2] is no longer maintained. wiley/asn/etc. - What's the proper way to remove this dependency, but make it easy for fteproxy to be a PT? -Kevin [1] https://github.com/kpdyer/fteproxy/blob/master/fteproxy/cli.py#L248 [2] https://pypi.python.org/pypi/pyptlib

4 7

[RFC] On new guard algorithms and data structures
by George Kadianakis 09 Sep '15

09 Sep '15

Hello there, recently we've been busy specifying various important improvements to entry guard security. For instance see proposals 250, 241 and ticket #16861. Unfortunately, the current guard codebase is dusty and full of problems (see #12466, #12450). We believe that refactoring and cleaning up the entry guard code is essential before we proceed to more advanced security improvements. We've been working on new algorithms and data structures for guard nodes as part of ticket #12595. In this mail I include some pseudocode for this new algorithm with the hope that it will act as a draft for implementing these changes. You can find the pseucode here: https://gitweb.torproject.org/user/asn/tor.git/tree/src/or/guardlist.c?h=bu… A short description of the algorithm is included on top, and then various methods and functions are prototyped underneath to make the logic more concrete. Apart from the comments and XXXs on the code, here are some more thoughts on this work: - This new design focuses on protecting against path bias attacks, by slightly damaging our reachability. Specifically, the old design is better at recovering in filtered networks, because it will keep on adding new nodes till one succeeds. In this new design, we will not try more than 80 relays per time. So if none of them passes the filtered network, bad luck no Tor. While this failure mode should not happen much, it's bad news for users behind FascistFirewalls which are actually quite frequent. A quick fix here would be to always add an 80/443 guard on our list, however as it stands only 30% of the guards are 80/443 guards, so this has bad anonymity consequences. - To improve our algorithm and make it more robust we need to understand further what kind of path bias attacks are relevant here. The adversary here is a network adversary (like a gateway) that can block our connections to certain guards. What nasty attacks can this adversary do? If we can't find bad attacks here, then maybe we should stop worrying about those path bias attacks so much. For example a threat here with the old guard logic, is that if we used this evil gateway just for 10 minutes (in an airport), the adversary could launch a path bias attack and force us to connect to her guard node. Then even after we left that airport, we would still stick to the evil guard node which is bad. Also, an adversary that manages to own our guard using path bias attacks, then has further possibilites for biasing the rest of the circuit. What can this adversary do? - Notice that the pseudocode contains no logic about bridges. I'm not sure how bridges should be handled here. - I tried to keep the dirguard logic very simple, hoping that we can eventually forget about dirguards entirely when #12538 is done. The main dirguard feature is that we assume that populate_live_entry_guards() and add_an_entry_guard() will return dirguards when the circuit is a directory circuit. Maybe we should consider introducing the "primary dirguard" concept as well. And maybe also add some logic where Tor will move on to the next dirguard if it failed to receive a document from the current dirguard. - I used the ATTEMPTED_THRESHOLD concept of prop241, but did not use the NET_THRESHOLD and CONNECTED_THRESHOLD ideas. I removed NET_THRESHOLD because I increased the value of ATTEMPTED_THRESHOLD to the point that it can also be used as a network down indicator. Also, I was not sure what CONNECTED_THRESHOLD was useful for, and there were certain engineering issues with it (Like, if that threshold is hit, we need a logic that will *only retry the successfully connected guards*, and not all guards). - There is no log message warning the user of path bias attacks or bad network or anything. That's because there is no way to figure out what's the problem, and issuing an alarming log message here would confuse and panic the user. If we want to inform the user anyhow, maybe if the user is *actively* trying to visit a destination, and we've been cycling through our guard list for ages, maybe we should then issue a log message telling the user that something is wrong with the network. - In general, I tried to keep the number of heuristics and kludges to the minimum to keep the logic simple. Unfortunately, it seems that without a "network down" indicator (#16120) there is no way to avoid edge cases and false positives here. We should try to fix all problems here that can occur frequently or have security consequences, but there will always be scenarios where Tor will end up thinking there is no network while it's actually on a filternet. For this reason, we should give plenty of testing to this feature before we ship it to real users! - Finally, all the constants & parameters in the pseudocode are subject to change. I tried to motivate some of them, but others are just arbitrary. Feedback is very welcome and please let me know of any issues with security or reachability that you find! Or of how the pseudocode should be altered to make it more useful for implementors. Cheers!

9 19

Erebus Code Review
by Damian Johnson 09 Sep '15

09 Sep '15

Hi Cristobal, just took a peek at Erebus and looks good! There's some low hanging fruit for improvement (unused functions and such) but generally looks good. Cheers! -Damian ====================================================================== erebus/server/handlers/log.py ====================================================================== 9 import erebus.util.log Perfectly fine for now, but obviously most of this is copy/paste from Nyx. Both Nyx and Erebus are smack in the middle of development so this is great! Do whatever you find most expedient. But obviously in the long run we'll want to revisit this. ---------------------------------------------------------------------- 16 def conf_handler(key, value): 17 if key == 'features.log.maxLinesPerEntry': 18 return max(1, value) 19 elif key == 'features.log.prepopulateReadLimit': 20 return max(0, value) 21 elif key == 'features.log.maxRefreshRate': 22 return max(10, value) 23 elif key == 'cache.log_panel.size': 24 return max(1000, value) I know this is copy-and-paste from Nyx but personally I wouldn't bother. Many of the config keys are unused, and at present users don't have a way of customizing them. Nyx's support for a nyxrc is a very rarely used feature, so I would leave this out for now. ---------------------------------------------------------------------- 40 TOR_EVENT_TYPES = { 41 'd': 'DEBUG', 42 'i': 'INFO', 43 'n': 'NOTICE', 44 'w': 'WARN', 45 'e': 'ERR', I did this for Nyx because because... well, curses. For Erebus we can do better. ;) Rather than 'key => event' mapping maybe Erebus should show all the event types, with checkboxes for selection? ---------------------------------------------------------------------- 155 def _register_event(self, event): 156 output = { 'message': '', 'type': ''} 157 if event.type not in self._logged_events: 158 return 159 #self._event_log.add(event) 160 #self._log_file.write(event.display_message) 161 162 # notifies the display that it has new content 163 if self._filter.match(event.display_message): 164 self._has_new_event = True 165 output['message'] = event.display_message 166 output['type'] = event.type.lower() 167 ws = ws_controller() 168 if ws is not None: 169 ws.send_data(WebSocketType.LOG, output) You can drop the line with self._has_new_event. It's an unused attribute, and doesn't appear outside of this line. The 'output' is only ever used on this last line, so might as well just call... ws.send_data(WebSocketType.LOG, {'message': event.display_message, 'type': event.type.lower()}) ---------------------------------------------------------------------- 171 def log_handler(): 172 """ 173 Singleton for getting our log hander. 174 175 :returns: :class:`~erebus.server.handlers.log.LogHandler` 176 """ 177 178 return LOG_HANDLER 179 180 def init_log_handler(logged_events): 181 """ 182 """ 183 global LOG_HANDLER 184 LOG_HANDLER = LogHandler(logged_events) 185 return LOG_HANDLER Seems from a quick look that these might be unused. The controller.py and starter.py call init_log_handler, but not spotting anything that ever uses LOG_HANDLER. ---------------------------------------------------------------------- 195 response = None 196 if controller is not None: 197 response = controller.get_info('events/names', None) 198 if response is None: 199 return [] # GETINFO query failed This is equivalent to... response = controller.get_info('events/names', []) if controller else [] ====================================================================== erebus/util/controller.py ====================================================================== 29 def erebus_controller(): 30 """ 31 Provides the erebus controller instance. 32 """ 33 34 return EREBUS_CONTROLLER 35 36 def init_controller(control_port, control_socket, logged_events): 37 """ 38 Spawns the controller 39 """ 40 41 global EREBUS_CONTROLLER 42 EREBUS_CONTROLLER = Controller(control_port, control_socket, logged_events) Again, you call init_controller() but looks as though EREBUS_CONTROLLER is never used. ---------------------------------------------------------------------- 61 self._logged_events = logged_events 62 63 self._bw_handler = None 64 self._status_handler = None 65 self._log_handler = None All unused. ---------------------------------------------------------------------- 86 def _stop_loop_check(self): 87 """ 88 Stop the looping task when is not necessary 89 """ 90 91 self._loop_call.stop() Only called once, in _conn_starter(). Might as well just call this there (it's just a minor alias). ---------------------------------------------------------------------- 156 def heartbeat_check(is_unresponsive): 157 """ 158 Logs if its been ten seconds since the last BW event. 159 160 Arguments: 161 is_unresponsive - flag for if we've indicated to be responsive or not 162 """ 163 164 controller = tor_controller() 165 last_heartbeat = controller.get_latest_heartbeat() 166 167 if controller.is_alive(): 168 if not is_unresponsive and (time.time() - last_heartbeat) >= 10: 169 is_unresponsive = True 170 log.notice('Relay unresponsive (last heartbeat: %s)' % time.ctime(last_heartbeat)) 171 elif is_unresponsive and (time.time() - last_heartbeat) < 10: 172 # really shouldn't happen (meant Tor froze for a bit) 173 is_unresponsive = False 174 log.notice('Relay resumed') 175 176 return is_unresponsive Unused. ====================================================================== erebus/util/websockets.py ====================================================================== 41 def add_websocket(self, ws_type, ws): 42 """ 43 Add an object to the list of active websockets of certain type. 44 45 Arguments: 46 ws_type - type of websocket to be fetched 47 ws - websocket object to be added 48 """ 49 50 if self._websockets[ws_type] is not None: 51 if ws not in self._websockets[ws_type]: 52 self._websockets[ws_type].append(ws) 53 else: 54 log.notice('Undefined type (%s) when adding websocket' % ws_type) Your 'if self._websockets[ws_type] is not None' check will never evaluate to False. You default all WebSocketType values to an empty list, and if it doesn't belong to that enum then it'll raise a KeyError. You want... if ws_type in self._websockets: Also, if self._websockets had sets rather than lists then you could drop the second check. Btw, lets use Sphinx pydocs. The documentation style you're using here is adopted from Nyx before I started using Sphinx, and I'm replacing that documentation style as I rewrite it.

2 1

Troubleshooting Tor 0.2.7.1+ ephemeral hidden services
by Micah Lee 09 Sep '15

09 Sep '15

I'm trying to add support for ephemeral hidden services to OnionShare. They'll solve a wide variety of problems related to OnionShare using a system Tor instead of Tor browser, and running in environments like Tails or Whonix. I'd rather never write the hidden service info to disk anyway. I noticed that Tor Browser 5.5a2 comes with Tor 0.2.7.2-alpha, so it's easy to test. But so far I'm not having any success, at least with using stem. Before I open a bug report, either against stem or against tor, am I doing this right? I've made a simple test project: https://github.com/micahflee/ephemeral-hs-test It successfully appears to create the hidden service, but it never succeeds in connecting to itself. I also can't connect to it using Tor Browser. -- Micah Lee

2 2