- tor-dev - lists.torproject.org

Questions about censorship detection paper
by John 13 Jan '16

13 Jan '16

Hi, I ran into the technical report from George Danezis about an anomaly-based censorship-detection system for Tor. I have a few questions that I hope you can help me with. Is there an implementation available of the approach described in the paper? The paper talks about finding anomalies in the time series of the number of requests to directory servers from certain jurisdictions. Where can I find this data about the requests to directory servers? --john [1] https://research.torproject.org/techreports/detector-2011-09-09.pdf

2 3

Fwd: Orbot roadmap feedback
by Nathan Freitas 12 Jan '16

12 Jan '16

I've got big plans for Orbot in 2016... any thoughts on the roadmap below, particularly how it aligns with Tor core roadmap, would be appreciated. ----- Original message ----- From: Nathan of Guardian <nathan(a)guardianproject.info> To: guardian-dev(a)lists.mayfirst.org Subject: Orbot roadmap feedback Date: Mon, 11 Jan 2016 22:58:05 -0500 Here is my current set of brainstormed ideas for Orbot that I am currently working through... please add, edit, improve and prioritize as you are willing and able. - Continued inclusion of the latest and greatest pluggable transports (Obfs*, Meek, ???) - Improved support for requesting and configuring Obfs4 bridges - directly integrate with BridgeDB to provide streamlined way for people to get bridges where Tor is blocked - More Orbot VPN features including better UI for enabling/disabling Tor routing, blocking specific apps, ports and/or hosts, and general "firewall" or little snitch type features within the Orbot VPN - OrbotShare: an OnionShare inspired file sharing and download manager service to enable easy hidden service .onion setup and sharing (including support for new ephemeral onions) - OrbotRelay/Bridge: improved UI for running a bridge or relay from within Orbot, including support for scheduled times (night only) or states (only when plugged in, on wifi, and screen is off, etc) - Removal of Polipo, in favor of new streamlined in process HTTP proxy (less memory use, and more stable) - Creation of LibOrbot module/library for use by any app to embed Tor into their app - Overall improved configuration / settings UI to make tuning Orbot a better, simpler experience... this is an expansion of the new exit country selector in Orbot v15.1, but also includes managing things like network usage and so on. - OrbotWear / OrbotTV: Should we expand Orbot support to other Android platforms like watches and Smart TVs? Perhaps Orbot relay/bridge mode could run on a TV or set top box? - Some sort of "Pro" or Donation-ware version to allow a pay-what-you-can concept within the app Good, bad, ugly? Let me know! -- Nathan of Guardian nathan(a)guardianproject.info

3 2

Propsal 263 Quantum-safe Hybrid handshake for Tor, updated feature request
by Zhenfei Zhang 12 Jan '16

12 Jan '16

Hi list, Thanks for all your valuable comments. We have updated the feature request following your comments. Please see the attachment for the updated feature request, and see https://github.com/zhenfeizhang/ntru-tor/commit/0354fe1d61b2b79615301ff387e… for the modification over last version. Thanks for your time, and please let us know if you have any further comments/suggestions. Cheers, Zhenfei

2 1

Summary of meek's costs, December 2015
by David Fifield 11 Jan '16

11 Jan '16

Here's the summary of meek's CDN fees for December 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 September 2015 $617.08 + $710.75 + $490.58 = $1818.41 October 2015 $672.01 + $110.72 + $300.64 = $1083.37 November 2015 $602.35 + $474.13 + $174.18 = $1250.66 December 2015 $561.29 + $603.27 + $172.60 = $1337.16 -- total by CDN $8607.22 + $9064.72 + $1292.89 = $18964.83 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… The number of users increased by about 1,000 in December 2015. meek-google had an outage as a result of a server migration between 23 November and 4 December. I set up a bridge that is not rate-limited. If you use it, your meek will go a lot faster. https://bugs.torproject.org/17890 The catch is that you have to set up a CDN or upload a PHP file yourself. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2528 GB $303.39 5158 instance hours $257.90 Compared to the previous month: 2162 GB $259.52 6856 instance hours $342.83 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 56M requests $67.98 425 GB $58.83 Asia Pacific (Sydney) 2M requests $3.37 10 GB $1.41 Asia Pacific (Tokyo) 30M requests $37.14 242 GB $31.64 EU (Ireland) 142M requests $171.48 968 GB $76.50 South America (Sao Paulo) 5M requests $11.03 31 GB $7.46 US East (Northern Virginia) 86M requests $86.54 635 GB $49.88 -- total 324M requests $377.54 2314 GB $225.72 https://globe.torproject.org/#/bridge/F4AD82B2032EDEF6C02C5A529C42CFAFE5165… == Azure a.k.a. meek-azure == Zone 1 1560 GB $135.71 Zone 2 267 GB $ 36.89 -- total 1827 GB $172.60 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://trac.torproject.org/projects/tor/wiki/doc/meek#Costs

2 5

Transparent proxying: automagically add firewall rules
by Rene Bartsch 11 Jan '16

11 Jan '16

Hi, transparent proxying to TOR Hidden Services is a great feature of the TOR daemon when it comes to other applications/protocols than HTTP and surfing. It would also be great with privacy appliances (e.g. Mailpile using TOR as secure SMTP transport channel). John Does have problems with such a setup because of the NAT firewall rules. So I suggest the TOR daemon should automagically set the necessary NAT-rules on Windows, Linux and BSD when "TransPort" and "VirtualAddrNetworkIPv[4|6]" are configured in torrc. -- Best regards, Renne

2 1

Tor Control Protocol: multiple commands in a single line possible?
by Patrick Schleizer 11 Jan '16

11 Jan '16

TLDR: Does Tor's control protocol actually support something like ; ? I.e. something like signal newnym ; getinfo address ? How many of there separators are there? Can you provide examples please? Background: At the moment we are implementing support for whitelisting wildcards for control-port-filter-proxy-python [0] to support Onionshare and Ricochet. [1] [2] [3] [4] Does Tor's control protocol actually support something like ; ? If it does, then this would complicate the wildcard feature. If we whitelisted the wildcard add_onion *, we don't want it to match some hypothetical feature add_onion * ; .... I.e. not add_onion * ; GETINFO address. (If it was the case, then we would have to limit the wildcard (*) from example SETCONF HiddenServicePort * to exactly one [numeric] argument etc. Or better, correctly parse multi lined commands. Or as stopgap, reject these separators since no applications using Tor's control protocol are currently using those.) Cheers, Patrick [0] https://github.com/Whonix/control-port-filter-python https://www.whonix.org/wiki/Dev/Control_Port_Filter_Proxy [1] https://phabricator.whonix.org/T446 [2] https://phabricator.whonix.org/T445 [3] https://www.whonix.org/wiki/Next#onionshare [4] https://phabricator.whonix.org/T448 (Asked Roger by mail, said it's okay to post this on tor-dev.)

4 3

Tor metrics: Questionable relays-by-platform accounting
by Fabian Keil 09 Jan '16

09 Jan '16

Looking at "Tor metrics - Relays by platform" [0] one could think that the number of "FreeBSD" relays is only slightly lower than the number of "Windows" relays. It also looks like the number of "other" platforms is constantly zero while the number of "Darwin" relays is unclear (but potentially zero as well). Obviously that's incorrect: fk@r500 ~ $sudo -u _tor grep platform /usr/jails/tor-jail/var/db/tor/cached-descriptors | cut -w -f 5 | sort | uniq -c | sort -rn 6662 Linux 555 Windows 286 FreeBSD 83 OpenBSD 25 Darwin 10 NetBSD 4 SunOS 4 ElectroBSD 4 Bitrig 2 DragonFly 1 GNU/kFreeBSD 1 CYGWIN_NT-10.0-WOW Is it possible that the "Tor metrics" count everything but "Linux" and "Windows" as FreeBSD? Fabian [0] https://metrics.torproject.org/platforms.html

3 6

Proposal: Stop giving Exit flags when only unencrypted traffic can exit
by Tom van der Woerdt 08 Jan '16

08 Jan '16

I've had this on my todo list for a while, finally wrote it down. Honestly, it's a minor change, but something that imho needs to be done. Torspec branch: https://github.com/TvdW/torspec/commits/exit-flag-not-all-plaintext Full text below, tldr first: replace [80,443,6667] with [80,443,5222] for Exit flagging. =================== Filename: 264-exit-flag-not-all-plaintext.txt Title: Stop giving Exit flags when only unencrypted traffic can exit Author: Tom van der Woerdt Created: 2016-01-05 Status: Open 1. Introduction Tor's Exit flags are assigned to relays that have an exit policy that allows exiting to at least two out of three pre-defined ports: 80, 443 and 6667. Since 80 and 6667 (resp. http and irc) are generally used for unencrypted traffic, an attacker could construct an exit policy that relays only unencrypted data. 2. Changes 2.1. Exit flagging By replacing the port 6667 (IRC) entry with a port 5222 (XMPP) entry, Exit flags can no longer be assigned to relays that exit only to unencrypted ports. 2.2. dir-spec.txt A change to dir-spec.txt will be needed to change port 6667 to 5222. 3. Migration This change only needs to be rolled out to directory authorities. Since the flagging system is simple, no special migration is needed for this change, and it will take effect as soon as the deployment of the change has reached a sufficient number of directory authorities. 4. Other considerations While it would have been ideal to drop the port 80 condition as well, in the current state of the internet this is not likely to be a good idea. Too much websites still use unencrypted connections. However, this may be worth reconsidering every few years. XMPP was chosen to replace IRC because nowadays unencrypted XMPP is rare, and because the XMPP protocol is slowly gaining popularity within the communities on the internet. Other popular ports have been considered, such as 22 (SSH), 465 (SMTP), or 995 (POP3), but these are unlikely to be good candidates because of wide spread bruteforce attacks on these ports. =================== Tom

8 16

Proposal 264: Putting version numbers on the Tor subprotocols
by Nick Mathewson 08 Jan '16

08 Jan '16

Filename: 264-subprotocol-versions.txt Title: Putting version numbers on the Tor subprotocols Author: Nick Mathewson Created: 6 Jan 2016 Status: Open 1. Introduction At various points in Tor's history, we've needed to migrate from one protocol to another. In the past, we've mostly done this by allowing relays to advertise support for various features. We've done this in an ad-hoc way, though. In some cases, we've done it entirely based on the relays' advertised Tor version. That's a pattern we shouldn't continue. We'd like to support more live Tor relay implementations, and that means that tying "features" to "tor version" won't work going forwards. This proposal describes and alternative method that we can use to simplify the advertisement and discovery of features, and the transition from one set of features to another. 1.1. History: "Protocols" vs version-based gating. For ages, we've specified a "protocols" line in relay descriptors, with a list of supported versions for "relay" and "link" protocols. But we've never actually looked at it, and instead we've relied on tor version numbers to determine which features we could rely upon. We haven't kept the relay and link protocols listed there up-to-date either. Clients have used version checks for three purposes historically: checking relays for bugs, checking relays for features, and implementing bug-workarounds on their own state files. In this design, feature checks are now performed directly with subprotocol versions. We only need to keep using Tor versions specifically for bug workarounds. 2. Design: Advertising protocols. We revive the "Protocols" design above, in a new form. "proto" Entries NL Entries = Entries = Entry Entries = Entry SP Entries Entry = Keyword "=" Values Values = Value Values = Value "," Values Value = Int Value = Int "-" Int Int = NON_ZERO_DIGIT Int = Int DIGIT Each 'Entry' in the "proto" line indicates that the Tor relay supports one or more versions of the protocol in question. Entries must be sorted by keyword. Values must be numerically ascending within each entry. (This implies that there are no overlapping ranges.) Ranges must be represented as compactly as possible. Ints must be no more than 2^32 - 1. The semantics for each keyword must be defined in a Tor specification. Extension keywords are allowed if they begin with "x-" or "X-". Keywords are case-sensitive. During voting, authorities copy these lines immediately below the "v" lines. When a descriptor does not contain a "proto" entry, the authorities should reconstruct it using the approach described below in section A.1. They are included in the consensus using the same rules as currently used for "v" lines, if a sufficiently late consensus method is in use. 2.1. An alternative: Moving 'v' lines into microdescriptors. [[[[[ Here's an alternative: we could put "v" and "proto" lines into microdescriptors. When building microdescriptors, authorities could copy all valid "proto" entries verbatim if a sufficiently late consensus method is in use. When a descriptor does not contain a "proto" entry, the authorities should reconstruct it using the approach described below in section A.1. Tor clients that want to use "v" lines should prefer those in microdescriptors if present, and ignore those in the consensus. (Existing maintined client versions can be adapted to never look at "v" lines at all; the only versions that they still check for are ones not allowed on the network. The "v" line can be dropped from the consensus entirely when current clients have upgraded.) ]]]]] [I am rejecting this alternative for now, since proto lines should compress very well, given that the proto line is completely inferrable from the v line. Removing all the v lines from the current consensus would save only 1.7% after gzip compression.] 3. Using "protos" and "v" lines Whenever possible, clients and relays should use the list of advertised protocols instead of version numbers. Version numbers should only be used when implementing bug-workarounds for specific Tor versions. Every new feature in tor-spec.txt, dir-spec.txt, and rend-spec.txt should be gated on a particular protocol version. 4. Required protocols The consensus may contain three lines: RequiredRelayProtocols, RecommendedClientProtocols, and RequiredClientProtocols. Each has the same format as the "protos" line. To vote on these entries, a protocol/version combination is included only if it is listed by a majority of the voters. When a relay lacks a protocol listed in RequiredRelayProtocols, it must not attempt to join the network. When a client lacks a protocol listed in RecommendedClientProtocols, it should warn the user that the client is obsolete. When a client lacks a protocol listed in RequiredClientProtocols, it must not use the corresponding feature. If the corresponding feature is required to operate a Tor client, the client must must log an error message and shut down without connecting to the network, even if the cached consensus is out-of-date. This implements a "safe forward shutdown" mechanism for zombie clients. The above features should be backported to 0.2.4 and later. 5. Current protocols (See "6. Maintaining the protocol list" below for information about how I got these, and why version 0.2.4.19 comes up so often.) 5.1. "Link" The "link" protocols are those used by clients and relays to initiate and receive OR connections and to handle cells on OR connections. The "link" protocol versions correspond 1:1 to those versions. Two Tor instances can make a connection to each other only if they have at least one link protocol in common. The current "link" versions are: "1" through "4"; see tor-spec.txt for more information. All current Tor versions support "1-3"; version from 0.2.4.11-alpha and on support "1-4". Eventually we will drop "1" and "2". 5.2. "LinkAuth" LinkAuth protocols correspond to varieties of Authenticate cells used for the v3+ link protocools. The currrent version is "1". 5.3. "Relay" The "relay" protocols are those used to handle CREATE cells, and those that that handle the various RELAY cell types received after a CREATE cell. (Except, relay cells used to manage introduction and rendezvous points are managed with the "HSMid" protocols.) "1" -- supports the TAP key exchange, with all features in Tor 0.2.3. Support for CREATE and CREATED and CREATE_FAST and CREATED_FAST and EXTEND and EXTENDED. "2" -- supports the ntor key exchange, and all features in Tor 0.2.4.19. Includes support for CREATE2 and CREATED2 and EXTEND2 and EXTENDED2. 5.3. "HSMid" The "HSMid" protocols are those that handle introduction points and rendezvous points. "1" -- supports all features in Tor 0.2.4.19 5.4. "DirCache" The "DirCache" protocols are the set of documents available for download from a directory cache via BEGIN_DIR, and the set of URLs available to fetch them. (This excludes URLs for hidden service objects.) "1" -- supports all features in Tor 0.2.4.19. 5.5. "HSDir" The HSDir protocols are the set of hidden service document types that can be uploaded to, understood by, and downloaded from a tor relay, and the set of URLs available to fetch them. "1" -- supports all features in Tor 0.2.4.19. 5.6. "Desc" Describes features present or absent in descriptors. Most features in descriptors don't require a "Desc" update -- only those that need to someday be required. For example, someday clients will need to understand ed25519 identities. "1" -- supports all features in Tor 0.2.4.19. "2" -- cross-signing with onion-keys, signing with ed25519 identities. 5.7. "Microdesc" Describes features present or absent in microdescriptors. Most features in descriptors don't require a "MircoDesc" update -- only those that need to someday be required. These correspond more or less with consensus methods. "1" -- consensus methods 9 through 20. "2" -- consensus method 21 (adds ed25519 keys to microdescs). 5.8. "Cons" Describes features present or absent in consensus documents. Most features in consensus documents don't require a "Cons" update -- only those that need to someday be required. These correspond more or less with consensus methods. "1" -- consensus methods 9 through 20. "2" -- consensus method 21 (adds ed25519 keys to microdescs). 6. Maintaining the protocol lists What makes a good fit for a "protocol" type? Generally, it's a set of communications functionality that tends to upgrade in tandem, and in isolation from other parts of the Tor protocols. It tends to be functionality where it doesn't make sense to implement only part of it -- though omitting the whole thing might make sense. (Looking through our suite of protocols, you might make a case for splitting DirCache into sub-protocols.) We shouldn't add protocols for features where others can remain oblivious to their presence or absence. For example, if some directory caches start supporting a new header, and clients can safely send that header without knowing whether the directory cache will understand it, then a new protocol version is not required. Because all relays currently on the network are 0.2.4.19 or later, we can require 0.2.4.19, and use 0.2.4.19 as the minimal version so we we don't need to do code archeology to determine which number Adding new protocol types is pretty cheap, given compression. A.1. Inferring missing protos lines. The directory authorities no longer allow versions of Tor before 0.2.4.8-alpha. But right now, there is no version of Tor in the consensus before 0.2.4.19. Therefore, we should disallow versions of Tor earlier than 0.2.4.19, so that we can have the protocol list for all current Tor versions include: DirCache=1 HSDir=1 HSMid=1 Link=1-4 LinkAuth=1 Relay=1-2 For Desc, Tor versions before 0.2.7.stable should be taken to have Desc=1 and versions 0.2.7.stable or later should have Desc=1-2. For Microdesc and Cons, Tor versions before 0.2.7.stable should be taken to support version 1; 0.2.7.stable and later should have 1-2. A.2. Initial required protocols For clients we will Recommend and Require: DirCache=1 HSDir=1 Desc=1 Cons=ANYOF(9-20) Microdesc=ANYOF(9-20) HSMid=1 Link=4 LinkAuth=1 Relay=2 For relays we will Require: DirCache=1 HSDir=1 Desc=1 Cons=ANYOF(9-20) Microdesc=ANYOF(9-20) HSMid=1 Link=3-4 LinkAuth=1 Relay=1-2 A.3. Example integration with other open proposals In this appendix, I try to show that this proposal is viable by showing how it can integrate with other open proposals to avoid version-gating. I'm looking at open/draft/accepted proposals only. 140 Provide diffs between consensuses This proposal doesn't affect interoperability, though we could add a DirCache protocol version for it if we think we might want to require it someday. 164 Reporting the status of server votes Interoperability not affected; no new protocol. 165 Easy migration for voting authority sets Authority-only; no new protocol. 168 Reduce default circuit window Interoperability slightly affected; could be a new Relay protocol. 172 GETINFO controller option for circuit information 173 GETINFO Option Expansion Client/Relay interop not affected; no new protocol. 177 Abstaining from votes on individual flags Authority-only; no new protocol. 182 Credit Bucket No new protocol. 188 Bridge Guards and other anti-enumeration defenses No new protocol. 189 AUTHORIZE and AUTHORIZED cells This would be a new protocol, either a Link protocol or a new LinkAuthz protocol. 191 Bridge Detection Resistance against MITM-capable Adversaries No new protocol. 192 Automatically retrieve and store information about bridges No new protocol. 195 TLS certificate normalization for Tor 0.2.4.x Interop not affected; no new protocol. 201 Make bridges report statistics on daily v3 network status requests No new protocol. 202 Two improved relay encryption protocols for Tor cells This would be a new Relay protocol. 203 Avoiding censorship by impersonating an HTTPS server Bridge/PT only; no new protocol. 209 Tuning the Parameters for the Path Bias Defense Client behavior only; no new protocol. 210 Faster Headless Consensus Bootstrapping Client behavior only; no new protocol. 212 Increase Acceptable Consensus Age Possibly add a new DirCache protocol version to describe the "will hold older descriptors" property. 219 Support for full DNS and DNSSEC resolution in Tor New relay protocol, or new protocol class (DNS=2?) 220 Migrate server identity keys to Ed25519 Once link authentication is supported, that's a new LinkAuth protocol version. No new protocol version is required for circuit extension, since it's a backward-compatible change. 224 Next-Generation Hidden Services in Tor Adds new HSDir and HSMid protocols. 226 "Scalability and Stability Improvements to BridgeDB: Switching to a Distributed Database System and RDBMS" No new protocol. 229 Further SOCKS5 extensions Client-only; no new protocol. 233 Making Tor2Web mode faster No new protocol. 234 Adding remittance field to directory specification Could be a new protocol; or not. 237 All relays are directory servers No new protocol. 239 Consensus Hash Chaining No new protocol. 242 Better performance and usability for the MyFamily option New Desc protocol. 244 Use RFC5705 Key Exporting in our AUTHENTICATE calls Part of prop220. Also adds another LinkAuth protocol version. 245 Deprecating and removing the TAP circuit extension protocol Removes Linkauth protocol 1. Removes a Desc protocol. 246 Merging Hidden Service Directories and Introduction Points Possibly adds a new HSMid protocol. 247 Defending Against Guard Discovery Attacks using Vanguards No new protocol. 248 Remove all RSA identity keys Adds a new Desc protocol version and a new Cons protocol version; eventually removes a version of each. 249 Allow CREATE cells with >505 bytes of handshake data Adds a new Link protocol version for CREATE2V. Adds a new Relay protocol version for new EXTEND2 semantics. 250 Random Number Generation During Tor Voting No new protocol. 251 Padding for netflow record resolution reduction No new protocol. 252 Single Onion Services No new protocol. 253 Out of Band Circuit HMACs New Relay protocol. 254 Padding Negotiation New Link protocol, new Relay protocol. 255 Controller features to allow for load-balancing hidden services No new protocol. 256 Key revocation for relays and authorities New Desc protocol. 257 Refactoring authorities and taking parts offline No new protocol. 258 Denial-of-service resistance for directory authorities No new protocol. 259 New Guard Selection Behaviour No new protocol 260 Rendezvous Single Onion Services No new protocol 261 AEZ for relay cryptography New Relay protocol version. 262 Re-keying live circuits with new cryptographic material New Relay protocol version 263 Request to change key exchange protocol for handshake New Relay protocol version.

2 1

[Win32] test_util.c + test_checkdir.c
by Gisle Vanem 07 Jan '16

07 Jan '16

The src/test/test_util.c have this statement: #ifdef _WIN32 #define UTIL_TEST_NO_WIN(n, f) { #n, NULL, TT_SKIP, NULL, NULL } #define UTIL_TEST_WIN_ONLY(n, f) UTIL_TEST(n, (f)) #define UTIL_LEGACY_NO_WIN(n) UTIL_NO_WIN(n) But I fail to see where 'UTIL_NO_WIN(n)' is defined. Should not the above read: --- a/test/test_util.c 2016-01-05 13:19:07 +++ b/test/test_util.c 2016-01-05 14:36:52 @@ -4676,7 +4676,7 @@ #ifdef _WIN32 #define UTIL_TEST_NO_WIN(n, f) { #n, NULL, TT_SKIP, NULL, NULL } #define UTIL_TEST_WIN_ONLY(n, f) UTIL_TEST(n, (f)) -#define UTIL_LEGACY_NO_WIN(n) UTIL_NO_WIN(n) +#define UTIL_LEGACY_NO_WIN(n) { #n, NULL, TT_SKIP, NULL, NULL } #else #define UTIL_TEST_NO_WIN(n, f) UTIL_TEST(n, (f)) With this, my test.exe runs fine. Although 36 SKIPPED tests. Another thing regarding MSVC. In test/test_checkdir.c, <dirent.h> is included for _WIN32. MSVC does not have this header. Hence I think this patch is needed: --- a/test/test_checkdir.c 2015-08-31 13:24:33 +++ b/test/test_checkdir.c 2015-08-31 14:50:53 @@ -4,9 +4,7 @@ #include "orconfig.h" #include "or.h" -#ifdef _WIN32 -#include <direct.h> -#else +#ifndef _MSC_VER #include <dirent.h> #endif Since <direct.h> is already included in "or.h", it's not needed here too. -- --gv

2 1

Applications Team meeting today (Tuesday Jan 5) 19:00 UTC
by Mike Perry 05 Jan '16

05 Jan '16

Hi, Just a reminder that the applications team is having its monthly meeting today at 19:00 UTC in #tor-project on irc.oftc.net. We'll likely recap the State of the Onion, and generally talk about UI/UX as per usual. Please come up with suggestions if you feel there is something else we should discuss. -- Mike Perry

1 0

Re: [tor-dev] Quantum-safe Hybrid handshake for Tor
by Zhenfei Zhang 04 Jan '16

04 Jan '16

Hi all, Thanks for all the comments. Sorry I wasn't able to reply immediately. Please allow me to summarize the comments. I see mainly the following questions. 1. Quantum-safe authentication. As Yawning has pointed out, > I personally don't think that any of the PQ signature schemes are usable > for us right now, because the smallest key size for an algorithm that > isn't known to be broken is ~1 KiB (SPHINCS256), and we probably can't > afford to bloat our descriptors/micro-descriptors that much. This is also the reason we wanted to roll out QSH first and add the quantum-safe signature schemes later. We have several good candidate for quantum-safe encryption algorithms. But for signatures, they are not mature imho. Another reason that we did not include authentication is due to the attack model. As I have mentioned (but probably didn't explain very clearly) in previous email, we are facing two types of attackers. Attacker type I, passive attacker at present, who cannot break classical authentication nor encryption, record the data anyway, and decrypt it when quantum computer become available in future. Attack type II, active attacker who tries to attack the authentication while the communication is taking place. This attack will be possible in future when quantum computer arrives. But for now, they will not be successful. As we believe that there does not exist a general purpose quantum computer at present (and maybe several years away), we have time to deal with attacker type II. But the attacker type I is the real threat at the present day. Our proposal is to address this threat. 2. On NTRU vs NTRU-Prime vs R-LWE and others. The QSH is modular designed to suite any quantum-safe encryption algorithm. So we can chose any one we want for trail. And furthermore, we can also hybrid, say ECC, NTRU and R-LWE, to give a bit more confidence in case one of the quantum-safe encryption algorithm turns out to be not quantum safe, or broken. That been said, we chose NTRU for several reasons. NTRU is more mature than R-LWE from our point o view. NTRU has a full spec, a reference implementation, and is standardized by several bodies; while for R-LWE, since it enables many interesting cryptographic primitives, such as FHE, there has been many different parameter proposals, which leads to some kind of confusion as to which one should reference to. > However, if we were to go the route of using NTRU, we'd likely want to instead > use Dan Bernstein's NTRU Prime parameters, in order to eliminate some of the > inherent algebraic structure of the ideal lattice which might possibly be > exploited. [0] [1] As for NTRU-Prime, I am not aware there is a specific instantiation of this parameter sets, nor any paper that considers the security of a specific parameter set. Also, this NTRU-Prime would require some extensive scrutiny before we can use it. We are happy to roll out any above encryption algorithm as you see fit. But our proposal is mainly about the QSH approach. I think the best option for now is to buildin a QSH for Tor, with a flexible API that allows us to switch between algorithms when fit. And for now use any quantum-safe encryption algorithm that is ready to be used. After all, any QS encryption is better than nothing. 3. License I am sorry I am not familiar with the license. But my general feeling is that Security Innovation is willing to let Tor to use NTRU for free. We just need to work out the suitable license to make this happen. 4. Misc. > Post-quantum forward-secrecy is what I've been using to describe this > property. We will use this terminology. Thanks. > As I recall, the product form parameter sets are extra encumbered. > Apart from key/ciphertext size and a minor performance differential, is > there any reason to not use one of the X9.98 parameter sets (Eg: > EES613EP1) Yes we can use non-product form polynomials, if everyone agrees on it. Non-product form polynomials will make key generation and decryption a bit slower, but those cost are on the client side. It has no impact on the load of server side. > * "For 128 bits quantum security, use NTRU_EESS743EP1." should be > "For 256 bits" (Section 2.3). NTRU_EESS743EP1 provides 256 classical security and 128 bits quantum security. Please see https://eprint.iacr.org/2015/708.pdf for arguments of those security levels. > I'm a little confused about what exactly is meant by "disaster resilience" here. We will remove "disaster resilience". Those are most comments I saw. Sorry if I missed some of your comments. Please let me know if you have questions that I failed to answer. Happy new year to everyone. Cheers, Zhenfei On Sun, Jan 3, 2016 at 8:32 PM, Ryan Carboni <ryacko(a)gmail.com> wrote: > Wasn't there a transition period in migrating from RSA to ECC? > > Maybe I'm just confused. Or you are confused. But I think it is best plan > for a five or ten year transition period. > > _______________________________________________ > tor-dev mailing list > tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > >

2 2

Re: [tor-dev] tor-dev Digest, Vol 60, Issue 2
by Flipchan 04 Jan '16

04 Jan '16

How would u add quantum-safe crypto? I havent seen anyone puttin a pub lib that anyone can import tor-dev-request(a)lists.torproject.org skrev: (2 januari 2016 13:00:02 CET) >Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > >To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > >You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of tor-dev digest..." > > >Today's Topics: > > 1. Re: Quantum-safe Hybrid handshake for Tor (Ryan Carboni) > 2. Re: Quantum-safe Hybrid handshake for Tor (Yawning Angel) > > >---------------------------------------------------------------------- > >Message: 1 >Date: Fri, 1 Jan 2016 19:33:31 -0800 >From: Ryan Carboni <ryacko(a)gmail.com> >To: tor-dev(a)lists.torproject.org >Subject: Re: [tor-dev] Quantum-safe Hybrid handshake for Tor >Message-ID: > <CAO7N=i2MspE1N5eOczCyT9RCPORgUJboSOY3vUMGKL5FSzAPnw(a)mail.gmail.com> >Content-Type: text/plain; charset="utf-8" > >The first step should be replacing the long-term keys with quantum-safe >crypto. >

5 6

Go version in Gitian descriptors
by Jeremy Rand 04 Jan '16

04 Jan '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello, I noticed that it looks like Tor Project is using Go 1.4.2 to build the pluggable transports in Gitian. I'm curious why a newer version of Go isn't used. My understanding is that Go 1.4.2 (or earlier) is needed to build Go 1.5 because 1.5's source code is itself in Go. Would using Go 1.5 be as simple as building 1.4.2 in Gitian (as is done now), and then using 1.4.2 to build 1.5, and then placing 1.5 in PATH instead of 1.4.2 as is done now? Have obstacles been identified in such a configuration, or is it just that no one tried it? Cheers, - -Jeremy Rand -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWifxjAAoJEAHN/EbZ1y0686cP/3kdRz3va8dZ056j4x9k+sd7 XJsfsj9eWJ/tPxdpppPsBIoseTVNrPSCOB/cJ/bc0Xl75KGQkHFsmGusnUp6kg9L bwJK8wcNoO+s5EjjiY/Rk39UQTNJ3YKOaanvUnGXolYy4LtgUTTI7jdSWcpyZmd7 zxQVzvIXQEawgbu9SjTzXFkBQVHkZzkCDmTPcQNEPtIZambj7fFU0EMBWSmrYkVa /Lzz0bq2wMcZBpdMR07JT0WcrXNgkun48gJlBCV2SuScgs+FycmQi1dVCapwptrQ H0WeCv61XnhXTlens60gfhjrXMcUf4Xud2p71pG0N3OYrF8eOJGpSzTdnBnOFNGw IA2B+CB8u6l8PoZAqeJg0qhFjwm/BoG9o8b8WI9WJOvqDsSfo9Mjmr2lfuoxEGI7 /Uu7aZgRboFI899/xfKJUjzZpQs3hVrtpgukjNEShDBBhsZDOQUkbfzw097NttxB Nst6RW8WmRpbvFMGMrLiYrUKxGKJkzOUVaexU2C9y6y5asq3i2qotTmXpzEHkcP0 XL4KR9/JC6MeuBqSszEb2gpcQsHVsPxk7Wo1cJwLDSeS2YniZnxlpre5DjF2Exez H54PqwtXTezThc8TZuTzfJZrkOA/X5ggL7/wnpUaDrF8T9r4fn1rosP6m8PdUeOL RhVNDX3NeTjV6KywVJX4 =ELuw -----END PGP SIGNATURE-----

5 4

Re: [tor-dev] Quantum-safe Hybrid handshake for Tor
by Ryan Carboni 04 Jan '16

04 Jan '16

Wasn't there a transition period in migrating from RSA to ECC? Maybe I'm just confused. Or you are confused. But I think it is best plan for a five or ten year transition period.

1 0

Re: [tor-dev] Quantum-safe Hybrid handshake for Tor
by Ryan Carboni 04 Jan '16

04 Jan '16

> > We had a GSOC project to produce "consensus diffs", so that clients could download the differences between each consensus each hour, rather than downloading a full consensus (~1.5MB). > > It showed some great results, but still needs a little work before we merge it.https://trac.torproject.org/projects/tor/ticket/13339 <https://trac.torproject.org/projects/tor/ticket/13339> > > Still, one doesn't need to download the full consensus. 3/16ths of the consensus could work and would be more beneficial for low bandwidth clients. If Tor became popular, it wouldn't be usable. Right now it consumes the bandwidth of several colleges. If Tor got anywhere near consuming the bandwidth and popularity of a major datacenter (such as the five million user botnet spike of 2013), Tor wouldn't work too well. Facebook has hundreds of millions of active users. A few billion people live under a dictatorship. How well would Tor work if it's infrastructure and usage was scaled up a hundred times? It's not an immediate issue, but if one was to add quantum cryptography, this would be one of the issues that needs to be addressed (as it is one of many long-term issues). nb: Migrating to X448 would possibly hold up longer than Curve25519 > would since it requires a bigger quantum computer. But performance > isn't that great without using vectorization. > > Given the slow time it takes to roll things out, a timeline which begins with trusted directory keys include post-quantum crypto first, and which ends in enabling clients to use post-quantum crypto would be best.

3 2

Re: [tor-dev] Quantum-safe Hybrid handshake for Tor
by Ryan Carboni 03 Jan '16

03 Jan '16

And yet the NSA is moving to prime numbers. A large public key isn't a very good reason to not adopt quantum-safe crypto, it just means that it requires having the Tor project to be able to scale to a larger degree. I suggest hash tables, a percentage of which are pseudorandomly downloaded. Otherwise the Tor project won't scale to 10x the relays ... even ignoring quantum cryptography.

4 4

trac account 'proper' - Full Disclosure
by Patrick Schleizer 02 Jan '16

02 Jan '16

Hi, I've learned that using the trac account 'proper' [1] has lead to some confusion and/or curiosity. That account name is really old, it's been created while I was still anonymous. Tickets created by the trac account are created by me, Patrick Schleizer, nickname adrelanos. The arguments made in these tickets should stand or fall independently of my identity. Those are either of interest for me personally, for my general contribution to Libre Software and/or for the Whonix project. >From now, I will be using my new trac account 'adrelanos'. Cheers, Patrick [1] https://trac.torproject.org/projects/tor/query?reporter=proper&col=id&col=s…

2 1

Re: [tor-dev] Quantum-safe Hybrid handshake for Tor
by Ryan Carboni 02 Jan '16

02 Jan '16

The first step should be replacing the long-term keys with quantum-safe crypto.

2 1

Proposal 262: Re-keying live circuits with new cryptographic material
by Nick Mathewson 29 Dec '15

29 Dec '15

[Proposal 261 probably needs this.] Filename: 262-rekey-circuits.txt Title: Re-keying live circuits with new cryptographic material Author: Nick Mathewson Created: 28 Dec 2015 Status: Open 1. Summary and Motivation Cryptographic primitives have an upper limit of how much data should be encrypted with the same key. But currently Tor circuits have no upper limit of how much data they will deliver. While the upper limits of our AES-CTR crypto is ridiculously high (on the order of hundreds of exabytes), the AEZ crypto we're considering suggests we should rekey after the equivalent in cells after around 280 TB. 280 TB is still high, but not ridiculously high. So in this proposal I explain a general mechanism for rekeying a circuit. We shouldn't actually build this unless we settle on 2. RELAY_REKEY cell operation To rekey, the circuit initiator ("client") can send a new RELAY_REKEY cell type: struct relay_rekey { u16 rekey_method IN [0, 1]; u8 rekey_data[]; } const REKEY_METHOD_ACK = 0; const REKEY_METHOD_SHAKE128_CLIENT = 1; This cell means "I am changing the key." The new key material will be derived from SHAKE128 of the aez_key concatenated with the rekey_data field, to fill a new shake_output structure. The client should set rekey_data at random. After sending one of these RELAY_REKEY cells, the client uses the new aez_key to encrypt all of its data to this hop, but retains the old aez_key for decrypting the data coming back from the relay. When the relay receives a RELAY_REKEY cell, it sends a RELAY_REKEY cell back towards the client, with empty rekey_data, and relay_method==0, and then updates its own key material for all additional data it sends and receives to the client. When the client receives this reply, it can discard the old AEZ key, and begin decrypting subsequent inbound cells with the new key. So in summary: the client sends a series of cells encrypted with the old key, and then sends a REKEY cell, followed by relay cells encrypted with the new key: OldKey[data data data ... data rekey] NewKey[data data data...] And after the server receives the REKEY cell, it stops sending relay cells encrypted with the old keys, sends its own REKEY cell with the ACK method, and starts sending cells encrypted with the new key. REKEY arrives somewhere in here I V OldKey[data data data data rekey-ack] NewKey[data data data ...] 2.1. Supporting other cryptography types Each relay cipher must specify its own behavior in the presence of a REKEY cell of each type that it supports. In general, the behavior of method 1 ("shake128-client") is "regenerate keys as if we were calling the original KDF after a CREATE handshake, using SHAKE128 on our current static key material and on a 32-byte random input." The behavior of any unsupported REKEY method must be to close the circuit with an error. The AES-CTR relay cell crypto doesn't support rekeying. See 3.2 below if you disagree. 2.2. How often to re-key? Clients should follow a deterministic algorithm in deciding when to re-key, so as not to leak client differences. This algorithm should be type-specific. For AEZ, I recommend that clients conservatively rekey every 2**32 cells (about 2 TB). And to make sure that this code actually works, the schedule should be after 2**15 cells, and then every 2**32 cells thereafter. It may be beneficial to randomize these numbers. If so, let's try subtracting between 0 and 25% at random. 2.3. How often to allow re-keying? We could define a lower bound to prevent too-frequent rekeying. I'm not sure I see the point here; the process described above is not that costly. 3. Alternative designs 3.1. Should we add some public key cryptography here? We could change the body of a REKEY cell and its ack to be more like CREATE/CREATED. Then we'd have to add a third step from the client to the server to acknowledge receipt of the 'CREATED' cell and changing of the key. So, what would this added complexity and computational load buy us? It would defend against the case where an adversary had compromised the shared key material for a circuit, but was not able to compromise the rekey process. I'm not sure that this is reasonable; the likeliest cases I can think of here seem to be "get compromised, stay compromised" for a circuit. 3.2. Hey, could we use this for forward secrecy with AES-CTR? We could, but the best solution to AES-CTR's limitations right now is to stop using our AES-CTR setup. Anything that supports REKEY will also presumably support AEZ or something better. 3.3. We could upgrade ciphers with this! Yes we could. We could define this not only to change the key, but to upgrade to a better ciphersuite. For example, we could start by negotiating AES-CTR, and then "secretly" upgrade to AEZ. I'm not sure that's worth the complexity, or that it would really be secret in the presence of traffic analysis.

2 1

Proposal 261: AEZ for relay cryptography
by Nick Mathewson 29 Dec '15

29 Dec '15

[Okay, I'm pulling this out of draft stage. Still not sure this is the right primitive, but I'm pretty sure this would be about the right way to do it.] Filename: 261-aez-crypto.txt Title: AEZ for relay cryptography Author: Nick Mathewson Created: 28 Oct 2015 Status: Open 0. History I wrote the first draft of this around October. This draft takes a more concrete approach to the open questions from last time around. 1. Summary and preliminaries This proposal describes an improved algorithm for circuit encryption, based on the wide-block SPRP AEZ. I also describe the attendant bookkeeping, including CREATE cells, and several variants of the proposal. For more information about AEZ, see http://web.cs.ucdavis.edu/~rogaway/aez/ For motivations, see proposal 202. 2. Specifications 2.1. New CREATE cell types. We add a new CREATE cell type that behaves as an ntor cell but which specifies that the circuit will be created to use this mode of encryption. [TODO: Can/should we make this unobservable?] The ntor handshake is performed as usual, but a different PROTOID is used: "ntor-curve25519-sha256-aez-1" To derive keys under this handshake, we use SHAKE256 to derive the following output: struct shake_output { u8 aez_key[48]; u8 chain_key[32]; u8 chain_val_forward[16]; u8 chain_val_backward[16]; }; The first two two fields are constant for the lifetime of the circuit. 2.2. New relay cell payload We specify the following relay cell payload format, to be used when the exit node circuit hop was created with the CREATE format in 2.1 above: struct relay_cell_payload { u32 zero_1; u16 zero_2; u16 stream_id; u16 length IN [0..498]; u8 command; u8 data[498]; // payload_len - 11 }; Note that the payload length is unchanged. The fields are now rearranged to be aligned. The 'recognized' and 'length' fields are replaced with zero_1, zero_2, and the high 7 bits of length, for a minimum of 55 bits of unambigious verification. (Additional verification can be done by checking the other fields for correctness; AEZ users can exploit plaintext redundancy for additional cryptographic checking.) When encrypting a cell for a hop that was created using one of these circuits, clients and relays encrypt them using the AEZ algorithm with the following parameters: Let Chain denote chain_val_forward if this is a forward cell or chain_val_backward otherwise. tau = 0 # We set tau=0 because want no per-hop ciphertext expansion. Instead # we use redundancy in the plaintext to authenticate the data. Nonce = struct { u64 cell_number; u8 is_forward; u8 is_early; } # The cell number is the number of relay cells that have # traveled in this direction on this circuit before this cell. # ie, it's zero for the first cell, two for the second, etc. # # is_forward is 1 for outbound cells, 0 for inbound cells. # is_early is 1 for cells packaged as RELAY_EARLY, 0 for # cells packaged as RELAY. # # Technically these two values would be more at home in AD # than in Nonce; but AEZ doesn't actually distinguish N and AD # internally. Define CELL_CHAIN_BYTES = 32 AD = [ XOR(prev_plaintext[:CELL_CHAIN_BYTES], prev_ciphertext[:CELL_CHAIN_BYTES]), Chain ] # Using the previous cell's plaintext/ciphertext as additional data # guarantees that any corrupt ciphertext received will corrupt the # plaintext, which will corrupt all future plaintexts. Set Chain = AES(chain_key, Chain) xor Chain. # This 'chain' construction is meant to provide forward # secrecy. Each chain value is replaced after each cell with a # (hopefully!) hard-to-reverse construction. This instantiates a wide-block cipher, tweaked based on the cell index and direction. It authenticates part of the previous cell's plaintext, thereby ensuring that if the previous cell was corrupted, this cell will be unrecoverable. 3. Design considerations 3.1. Wide-block pros and cons? See proposal 202, section 4. 3.2. Given wide-block, why AEZ? It's a reasonably fast probably secure wide-block cipher. In particular, it's performance-competitive with AES_CTR, and far better than what we're doing now. See performance appendix. It seems secure-ish too. Several cryptographers I know seem to think it's likely secure enough, and almost surely at least as good as AES. [There are many other competing wide-block SPRP constructions if you like. Many require blocks be an integer number of blocks, or aren't tweakable. Some are slow. Do you know a good one?] 3.3. Why _not_ AEZ? There are also some reasons to consider avoiding AEZ, even if we do decide to use a wide-block cipher. FIRST it is complicated to implement. As the specification says, "The easiness claim for AEZ is with respect to ease and versatility of use, not implementation." SECOND, it's still more complicated to implement well (fast, side-channel-free) on systems without AES acceleration. We'll need to pull the round functions out of fast assembly AES, which is everybody's favorite hobby. THIRD, it's really horrible to try to do it in hardware. FOURTH, it is comparatively new. Although several cryptographers like it, and it is closely related to a system with a security proof, you never know. FIFTH, something better may come along. 4. Alternative designs 4.1. Two keys. We could have a separate AEZ key for forward and backward encryption. This would use more space, however. 4.2. Authenticating things differently In computing the AD, we could replace xor with concat. In computing the AD, we could replace CELL_CHAIN_BYTES with 16, or 509. (Another thing we might dislike about the current proposal is that it appears to requires us to remember 32 bytes of plaintext until we get another cell. But that part is fixable: note that in the structure of AEZ, the AD is processed in the AEZ-hash() function, and then no longer used. We can compute the AEZ-hash() to be used for the next cell after each cell is en/de crypted.) 4.3. Other hashes. We could update the ntor definition used in this to use a better hash than SHA256 inside. 4.4. Less predictable plaintext. A positively silly option would be to reserve the last X bytes of each relay cell's plaintext for random bytes, if they are not used for payload. This might help a little, in a really doofy way. A.1. Performance notes: memory requirements Let's ignore Tor overhead here, but not openssl overhead. IN THE CURRENT PROTOCOL, the total memory required at each relay is: 2 sha1 states, 2 aes states. Each sha1 state uses 96 bytes. Each aes state uses 244 bytes. (Plus 32 bytes counter-mode overhead.) This works out to 704 bytes at each hop. IN THE PROTOCOL ABOVE, using an optimized AEZ implementation, we'll need 128 bytes for the expanded AEZ key schedule. We'll need another 244 bytes for the AES key schedule for the chain key. And there's 32 bytes of chaining values. This gives us 404 bytes at each hop, for a savings of 42%. If we used separate AES and AEZ keys in each direction, we would be looking at 776 bytes, for a space increase of 10%. A.2. Performance notes: CPU requirements on AESNI hosts The cell_ops benchmark in bench.c purports to tell us how long it takes to encrypt a tor cell. But it wasn't really telling the truth, since it only did one SHA1 operation every 2^16 cells, when entries and exits really do one SHA1 operation every end-to-end cell. I expanded it to consider the slow (SHA1) case as well. I ran this on my friendly OSX laptop (2.9 GHz Intel Core i5) with AESNI support: Inbound cells: 169.72 ns per cell. Outbound cells: 175.74 ns per cell. Inbound cells, slow case: 934.42 ns per cell. Outbound cells, slow case: 928.23 ns per cell. Note that So for an n-hop circuit, each cell does the slow case and (n-1) fast cases at the entry; the slow case at the exit, and the fast case at each middle relay. So for 3 hop circuits, the total current load on the network is roughly 425 ns per hop, concentrated at the exit. Then I started messing around with AEZ benchmarks, using the aesni-optimized version of AEZ on the AEZ website. (Further optimizations are probably possible.) For the AES256, I used the usual aesni-based aes construction. Assuming xor is free in comparison to other operations, and CELL_CHAIN_BYTES=32, I get roughly 270 ns per cell for the entire operation. If we were to pick CELL_CHAIN_BYTES=509, we'd be looking at around 303 ns per cell. If we were to pick CELL_CHAIN_BYTES=509 and replace XOR with CONCAT, it would be around 355 ns per cell. If we were to keep CELL_CHAIN_BYTES=32, and remove the AES256-chaining, I see values around 260 ns per cell. (This is all very spotty measurements, with some xors left off, and not much effort done at optimization beyond what the default optimized AEZ does today.) A.3. Performance notes: what if we don't have AESNI? Here I'll test on a host with sse2 and ssse3, but no aesni instructions. From Tor's benchmarks I see: Inbound cells: 1218.96 ns per cell. Outbound cells: 1230.12 ns per cell. Inbound cells, slow case: 2099.97 ns per cell. Outbound cells, slow case: 2107.45 ns per cell. For a 3-hop circuit, that gives on average 1520 ns per cell. [XXXX Do a real benchmark with a fast AEZ backend. First, write one. Preliminary results are a bit disappointing, though, so I'll need to invetigate alternatives as well.]

2 1

RFC: AEZ for relay cryptography, v2
by Nick Mathewson 29 Dec '15

29 Dec '15

[This is an improvement over my last draft in this area; it makes concrete proposals about forward secrecy and chaining, and tries to start getting performance numbers for some platforms. I still need to compute plausible performance numbers on non-aesni platforms, but I might not get to that immediately. For the last draft, see https://www.mail-archive.com/tor-dev@lists.torproject.org/msg07104.html .] Filename: xxx-aez-relay.txt Title: AEZ for relay cryptography Author: Nick Mathewson Created: 13 Oct 2015 Last-changed: 29 Nov 2015 Status: Draft 0. History I wrote the first draft of this around October. This draft takes a more concrete approach to the open questions from last time around. 1. Summary and preliminaries This proposal describes an improved algorithm for circuit encryption, based on the wide-block SPRP AEZ. I also describe the attendant bookkeeping, including CREATE cells, and several variants of the proposal. For more information about AEZ, see http://web.cs.ucdavis.edu/~rogaway/aez/ For motivations, see proposal 202. 2. Specifications 2.1. New CREATE cell types. We add a new CREATE cell type that behaves as an ntor cell but which specifies that the circuit will be created to use this mode of encryption. [TODO: Can/should we make this unobservable?] The ntor handshake is performed as usual, but a different PROTOID is used: "ntor-curve25519-sha256-aez-1" To derive keys under this handshake, we use SHAKE128 to derive the following output: struct hkdf_output { u8 aez_key[48]; u8 chain_key[32]; u8 chain_val_forward[16]; u8 chain_val_backward[16]; }; The first two two fields are constant for the lifetime of the circuit. 2.2. New relay cell payload We specify the following relay cell payload format, to be used when the exit node circuit hop was created with the CREATE format in 2.1 above: struct relay_cell_payload { u32 zero_1; u16 zero_2; u16 stream_id; u16 length IN [0..498]; u8 command; u8 data[498]; // payload_len - 11 }; Note that the payload length is unchanged. The fields are now rearranged to be aligned. The 'recognized' and 'length' fields are replaced with zero_1, zero_2, and the high 7 bits of length, for a minimum of 55 bits of unambigious verification. (Additional verification can be done by checking the other fields for correctness; AEZ users can exploit plaintext redundancy for additional cryptographic checking.) When encrypting a cell for a hop that was created using one of these circuits, clients and relays encrypt them using the AEZ algorithm with the following parameters: Let Chain denote chain_val_forward if this is a forward cell or chain_forward_backward otherwise. tau = 0 # We set tau=0 because want no per-hop ciphertext expansion. Instead # we use redundancy in the plaintext to authenticate the data. Nonce = struct { u64 cell_number; u8 is_forward; u8 is_early; } # The cell number is the number of relay cells that have # traveled in this direction on this circuit before this cell. # ie, it's zero for the first cell, two for the second, etc. # # is_forward is 1 for outbound cells, 0 for inbound cells. # is_early is 1 for cells packaged as RELAY_EARLY, 0 for # cells packaged as RELAY. # # Technically these two values would be more at home in AD # than in Nonce; but AEZ doesn't actually distinguish N and AD # internally. Define CELL_CHAIN_BYTES = 32 AD = [ XOR(prev_plaintext[:CELL_CHAIN_BYTES], prev_ciphertext[:CELL_CHAIN_BYTES]), Chain ] # Using the previous cell's plaintext/ciphertext as additional data # guarantees that any corrupt ciphertext received will corrupt the # plaintext, which will corrupt all future plaintexts. Set Chain = AES256(chain_key, Chain) xor Chain. # This 'chain' construction is meant to provide forward # secrecy. Each chain value is replaced after each cell with a # (hopefully!) hard-to-reverse construction. This instantiates a wide-block cipher, tweaked based on the cell index and direction. It authenticates part of the previous cell's plaintext, thereby ensuring that if the previous cell was corrupted, this cell will be unrecoverable. 3. Design considerations 3.1. Wide-block pros and cons? See proposal 202, section 4. 3.2. Given wide-block, why AEZ? It's a reasonably fast probably secure wide-block cipher. In particular, it's performance-competitive with AES_CTR, and far better than what we're doing now. See performance appendix. It seems secure-ish too. Several cryptographers I know seem to think it's likely secure enough, and almost surely at least as good as AES. [There are many other competing wide-block SPRP constructions if you like. Many require blocks be an integer number of blocks, or aren't tweakable. Some are slow. Do you know a good one?] 3.3. Why _not_ AEZ? There are also some reasons to consider avoiding AEZ, even if we do decide to use a wide-block cipher. FIRST it is complicated to implement. As the specification says, "The easiness claim for AEZ is with respect to ease and versatility of use, not implementation." SECOND, it's still more complicated to implement well (fast, side-channel-free) on systems without AES acceleration. We'll need to pull the round functions out of fast assembly AES, which is everybody's favorite hobby. THIRD, it's really horrible to try to do it in hardware. FOURTH, it is comparatively new. Although several cryptographers like it, and it is closely related to a system with a security proof, you never know. FIFTH, something better may come along. 4. Alternative designs 4.1. Two keys. We could have a separate AEZ key for forward and backward encryption. This would use more space, however. 4.2. Authenticating things differently In computing the AD, we could replace xor with concat. In computing the AD, we could replace CELL_CHAIN_BYTES with 16, or 509. (Another thing we might dislike about the current proposal is that it appears to requires us to remember 32 bytes of plaintext until we get another cell. But that part is fixable: note that in the structure of AEZ, the AD is processed in the AEZ-hash() function, and then no longer used. We can compute the AEZ-hash() to be used for the next cell after each cell is en/de crypted.) 4.3. A forward-secure variant. We might want the property that after every cell, we can forget some secret that would enable us to decrypt that cell if we saw it again. One way to do this, at a little extra expense, is to keep a 16 or 32 byte 'chaining' value that changes after each cell. The initial chaining value in each direction would be another output of the HKDF. We could use it as an extra AD for the AEZ encryption. To update the chaining value, we need a one-way function. One option would be your-favorite-hash-function; blake2b isn't _that_ bad, right? We could also try to XOR it with a function of some hidden value from AEZ: E(S,-1,?) is promising, but it would require that we get our hands inside of our AEZ implementation. Also it would require a real cryptographer to come up with it. :) A more severe option is to update the entire key after each cell. This would conflict with 4.1 above, and cost us a bit more. A positively silly option would be to reserve the last X bytes of each relay cell's plaintext for random bytes, if they are not used for payload. This would help forward secrecy a little, in a really doofy way. Any other ideas? 4.4. Other hashes. We could update the ntor definition used in this to use a better hash than SHA256 inside A.1. Performance notes: memory requirements Let's ignore Tor overhead here, but not openssl overhead. IN THE CURRENT PROTOCOL, the total memory required at each relay is: 2 sha1 states, 2 aes states. Each sha1 state uses 96 bytes. Each aes state uses 244 bytes. (Plus 32 bytes counter-mode overhead.) This works out to 704 bytes at each hop. IN THE PROTOCOL ABOVE, using an optimized AEZ implementation, we'll need 128 bytes for the expanded AEZ key schedule. We'll need another 244 bytes for the AES key schedule for the chain key. And there's 32 bytes of chaining values. This gives us 404 bytes at each hop, for a savings of 42%. If we used separate AES and AEZ keys in each direction, we would be looking at 776 bytes, for a space increase of 10%. A.2. Performance notes: CPU requirements on AESNI hosts The cell_ops benchmark in bench.c purports to tell us how long it takes to encrypt a tor cell. But it wasn't really telling the truth, since it only did one SHA1 operation every 2^16 cells, when entries and exits really do one SHA1 operation every end-to-end cell. I expanded it to consider the slow (SHA1) case as well. I ran this on my friendly OSX laptop (2.9 GHz Intel Core i5) with AESNI support: Inbound cells: 169.72 ns per cell. Outbound cells: 175.74 ns per cell. Inbound cells, slow case: 934.42 ns per cell. Outbound cells, slow case: 928.23 ns per cell. Note that So for an n-hop circuit, each cell does the slow case and (n-1) fast cases at the entry; the slow case at the exit, and the fast case at each middle relay. So for 3 hop circuits, the total current load on the network is roughly 425 ns per hop, concentrated at the exit. Then I started messing around with AEZ benchmarks, using the aesni-optimized version of AEZ on the AEZ website. (Further optimizations are probably possible.) For the AES256, I used the usual aesni-based aes construction. Assuming xor is free in comparison to other operations, and CELL_CHAIN_BYTES=32, I get roughly 270 ns per cell for the entire operation. If we were to pick CELL_CHAIN_BYTES=509, we'd be looking at around 303 ns per cell. If we were to pick CELL_CHAIN_BYTES=509 and replace XOR with CONCAT, it would be around 355 ns per cell. If we were to keep CELL_CHAIN_BYTES=32, and remove the AES256-chaining, I see values around 260 ns per cell. (This is all very spotty measurements, with some xors left off, and not much effort done at optimization beyond what the default optimized AEZ does today.) A.3. Performance notes: what if we don't have AESNI? Here I'll test on a host with sse2 but no aesni instructions. [....XXXX writeme ....]

3 5

Scaling Tor Metrics, Round 2
by Karsten Loesing 28 Dec '15

28 Dec '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi everyone, I posted some thoughts on Scaling Tor Metrics [0] almost two weeks ago and received very useful feedback from George, David, Thomas, and Letty. Thanks for that! In fact, this is such a big topic and this was so much feedback that I decided not to respond to every idea individually but instead start over and suggest a new plan that incorporates all feedback I saw. Otherwise, I'd be worried that we'd lose ourselves in the details and miss the big picture. Maybe this also makes this topic somewhat easier to follow and respond to, which I hope many people will do. The problem to solve is still that the Tor Metrics website has a huge product backlog and that we want it to remain "the primary place to learn interesting facts about the Tor network", either by making it better and bigger internally, or by adding more ways to let others contribute to it externally. - From the feedback in round 1, I observe three major areas where we need to improve Tor Metrics: 1 Metrics frontend 2 Metrics backend 3 External contributions There are low-hanging fruit in each area, but there are many fruit overall and some are hanging higher than we might think. I'll go through them area by area and assign numbers to the tasks. 1 Metrics frontend The frontend is the part of the Tor Metrics website that takes pre-aggregated data in the .csv format as input and produces somewhat interactive visualizations. The current frontend uses Java servlets/JSP as web framework and R/ggplot2 for graphs. This code is hard to extend, even for me, and the result isn't pretty, but therefore the website doesn't require any JavaScript. So, one task would be: #1 decide whether we can still ignore JavaScript and what it has to offer. I agree that D3.js is cool, I even used it myself in the past, though I know very little about it. This decision would mean that we develop new visualizations in D3.js and phase out the existing R/ggplot2 visualizations one by one. This is a tough decision, but one with a lot of potential. I understand how we're excited about this as developers, but I'd want to ask Metrics users about this first. Another task would be: #2 website redesign. In fact, what you see on the website right now is a redesign half-way through. Believe me, the website was even less readable a year or two ago, and this statement alone tells you how slow things are moving. But the remaining step is just to replace the start page with a gallery, well, and to apply a Bootstrap design to everything, because why not. One challenge here is that the current graphs all look quite similar making them hard to distinguish in a gallery, but that's probably still more useful than putting text there. It sounds like Letty and Thomas might be willing to help out with this, which would be great. Yet another task would be: #3 replace the website framework with something more recent. This can be something simple, as long as it supports some basic filtering and maybe searching on the start page. I'd say let's pick something Python-based here. However, maybe we should first replace the existing graphs that are deeply tied into the current website framework. If we switch to D3.js and have replaced all existing graphs, this switch to a new website framework will hurt a lot less. Another high-hanging fruit would be: #4 build something like Thomas' Visionion, where users can create generic visualizations on the fly. This is an exciting idea, really, but I think we have to accept that it's out of reach for now. 2 Metrics backend The backend of Tor Metrics consists of a bunch of programs that run once per day to fetch new data from CollecTor and produce a set of .csv files for the frontend. There are no strict requirements to languages and databases, as long as tools are available in Debian stable. Some programs use PostgreSQL, but most of them just use files. Ironically, it's the database-based tools that have major performance problems, whereas the file-based ones work just fine. Most programs are written in Java, very few in Python. One rather low-hanging fruit would be: #5 document backend programs and say what's required to add one more to the bunch. The biggest challenge in writing such a program is that it needs to stay reasonably fast even over the next couple of years and even if the network doubles or triples in size. I started documenting things a while ago, but got distracted by other things. I wouldn't mind help. Another rather low-hanging fruit would be: #6 use Big Data to produce pre-aggregated data for the frontend. As said before, it doesn't matter whether a backend program uses files or PostgreSQL or another database engine. What matters is that it reads data from CollecTor and produces a data that the frontend can use. This could be a CSV or JSON file. We should probably pick the next visualization project as test case for applying Big Data tools, or we could rewrite one that needs to be rewritten for performance reasons anyway. Here's a not-so-low-hanging fruit for the backend: #7 have the backend provide an API to the frontend. This is potentially more difficult. The part that I'm very much afraid of is performance. It's just too easy to build a tool that performs reasonable during testing but that can't handle the load of 10 or 100 people looking at a frontend visualization at once. In particular, it's easy to build an API that works just fine and then add another feature that looks harmless, which later turns out to hurt performance a lot. I'd say postpone. 3 External contributions Most of the discussion in round 1 circled around how external contributions are great, but that they need to be handled with care. I understand the risks here, and I think I'll postpone the part where we're including externally developed websites after "redressing" them. Let's instead try to either keep those contributions as external links or properly integrate them into Tor Metrics. The lowest-hanging fruit here is: #8 keep adding links to external websites as we already do, assuming that we clearly mark these contributions as external. Another low-hanging fruit is: #9 add static data or static graphs. I heard a single +1, but I guess once we have concrete suggestions what data or graphs to add, we'll have more discussion. That's fine. One important and still somewhat low-hanging fruit is: #10 give external developers more support when developing visualizations that could later be added to Metrics. This requires better documentation, but it also requires making it easier to install Tor Metrics locally and test new additions before submitting them. The latter is a good goal, but we're not there yet. The documentation part doesn't seem crazy though. David, if you don't mind being the guinea pig yet once more, I'd want to try this out with your latest visualizations. This is pending on the JavaScript decision though. Now to the higher-hanging fruit: #11 Build or adapt a tool like Munin to prototype new visualizations quickly. I didn't fully understand how Munin makes it easier to prototype a visualization than just writing a Python/Stem script for the data pre-processing and using any graphing engine for the visualization. But maybe there's something to learn here. Still, this seems like a quite high goal at the moment. Another one: #12 provide a public API like Onionoo but for Metrics data. This seems somewhat out of reach for the moment. It's a cool idea, but doing it right is not at all trivial. I'd want to put this to the lower end of the list. Another high-hanging fruit: #13 adapt the Gist idea where external contributors write some code that magically turn into new visualizations. I think that most new visualizations require writing backend code to aggregate different parts of the available data or to aggregate the same data differently. It's a neat idea that external contributors could simply write some code that then magically turns into new visualizations. But I think it's a long way until we're there. 4 Summary Here's the list of low-hanging fruit: #1 decide whether we can still ignore JavaScript #2 website redesign #5 document backend programs #6 use Big Data to produce pre-aggregated data #8 keep adding links to external websites #9 add static data or static graphs #10 give external developers more support And here are the tasks that I think we should postpone a bit longer: #3 replace the website framework #4 build something like Thomas' Visionion #7 have the backend provide an API to the frontend #11 Build or adapt a tool like Munin #12 provide a public API like Onionoo but for Metrics data #13 adapt the Gist idea How does this sound? What did I miss (sorry!), on a high level without going into all the details just yet? Who wants to help? All the best, Karsten [0] https://lists.torproject.org/pipermail/tor-dev/2015-November/009983.html -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWZFnNAAoJEJD5dJfVqbCrowMIAIcx5kd7BfHN9HsgEj+JZ6e6 YsIUfrrCcTeMlZyHHG9amWodac+spE11fZCAkZ2B2gYf8I6KT23pex4w23WCVlxo 0qq5pZSADedAYBsocEhZLh2bZyzIhEPueGJW2/HeyW2Smd9T+YJ8hA1kkoebX/Xv xb/2051bc48sQW767gst3ClTv4va8+24pKytJxvfRYDrs3xv3VxPlyCvLsrgz9ji pnxd2C+Syo4sT3vHFb9JNhvd1ZaEmqjXBzywQ2SNnOqZNQE77zfvB766FFSJEg6a MMgwt0RcjElTnZRZR2utA2WLYzJDiof4ULfw51xoYYcT6kmGQw9IOqOuzgBvSr4= =Yo+I -----END PGP SIGNATURE-----

4 12

metrics-lib 1.1.0 is released
by Karsten Loesing 28 Dec '15

28 Dec '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello devs, I just released metrics-lib 1.1.0: https://dist.torproject.org/descriptor/1.1.0/ - From the change log: # Changes in version 1.1.0 - 2015-12-28 * Medium changes - Parse flag thresholds in bridge network statuses, and parse the "ignoring-advertised-bws" flag threshold in relay network status votes. - Support parsing of .xz-compressed tarballs using Apache Commons Compress and XZ for Java. Applications only need to add XZ for Java as dependency if they want to parse .xz-compressed tarballs. - Introduce a new ExitList.Entry type for exit list entries instead of the ExitListEntry type which is now deprecated. The main difference between the two is that ExitList.Entry can hold more than one exit address and scan time which were previously parsed as multiple ExitListEntry instances. - Introduce four new types to distinguish between relay and bridge descriptors: RelayServerDescriptor, RelayExtraInfoDescriptor, BridgeServerDescriptor, and BridgeExtraInfoDescriptor. The existing types, ServerDescriptor and ExtraInfoDescriptor, are still usable and will not be deprecated, because applications may not care whether a relay or a bridge published a descriptor. - Support Ed25519 certificates, Ed25519 master keys, SHA-256 digests, and Ed25519 signatures thereof in server descriptors and extra-info descriptors, and support Ed25519 master keys in votes. - Include RSA-1024 signatures of SHA-1 digests of extra-info descriptors, which were parsed and discarded before. - Support hidden-service statistics in extra-info descriptors. - Support onion-key and ntor-onion-key cross certificates in server descriptors. * Minor changes - Start using Java 7 features like the diamond operator and switch on String, and use StringBuilder correctly in many places. Thanks to iwakeh for driving many of the changes contained in this release and to thms for spotting missing features that are now included. Happy descriptor parsing! All the best, Karsten -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWgPSdAAoJEJD5dJfVqbCrYdEIAI+zZzM8iDRplpmlJ62ehkGi r07yo14VfFHxa2od+abh8A6pOM+hdwJjeYOQ1dz/HGf6xIl7GmdDt1Tt+KMM0bAc F7r+1Yi4eo8G7ZqNIDIsJdey59orMLjpDafUFuXYv4RM22ol5zj5b3Hj94OIQM9A +gZXReoCxJQS+WxOfsMVP2yPjdQsoptZCJQjIJMiu5fL0lQF3tkCLuJ6r5teLnLj 1VGx+YsJWKE8czM42yfpY4/SmnTkUtjmYO2Ivq/WOMuCh7RcbBSGkr09PB7fxrHT ZmoML92TqI0+zwEY7yI52pjn+Q+EX5BY8qvKeCen14b/BwKCIKXCRqaye15RTX4= =VkEL -----END PGP SIGNATURE-----

1 0

Traffic correlation attacks on Hidden Services
by Virgil Griffith 24 Dec '15

24 Dec '15

I've been looking into simple graph-theoretic metrics for Roster to quantifying Tor's susceptibility to traffic correlation attacks, mostly using BGPStream, https://bgpstream.caida.org/ . All of the academic literature I've read talks about the risk to Tor users of an AS being in the path between client <-> guard + exit <-> destination. Questions: (1) To ensure I'm not measuring the wrong thing, can someone be more specific on the correlation attack scenario for Tor hidden services? (2) Just guessing, but would be it be the same but replace "exit <-> destination" with: "HS server <-> HS guard" ? (3) If yes to (2), the natural solution is simply to install a Tor relay on the HS server itself so that there's no ASpath between the two? Comments greatly appreciated. I'm not an internet routing expert and I want to ensure Roster is incentivizing the right things to harden the network. -V

2 1