July 2017 - tor-dev - lists.torproject.org

non-anonymous ephemeral onion services with stem
by Micah Lee 30 Jan '18

30 Jan '18

The stem documentation for create_ephemeral_hidden_service [1] says: "Changed in version 1.5.0: Added support for non-anonymous services." But I can't figure out to actually use this feature. There doesn't seem to be a new argument to say if you want your onion service to be non-anonymous. It also says, "Changed in version 1.5.0: Added the basic_auth argument." But there's a new basic_auth argument you can pass into the function to use that. [1] https://stem.torproject.org/api/control.html#stem.control.Controller.create…

5 12

onionoo.tpo stuck at 2017-06-12 12:00?
by nusenu 26 Oct '17

26 Oct '17

Hi, just wanted to let you know that the delta between relays_published and current time is unusually high. https://onionoo.torproject.org/details?limit=0 {"version":"4.0", "relays_published":"2017-06-12 12:00:00", -- https://mastodon.social/@nusenu https://twitter.com/nusenu_

2 4

Pluggable Transports 2.0 Specification, Draft 2
by Brandon Wiley 10 Oct '17

10 Oct '17

Attached is the second draft of the Pluggable Transport 2.0 Specification. If you have feedback on this draft, please send me your comments by July 20.

4 11

Need help on my new work related to TOR Project
by Nima Azizzadeh 25 Jul '17

25 Jul '17

Hello My name is Nima Azizzadeh and I'm a second year of mastering Cyber Security major at University of Tehran, Iran. I am extremely interested in pursuing independent research. An extensive research experience will greatly help me consolidate my future career choice. Recently I'm interested in tor project vulnerabilities and I looked at some papers about it. This paper "The Effect of DNS on Tor’s Anonymity, NDSS, February 2017" attracted a great deal of my attention. A lot of research has gone into improving the Tor network, but its use of DNS has received little attention. In this research project, they set out to learn how DNS can harm the anonymity of Tor users, and how adversaries can leverage the DNS protocol to deanonymize users. They studied (i) how exposed the DNS protocol is compared to web traffic, (ii) how Tor exit relays are configured to use DNS, (iii) how existing website fingerprinting attacks can be enhanced with DNS, and (iv) how effective these enhanced website fingerprinting attacks are at Internet-scale. Also, I found another project that I think can defend against these attacks to some degree. The project is T-DNS (Connection-Oriented DNS to Improve Privacy and Security - DNS over TCP and TLS). It combines TCP to smoothly support large payloads and mitigate spoofing and amplification for DoS. T-DNS uses transport-layer security (TLS) to provide privacy from users to their DNS resolvers and optionally to authoritative servers. I want to integrate T-DNS server proxy and client proxy with Tor project source code to add some security feature to Tor project. I also implement T-DNS server proxy and client proxy I just need to add this feature in Tor source code. Something like creating a built-in standalone T-DNS client proxy and server proxy in Tor relays, and encourage people to use them with Tor Exits (and other DNS clients and servers). As you have a lot of exprience on Tor project, I would be happy if you could help me on this or give me some suggestion to clarify the way for me. I look forward to hearing from you! Best Regards, Nima Azizzadeh

2 1

prop224: Deprecating SHA1 circuit digests
by teor 24 Jul '17

24 Jul '17

Hi all, At the moment, Tor uses SHA1 for the running digests of circuit cell payloads. Some of the prop224 code seems to use SHA256 for the digests for client to service rendezvous circuits. But that's not in the spec yet (see #22995 at [0]). How and when do we plan to move away from using SHA1 in Tor circuits? For non-onion service circuits, this would mean: * implementing support for SHA256 [1] circuit digests * adding a Relay protocol version 3 * teaching clients to use SHA256 digests with relays with Relay protocol >= 3 For onion service circuits, it's more complicated, because the following circuit types can't use relay versions from the consensus: * client to intro * service to rend * client to service (Using relay versions from the consensus leaks which consensus clients and services have, which reduces the anonymity set.) Here are the upgrade mechanisms in prop224 at the moment, for both circuit protocol versions and any necessary handshake material: client to intro: * the protocol version could be in a proto line to each intro point, but this isn't implemented yet * the handshake data can be in the link-specifiers (I think?) service to rend * the protocol version could be in the EXT_FIELD in the INTRODUCE cell, but this isn't implemented yet * the handshake data can be in the link-specifiers (I think?) client to service: * the protocol version is in the create2-formats in the descriptor * the handshake data is in HANDSHAKE_INFO in the RENDEZVOUS cells * SHA256 digests are implemented, but not documented in prop224 [0] I suggest we make the following changes to prop224 to make this happen: Protocol version information: * add the relevant relay protocol versions to the intro point section of the descriptor * put the relevant relay protocol versions in an EXT_FIELD in the INTRODUCE cell * check create2-formats contains all the version info we will need to change the client to service circuit protocol version Downgrade resistance: * teach clients and services to use the highest common protocol between client/service and relay, excluding protocols that are below the minimum required protocol versions * work out how we will tell clients to no longer accept an old create2-formats line from a service [0]: https://trac.torproject.org/projects/tor/ticket/22995 [1]: By SHA256, I mean "a good secure hash at the time" T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------

3 5

Thoughts on the Counter-RAPTOR paper
by Roger Dingledine 20 Jul '17

20 Jul '17

Hi Prateek, Yixin, (and please involve your other authors as you like), (I'm including tor-dev here too so other Tor people can follow along, and maybe even get involved in the research or the discussion.) I looked through "Counter-RAPTOR: Safeguarding Tor Against Active Routing Attacks": https://arxiv.org/abs/1704.00843 For the tl;dr for others here, the paper: a) comes up with metrics for how to measure resilience of Tor relays to BGP hijacking attacks, and then does the measurements; b) describes a way that clients can choose their guards to be less vulnerable to BGP hijacks, while also considering performance and anonymity loss when guard choice is influenced by client location; and c) builds a monitoring system that takes live BGP feeds and looks for routing table anomalies that could be hijack attempts. Here are some hopefully useful thoughts: ----------------------------------------------------------------------- 0) Since I opted to write these thoughts in public, I should put a little note here in case any journalists run across it and wonder. Yay research! We love research on Tor -- in fact, research like this is the reason Tor is so strong. For many more details about our perspective on Tor research papers, see https://blog.torproject.org/blog/tor-heart-pets-and-privacy-research-commun… ----------------------------------------------------------------------- 1a) The "live BGP feed anomaly detection" part sounds really interesting, since in theory we could start using it really soon now. Have you continued to run it since you wrote the paper? Have you done any more recent analysis on its false positive rate since then? I guess one of the real challenges here is that since most of the alerts are false positives, we really need a routing expert to be able to look at each alert and assess whether we should be worried about it. How hard is it to locate such an expert? Is there even such a thing as an expert in all routing tables, or do we need expertise in "what that part of the network is supposed to look like", which doesn't easily scale to the whole Internet? Or maybe said another way, how much headway can we make on automating the analysis, to make the frequency of alerts manageable? I ask because it's really easy to write a tool that sends a bunch of warnings, and if some of them are false positives, or heck even if they're not but we don't know how to assess how bad they really are, then all we've done is make yet another automated emailer. (We've made a set of these already, to e.g. notice when relays change their identity key a lot: https://gitweb.torproject.org/doctor.git/tree/ but often nobody can figure out whether such an anomaly is really an attack or what, so it's a constant struggle to keep the volume low enough that people don't just ignore the mails.) The big picture question is: what steps remain from what you have now to something that we can actually use? 1b) How does your live-BGP-feed-anomaly-detector compare (either in design, or in closeness to actually being usable ;) to the one Micah Sherr was working on from their PETS 2016 paper? https://security.cs.georgetown.edu/~msherr/reviewed_abstracts.html#tor-data… 1c) Your paper suggests that an alert from a potential hijack attempt could make clients abandon the guard for a while, to keep clients safe from hijack attempts. What about second-order effects of such a design, where the attacker's *goal* is to get clients to abandon a guard, so they add some sketchy routes somewhere to trigger an alert? Specifically, how much easier is it to add sketchy routes that make it look like somebody is attempting an attack, compared to actually succeeding at hijacking traffic? I guess a related question (sorry for my BGP naivete) is: if we're worried about false positives in the alerts, how much authentication and/or attribution is there for sketchy routing table entries in general? Can some jerk drive up our false positive rate, by adding scary entries here and there, in a way that's sustainable? Or heck, can some jerk DDoS parts of the Internet in a way that induces routing table changes that we think look sketchy? These are not reasons to not take the first steps in the arms race, but it's good to know what the later steps might be. ----------------------------------------------------------------------- 2a) Re changing guard selection, you should check out proposal 271, which resulted in the new guard-spec.txt as of Tor 0.3.0.x: https://gitweb.torproject.org/torspec.git/tree/guard-spec.txt I don't fully understand it yet (so many things!), but I bet any future guard selection change proposal should be relative to this design. 2b) Your guard selection algorithm makes the assumption that relays with the Guard flag are the only ones worth choosing from, and then describes a way to choose from among them with different weightings. But you could take a step back, and decide that resilience to BGP hijack should be one of the factors for whether a relay gets the Guard flag in the first place. It sounded from your analysis like some ASes, like OVH, are simply bad news for (nearly) all Tor clients. Your proposed guard selection strategy reduced, but did not eliminate, the chances that clients would get screwed by picking one of these OVH relays. And the tradeoff was that by only reducing the chances, you left the performance changes not as extreme as you might have otherwise. How much of the scariness of a relay is a function of the location of the particular client who is considering using it, and how much is a function of the average (expected) locations of clients? That is, can we identify relays that are likely to be bad news for many different clients, and downplay their weights (or withhold the Guard flag) for everybody? The advantage of making the same decision for all clients is that you can get rid of the "what does guard choice tell you about the client" anonymity question, which is a big win if the rest of the effects aren't too bad. Which leads me to the next topic: ----------------------------------------------------------------------- 3) I think you're right that when analyzing a new path selection strategy, there are three big things to investigate: a) Does the new behavior adequately accomplish the goal that made you want a new path selection strategy (in this case resilience to BGP attacks)? b) What does the new behavior do to anonymity, both in terms of the global effect (e.g. by flattening the selection weights or by concentrating traffic in fewer relays or on fewer networks) and on the individual epistemic side (e.g. by leaking information about the user because of behavior that is a function of sensitive user details)? c) What are the expected changes to performance, and are there particular scenarios (like high load or low load) that have higher or lower impact? I confess that I don't really buy your analysis for 'b' or 'c' in this paper. Average change in entropy doesn't tell me whether particular user populations are especially impacted, and a tiny Shadow simulation with one particular network load and client behavior doesn't tell me whether things will or won't get much worse under other network loads or other client behavior. I can't really fault this paper though, because the structure of an academic research paper means you can only do so much in one paper, and you did a bunch of other interesting things instead. We, the Tor research community, really need better tools for reasoning about the interaction between anonymity and performance. In fact, there sure have been a lot of Tor path selection papers over the past decade which each invent their own ad hoc analysis approach for showing that their proposed change doesn't impact anonymity or performance "too much". Is it time for a Systemization of Knowledge paper on this area -- with the goal of coming up with best practices that future papers can use to provide more convincing analysis? --Roger

2 1

[release] CollecTor 1.2.0
by iwakeh 13 Jul '17

13 Jul '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! a new CollecTor version is available: https://dist.torproject.org/collector/1.2.0/ The main new features are the switch from Torperf to Onionperf (see [0] for background) and the upgrade to metrics-lib-2.0.0. For more see the change log [1]. In addition, CollecTor's web-pages are now part of Tor Metrics web-site: https://metrics.torproject.org/collector.html. Please update any bookmarks and report any wrong links to the old page. All data is also still available directly at collector.torproject.org. Please direct comments and questions to the metrics-team mailing list [2]. And, of course, bug reports [3] or feature requests [4] can be filed in trac. Cheers, iwakeh [0] https://metrics.torproject.org/stats.html#torperf-1.1 [1] https://gitweb.torproject.org/collector.git/plain/CHANGELOG.md?id=collector… [2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team [3] https://trac.torproject.org/projects/tor/newticket?component=Metrics/Collec… [4] https://trac.torproject.org/projects/tor/newticket?component=Metrics/Collec… -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJZZ200AAoJEPeEx9Sa/gvi7oEQAK1KGYwtjbPf81dPjOLg3aXo 9yPDylFT6h6yQ2xSadfr895prrvvNs34w+gmCtVv/SoTt9M/NfBS+Sj6vj6BEVOO E6e9gLty64sZYV8dlvGksa1RcGbMIpFRqLKk38agqF4h5074Wg2f1fk+KdzXuQ4G Q70Upp8YtOFaEf0dyIc6a2VUIPtv3qoRuj5bQNOHuPyO40l8nGh4qm7kYYVjB7t/ lRS3byJl2JTplmX8+9ttJlmwyjCi7K14xUMJ/tZ8TjV7YRTIUAVlSTSzxz7AKxXw NwtyPNIAQi6d1QI7HT1fWOHwqvWC1RXLelnS5poQ89LmQWHF5ONDED62q4Oxsc9j JkivmhxW0akp9ijck8K2G581sO3TNZBnNVp1eOL3Ymg+G55HGog+DYM9X1hm2geB olUkl3PvIIYPVot2s8E/gslOi3QMAYhkoGO0+kHoy4TkWveclXgFJi/o+QKKzora W2cIuM1hO5qLlQ6sNEHBW6fMLcmgvDJNVAR/+CWUYz1QhesL92hCGpukAvTkYsKv jwMFAiXBLxbSkzxDrAI37Ah5/QZRU+Eu17UkyNQb5QGnRoI8t58pJjoY2+POSBZ+ TtvezXD//xq0Wpy+Erpj/OxOfpPItTrH7LF1IrmpGZ3JW+KaZJzItve7VvEG8NVR 9FWp8aT1usWmHA2EvWfg =EDHj -----END PGP SIGNATURE-----

1 0

Fuzzing tor_inet_pton(): A fuzzing example
by Nick Mathewson 05 Jul '17

05 Jul '17

So, there was a bug report of an assertion failure in tor_inet_pton(), and I wanted to track it down. Here's my reconstruction of my process, in case it helps anybody else write fuzzing code. First, I wrote a new fuzzing target in src/test/fuzz/fuzz_pton.c: ====== #include "orconfig.h" #include "or.h" #include "fuzzing.h" int fuzz_init(void) { return 0; } int fuzz_cleanup(void) { return 0; } int fuzz_main(const uint8_t *stdin_buf, size_t data_size) { struct in_addr *in = tor_malloc(sizeof(*in)); struct in6_addr *in6 = tor_malloc(sizeof(*in6)); char *cp = tor_memdup_nulterm(stdin_buf, data_size); int in_ok = tor_inet_pton(AF_INET, cp, in); int in6_ok = tor_inet_pton(AF_INET6, cp, in6); tor_free(cp); tor_free(in); tor_free(in6); log_info(LD_GENERAL, "%d %d", in_ok, in6_ok); return 0; } ====== Notes: * The init and cleanup functions don't have anything to do above, but the test harness in fuzzing_common.c expects to find them. * This fuzzer tries to parse each address twice: first as an IPv4 address, then as an IPv6 address. * The fuzzer puts its outputs into heap-allocated RAM to improve the odds that any bugs will get caught by asan, though this shouldn't really be necessary. * Because tor_inet_pton() takes a nul-terminated string, this fuzzer nul-term inates its input. You shouldn't do that unless you're fuzing a function that requires nul-terminated inputs. Next, I edited the script in scripts/codegen/fuzzing_include_am.py to add "pton" to the "FUZZERS" list at the top of the file, and I re-ran the script and used its output to replace src/test/fuzz/include.am, so that our build system can know about the new fuzzer. I knew I would need a corpus of pton examples, so I started one in fuzzing-corpora (fuzzing-corpora is a separate repository). I made a new "pton" directory there, and filled it with 4 or 5 examples of things that looked like they might be addresses. I chose: 127.0.0.1 1.2.3.4 ::1 80f0:9999:ffff::3 Each example went into its own file, without a terminating newline. I tried the fuzzing corpora examples out with the fuzzing program, to make sure that I got the expected results $ ./src/test/fuzz/fuzz-pton --info < ../tor-fuzz-corpora/pton/b Jul 05 15:00:56.371 [info] fuzz_main(): 1 0 ===== AFL Then I re-built Tor for use with AFL: ./configure CC=afl-gcc --enable-fragile-hardening make clean AFL_HARDEN=1 make fuzzers To fuzz it, I started by telling afl-fuzz to run with no memory limit, writing results to pton-output, taking inputs from the corpus: afl-fuzz -i ../tor-fuzz-corpora/pton -o ./pton-output/ -m none \ -- ./src/test/fuzz/fuzz-pton AFL wound up complaining about some kernel settings, so I had to do this as root. YMMV. AFL told me to do this: echo core >/proc/sys/kernel/core_pattern cd /sys/devices/system/cpu echo performance | tee cpu*/cpufreq/scaling_governor Then AFL started running! I watched the "total paths" value climb. That's how many different distinct paths through the code that the fuzzer was able to find. I didn't see any "uniq crashes" or "uniq hangs" values, so I knew the fuzzer hadn't found anything. ===== Libfuzzer After that had run for a day, I stopped it and tried again with libfuzzer: ./configure --enable-libfuzzer CC=/home/nickm/build/llvm-build/bin/clang make clean make fuzzers Obviously, you'll need to point CC to your own clang here. I wasn't able to use the system clang because libfuzzer needs features I didn't have. Then I made a new file for the libfuzzer results, and ran libfuzzer using the original corpus and the output from AFL: mkdir pton-output-libfuzzer ./src/test/fuzz/lf-fuzz-pton pton-output-libfuzzer/ \ ./pton-output/queue/ ../tor-fuzz-corpora/pton/ And libfuzzer began giving results. Every "NEW" line was a new path; every "pulse" line meant that time had passed but no new paths were found.

1 0

Open topics of prop247: Defending Against Guard Discovery Attacks using Vanguards
by George Kadianakis 04 Jul '17

04 Jul '17

Hello, here is some background information and summarizing of proposal 247 "Defending Against Guard Discovery Attacks using Vanguards" for people who plan to work on this in the short-term future. I include a list of open design topics (probably not exhaustive) and a list of engineering topics. Some engineering stuff can be done parallel to the design stuff. ==================== Background info ==================== * Proposal: https://gitweb.torproject.org/torspec.git/tree/proposals/247-hs-guard-disco… * Discussion: ** Initial prop247 thread: https://lists.torproject.org/pipermail/tor-dev/2015-July/009066.html ** Recent prop247 thread: https://lists.torproject.org/pipermail/tor-dev/2015-September/009497.html ** Reading group notes of prop247: https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html ==================== Design topics ==================== * Optimize proposal parameters ** Optimize guardset sizes ** Optimize guardset lifetimes and prob distributions (minXX/maxXX/uniform?) ** To take informed decision, we might need a prop247 simulator, or an actual PoC with txtorcon * HOW to choose second-layer and third-layer guards? ** Should they be Guards? middles? Vanguards? Serious security / load balancing implications! ** Can guardsets share guards between them or are they disjoint? Particularly third-layer sets ** background: https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html * HOW to avoid side-channel guard discovery threats? ** Can IP/RP be the same as first-layer guard? ** Can first-layer guard be the same as third-layer guard? ** background: https://gitweb.torproject.org/user/mikeperry/torspec.git/commit/?h=guard_di… * Change path selection for IP circs to avoid third-layer guard linkability threats. ** Switch from [HS->G1->M->IP] to [HS->G1->G2->G3->IP] or even to [HS->G1->G2->G3->M->IP]. ** Consider the latter option for HSDir circs as well? ** background: https://gitweb.torproject.org/user/mikeperry/torspec.git/commit/?h=guard_di… * Should prop247 be optional or default? ** Consider making it optional for a testing period? * How does prop247 affects network performance and load balancing? ** especially if it's enabled by default? ** Update load balancing proposal? * Correct behavior for multiple HSes on single host? * Does prop247 influence guard fingerprinting (#10969) and should we care enough? ==================== Engineering topics ==================== * What's a good entrynodes API to implement prop247? * What's a good state file API to implement prop247? * Write prop247 simulator to verify security goals and optimize proposal parameters (see above). * Write PoC with txtorcon! * Write PoC with little-t-tor! ============================================================

5 7

Re: [tor-dev] Default parsing order of config files in Debain tor package
by iry 03 Jul '17

03 Jul '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Daniel! Thank you so much for your immediate, detailed and informative answer. I really like the example you offered as it clearly illustrates the parsing order. Your answer is very helpful for me to decide where I should place the %include line and Tor config files. I really appreciate your help. Best, iry Daniel: > The precedence for tor options is the following (1 overrides 2, > etc...): > > 1. Command line options. 2. Configuration file options (your > /etc/torrc). 3. Defaults file options (your > /usr/share/tor/tor-service-defaults-torrc). > > In the same file, options that appear later override earlier > options. > > Currently, there is no torrc.d directory created when you install > the tor package. However, you can use a %include in the > configuration file or in the defaults file. When you insert a > %include in a file, it works as if all the options for the included > file or folder were written on the line of the %include. If you're > including a folder, the files will be processed in lexicographic > order and files starting with a dot will be ignored. > > Here is an example: > > tor-service-defaults-torrc: SomeOption 0 %include /etc/tor/torrc.d/ > # SomeOption is now 2 SomeOption 3 # SomeOption is now 3 > > /etc/tor/torrc.d/01_one: SomeOption 1 > > /etc/tor/torrc.d/02_two: SomeOption 2 > > > With this configuration, the value for some option is 3. But we can > have a torrc with %include too: > > /etc/torrc: SomeOption 4 # SomeOption is now 4 %include > /etc/tor/foo.torrc # SomeOption is now 5 SomeOption 6 # SomeOption > is now 6 > > /etc/tor/foo.torrc: SomeOption 5 > > With both these files, the value for SomeOption is 6. > > There are also different types of options and some can take > multiple values. For more information see the section "Mid-level > semantics" on this file: > https://gitweb.torproject.org/tor.git/tree/doc/torrc_format.txt > > Best regards, Daniel -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZWZ1gAAoJEKFLTbxtzdU84gAQAKVu8UcbYHXIaZ+sUkiENUeW lSBnvOHeRcIBAOe1LfClgkAFiVWhHwfG2ll7cGDTiMxn7IuDlv+sUFtwpqRVFgYH kOxJiVS1kORQJ8t8DuXh3rQZTUTKOkhCCepb8maHeCSED0yUGSXs+wzpFMAyKJk+ 2B73YR/hQd2XgpZx9LcpfkdznTXF/jpOPZthZWFAkwm1yNAlvwAmfXgADV4lQy5K RRCwYmAgdSS9OLrDMj0G1lVnvsr7qUgfePzLpp1FBkMf/E8nhL6NBCK+jJc+BA+R 3YKn5pKtJ5/KaAwutQ5nlI3+mdVoDnJI0wWGHxL6ZJPwfCMANqkrbRa8/Wq0i7ZB u1sG9aWJzTXPaI/OZQ8VE5bvh9mLSfwo0RpzYv8CVsFgu2VnMPk5EW13d33+Dzjz 5GVcA0s0V1AA7uZ/NGUrcUZ/jO+z5Argre7RxkCnAgjItKb2vJK5hGnj7gVQO0il bcx+e47MyLVNoi7tOaoJmOusEEMQB1wMG6S1Pc+o/apqW6p2VEUKrTh+DSp72+LX zDPOxAleJjbntXZkGrDO0nay21y1LWYL2bOHpWomjd03f9v6br/n9oofNX6pN7Xn B9FIOa0PxzCT2BwcROnEDEB9mCHDbIRSEvuwpbbdz3StRW3pLzU1Q2e6WO+eF7dH OpDeWlO5/Z6IJKV4eesz =9riT -----END PGP SIGNATURE-----

1 0