- tor-dev - lists.torproject.org

[release] CollecTor 1.2.0
by iwakeh 13 Jul '17

13 Jul '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! a new CollecTor version is available: https://dist.torproject.org/collector/1.2.0/ The main new features are the switch from Torperf to Onionperf (see [0] for background) and the upgrade to metrics-lib-2.0.0. For more see the change log [1]. In addition, CollecTor's web-pages are now part of Tor Metrics web-site: https://metrics.torproject.org/collector.html. Please update any bookmarks and report any wrong links to the old page. All data is also still available directly at collector.torproject.org. Please direct comments and questions to the metrics-team mailing list [2]. And, of course, bug reports [3] or feature requests [4] can be filed in trac. Cheers, iwakeh [0] https://metrics.torproject.org/stats.html#torperf-1.1 [1] https://gitweb.torproject.org/collector.git/plain/CHANGELOG.md?id=collector… [2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team [3] https://trac.torproject.org/projects/tor/newticket?component=Metrics/Collec… [4] https://trac.torproject.org/projects/tor/newticket?component=Metrics/Collec… -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJZZ200AAoJEPeEx9Sa/gvi7oEQAK1KGYwtjbPf81dPjOLg3aXo 9yPDylFT6h6yQ2xSadfr895prrvvNs34w+gmCtVv/SoTt9M/NfBS+Sj6vj6BEVOO E6e9gLty64sZYV8dlvGksa1RcGbMIpFRqLKk38agqF4h5074Wg2f1fk+KdzXuQ4G Q70Upp8YtOFaEf0dyIc6a2VUIPtv3qoRuj5bQNOHuPyO40l8nGh4qm7kYYVjB7t/ lRS3byJl2JTplmX8+9ttJlmwyjCi7K14xUMJ/tZ8TjV7YRTIUAVlSTSzxz7AKxXw NwtyPNIAQi6d1QI7HT1fWOHwqvWC1RXLelnS5poQ89LmQWHF5ONDED62q4Oxsc9j JkivmhxW0akp9ijck8K2G581sO3TNZBnNVp1eOL3Ymg+G55HGog+DYM9X1hm2geB olUkl3PvIIYPVot2s8E/gslOi3QMAYhkoGO0+kHoy4TkWveclXgFJi/o+QKKzora W2cIuM1hO5qLlQ6sNEHBW6fMLcmgvDJNVAR/+CWUYz1QhesL92hCGpukAvTkYsKv jwMFAiXBLxbSkzxDrAI37Ah5/QZRU+Eu17UkyNQb5QGnRoI8t58pJjoY2+POSBZ+ TtvezXD//xq0Wpy+Erpj/OxOfpPItTrH7LF1IrmpGZ3JW+KaZJzItve7VvEG8NVR 9FWp8aT1usWmHA2EvWfg =EDHj -----END PGP SIGNATURE-----

1 0

Fuzzing tor_inet_pton(): A fuzzing example
by Nick Mathewson 05 Jul '17

05 Jul '17

So, there was a bug report of an assertion failure in tor_inet_pton(), and I wanted to track it down. Here's my reconstruction of my process, in case it helps anybody else write fuzzing code. First, I wrote a new fuzzing target in src/test/fuzz/fuzz_pton.c: ====== #include "orconfig.h" #include "or.h" #include "fuzzing.h" int fuzz_init(void) { return 0; } int fuzz_cleanup(void) { return 0; } int fuzz_main(const uint8_t *stdin_buf, size_t data_size) { struct in_addr *in = tor_malloc(sizeof(*in)); struct in6_addr *in6 = tor_malloc(sizeof(*in6)); char *cp = tor_memdup_nulterm(stdin_buf, data_size); int in_ok = tor_inet_pton(AF_INET, cp, in); int in6_ok = tor_inet_pton(AF_INET6, cp, in6); tor_free(cp); tor_free(in); tor_free(in6); log_info(LD_GENERAL, "%d %d", in_ok, in6_ok); return 0; } ====== Notes: * The init and cleanup functions don't have anything to do above, but the test harness in fuzzing_common.c expects to find them. * This fuzzer tries to parse each address twice: first as an IPv4 address, then as an IPv6 address. * The fuzzer puts its outputs into heap-allocated RAM to improve the odds that any bugs will get caught by asan, though this shouldn't really be necessary. * Because tor_inet_pton() takes a nul-terminated string, this fuzzer nul-term inates its input. You shouldn't do that unless you're fuzing a function that requires nul-terminated inputs. Next, I edited the script in scripts/codegen/fuzzing_include_am.py to add "pton" to the "FUZZERS" list at the top of the file, and I re-ran the script and used its output to replace src/test/fuzz/include.am, so that our build system can know about the new fuzzer. I knew I would need a corpus of pton examples, so I started one in fuzzing-corpora (fuzzing-corpora is a separate repository). I made a new "pton" directory there, and filled it with 4 or 5 examples of things that looked like they might be addresses. I chose: 127.0.0.1 1.2.3.4 ::1 80f0:9999:ffff::3 Each example went into its own file, without a terminating newline. I tried the fuzzing corpora examples out with the fuzzing program, to make sure that I got the expected results $ ./src/test/fuzz/fuzz-pton --info < ../tor-fuzz-corpora/pton/b Jul 05 15:00:56.371 [info] fuzz_main(): 1 0 ===== AFL Then I re-built Tor for use with AFL: ./configure CC=afl-gcc --enable-fragile-hardening make clean AFL_HARDEN=1 make fuzzers To fuzz it, I started by telling afl-fuzz to run with no memory limit, writing results to pton-output, taking inputs from the corpus: afl-fuzz -i ../tor-fuzz-corpora/pton -o ./pton-output/ -m none \ -- ./src/test/fuzz/fuzz-pton AFL wound up complaining about some kernel settings, so I had to do this as root. YMMV. AFL told me to do this: echo core >/proc/sys/kernel/core_pattern cd /sys/devices/system/cpu echo performance | tee cpu*/cpufreq/scaling_governor Then AFL started running! I watched the "total paths" value climb. That's how many different distinct paths through the code that the fuzzer was able to find. I didn't see any "uniq crashes" or "uniq hangs" values, so I knew the fuzzer hadn't found anything. ===== Libfuzzer After that had run for a day, I stopped it and tried again with libfuzzer: ./configure --enable-libfuzzer CC=/home/nickm/build/llvm-build/bin/clang make clean make fuzzers Obviously, you'll need to point CC to your own clang here. I wasn't able to use the system clang because libfuzzer needs features I didn't have. Then I made a new file for the libfuzzer results, and ran libfuzzer using the original corpus and the output from AFL: mkdir pton-output-libfuzzer ./src/test/fuzz/lf-fuzz-pton pton-output-libfuzzer/ \ ./pton-output/queue/ ../tor-fuzz-corpora/pton/ And libfuzzer began giving results. Every "NEW" line was a new path; every "pulse" line meant that time had passed but no new paths were found.

1 0

Open topics of prop247: Defending Against Guard Discovery Attacks using Vanguards
by George Kadianakis 04 Jul '17

04 Jul '17

Hello, here is some background information and summarizing of proposal 247 "Defending Against Guard Discovery Attacks using Vanguards" for people who plan to work on this in the short-term future. I include a list of open design topics (probably not exhaustive) and a list of engineering topics. Some engineering stuff can be done parallel to the design stuff. ==================== Background info ==================== * Proposal: https://gitweb.torproject.org/torspec.git/tree/proposals/247-hs-guard-disco… * Discussion: ** Initial prop247 thread: https://lists.torproject.org/pipermail/tor-dev/2015-July/009066.html ** Recent prop247 thread: https://lists.torproject.org/pipermail/tor-dev/2015-September/009497.html ** Reading group notes of prop247: https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html ==================== Design topics ==================== * Optimize proposal parameters ** Optimize guardset sizes ** Optimize guardset lifetimes and prob distributions (minXX/maxXX/uniform?) ** To take informed decision, we might need a prop247 simulator, or an actual PoC with txtorcon * HOW to choose second-layer and third-layer guards? ** Should they be Guards? middles? Vanguards? Serious security / load balancing implications! ** Can guardsets share guards between them or are they disjoint? Particularly third-layer sets ** background: https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html * HOW to avoid side-channel guard discovery threats? ** Can IP/RP be the same as first-layer guard? ** Can first-layer guard be the same as third-layer guard? ** background: https://gitweb.torproject.org/user/mikeperry/torspec.git/commit/?h=guard_di… * Change path selection for IP circs to avoid third-layer guard linkability threats. ** Switch from [HS->G1->M->IP] to [HS->G1->G2->G3->IP] or even to [HS->G1->G2->G3->M->IP]. ** Consider the latter option for HSDir circs as well? ** background: https://gitweb.torproject.org/user/mikeperry/torspec.git/commit/?h=guard_di… * Should prop247 be optional or default? ** Consider making it optional for a testing period? * How does prop247 affects network performance and load balancing? ** especially if it's enabled by default? ** Update load balancing proposal? * Correct behavior for multiple HSes on single host? * Does prop247 influence guard fingerprinting (#10969) and should we care enough? ==================== Engineering topics ==================== * What's a good entrynodes API to implement prop247? * What's a good state file API to implement prop247? * Write prop247 simulator to verify security goals and optimize proposal parameters (see above). * Write PoC with txtorcon! * Write PoC with little-t-tor! ============================================================

5 7

Re: [tor-dev] Default parsing order of config files in Debain tor package
by iry 03 Jul '17

03 Jul '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Daniel! Thank you so much for your immediate, detailed and informative answer. I really like the example you offered as it clearly illustrates the parsing order. Your answer is very helpful for me to decide where I should place the %include line and Tor config files. I really appreciate your help. Best, iry Daniel: > The precedence for tor options is the following (1 overrides 2, > etc...): > > 1. Command line options. 2. Configuration file options (your > /etc/torrc). 3. Defaults file options (your > /usr/share/tor/tor-service-defaults-torrc). > > In the same file, options that appear later override earlier > options. > > Currently, there is no torrc.d directory created when you install > the tor package. However, you can use a %include in the > configuration file or in the defaults file. When you insert a > %include in a file, it works as if all the options for the included > file or folder were written on the line of the %include. If you're > including a folder, the files will be processed in lexicographic > order and files starting with a dot will be ignored. > > Here is an example: > > tor-service-defaults-torrc: SomeOption 0 %include /etc/tor/torrc.d/ > # SomeOption is now 2 SomeOption 3 # SomeOption is now 3 > > /etc/tor/torrc.d/01_one: SomeOption 1 > > /etc/tor/torrc.d/02_two: SomeOption 2 > > > With this configuration, the value for some option is 3. But we can > have a torrc with %include too: > > /etc/torrc: SomeOption 4 # SomeOption is now 4 %include > /etc/tor/foo.torrc # SomeOption is now 5 SomeOption 6 # SomeOption > is now 6 > > /etc/tor/foo.torrc: SomeOption 5 > > With both these files, the value for SomeOption is 6. > > There are also different types of options and some can take > multiple values. For more information see the section "Mid-level > semantics" on this file: > https://gitweb.torproject.org/tor.git/tree/doc/torrc_format.txt > > Best regards, Daniel -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZWZ1gAAoJEKFLTbxtzdU84gAQAKVu8UcbYHXIaZ+sUkiENUeW lSBnvOHeRcIBAOe1LfClgkAFiVWhHwfG2ll7cGDTiMxn7IuDlv+sUFtwpqRVFgYH kOxJiVS1kORQJ8t8DuXh3rQZTUTKOkhCCepb8maHeCSED0yUGSXs+wzpFMAyKJk+ 2B73YR/hQd2XgpZx9LcpfkdznTXF/jpOPZthZWFAkwm1yNAlvwAmfXgADV4lQy5K RRCwYmAgdSS9OLrDMj0G1lVnvsr7qUgfePzLpp1FBkMf/E8nhL6NBCK+jJc+BA+R 3YKn5pKtJ5/KaAwutQ5nlI3+mdVoDnJI0wWGHxL6ZJPwfCMANqkrbRa8/Wq0i7ZB u1sG9aWJzTXPaI/OZQ8VE5bvh9mLSfwo0RpzYv8CVsFgu2VnMPk5EW13d33+Dzjz 5GVcA0s0V1AA7uZ/NGUrcUZ/jO+z5Argre7RxkCnAgjItKb2vJK5hGnj7gVQO0il bcx+e47MyLVNoi7tOaoJmOusEEMQB1wMG6S1Pc+o/apqW6p2VEUKrTh+DSp72+LX zDPOxAleJjbntXZkGrDO0nay21y1LWYL2bOHpWomjd03f9v6br/n9oofNX6pN7Xn B9FIOa0PxzCT2BwcROnEDEB9mCHDbIRSEvuwpbbdz3StRW3pLzU1Q2e6WO+eF7dH OpDeWlO5/Z6IJKV4eesz =9riT -----END PGP SIGNATURE-----

1 0

Default parsing order of config files in Debain tor package
by iry 02 Jul '17

02 Jul '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, Tor developers! With the implementation of torrc.d, there will be at least two Tor config files and one Tor config directory by default in Debian tor package. However, I am not able to find the documents on what the parsing order Tor follows. My guess is that: 1. /usr/share/tor/tor-service-defaults-torrc is parsed first 2. /etc/torrc.d is parsed next (in lexical order) 3. /etc/torrc is parsed at last 4. and the lines in config files parsed later will overwrite similar lines in config files that are parsed earlier Could anyone familiar with the problem help me please? This will be really helpful to my future work and I really appreciate your help! Thank you very much! Best, iry -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZWPgFAAoJEKFLTbxtzdU8HpoP/3dz869rQmlAMC4v/2UAO0F+ hPtC/L27ILeICraQRc2sJOsf7FTueKD/a6GadUbCbWBwWyyzdBXotqADnmJdvNnm hSgkE1mYz27+eI2NUd7UpHs+NrP7FXdjufGkHxZfJPrhTbGKPCC66HouT+T1vdh5 PTRBjc6M/+C2yRuteGRcAW6Be3NJ+3HRuFC5THJd6b+p0WmP+OYXdVlq7cXuY3zj ffaXvX7WGOH2J+ALIaspc0wjLiieUCq5wrO7REkeUM0TuqjhWWGBhANqVM1Qrfs+ 0jWPRklFq/7BkfySUZA88H6D5pkpBX3eVDsUlduxvoSS/F8ksWzAIdMMFaz9a7tR GpCEBBmbJIN3d4ApF78h//Y9XkaBsAotf40YEC4uZ4h3iW4rlyojWuU7N78A36PL MJ0IdUm52hWy7ySRDFi9lr7y1xtGQ9qO1gAYCq7D+qeKte+E7UO/QKZjOkAMBBir hxLimroNtL708rcDzLegstWt5Rpf2iJOYP1wOYnAeYfkSeXXW5KQBDJEd1botJg6 YRa69iICoLBNY+4z6i3t5lVlMolqPyAAwEfA1j4yXBrtBed8ISKx4V44y5JeGnZ2 SjMfHM6XJB95Bu8sWVQc1dQ+eBBiFYHbP4s4n1+jsTbmImg2bc+LauxChwRmAHek o9KiSasLITPC7zIv1m2o =KrAy -----END PGP SIGNATURE-----

2 1

[release] metrics-lib 2.0.0
by iwakeh 29 Jun '17

29 Jun '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! As planned in May [0] the next a major release of Tor Metrics Library is available: https://dist.torproject.org/metrics-lib/2.0.0/ This time we have a special blog post about this release and Tor Metrics Library [1]. Please direct comments and questions to the metrics-team mailing list [2], or comment on the blog post. Cheers, iwakeh [0] https://lists.torproject.org/pipermail/tor-dev/2017-May/012261.html [1] https://blog.torproject.org/blog/tor-descriptors-la-carte-tor-metrics-libra… [2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJZVR3LAAoJEPeEx9Sa/gvi/UkP/0X0mB7fkjfnbQDHOBrrzJdL CIM1pxubdoTvfr3I9Gy/3EYBP2ps/eqEa1bPP6WB0eg/BnLkEhuRtu4mFd35oEDB 67XPGfBwv7ICww4/9NXBQ37aNeBpwuQ6yZopoymY6Pis/RYiO1NG89GWCIMeq2I8 w4rDf2ElMAjYY2Vw0lLlKxL+v1eja2ChuaYpXhJvqxZV7qQsmvi9oOtQ5YYHNJyW +c5c0ajbO8LtC6Bd40XNjVBV3ayhp9iwbKFKbsvnkrlHHUvdMwRwk6CeE8iiK0FG YVObe6hljLR9IQjfaRYBf4moMg/rZdacJtu2rqBR9MubL/i7//+r0wgxk3za/rok kzNaoO9iz13FGBE/4Rum9CsN/8gAJuOn73JyeApB6HohXlAQfIwUtmpVcg/EaWXb flyJDRrxUIWilwuOTfyunpu/BEuR3bSawYorvLasna2hIPG14DtVrHzNCY3TlxFz 2GJn0vegDGJI9QpgTW8J2VCGQCPXcFqvIKfl0IM3p6lJWw8VuE4K6XchlebHgJaH qzYwYvfgtGD5GbpdssBOFocYPwOP0jQqgkEiZPMpkdl9U27o7DmHrnd9E54NCDYw cOAfu9s1OpGjgiQuwli0UoZHVoZtDIghmZMOC1hUzmlz5W/be7GgThRGix9ZMfXF BSAGkbAy1AlwN1kGEJMx =i4rq -----END PGP SIGNATURE-----

1 0

moving toward requiring regression tests for bug fixes
by Taylor Yu 28 Jun '17

28 Jun '17

Hi all, I've opened a Trac ticket at https://trac.torproject.org/projects/tor/ticket/22745 about possibly requiring all bug fixes to have associated regression tests. This is aimed at Core Tor (starting with 0.3.2.x) but other Tor-related software might want to consider a policy like this as well. Ideally all bug fixes will have automated regression tests so we can promptly recognize when they've regressed and fix them. I realize that some of our code may be too complex for an automated regression test to be feasible, so we would have a procedure for exceptions from this requirement. (This would become part of the patch review process.) Please comment on the ticket if you have opinions about this idea. (Or respond in email if your reply wouldn't work well in the form of a ticket comment.) Thanks! -Taylor

1 0

prop224: Time Period Overlaps and Blinded Keys
by teor 27 Jun '17

27 Jun '17

Hi, The time period overlap section 2.2.4 in prop224 is under-specified: https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.t… 1. During the overlap period, does the service use the new blinded key for the new period, and the old blinded key for the old period? I think the answer is yes, but this requires some deduction to work out. 2. If the overlap period starts when a service sees the first consensus with a new SRV, does the service stop using that SRV and blinded key: * at the end of the period? (that is, exactly 36 hours after the earliest the overlap period could possibly have started.) * exactly 36 hours after the SRV was first seen? (that is, exactly 36 hours after the service started the overlap period. For example, if the service fetched the consensus 2 hours after it was created, it would end 2 hours after the end of the period.) * when the first reveal consensus is received with that SRV as the previous SRV? (or some similar consensus-driven event) Does every service on a tor instance start the overlap at the same time? https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.t… T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------

4 3

Re: [tor-dev] Problems on torrc.d-style configuration directories
by iry 27 Jun '17

27 Jun '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi teor and Daniel! Thank you so much for your reply! Your instructions are really helpful to me. teor: > You probably need a %include directive in /etc/tor/torrc. I tried to add %include directive in /etc/tor/torrc and /usr/share/tor/tor-service-defaults-torrc separately. And both of them worked well :) teor: > If you get this working, please submit a patch to the Debian bug > tracker. No problem! But please forgive my ignorance, could you please explain a little bit more to me that why I should report it to Debian BTS, instead of tpo? In other words, what is the relationship between packages.debian.org and deb.torproject.org? According to cypherpunks[0]: > The first released tor version with this feature is 0.3.1.1-alpha. > As usual there will be alpha packages on deb.torproject.org > > If you want this feature _now_ you can use the nightly builds: > https://deb.torproject.org/torproject.org/dists/tor-nightly-master-str etch However, > the highest tor version in Debian BTS right now is 3.0.8-1.[1], which means the feature has not been included into Debian? My current thought is deb.torproject.org is the upstream of packages.debian.org in terms of tor package. So a change made in deb.torproject.org will be adopted by packages.debian.org after a while. The following is my testing environment which may be helpful to the problem: I tested the torrc.d feature in both Debian 8 and Whonix13(based on Debian8). Instead of downloading from packages.debian.org, I download tor from: > deb http://deb.torproject.org/torproject.org jessie main deb-src > http://deb.torproject.org/torproject.org jessie main deb > http://deb.torproject.org/torproject.org tor-nightly-master-jessie > main The Tor version I tested was: > Tor version 0.3.1.3-alpha-dev (git-a73d0fe9a87df762+b433dff) Again, thank you very much, teor and Daniel! I really appreciate your help! Best, iry [0]: https://trac.torproject.org/projects/tor/ticket/1922 [1]: https://packages.debian.org/source/experimental/tor -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZUb3/AAoJEKFLTbxtzdU8jQsP/2B29N3WpkypE7DQ1A2wtjd9 MP3Blz9lbJH8LQJw+juaHhzslokCXwpGJSl7OfyqhRL44VkXnTllfd88HacW5luM PqZ4OeljB2UYrpDM7TypjdA+RXmSLTKNCmFifCESQPcmsu97qxlcvkgtF69fJ8eX p/22PEOjsqnows37rYp0AZiLa7x9I1dZdjzD6tHfKNVylM5AuuyOExAZKlx90rlW lL1nCJEuw/d6HBtVWWBgYDjeAMg5G0TpO2gU1j/A7kEzgZxWID6q/r1TvpDp1pi3 aJhjmyeZn2rVwdGPFX03pp7DWJetbYA5CuhGFucjPtkdXDCh2guuSsivcM6obpiA EmI4QtedGGhy2Xp1ufLAHP89TuuSU1n9VaioHSpLlGkQA9v2PRjW4ChBVJwP6mVt 9XGhY7TgEQMVZWL/brCNI6aeCf8rfaBSHtMwAR6xQ6ofvYAjl7X1BpNOSx4s73B/ CWnGxvMNDPuu1scxkOF0y3M68p+xgT0UuHTx4JjUTQtESFTqtdua5JxLyzE2qGyz pwzFjFzwoPBav5TPY7MB8WN79GImZFUiELA6vOL4CLaXEMCepsXYDhdP/3JLkD0e GQfliduoa2KGUijKrzy+ZVrKwBjF63ylbKGrxJHnX1SB8OftYbdlIQRvhzwHwaOL uSfnWGIg3Fo0fKs1/F/Q =L3H9 -----END PGP SIGNATURE-----

3 2

Names for your onions
by heddha 25 Jun '17

25 Jun '17

Good day everyone! A friend and I are looking into implementing a few of the ideas listed in [0] as part of a university project. We saw quite a lot of discussion on idea 1, but we're mainly interested in working on ideas 2.5 and 3, and additionally either 2.1 or 4. Has there been discussion and work on these yet? Where can we find it? Is there anything else that you can tell us that we should know? Thank you very much for your time! Kind regards, heddha [0]: https://blog.torproject.org/blog/cooking-onions-names-your-onions

3 3