- tor-dev - lists.torproject.org

onionoo details document deterministic output
by nusenu 28 Apr '15

28 Apr '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, if I ask two onionoo instances running the same version (as given in the header of the document) for the same details document, to what extend should they match if relays_published matches, sorting is been taken care of and lets assume that both instances use the same maxmind files and their DNS servers provide the same answers? One example: Torprojects instance says: "flags":[] cthulhu's instance says: "flags":[""] thanks, nusenu -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVN+saAAoJEFv7XvVCELh0ywEP/RNsdjuOywrJMAuaSlLLwCYs eHK0EUcmq/DDyq2OaAK9wOgAYqtusgBp5+YhiL11dBAvuW2siTZJPiKcoj8UQk84 m6xq9FCImlxLIEe52YP+wz2t8y6p42oAiSqlMLgtBVjW6QvgK3ov0e++iEhbJbvj hKLeSPsVLHz5XWtUdSjQ5fbbpsuItzTphOc9bi/nWGGYzeRzBNdtXilU5afCysi5 GvxV2SdttRrZw0IVzbLs0CjZBBwxhiuR8yemjRTaL7QDBgS2x0TMCxqhDmVXbp70 eqwGf5Rvf9AYgGu6i+3CJk9YAPh4vcLx1XS8hKxwreIXsUpWGGOfZs3VCGVFcZKe T9oQKQgy/EX0DSi5CSU5mbAUB0XT6S6gzh2Q7ZrsYfMWOaZXuawfeyBcvp3l/gSQ vkZ496ilDsc0DK+z0rrGLNbabx3CoQmHBr8kJQS9ZqAzZEMuDETXKwOBD+IVvSYh wo7nCU9o0NrzuwQCLdjGWr7WoInMLnASgySit8sGsbjzGTigwMvbg6LD4VlUQmcj NFaofSw/KcTUDf2+ZFP5TdXOizaPHeYxfyXn+B8ntqktYav86ePpaCUtq50HMEl9 6BTKvVFcCHkpUbgYTTDFW6YMM1qSvwmODiyKQbkWaB3sItdiPZaHy5ejPSEfDRCz FOY1pHCNGjCJWjrmY5o6 =YbXp -----END PGP SIGNATURE-----

2 10

onionoo: increasing first_seen granularity + providing document fingerprints
by nusenu 28 Apr '15

28 Apr '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, timestamps are useful to link relays, unfortunately onionoo only says in which consensus the relay was first seen (granularity: 1hour) but does not include the timestamp of the first seen descriptor that ended up in the consensus (published line). That timestamp would have a higher granularity (seconds). What do you think about adding such a timestamp? (I acknowledge that I might be the only user of this field) I might want to prove to third parties that I'm indeed processing/providing authentic historic onionoo documents from onionoo.tpo. What do you think of signing them? thanks -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVOpLqAAoJEFv7XvVCELh0WaIP/jqLtHaeno0EEUNPeqT8rJQp XRV76Vfq6LOciexFBRfcN2p7kkG26Y1LunRs12QuRpJngNfSW/nu3/WNUruR205D PrFJCI2Olxn3s8Fktd/N5kNLJbtPDmzqkghhvWUP+f1MBuW0FstMWi4fZ+TyerE9 FTD8eTF8woFb/JEHIrKsH4LB5Os/uGCoJbWT+AlxKHQcU/mQn0rb4ZIIllbUr9LJ uuFUV7ajZAR/rBX2AWvpkJgaS3BfOk3cMOVd3dQgweDjtUlS66Q5v+vtvBHXYk1V jd+fXyQ3txFmFVj/oQgnJJFRmGSmqpm5hRWEkzOXjXlCdsnqMPUBBjwjrgU+vmGB sNQOi/NdXUImQ0bT+ryjPTOT7Gf5EXOihYyH4e4ebn5OEEEVTQh/86iXxrLB23I2 z7sciEw3q/Tt7mM21cSMRStxavBU5InixIvQz78TgK1sE9PpeohzhToxNedm8aDE Ny/O+KYBBBmKOuRwjmbyJQ86MxQHnmEwLhksVNSJ3MUuG0QmfFTrht3thkmlew6z UfFBS9YzfBIoztL7X5Pu4nFsNFy2ffxZIV/bs4Dmx1PlweIw0eUeCgUdcdZTcvA+ v4OjCukdzLupDpb8lhDuX5EqrmeYUnczlopMX3szeX8udLKr5G+MWaihGdKasdKw Fy6T4z11bbzuW/nBf6Ra =oJ6y -----END PGP SIGNATURE-----

2 1

ANN: OnionPy 0.3.0
by Lukas Erlacher 28 Apr '15

28 Apr '15

Hello all, I've updated OnionPy to respect the new "version" field in query responses. There are one or two new objects OnionPy doesn't parse fully, which I hope I can get to soon. Signed builds are on PyPI. [1] OnionPy is a pure-python OnionOO request wrapper that supports transparent caching. If you are planning to make something in python that uses the tor network status, accessing OnionOO [2] using OnionPy might be exactly what you need. If you are interested, check out the repo and readme! [3] Of course, criticism, suggestions, and pull requests are always welcome. Best regards, Luke PS: Yes, I know, the last update to the OnionOO protocol happened in December, so "new" is relative ;-) [1] https://pypi.python.org/pypi/OnionPy [2] https://onionoo.torproject.org/ [3] https://github.com/duk3luk3/onion-py

1 0

tor's "make test" fails when run on tails
by warms0x 27 Apr '15

27 Apr '15

I am trying to execute tor's tests from git (63a90f2) on tails but the socketpair_ersatz test keeps failing util/socketpair: [forking] OK util/socketpair_ersatz: [forking] FAIL src/test/test_util.c:4212: assert(0 OP_EQ tor_socketpair_fn(family, SOCK_STREAM, 0, fds)): 0 vs -111 [socketpair_ersatz FAILED] util/max_mem: OK The failure seems to be in connect(): https://gitweb.torproject.org/tor.git/tree/src/common/compat.c#n1492 armadev said on IRC that he/she could make the tests pass, so I wonder if there might be some specific behavior on tails that prevents the test from passing successfully. Is there a suggested workaround? Or perhaps is it possible to easily ignore this one test?

3 2

Bridge Clients à la Vidalia
by Kenneth Freeman 25 Apr '15

25 Apr '15

This may be a naïve question, but how do I parse the clients who have used my bridge relay on my Debian box à la Vidalia? Just set the syslog to info and run a search or script? I routinely tell my friends which nation-states have become especially active (an interesting metric of geopolitical turbulence in and of itself). Thanks!

2 1

HTTPS Everywhere harmful
by Maciej Soltysiak 25 Apr '15

25 Apr '15

http://www.w3.org/DesignIssues/Security-NotTheS.html Maciej

4 3

Impending Facebook Onion-Site Certificate Changes
by Alec Muffett 24 Apr '15

24 Apr '15

Hello All! Details: https://www.facebook.com/facebookcorewwwi/posts/703469239759799 <https://www.facebook.com/facebookcorewwwi/posts/703469239759799> Summary: new EV-style SSL certificate rolls out to facebookcorewwwi.onion next week. - alec — Alec Muffett Security Infrastructure Facebook Engineering London

1 0

Re: [tor-dev] Experimenting with private tor setup
by teor 23 Apr '15

23 Apr '15

> Date: Wed, 22 Apr 2015 01:05:44 -0400 > From: CJ Ess <zxcvbn4038(a)gmail.com> > > On Tue, Apr 21, 2015 at 9:01 AM, teor <teor2345(a)gmail.com> wrote: > > > > Date: Tue, 21 Apr 2015 02:13:48 -0400 > > > From: CJ Ess <zxcvbn4038(a)gmail.com> > > > > > > I've been experimenting with a private tor setup - I've managed to setup a > > > couple directory authorities, six routers/exit nodes (which seemed to be > > > the minimum to bootstrap everything), and a client. … > > > So I'm wondering what would happen if I set TestingTorNetwork to 0, so I > > > picked one onion router instance and made that change. > > > > > > Because its all running on one box I had to keep some of the special > > > settings: > > > DirAllowPrivateAddresses 1 > > > EnforceDistinctSubnets 0 > > > AuthDirMaxServersPerAddr 0 > > > AuthDirMaxServersPerAuthAddr 0 > > > ExtendAllowPrivateAddresses 1 > > > > > > And that almost works, I got this far: > > > > > > Apr 21 00:50:09.000 [notice] Bootstrapped 100%: Done > > > Apr 21 00:50:09.000 [notice] Now checking whether ORPort > > > xxx.18.110.101:5106 is reachable… (this may take up to 20 minutes -- look > > > for log messages indicating success) > > > Apr 21 01:10:09.000 [warn] Your server (xxx.18.110.101:5106) has not > > > managed to confirm that its ORPort is reachable. Please check your > > > firewalls, ports, address, /etc/hosts file, etc. > > > Apr 21 01:30:09.000 [warn] Your server (xxx.18.110.101:5106) has not > > > managed to confirm that its ORPort is reachable. Please check your > > > firewalls, ports, address, /etc/hosts file, etc. > > > > > > It looks like the "is reachable" is determined by opening a circuit. … > > > I don't see any warnings or errors, it looks like the circuits are being > > > opened successfully. Any ideas why this doesn't translate to getting past > > > the ORPort being reachable test? > > > > If you're using a version of tor before 0.2.6.1, then it's possible that > > bug #13924 may be the culprit: > > https://trac.torproject.org/projects/tor/ticket/13924 > > "Reachability testing and channel is_local assume private addresses are > > local" (and therefore discount the local/private addresses for the purposes > > of reachability testing). … > I found the problem here: > https://github.com/torproject/tor/blob/cc10f13408e25eaf04f849d0f761680f383f… > > The check for channel_is_local returns true because my dev box has an > 172.16/12 address: > > https://github.com/torproject/tor/blob/725d6157df150ec9151450dc2422d9838c20… > > And thats what is keeping everything from working. The 172.16/12 block is a private address block. https://en.wikipedia.org/wiki/Private_network tor assumes that connections from private addresses are local. It doesn't update its reachability self-testing flag unless it gets a connection from a non-local address. This restriction is relaxed when TestingTorNetwork is set, as many test networks use private address blocks. > I commented out that > address block and now orport, dirport, and bandwidth tests are running. Commenting out a private address range is a dangerous change that has at least the following security implications: * "ExitPolicyRejectPrivate 1" won't stop clients exiting to the block you've commented out (warning, you have this set to the default, 1) * "ExtendAllowPrivateAddresses 0" won't stop clients extending to the block you've commented out (but you have this set to 1) * "DirAllowPrivateAddresses 0" won't exclude directory nodes with IPs in the block you've commented out (but you have this set to 1) As long as you trust your entire network, including any clients that can connect to any open ports, this should not be a problem. But please don't use versions of tor with this change on the public Tor network, you'd be opening yourself up to attack. It appears that my original fix to bug #13924 didn't anticipate anyone using "ExtendAllowPrivateAddresses 1" without "TestingTorNetwork 1". I'll submit a patch which swaps TestingTorNetwork for ExtendAllowPrivateAddresses. This will preserve the existing fix, because "TestingTorNetwork 1" implies "ExtendAllowPrivateAddresses 1". The relevant line is here: https://github.com/torproject/tor/blob/cc10f13408e25eaf04f849d0f761680f383f… In the interim, if you'd like a safer option, you could make the same code change to that line, or disable reachability testing entirely using "AssumeReachable 1". teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7

2 3

Re: [tor-dev] tor-dev Digest, Vol 51, Issue 34
by Rekha Tokas 23 Apr '15

23 Apr '15

Can I get the list of websites which are blocking tor traffic to view or edit their page ?? On 23 Apr 2015 17:30, <tor-dev-request(a)lists.torproject.org> wrote: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Re: Experimenting with private tor setup (teor) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 23 Apr 2015 21:27:40 +1000 > From: teor <teor2345(a)gmail.com> > To: "tor-dev(a)lists.torproject.org" <tor-dev(a)lists.torproject.org> > Subject: Re: [tor-dev] Experimenting with private tor setup > Message-ID: <4942EA68-C681-4223-8A3B-A5E0C97E296F(a)gmail.com> > Content-Type: text/plain; charset="utf-8" > > > Date: Wed, 22 Apr 2015 23:14:07 -0400 > > From: CJ Ess <zxcvbn4038(a)gmail.com> > ? > > I think that just leaves a couple questions for now: > > > > - I know the directory authority's certificate needs to be > > renewed periodically (looks like 12 months default). Since it asked for a > > password that is something I need to do manually then send a HUP, > correct? > > Later on I'll try to hack generating a cert with days or hours of time > just > > to see what happens, but would like to know what the correct way to > handle > > it is. > > Yes, you will need to generate new authority certificates when they expire. > > Look at "man tor-gencert" for more details - you may be able to enter a > blank password, or generate a new certificate and send a HUP on a cron job. > Alternately, look at the chutney source code for how it's done there. (This > isn't my area of expertise.) > > > - All of my routers have done the bandwidth tests, but I see this warning > > on the DAs with each vote: > > > > Apr 22 22:39:41.000 [warn] Consensus with empty bandwidth: G=0 M=5 E=0 > > D=243 T=248 > > > > Is this something that will fill out as the uptime of the instances > > increase? Or is this something I should look into? > > I think you need to run a separate bandwidth authority to get bandwidths > in your consensus. This probably isn't necessary for your purposes, as your > instances are all on the same host. > > It's also not something that chutney does, and chutney networks function > fine. > > teor > > teor2345 at gmail dot com > pgp 0xABFED1AC > https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 > > teor at blah dot im > OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7 >

1 0

Re: [tor-dev] Experimenting with private tor setup
by teor 23 Apr '15

23 Apr '15

> Date: Wed, 22 Apr 2015 23:14:07 -0400 > From: CJ Ess <zxcvbn4038(a)gmail.com> … > I think that just leaves a couple questions for now: > > - I know the directory authority's certificate needs to be > renewed periodically (looks like 12 months default). Since it asked for a > password that is something I need to do manually then send a HUP, correct? > Later on I'll try to hack generating a cert with days or hours of time just > to see what happens, but would like to know what the correct way to handle > it is. Yes, you will need to generate new authority certificates when they expire. Look at "man tor-gencert" for more details - you may be able to enter a blank password, or generate a new certificate and send a HUP on a cron job. Alternately, look at the chutney source code for how it's done there. (This isn't my area of expertise.) > - All of my routers have done the bandwidth tests, but I see this warning > on the DAs with each vote: > > Apr 22 22:39:41.000 [warn] Consensus with empty bandwidth: G=0 M=5 E=0 > D=243 T=248 > > Is this something that will fill out as the uptime of the instances > increase? Or is this something I should look into? I think you need to run a separate bandwidth authority to get bandwidths in your consensus. This probably isn't necessary for your purposes, as your instances are all on the same host. It's also not something that chutney does, and chutney networks function fine. teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7

1 0

Summer of Privacy: Enhance GetTor
by ilv 22 Apr '15

22 Apr '15

Hi people, My name is Israel Leiva. I'm a student of BSc. in Computer Science at Universidad de Santiago de Chile. I'm also the maintainer of GetTor and one of the students selected for Tor Summer of Privacy. This summer I'll be working on GetTor, improving what I did in last year's GSoC and adding new stuff. In short, I will: * Improve stats and debugging. * Deploy more providers: GDrive, GitHub. * Enable more distributions channels: XMPP, Twitter. * Create tests for GetTor. * New features, like send mirrors list and localized Tor Browser. I started to work on most of this stuff in last year's GSoC, and I will finish it during this TSoP. For more details, you can read my proposal 'Enhance GetTor' on [0]. My mentors will be Sukhbir and Nima. I'm really happy to be working at the Tor Project again and I hope to do my best. If you have comments or new ideas about [0], I'm all ears [0] https://people.torproject.org/~ilv/sop_proposal_2015.html Have a nice day! --ilv

1 0

Re: [tor-dev] Experimenting with private tor setup
by teor 22 Apr '15

22 Apr '15

> Date: Tue, 21 Apr 2015 02:13:48 -0400 > From: CJ Ess <zxcvbn4038(a)gmail.com> > > I've been experimenting with a private tor setup - I've managed to setup a > couple directory authorities, six routers/exit nodes (which seemed to be > the minimum to bootstrap everything), and a client. With the latest versions of tor and chutney, the "basic-min" configuration bootstraps a testing tor network with 3 authorities (non-exits), 1 exit relay, and 1 client. I'm pretty sure this is the minimum number of tor instances to bootstrap, as bootstrapping requires each relay to create a 3-hop path through other relays which have already bootstrapped (or, in this case, the 3 authorities, which assume their own reachability). I'm not sure if you're using chutney to set up your network. chutney simplifies the setup of local tor networks using a templating and launch system. The latest version can be downloaded using: git clone https://git.torproject.org/chutney.git > Its a pretty normal > setup (aside from everything running on my development box) and passes > traffic as expected. > > So I'm wondering what would happen if I set TestingTorNetwork to 0, so I > picked one onion router instance and made that change. > > Because its all running on one box I had to keep some of the special > settings: > DirAllowPrivateAddresses 1 > EnforceDistinctSubnets 0 > AuthDirMaxServersPerAddr 0 > AuthDirMaxServersPerAuthAddr 0 > ExtendAllowPrivateAddresses 1 > > And that almost works, I got this far: > > Apr 21 00:50:09.000 [notice] Bootstrapped 100%: Done > Apr 21 00:50:09.000 [notice] Now checking whether ORPort > xxx.18.110.101:5106 is reachable... (this may take up to 20 minutes -- look > for log messages indicating success) > Apr 21 01:10:09.000 [warn] Your server (xxx.18.110.101:5106) has not > managed to confirm that its ORPort is reachable. Please check your > firewalls, ports, address, /etc/hosts file, etc. > Apr 21 01:30:09.000 [warn] Your server (xxx.18.110.101:5106) has not > managed to confirm that its ORPort is reachable. Please check your > firewalls, ports, address, /etc/hosts file, etc. > > It looks like the "is reachable" is determined by opening a circuit. > > I see this message several times in the logs: > > Apr 21 01:38:56.000 [info] channel_tls_process_netinfo_cell(): Got good > NETINFO cell from xxx.18.110.101:5002; OR connection is now open, using > protocol version 4. Its ID digest is > 41373151BCC461FEFEFCC1BAF6DCEFD89922014C. Our address is apparently > xxx.18.110.101. > > I don't see any warnings or errors, it looks like the circuits are being > opened successfully. Any ideas why this doesn't translate to getting past > the ORPort being reachable test? If you're using a version of tor before 0.2.6.1, then it's possible that bug #13924 may be the culprit: https://trac.torproject.org/projects/tor/ticket/13924 "Reachability testing and channel is_local assume private addresses are local" (and therefore discount the local/private addresses for the purposes of reachability testing). Alternately, you could have discovered a similar bug which occurs when all tor instances in a network are on the same, public IP address. Please feel free to log a bug if this behaviour is still showing up in tor versions after 0.2.6.1 - and reference #13924 so we know the history of the issue. https://trac.torproject.org/projects/tor/newticket You could also disable reachability testing entirely using: AssumeReachable 1 TestingTorNetworks also allow you to skip several boring minutes of bootstrap by just voting everything a guard and an exit (only in recent versions of tor): TestingDirAuthVoteExit * TestingDirAuthVoteHSDir * Let us know how you go. teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7

2 2

Experimenting with private tor setup
by CJ Ess 21 Apr '15

21 Apr '15

I've been experimenting with a private tor setup - I've managed to setup a couple directory authorities, six routers/exit nodes (which seemed to be the minimum to bootstrap everything), and a client. Its a pretty normal setup (aside from everything running on my development box) and passes traffic as expected. So I'm wondering what would happen if I set TestingTorNetwork to 0, so I picked one onion router instance and made that change. Because its all running on one box I had to keep some of the special settings: DirAllowPrivateAddresses 1 EnforceDistinctSubnets 0 AuthDirMaxServersPerAddr 0 AuthDirMaxServersPerAuthAddr 0 ExtendAllowPrivateAddresses 1 And that almost works, I got this far: Apr 21 00:50:09.000 [notice] Bootstrapped 100%: Done Apr 21 00:50:09.000 [notice] Now checking whether ORPort xxx.18.110.101:5106 is reachable... (this may take up to 20 minutes -- look for log messages indicating success) Apr 21 01:10:09.000 [warn] Your server (xxx.18.110.101:5106) has not managed to confirm that its ORPort is reachable. Please check your firewalls, ports, address, /etc/hosts file, etc. Apr 21 01:30:09.000 [warn] Your server (xxx.18.110.101:5106) has not managed to confirm that its ORPort is reachable. Please check your firewalls, ports, address, /etc/hosts file, etc. It looks like the "is reachable" is determined by opening a circuit. I see this message several times in the logs: Apr 21 01:38:56.000 [info] channel_tls_process_netinfo_cell(): Got good NETINFO cell from xxx.18.110.101:5002; OR connection is now open, using protocol version 4. Its ID digest is 41373151BCC461FEFEFCC1BAF6DCEFD89922014C. Our address is apparently xxx.18.110.101. I don't see any warnings or errors, it looks like the circuits are being opened successfully. Any ideas why this doesn't translate to getting past the ORPort being reachable test?

1 0

Possible anomaly in meek user graph circa April 15, 2015
by David Fifield 21 Apr '15

21 Apr '15

The latest meek user graph shows two recent large increases. The first increase from 2000 to 3000 is around April 9. The second from 3000 to 5000 is all on April 15. The first increase makes sense; it corresponds with the removal of a bottleneck on meek-azure: https://lists.torproject.org/pipermail/tor-dev/2015-April/008637.html The second one happens all in one day and I don't have an explanation for it. Here are the day-by-day numbers, the same numbers from which the graph is derived: ↓↓ 2015-04-09,bridge,,meek,,,,2584,54 2015-04-10,bridge,,meek,,,,3363,54 2015-04-11,bridge,,meek,,,,3392,51 2015-04-12,bridge,,meek,,,,3531,52 2015-04-13,bridge,,meek,,,,3344,49 ← 2015-04-14,bridge,,meek,,,,5385,53 ← 2015-04-15,bridge,,meek,,,,5113,52 2015-04-16,bridge,,meek,,,,4990,52 2015-04-17,bridge,,meek,,,,5173,51 2015-04-18,bridge,,meek,,,,5422,33 I at first suspected conflation with the statistics of some other transport, like we observed with meek and websocket back in September 2014. https://lists.torproject.org/pipermail/tor-dev/2014-September/007483.html There is a nearly simultaneous increase in obfs4 (also attached); I didn't see anything similar in any other transports. However I checked Onionoo and the only bridges running meek are ones I know about, and they aren't running any other transports. https://onionoo.torproject.org/details?type=bridge {"nickname":"UtahMeekBridge","hashed_fingerprint":"88F745840F47CE0C6A4FE61D827950B06F9E4534","or_addresses":["10.45.162.225:8000"],"last_seen":"2015-04-20 19:48:46","first_seen":"2014-10-24 20:37:04","running":true,"flags":["Fast","Guard","Running","Stable","Valid"],"last_restarted":"2015-04-13 19:14:56","advertised_bandwidth":6517470,"platform":"Tor 0.2.5.12 on Linux","transports":["meek","meek"]}, {"nickname":"TorLandMeek","hashed_fingerprint":"55CD4494A9EC14B74A677A7B7DD356E3417F9E69","or_addresses":["10.60.60.122:443"],"last_seen":"2015-04-16 17:45:30","first_seen":"2015-04-15 18:44:44","running":false,"flags":["Valid"],"last_restarted":"2015-04-15 18:24:47","advertised_bandwidth":3540992,"platform":"Tor 0.2.5.10 on Linux","transports":["meek"]}, {"nickname":"starman","hashed_fingerprint":"AA033EEB61601B2B7312D89B62AAA23DC3ED8A34","or_addresses":["10.243.156.101:9002","[fd9f:2e19:3bcf::24:d1d2]:9002"],"last_seen":"2015-04-20 19:48:46","first_seen":"2014-09-16 06:37:04","running":true,"flags":["Fast","Guard","Running","Stable","Valid"],"last_restarted":"2015-04-09 16:19:58","advertised_bandwidth":2434220,"platform":"Tor 0.2.5.12 on Linux","transports":["meek","meek"]}, {"nickname":"TorLandMeek","hashed_fingerprint":"3FD131B74D9A96190B1EE5D31E91757FADA1A4F3","or_addresses":["10.116.43.196:443"],"last_seen":"2015-04-20 19:48:46","first_seen":"2014-10-14 06:37:04","running":true,"flags":["Fast","Guard","Running","Stable","Valid"],"last_restarted":"2015-04-15 19:02:22","advertised_bandwidth":4125225,"platform":"Tor 0.2.6.7 on Linux","transports":["meek"]}, David

1 0

Visualization of the lizard Sybil attack
by David Fifield 20 Apr '15

20 Apr '15

Check out these pictures of relays in the consensus during November and December. (Warning, the linked images are huge, just look at the downscaled attachments if you want.) November https://people.torproject.org/~dcf/graphs/microdescs/microdescs-2014-11.png (720×13242 pixels) December (spot the anomaly) https://people.torproject.org/~dcf/graphs/microdescs/microdescs-2014-12.png (744×18136 pixels) Each row is one descriptor, and each column is one hour. The rows are sorted so that descriptors with similar uptimes are nearby. The conspicuous narrow vertical stripe in the lower right of the December graph is a few thousand descriptors entering the consensus all at once and then leaving it again. If you want to dig into any patterns you see, the lines of these text files correspond 1:1 to the rows of the images. https://people.torproject.org/~dcf/graphs/microdescs/microdescs-2014-11.txt https://people.torproject.org/~dcf/graphs/microdescs/microdescs-2014-12.txt Source code is in https://people.torproject.org/~dcf/graphs/microdescs/. wget https://collector.torproject.org/archive/relay-descriptors/microdescs/micro… tar xJf microdescs-2014-11.tar.xz tar xJf microdescs-2014-12.tar.xz ./microdescs --output microdescs-2014-11 microdescs-2014-11 ./microdescs --output microdescs-2014-12 microdescs-2014-12 David Fifield

5 7

Urdu & Hindi translations of Tor browser ?
by Griffin Boyce 19 Apr '15

19 Apr '15

Hello all, Whenever I attend events with a large Pakistani or Indian contingent, I'm asked why there isn't an Urdu or Hindi translation of Tor Browser. And I'm not totally sure what to say. There's clearly a large need, given Pakistan's history internet censorship. At a recent event in DC, an activist from Pakistan spoke with me about increased surveillance in recent years, as well as the shocking trend of targeting activists with charges of blasphemy for criticizing the government. (Blasphemy is punishable by death, and those accused frequently do not survive until trial due to mob violence). The situation in India is a bit different, but their need for online privacy much the same as like Pakistan they are subject to mass surveillance. Both populations also have a large number of speakers: ~300M for Hindi and ~66M for Urdu. What do you think? ~Griffin -- “Sometimes the questions are complicated and the answers are simple.” ― Dr. Seuss

4 6

Fwd: Games Without Frontiers: Investigating Video Games as a Covert Channel
by Rishab Nithyanand 19 Apr '15

19 Apr '15

Hey all, I just thought I'd share and get feedback about some recent work from our team at Stony Brook University. I posted this to the tor-talk list earlier and it was suggested to x-post here, too. Title: Games Without Frontiers: Investigating Video Games as a Covert Channel [ http://arxiv.org/pdf/1503.05904v1.pdf ] Abstract: The Internet has become a critical communication infrastructure for citizens to organize protests and express dissatisfaction with their governments. This fact has not gone unnoticed, with governments clamping down on this medium via censorship, and circumvention researchers working tirelessly to stay one step ahead. In this paper, we explore a promising new avenue for covert channels: using video games as a cover protocol. The popularity of platforms like Steam have given rise to a rich population of video games for use as cover. The common properties of games in the same genre simplify the process of adapting channels to evade detection. We demonstrate the feasibility of this approach using two real time strategy games (including a popular closed-source game). We show how common properties of these games can be used to design a coding scheme to translate data into game commands in a way that is general across games and requires little per-game customizations. We evaluate the security of Castle by quantifying its resilience to a censor-adversary, its similarity to real game traffic, and its ability to avoid common pitfalls in covert channel design. We use our prototype to demonstrate that Castle can provide throughput which is amenable to transfer of textual data (\eg e-mail, short articles, etc.). I thought this was a cool idea, and had a lot of fun exploring it. One of the key changes that Castle brings to the PT landscape is easy extensibility and large space diversity -- i.e., it takes very little skill and developer time to port Castle to any of 100's of existing (and perennially releasing) RTS games, in the event that it is detected and blocked for one game. Please check out the paper and feel free to post questions here, or email them to me. Rishab

3 2

Re: [tor-dev] shipping with fallbackdir sources
by teor 18 Apr '15

18 Apr '15

> Date: Sat, 18 Apr 2015 08:51:35 +0200 > From: Peter Palfrader <weasel(a)torproject.org> > >>> On Fri, 17 Apr 2015, Jacob Appelbaum wrote: >>> >>> On 4/17/15, Ian Goldberg <iang(a)cs.uwaterloo.ca> wrote: >>> Remember that the purpose is to help first-time clients, who have never >>> used the Tor network before, fetch their very first consensus. To do >>> that, they'll need addresses to contact bundled into the binary and/or >>> distro without being able to look at a consensus first. Disabling fallback directories in custom Tor networks (Trac #15642) tor currently doesn't disable the list of fallback directories when the DirAuthorities or Alternate{Bridge,Dir}Authorities options are set. This means that any fallback directories will still be used in test, private and custom Tor networks. This could mix descriptors from the public network in with private Tor network in certain circumstances (at the very least, when the private authorities are unreachable). I suspect this is an undiscovered bug in the fallback directories feature. It hasn't really been an issue until now, as the list of fallback directories has been blank. I am also concerned that this general area of the code lacks unit tests, which it might be wise to include before we effectively activate it for the first time. I've created a git branch with a fix in the trac issue #15642 [0]. I've also included unit tests for the function I've modified as a commit in that branch. But there's currently no coverage for the function that adds fallback directories. (In fact, I mock it in my unit tests, because I need it to do *something* so I know if it has been called or not.) Loading from an Authority-Signed file of Fallback Directories The function which loads fallback directories currently loads from a string array inside the function, so it would need to be modified to load from a signed file. I support the security benefits of signed fallback directories enough to write client code and unit tests for it, but I'm not sure how the code for the authorities would work - is the proposal to sign a section of the consensus, and output it as a separate file? If so, we would either need to backport, and/or wait until a majority of the authorities update to tor versions with the feature. And perhaps a majority of clients as well, controlled by a consensus parameter? (Otherwise, using any entry in the file itself would allow clients to effectively be partitioned from the rest of the network by their behaviour.) While I'm making a list, do we need to modify the existing proposal which describes fallback directories? Is this change proposed for 0.2.7? Or all currently supported releases? 0: https://trac.torproject.org/projects/tor/ticket/15642#comment:8 teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR C3C57B23 349825DE 929A1DEF C3531C25 A32287ED

1 0

shipping with fallbackdir sources
by Peter Palfrader 18 Apr '15

18 Apr '15

Hi, so, Tor has included a feature to fetch the initial consensus from nodes other than the authorities for a while now. We just haven't shipped a list of alternate locations for clients to go to yet. Reasons why we might want to ship tor with a list of additional places where clients can find the consensus is that it makes authority reachability and BW less important. At the last Tor dev meeting we came up with a list of arbitrary requirements that nodes should meet to be included in this list. We want them to have been around and using their current key, address, and port for a while now (120 days), and have been running, a guard, and a v2 directory mirror for most of that time. I have written a script to come up with a list of notes that match our criteria. It's currently at https://www.palfrader.org/volatile/fallback-dir/get-fallback-dir-candidates It currently produces https://www.palfrader.org/volatile/2015-04-17-VjBkc8DWV8c/list Discuss :) -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `- http://www.debian.org/

3 7

Release: obfs4proxy-0.0.5
by Yawning Angel 17 Apr '15

17 Apr '15

Hello all, I just tagged obfs4proxy-0.0.5, this time with improvements for both clients and servers. All users are recommended to upgrade. Special thanks to mvdan for various code cleanups. Tarball/Signature: https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.5… https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.5… Changes in version 0.0.5 - 2015-04-15: - Go vet/fmt fixes, and misc. code cleanups. Patches by mvdan. - Changed the go.net import path to the new location (golang.org/x/net) - Added limited support for detecting if the parent process crashes. - Support for tor feature #15335 (stdin based termination notification). - Moved the leveled logging wrappers into common/log so they are usable in transport implementations. - Added a DEBUG log level. - Use a bundled SOCKS 5 server instead of goptlib's SocksListener. The big features are: - obfs4proxy now tries really hard to detect if the parent process has crashed, using various system specific hacks (and the new and improved tor assisted mechanism in #15335). This should reduce defunct obfs4proxy processes when tor happens to crash without tearing down the obfs4proxy instance. - Instead of using goptlib's SOCKS4a server, obfs4proxy now includes a SOCKS5 implementation, bringing IPv6 support to clients. Note that running IPv6 bridges has always worked (though dual stack configs are currently somewhat broken due to a tor PT configuration protocol limitation). Notes for packagers: - Like in obfs4proxy-0.0.5, one of the upstream dependency locations has changed. (code.google.com/p/go.net -> golang.org/x/net) As far as I am aware, migrating to the new package immediately is not required though, should be done sooner rather than later due to the impending deprecation of code.google.com. To temporarily continue to build against the old go.net dependency, please revert: aed4b723891db1be34eb866a03c62806b58ac148 - It is *strongly* recommended that packages be built against goptlib-0.4 or newer to work around tor bug #15240. Without this workaround, certain bridges will fail to operate correctly when the ExtORPort is enabled (the Tor side fix is in tor-0.2.6.5-rc and newer). Questions, comments, feedback appreciated, -- Yawning Angel

3 3

Summer of Privacy application, Censorship Analyzer
by Miquel Llobet 16 Apr '15

16 Apr '15

Hey tor-dev, I'm a third year Computer Science student interested in doing the Censorship Analyzer project this summer. I just found out about the Summer of Privacy program so I'll do my best to be on time. I find the project very exciting as it can potentially help the TorProject and its users in a big way, specially if we can receive feedback from the tests. I've done a first read of the Censorship Analyzer paper and I'm currently working on the proposal, I expect to submit by tomorrow. As far as coding goes, I played a bit with OONI (did a scan, turns out I'm clean :-) ). and built it from source. What bugs to you recommend to work on as a start? Ideally I can write a patch before the submission is due to attest my skills. Let me know if you have any suggestion or advice on pursuing this project. Best, Miquel

3 3

Should popularity-hiding be a security property of hidden services?
by George Kadianakis 16 Apr '15

16 Apr '15

Hello, Tor hidden services are meant to primarily provide server anonymity but they also provide various other properties. For example, their addresses are self-authenticated and their connections punch NAT. This post is about another property, which is that Tor does not reveal the popularity of a hidden service by default. That is, you can't easily get the user count of a specific hidden service. This is not that surprising to hidden service operators, since that's also how the normal Internet works. In the normal Internet, someone cannot learn the user count of an IP or website, except if they are the operator or they control DNS or the site publishes analytics. Over the past years, people have suggested various features that would provide us with interesting information and optimizations but would have the side effect of revealing the user count of hidden services (or only of popular hidden services) to the public. Some examples of such popularity-leaking features: - Hidden Services dynamically set their number of Introduction Points depending on their popularity. Basically, they self evaluate their popularity, and use a formula to decide the number of Introduction Points between 3 and 10. For a while, we thought that this formula does not work properly (#4862, #8950), but we recently discovered that it seems to be working in some manner. While an interesting and useful feature on its own, it has the side effect that it leaks how popular your hidden service is. Since a hidden service publishes a descriptor every hour, you can monitor hourly usage patterns of hidden services. Of course, you can't get the exact user count, but you might be able to get rough approximate numbers (we still haven't analyzed the formula enough to know *exactly* how much). - During our recent work on hidden service statistics [0] people have suggested to gather statistics that would get us closer to learn the number of hidden service users [1]. The suggested way to do so is to have HSDirs or introduction points count the total number of introductions or descriptor fetches and publish that number in their extra-info descriptor. Since given a hidden service address you can easily learn its HSDirs or IPs, it should be possible to map those statistics to specific hidden services, which would leak their popularity (more on this later). There might be more examples that I'm missing, but this should be enough to demonstrate the leaks. For the rest of this post, I will be presenting various arguments for and against leaking popularity. Disclaimer: I still am not 100% decided here, but I lean heavily towards the "popularity is private information and we should not reveal it if we can help it" camp, or maybe in the "there needs to be very concrete positive outcomes before even considering leaking popularity". Hence, my arguments will be obviously biased towards the negatives of leaking popularity. I invite someone from the opposite camp to articulate better arguments for why popularity-hiding is something worth sacrificing. == Arguments for leaking popularity and reaping its benefits == Here are a few arguments that people use to shrug off popularity-hiding. I can relate to some of them, but I find the reasoning of some others funny or even dangerous. - "If we don't care about leaking popularity we can get useful statistics" Indeed, we've dismissed various statistics that we could collect because we were afraid that they would leak the popularity of hidden services. If we didn't have this fear, we would have a better idea on how much usage hidden services see, or whether people are conducting DoS attacks on hidden services. - "If we don't care about leaking popularity we can get nice optimizations" As an example, the dynamic IP calculation is one of those optimizations. I'm not aware of other optimizations, but I bet that we can think of a few more if we completely remove popularity-hiding from our threat model. Also, people have claimed that more statistics would reveal more optimizations that we could do. - "Popularity-hiding is just a side-effect of the Tor protocol, and not a stated security goal" People have claimed that popularity-hiding is not a stated security goal of hidden services, or that the name "hidden services" does not imply popularity-hiding in any way. - "There are no realistic attacks that could happen from leaking popularity" People have claimed that popularity is just a curiosity, and nothing bad can come from leaking it. They say that protecting popularity does not offer security against realistic or dangerous attacks. Other people claim that popularity-revealing attack vectors are too noisy and contain too much random data, hence it's hard to get targetted popularity values out of them. They say that it might only be possible for very popular hidden services, or for unlikely edge cases. - "There are probably other ways to reveal popularity. You can't fix them all" That's actually a big fear of mine. That we are nitpicking about 2-3 popularity revealing vectors, while there are hundreds more currently open. See #8742 for example, but I bet there are more vectors that we need to think about. == Arguments for protecting popularity == And here are arguments that make me believe that popularity is something that should be protected. - Popularity attracts attention Anonymity likes uniformity, but popularity attracts attention. There are literally infinite possible use cases where a hidden service wants to be public and still not attract attention. However, since the above argument has not been particularly successful and only attack demonstrations will persuade a true skeptic's mind, here is an attack scenario: Try hard to imagine a dystopian future where authorities are tracking down and hacking activist websites. They just received a big list of hidden services, the result of a messy interrogation, but they are all locked. Their hackers can hack some of them but not all. Not much time before revolution, end of dystopian future and happiness for all humanity. The dictator needs to decide which hidden services to hack to stop the revolution. Which?? With popularity being public , they can get the popularity of the biggest ones and target those first. - Popularity can be used to find patterns in group movements now and in the past. Even though you can't track specific users using popularity, you can still track group of users. Also, these statistics are forever: even if you didn't care about a group of users in the past, but you start caring about them now, you can still look back and see their development over time. Here is an attack scenario: Imagine a community that practices very dangerous urban climbing [2]. Imagine thousands of friends climbing away in happiness from all over the world, Imagine now that this community splinters in other smaller communities, if you monitor their popularity, it will be possible for you to observe the movement of that subculture. As a further point, imagine now that dystopian future comes and very dangerous urban climbing gets outlawed. The police catches an urban climber in New London and gets a list of hidden services from her. They can then check _historically_ how many users those hidden services had. They can basically notice all the trends of the urban climbing scene in the past years. Creepy, no? As you probably well know, anonymity is not a binary option. It's not like you are not either super anonymous, or not. It's more of a fuzzy variable that depends on many things. OPSEC is a big part of anonymity, and it seems to me that popularity has OPSEC consequences. - Statistics noise will get reduced. Attacks only get better. In the statistics we were talking about, each HSDir would reveal the number of descriptor fetches it received over the past day. We know that each HSDir serves about 150 hidden services, which means that the final value in the end will contain the popularity of 150 hidden services in one number. This is expected to be extremely noisy, and I think that's one of the main hopes of people who don't care about popularity hiding. That allows them to claim that popularity will only be leaked for very popular hidden services. While this indeed seems reasonable, my main intuition is that attacks can only get better. Here are some ways that noise can be reduced. I will focus on the HSDir case, but same arguments apply to other suggested statistics like number of introductions per IP. -- It's still early in the hidden services scene, so not many services get lots of traffic. I imagine that many of those 150 hidden services are going to be very inactive, and not provide much noise. -- Hidden services publish hidden service descriptors to 6 HSDirs. This means that every day you will learn 6 noisy values for your target hidden service, not just 1. It's easier to remove noise that way. -- Also, those 5 irrelevant hidden services that provide the noise will publish themselves to 6 HSDirs. Applying the same logic as above, you might be able to learn information about the noise, which makes it easier to remove. In a way, you can put all the statistics measurements in a big system of equations, and start solving it to reduce noise in the equation you are interested in. -- Think of crazy edge cases. Maybe an introduction point is very weak and unlikely to be picked and only got 10 HSes for a day. If one of them is the hidden service you are interested in, there is going to be much fewer noise than usual. -- There might be other techniques for reducing noise, by combining other statistics (like the number of hidden services per HSDir which is already a stat), or by influencing the statistics yourself (like Aaron's attack on the stats aggregation protocol [3]). What I'm trying to say here, is that if you thought that the urban climbing example was ridiculous because such a community cannot be big enough to be visible in noisy statistics, maybe by reducing noise you can actually make it distinguishable. - There are not that amazing benefits from ditching popularity-hiding. To be honest, I have not heard convincing enough arguments that would make me ditch popularity hiding. Some extra statistics or some small optimizations do not seem exciting enough to me. Please try harder. This could be a nice thread to demonstrate all the positive things that could happen if we ditch popularity-hiding. Also, there is a small difference here between the stats and the introduction point formula. The dynamic introduction point formula is something that we could disable by default, but also leave it as a configurable option for people who want to use it. That is, it will then be *the choice of the hidden service operator* whether he cares about popularity being hidden or not. With the statistics that have been proposed, you don't give any choice. You just do it for all hidden services forever. - Principle of least surprise Hidden service operators except that hidden services are at least as secure as the normal Internet plus more. On the normal Internet, popularity is private by default. Having this assumption violated on hidden services, might not be polite. - Popularity-hiding is crucial to maintain the deep sea security model of hidden services As I have mentioned in the past, some people think of the onion land as a very deep ocean. In some places of the ocean, you might be able to see some buoys (some more visible than others). To visit them, you need to wear your goggles and your snokrel, dive in and enter from underwater. This might not seem like a very concrete security model, but in any case popularity is not revealed at any point. The sea is opaque and you can't see the divers entering the hidden services. Anyway this post has grown to immense size, and I was really hoping it would be shorter. On a more practical note, over the next few weeks, we should decide what we want to do with the dynamic introduction point formula and whether we should keep it or not (#4862). My current intuition is that it should be disabled but also kept there as an option for people who want to enable it. In any case, I hope that this thread can stimulate discussion. Also, if you are a hidden service operator I'm curious to hear about whether you believe that popularity hiding is a security property that should be preserved if that's even possible. Cheers! [0]: https://blog.torproject.org/blog/some-statistics-about-onions [1]: https://lists.torproject.org/pipermail/tor-dev/2015-February/008247.html [2]: https://www.youtube.com/watch?v=kpS7vhvkIQM [3]: https://lists.torproject.org/pipermail/tor-dev/2015-March/008404.html

4 3

TOR SoP proposal: extending and improving TOR network anomaly detection
by Kibo Schaffer 15 Apr '15

15 Apr '15

Hi all, I'm applying for TOR Summer of Privacy. I'm curious if there are any developers here who are able and willing to become a mentor for this idea. I want to improve TOR's ability to detect anomalies such as sybil attacks, and make it easy to include other heuristics for other potential attacks. When a potential attack is detected, users and maintainers are notified (as necessary). There has been research and development with this field with TorDoctor, exitmap, and HoneyConnector. However, as far as I am aware, these projects could use some help being solidified and integrated into TOR. Please let me know what you think, if you want to suggest any improvements, and / or if you are a mentor interested in this idea. Cheers, Kibo [1] TorDoctor -- https://gitweb.torproject.org/doctor.git [2] exitmap / HoneyConnector -- http://www.cs.kau.se/philwint/spoiled_onions/

2 2

Tor Summer of Privacy deadline is soon!
by Damian Johnson 15 Apr '15

15 Apr '15

Hi all, just a quick friendly heads up that the deadline is coming up soon. If you've been thinking of making an application then now is the time! Application deadline for students is April 17th (end-of-day everywhere [1]). NO LATE APPLICATIONS WILL BE CONSIDERED. We're sorry about needing to be strict on this, but we have a really tight deadline for making selection decisions. Late applications make the process harder, so the only fair thing is to not allow any exceptions. Cheers! -Damian [1] https://en.wikipedia.org/wiki/Anywhere_on_Earth

1 0

Re: [tor-dev] TSOP proposal
by Alden Page 15 Apr '15

15 Apr '15

Hey Remi, It's great to see someone else interested in adversarial stylometry! I have been working on a project quite similar to your proposal since January as my undergraduate thesis. I am considering applying for TSoP in order to fund the continued development of the project, but I think the mentor issue is a strong indication that we should focus our efforts on something more related to Tor. Since I have made considerable progress implementing the tool you described, perhaps we should instead combine our efforts? Rachel Greenstadt, whose work I am sure you are familiar with, also expressed interest in collaboration when I announced this project in December. You will find that I have implemented many of your proposed features in my project on GitHub. Near-term work involves making the project easier to install and implementing Writeprint feature set extractors. If you manage to get it running (for now, you will have to install dependencies manually), I would love to hear some feedback from you! Until then, see my thesis. Contribute at: www.github.com/pagea/unstyle A preliminary version of my thesis can be found below. I will provide a more permanent URL on GitHub in the future. https://www.scribd.com/doc/261957061/Unstyle-A-Tool-for-Evading-Authorship-… Full disclosure: I am applying to TSoP this year, and I am still strongly considering proposing the continued development of Unstyle, in addition to a few other potential applications. Cheers, Alden Page

1 0