- tor-dev - lists.torproject.org

Sybil attack detection (was: Karsten's July 2014)
by Philipp Winter 07 Aug '14

07 Aug '14

On Tue, Aug 05, 2014 at 11:37:45AM +0200, Karsten Loesing wrote: > Started looking into better algorithms to detect Sybil attacks on the > Tor network. Current thinking is that we should define relay similarity > metrics like common IP address prefix length or time between first seen > in a consensus, go throw the consensus archives, and see which relays > look similar but are not defined to be in the same family. Do you already have some code or more details on that? I'm quite interested in this topic and I'm wondering if it wouldn't be best to start with something simple like cosine similarity [0]. We would have to transform a relay descriptor into a numerical vector and then calculate the cosine similarity to other relay vectors. However, this might scale poorly as we would need (n^2)/2 comparisons. We might also want to weigh the vector's elements differently as some of them are easy to control for an attacker (advertised bandwidth, uptime, flags, exit policy) and others require more effort (IP address, ASN, observed bandwidth). Like you mentioned, the key thing to look at might be time, i.e., uptime and derived features such as "total online time in last month" etc. [0] https://en.wikipedia.org/wiki/Cosine_similarity Cheers, Philipp

5 5

Damian's Status Report - July 2014
by Damian Johnson 06 Aug '14

06 Aug '14

For one brief, glorious moment in time I touched code. Really, it happened! House plants can vouch as my witness. Or that is to say, I've coded a lot less than I'd like. Barring a long overdue vacation to Sol Duc this month went toward a lot of random distractions... * Worked with Philipp on overhauling our procedure for flagging bad relays... https://trac.torproject.org/projects/tor/wiki/doc/ReportingBadRelays Philipp's taken the lead here and is doing a great job! While I find this space interesting I'm already spread too thin, so going forward I won't be a co-maintainer for it after all. :( * Looked over Nick Hopper's consensus validation change, adding crypto blob type validation to simplify his work. I now owe him a more thorough code review. (#11045) * Handful of DocTor issues including OOM troubles (#12457), unpleasant amount of noise from consensus parameters (#12708), and a several minor revisions. * Discussed adding a new X- extrainfo descriptor field with Virgil, suggesting revisions and helping him make it conformant with the spec. * System I do development on died a puzzling death. Even after twenty hours investigating I'm still not quite sure what happened. Cleaning out dust, removing the RAID controller, and starting over seems to have done the trick. * GSoC administrative work. We're now less than two weeks from the final evaluations! * Continued overhauling arm but as already mentioned focused on this a lot less on this than I'd like. So little coding makes me grumpy, so like many of you I'm taking a hard look at where my time's evaporating to. Maybe there's something I can change or extricate myself from to better focus. Cheers! -Damian

1 0

Building tbb-4.0a1-build2 fails
by tor-admin 06 Aug '14

06 Aug '14

Hi, when I try to build ttb-4 I run into an error on compiling openssl. I was able to compile previous versions of tbb. Any idea what I am missing for ttb-4? Thanks & regards, torland --- Building utils-linux for lucid i386 --- Stopping target if it is up Making a new image copy Formatting 'target-lucid-i386.qcow2', fmt=qcow2 size=11811160064 backing_file='base-lucid-i386.qcow2' encryption=off cluster_size=65536 Starting target Checking if target is up Preparing build environment Updating apt-get repository (log in var/install.log) Installing additional packages (log in var/install.log) Grabbing package manifest Creating build script (var/build-script) Running build script (log in var/build.log) ./bin/gbuild:21:in `system!': failed to run on-target setarch i386 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError) from ./bin/gbuild:122:in `build_one_configuration' from ./bin/gbuild:224 from ./bin/gbuild:219:in `each' from ./bin/gbuild:219 from ./bin/gbuild:217:in `each' from ./bin/gbuild:217 make: *** [build-alpha] Error 1 from ../../gitian-builder/var/build.log gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,--noexecstack -DL_ENDIAN -DTERMIO -O3 -fomit- frame-pointer -Wall -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 - DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM - DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM - DGHASH_ASM -c -o x86cpuid.o x86cpuid.s x86cpuid.s: Assembler messages: x86cpuid.s:8: Error: invalid instruction suffix for `push' x86cpuid.s:9: Error: invalid instruction suffix for `push' x86cpuid.s:10: Error: invalid instruction suffix for `push' x86cpuid.s:11: Error: invalid instruction suffix for `push' x86cpuid.s:13: Error: invalid instruction suffix for `pushf' x86cpuid.s:14: Error: invalid instruction suffix for `pop' x86cpuid.s:17: Error: invalid instruction suffix for `push' x86cpuid.s:18: Error: invalid instruction suffix for `popf' x86cpuid.s:19: Error: invalid instruction suffix for `pushf' x86cpuid.s:20: Error: invalid instruction suffix for `pop' x86cpuid.s:130: Error: invalid instruction suffix for `pop' x86cpuid.s:131: Error: invalid instruction suffix for `pop' x86cpuid.s:132: Error: invalid instruction suffix for `pop' x86cpuid.s:133: Error: invalid instruction suffix for `pop' x86cpuid.s:145: Error: invalid instruction suffix for `pop' x86cpuid.s:147: Error: relocated field and relocation type differ in signedness x86cpuid.s:161: Error: invalid instruction suffix for `pop' x86cpuid.s:163: Error: relocated field and relocation type differ in signedness x86cpuid.s:169: Error: invalid instruction suffix for `pushf' x86cpuid.s:170: Error: invalid instruction suffix for `pop' x86cpuid.s:174: Error: invalid instruction suffix for `push' x86cpuid.s:175: Error: invalid instruction suffix for `push' x86cpuid.s:192: Error: invalid instruction suffix for `pushf' x86cpuid.s:193: Error: invalid instruction suffix for `pop' x86cpuid.s:223: Error: invalid instruction suffix for `pop' x86cpuid.s:225: Error: relocated field and relocation type differ in signedness x86cpuid.s:253: Error: invalid instruction suffix for `push' x86cpuid.s:262: Error: invalid instruction suffix for `pop' x86cpuid.s:270: Error: invalid instruction suffix for `push' x86cpuid.s:289: Error: invalid instruction suffix for `pop'

2 2

3.6.3-meek-2 bundles that can use the Amazon CloudFront CDN
by David Fifield 05 Aug '14

05 Aug '14

I made some test meek bundles that are capable of using the Amazon CloudFront CDN as a backend, in addition to Google App Engine that was supported before. https://people.torproject.org/~dcf/pt-bundle/3.6.3-meek-2/ https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront To test it, go to the pluggable transport screen, and choose "meek-amazon" from the selection box. meek is a pluggable transports that hides the true destination of your communication (a Tor bridge) by hiding it behind a different domain name. When you select meek-amazon from the list, your traffic will appear to a censor to be going to the domain a0.awsstatic.com, but will actually be going to a Tor bridge on cloudfront.net. I signed up for the Amazon Web Services free tier, which limits use to 50 GB and 2 million HTTP requests per month. Please feel free to use it, but don't depend on it not going away. If it becomes too expensive, we'll have to shut it down until we can find a way to fund operation long-term. (The same goes for App Engine, BTW.) David Fifield

2 3

Using a Tap for Tor's Homebrew recipe to increase confidence in source code authenticity
by Mark Rushakoff 05 Aug '14

05 Aug '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I noticed that in the instructions for using Homebrew to install Tor on OSX, the instructions say: > As with any application, you should make sure it came unmodified from the orginal source. Unfortunately, Homebrew does not come with integrated verification for downloads, and anyone could submit a modified Tor! If you use a [Brew Tap](https://github.com/Homebrew/homebrew/wiki/brew-tap), you can have a separate repository (which must be on Github) to track the Tor formula. I believe that if the Tor Project owns that repository, there should be higher confidence that a user installing Tor with Homebrew is installing the intended source code, so long as the user runs `brew install TheTorProject/tor/tor` (instead of just `brew install tor`, which will use the formula in the main Homebrew repo). If this sounds like something the devs desire, I have a tap repository ready to go at https://github.com/mark-rushakoff/homebrew-tor. It includes the tor.rb formula from Homebrew, including its full history. If you fork it to the Tor project (or perhaps Github offers a way for me to transfer ownership), then only someone affiliated with Tor will be able to approve changes to where Homebrew retrieves source code to build Tor. The only extra work in adopting this solution would be finding a way to "deprecate" or redirect the main recipe under Homebrew. I'm not familiar enough with Homebrew to provide direction on that topic, unfortunately. Mark Rushakoff -----BEGIN PGP SIGNATURE----- Version: OpenPGP.js v0.6.1-dev Comment: http://openpgpjs.org wsBcBAEBCAAQBQJT4G8NCRDqKXFAudXBCAAAcKYIAJwa99tJQOHxvzn1DyDW DZD+ktBrAg0pZ4FEE58+n2KiH1hzeHGuhLh4mLlsD30CGlDqtjmYKiU7VR/P g9jsQTawqmACI5KQkMkOMkdBKsjKfNwCaLxA7mdSoCHsRcmSKhQH++rg1Bli JiQrHgi9DihNrUku2/Km7leiurBrKED1KK2KAJ9mKnVMF2iRjcV//VQ9Nbtp WJp92mydyTnBEGYQPrt6M57WZjYvrkvYV1/eHYZpulrGcPAcXzDYnHgRhRPL 9bolbQhwyPWU6gvEdLC/+NAlCpN1Lfd/RFCyhksgVr6RT8GJFSdpjg1UVkw4 U/63nEVJR8cF0boBtUWQReQ= =baFT -----END PGP SIGNATURE-----

1 0

carml: tasty treats from your Tor
by meejah 05 Aug '14

05 Aug '14

I've got a first super-alpha release of this thing that's been sitting around for a while. Turns out "sanitize a bit" turns into "refactor some things" and so forth... Anyway, carml does various command-line things with Tor and I thought it might be useful to others (plays nicely with grep, pipes, etc). I would really love feedback on whether the "downloadbundle" command is doing the right thing with certificate-checks. https://github.com/meejah/carml https://carml.readthedocs.org/en/latest/ You can "pip install carml" to try it out. Recommend doing this in a virtualenv: virtualenv trycarml ./trycarml/bin/pip install carml ./trycarml/bin/carml help To check signatures first, instead download the WHL file and associated signature from PyPI, gpg --verify it and then replace "install carml" with "install path/to/.whl" above. Some other things to try: carml downloadbundle --extract --system-keyring echo "hello darkweb" | carml pastebin wait for a new consensus to be published, dump it and exit: carml events --once NEWCONSENSUS Currently, the defaults work with a system Tor (i.e. localhost port 9051). Probably I'll change this to be TBB defaults. To connect to a Tor Browser Bundle instance, do this: carml --connect tcp:localhost:9151 monitor It is written using Twisted and txtorcon. Thanks, meejah

3 4

GSoC: making tor daemon using more cores
by Белоус Михаил 04 Aug '14

04 Aug '14

Hi all, This my first report since my return, from millitary camp. Last week I have been watching tor daemon code and make little refactor. I think I found place where I can use my threadpool to make daemon multicore. I'm going to implement it as soon as I disquise my idea with Nick. Cheers, Mikhail

1 0

Needed improvements to the PT website
by George Kadianakis 03 Aug '14

03 Aug '14

Hello list, in my monthly status report [0] I mentioned that some PT-related parts of the website are in need of improvement. As a response, a few people sent me a private email asking me what kind of improvements I was thinking about. Instead of replying individually, I decided to send a mail to this list. Here are some of the needed improvements: - In the past, our main PT page was the obfsproxy page: https://www.torproject.org/projects/obfsproxy.html.en However, now that the scene is growing and the obfsproxy fad is shrinking, the main PT page should actually be: https://www.torproject.org/docs/pluggable-transports.html.en . Users, bridge sysadmins and researchers who want to learn about PTs will probably encounter that page. I suggest we optimize that page towards users and bridge syadmins and let researchers get their information from the "unofficial" wiki page: https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports - To make the PT page a bit more end-user-friendly, we could remove non-Tor-deployed PTs from the list, like Stegotorus, Skypemorph and Dust. These are better suited for the wiki, I think. Things that users see in the PT page should be ready to use. Also, it would be great if there was a "How to use PTs" document linked by the "Download PT bundles" section. Unfortunately, I don't think there is such a document atm. Sheried wanted to write one (with screenshots etc.) but I'm not sure if it was ever made. - To make the PT page a bit more bridge-sysadmin friendly, we could link to a "How to become an obfsbridge" document. Currently, the only such document is linked from the obfsproxy page: https://www.torproject.org/projects/obfsproxy.html.en#instructions and it only suggests installing obfsproxy. Nowadays, such a guide should also be guiding readers to install fteproxy. And Real Soon Now it will need to be guiding readers to install Yawning's obfs4proxy or gobfsproxy. Also, weasel read the guide and was not very happy. For example, the guide _suggests_ using pip as root, this is probably not sound advice and some workaround should be found. - Another document that is needed for bridge sysadmins is a "Best practices for running obfsbridges" document. Such a document could have the following advices: - Which PTs should be installed. - How to install ExtORPort and geoip. - How to find geoip stats from your bridge. - How to check whether your bridge and PTs are used. - How to enable logging in your PTs and general debugging tips. - etc. There are probably other improvements that could be suggested, but this is a good start, I think. Also, those tasks don't have any tickets at the moment. Please feel welcome to make tickets if you'd like, or I can do it at a later time :) BTW, I don't know much about UI or product marketing (#11501), so my advices might be totally off :) [0]: https://lists.torproject.org/pipermail/tor-reports/2014-August/000608.html

1 0

GSoC : Rewrite Tor Weather (Status Update 5)
by Sreenatha Bhatlapenumarthi 03 Aug '14

03 Aug '14

Hi all, This report includes my 4th status update as well. Here is a list of things I've been working on regarding the weather rewrite project. - Fixed some django/database issues that were causing errors(seen with Debug=True in settings) during user subscription in the weather webform. - Added a config option to consider the deployment time only in production mode. This was causing an issue previously because not many relays are filtered(considered recent) due to the presence of deploy time in the consideration criteria. - Made use of django's setup_environ method to ensure the standalone backend scripts running as cron jobs can operate with the proper settings. - Updated the daily script to ensure Tshirt emails are not sent to relay operators more than once. For the changes made, check out https://github.com/meejah/tor-weather/pull/10/ Here's what I'll be working on in the coming weeks. - Squash any remaining bugs with the backend scripts. - Get rid of previous mechanism of fetching data like listeners, updaters etc. - Write down test cases for various usage scenarios(e.g., check if emails are being sent to subscribers) within the vagrant environment. Cheers, Sreenatha

1 0

[GSoC] Consensus diffs - Fifth report
by Daniel Martí 03 Aug '14

03 Aug '14

Hello everyone, This is the fifth status report of my Google Summer of Code project, which is to implement consensus diffs for Tor. My mentors - Sebastian and Nick - and myself usually hold meetings on IRC on wednesday at 16h UTC. As described in my last report, I have spent the past couple of weeks integrating my work into the main Tor repository. It builds fine with the rest, all gcc/clang warnings were solved (mostly just C89 declaration errors) and all make check-spaces warnings were also fixed. You can see the result on the repository I created a few weeks ago on github [1]. You want the branch 'consdiff-1', where you will find src/or/consdiff.{h,c} and src/test/test_consdiff.c. I also added some very basic logging, so that the public consdiff_* functions will log what went wrong when returning NULL. I used LD_GENERAL for now, which I assume is wrong - suggestions welcome. I also simplified some bits for easier readability and usage, which can be found in the commit history. But most notably, I improved the docs and comments. The idea is to be able to stumble upon consdiff.c for the first time and be able to understand what it does, how and why in just a few minutes. It's hard to tell whether I did a good job at that - criticism very much appreciated. Most of that happened during the first week, as I was on a quick holiday during the second. Nick gave me some pointers on how to continue with the integration, and that's what I'll be doing in the next few days. The first thing to do is figure out a way to store multiple consensuses on disk, to be used to fetch and create diffs accordingly. There are now just over two weeks left, but I am confident that I can finish all the coding early enough to be able to test the whole thing properly. Regards. [1] https://github.com/mvdan/tor -- Daniel Martí - mvdan(a)mvdan.cc - http://mvdan.cc/ PGP: A9DA 13CD F7A1 4ACD D3DE E530 F4CA FFDB 4348 041C

2 2