- tor-dev - lists.torproject.org

Damian's Status Report - August 2013
by Damian Johnson 07 Sep '13

07 Sep '13

Hi all, after a month of work I'm pleased to announce that the Stem based replacement for Doctor is now live! https://gitweb.torproject.org/doctor.git https://lists.torproject.org/pipermail/tor-consensus-health/2013-September/ For those who aren't familiar with it, Doctor is a service that pulls hourly consensus information and checks it for a host of issues (directory authority outages, expiring certificates, etc). In the case of a problem it notifies tor-consensus-health@, and we in turn give the authority operator a heads up. The new version of Doctor replaces Karsten's java based counterpart and is part of our ongoing scheme to eventually deprecate Metrics-lib in favor of Stem... https://trac.torproject.org/8252 Other news this month includes... * Arlo Breault added DNSEL exit list parsing to Stem! https://stem.torproject.org/api/descriptor/tordnsel.html https://trac.torproject.org/8255 * Stem now mirrors Tor's hardcoded directory authority information (both required for the stem.descriptor.remote module and handy for scripting)... https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remo… * Added an example of remote descriptor fetching to our tutorials... https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#where-… * FAQ entry describing other controller libraries... https://stem.torproject.org/faq.html#are-there-any-other-controller-librari… * FAQ entry for the various ways of instantiating a Controller... https://stem.torproject.org/faq.html#how-do-i-connect-to-tor * FAQ entry describing python's 'with' keyword (this has been a common point of confusion for new stem users)... https://stem.torproject.org/faq.html#what-is-that-with-keyword-i-keep-seein… * Looked a bit unreliability regarding Tor's DirPort. Unfortunately requests for compressed descriptor data frequently fail with a truncated response, so Doctor is now requesting uncompressed data (bad, but likely better than hammering authorities with retries). https://trac.torproject.org/9379 * Assigned joint copyright of Stem to The Tor Project so it can live on if I get struck by a meteor, swallowed by a dragon, etc. * Support for new Tor capabilities including TRANSPORT_LAUNCHED events and bridge-ip-transports lines in extrainfo descriptors. * Overhauled the layout of arm's codebase so it'll behave like a more proper python module. * Discussed using Stem for Bridgedb with sysrqb and isis. https://trac.torproject.org/9380 * Answered a few controller related questions on Stack Overflow... http://stackoverflow.com/questions/18561778/how-do-i-get-the-ip-address-of-… http://stackoverflow.com/questions/18372615/tors-stem-module-python-failing http://stackoverflow.com/questions/18578908/torctl-errors-out-with-torctl-c… Cheers! -Damian

1 0

I have a project that might interest you. OnionMail
by Liste 07 Sep '13

07 Sep '13

Hi, my name is EPTO. I am the founder of Anopticon Project and tramaci.org http://tramaci.org/anopticon (Map of Video surveillance cameras) http://tramaci.org (This listsmlm(a)tramacci.org is my mail address for mailinglists). I'm building a new project: A mail server for the tor netword to increase the privacy of mail communications. The project is named "OnionMail". OnionMail is a SMTP/POP3 compatible mail hidden server with some extended functions: * Multiple instances of server. (multiple indipendent hidden serivces). * SSL cryptography by default. (STARTTLS) * Encrypted filesystem with asimmetric keys for any user, any log and messages. * Multiple encryption everywhere, RSA + AES + RSA + AES with salt * Support for unicode password * Inhibition of store any message in relay server. (Only direct connection is allowed without multiple connections). Eg.: Client_Of_Sender -> SMTP1 -> SMTP2 <- Sever_Of_receiving * Metadata protection * SMTP Compatibility * AntiSpam, blacklist realtime filters * Decentralized trust system for SSL certificate and public keys, GPG keys, Spam blacklist, etc... * Mailing list support * Anonymous remailer * username(a)address.onion email addresses. Your onion mail ;) * Garbage collector to remove automatically old messages. * GPG Support and optionaly GPG compulsory. * Clock and time zone spoofing. * Bulit in server operations: Add / Remove mail address or mail server in black list. Mailing list Subscribe / Unsubscribe KeyServer Request for anonymous mail address Request of server "rulez". (All via mail message to the server directly "server@ xyz... .onion ") * JAVA Implementation for all platform. * Localhost control port and server API. * Protected server password and keys (optionaly not saved). * Localhost full net use: 127.0.0.1 127.0.0.2 etc... * IP BlackList * onion BlackList * tor connection verifycation. Optional: * SMTP Exit Route/Enter Route with mail alias to connect internet to tor and vice versa. * DNSBL for normal SMTP operations. (not user for .onion address). * Mindless compliant mode: Prevent bad use of DNS, direct internet connection, exit enter node, enable only tor operations. * Etc...... The project is not pubblic at this time. In a short time it will public. (Ver 0.0.1 Beta) I joined to get some advice. I read somewhere that you should contact for projects here. What do you think about OnionMail? Sorry if the message is too long. but the project is big. The slogan is: «In the future, maybe we will implement the anonymous coffee! Today, only OnionMail ;) »

3 2

GSoC Status Report - EvilGenius
by Johannes Fürmann 06 Sep '13

06 Sep '13

Hi *, In the past two weeks, I've been working on EvilGenius. I had some weird problems with vagrant, and in the meantime I switched to Debian, but that is another story. I spent most of my GSoC-time on some seriously weird routing issues, one of which I eventually solved today. I am confident that I will solve the other routing Issues early next week. What is left to do is write documentation. You all have a beautiful weekend! ~johannes

1 0

torsocks: 0.2.4.17 socksport isolate auth
by grarpamp 06 Sep '13

06 Sep '13

libtorsocks(): Need a password in torsocks.conf or $TORSOCKS_PASSWORD to authenticate with ... : Connection refused To use default torsocks 1.3 you'll need to set socksport noisolatesocksauth or prefersocksnoauth or perhaps configure a user:pass in torsocks beyond the unset default, as hinted above. Not sure if the latter socksport option will prevent isolation or block torsocks again when later setting a user:pass in torsocks. I think there may be a case for adding an environment/conf TORSOCKS_RNDAUTH=[0/1] to torsocks such that it will generate a random user:pass per invocation (ie: parallel) thereby isolating each one. While other unset invocations would ride as default over whichever circuits.

1 0

High load even with 0.2.4.17-rc
by Leo Unglaub 06 Sep '13

06 Sep '13

Hey guys, to fix the load issues on my 5 relays i tryed the suggested development version of tor. But there is not difference at all on the exit relays. They all burn on 99% CPU usage at 23 MB/s bandwidth. Is there something special i have to do in the config to benefit from the new changes mad to quicker peerform the handshakes between the circuits? Thanks and greetings Leo -- Website: http://www.leo-unglaub.net Twitter: http://twitter.com/LeoUnglaub Send with Mozilla Thunderbird on Linux (Debian)

2 1

Re: [tor-dev] Pluggable transport weekly meeting
by vmonmoonshine＠gmail.com 06 Sep '13

06 Sep '13

> From: Yawning Angel <yawning(a)schwanenlied.me> > > Assuming not today (2013/09/16) isn't "soon", the times specified works > for me and I will make an effort to attend. > No, I didn't mean today. I just wanted to send it as soon as found out it hasn't been out. Perhapse it was better to wait few hours. But I would like to get some feedback first. Once we agree on a time (which seems Friday works for most people), I'll send a reminder before the meeting to dev list (hopefully to the correct one). Thanks for the effort ;) Vmon

1 0

[GSoC 2013] Status report - Searchable metrics archive
by Kostas Jakeliunas 06 Sep '13

06 Sep '13

Hello, This status update is less extensive/dramatic compared to the last one, but I'm still happy to report to be slowly moving ahead towards a stable searchable archive system. In short, I've been working on what I said I should work on in the last report, more or less: I should now move on with implementing / extending the Onionoo API, in > particular, working on date range queries, and refining/rewriting the "list > status entries" API point (see below). Need to carefully plan some > things, and always keep an updated API document. (Also need to update and > publish a separate, more detailed specification document.) The searchable metrics backend (encompassing a part of the database and the Onionoo-like API) [2] is still happily chugging along online; quite a few folk ran some queries on it, including myself. I'm in the process of expanding my benchmark.py tool to generate realistic-looking parallel/asynchronous traffic for different kinds of relays and API points; from my limited parallelized benchmarking so far, everything looks good - the bottlenecks are still localized to the individual queries. I should try and generate human-readable more-or-less rigorous benchmarking reports and publish them. I've briefly run the backend on an EC2 instance again, to compare benchmark outputs and average query times. Natural database caching seems to be helping quite a bit for the current online database (at ts.mkj.lt), by which I mean, indexes and some query results get cached via natural usage of the backend/database. I've been tinkering on a simple system to pre-warm indexes upon application start (without the need for any PostgreSQL extensions), for more uniformly distributed query times. Overall though, we seem to be doing OK in regards to actual query times. Made updates to the Onionoo API, but haven't pushed them yet (hoped to do that until this report, but finally learning not to delay); expect them soon (and subsequent updates to the Onionoo API doc [3] as well, once that happens.) I'm now trying to incorporate the most recent Karsten's feedback in regards to API points, parameters and some preliminary simplistic caching. Basically, once the date range parameters are working nicely for all three types of documents currently provided by the API, and once the status entry API point is returning valid-after summaries/ranges in a more intuitive document format, the whole thing will hopefully be able to satisfy actual usefulness criteria to a significant extent. Together with caching, it will hopefully be able to be considered as an almost-proper (smallish, as of now) subset of the Onionoo API/backend. Did some experiments with PostgreSQL's pg_trgm module for full-text search (so that search strings matching only the middle of some descriptor field could work); I realize that's not priority now, but I was curious to see if it would work. Nothing conclusive thus far, unfortunately. Specification document explaining more detailed design and applicable use cases for the Onionoo-like API coming along. I should continue finalizing the Onionoo-like API into a working, non-hacky state; keep the codebase clean and maybe do some cleanup refactoring; continue observing database performance; write more documentation; and if all is well, expand the list of fields contained in the three status documents. Besides all this, - update the database to the latest consensuses and descriptors; - turn on the cronjob for rsync and import of newest archives; - caching; - hopefully later integration with the bandwidth and weights documents in Onionoo proper; - import 2008 year data, etc. These couple of weeks coincide with an (not quite planned for) apartment-moving period, but I *really* hope I have the vast majority of the things mentioned working in a decent state by the end of the next week, to leave ourselves some wiggle space to ensure the resulting system is as stable as possible before the official date for the end of the coding period. I'll probably be available throughout the weekend/s, just in case. Cheers to you all Kostas. [1]: https://lists.torproject.org/pipermail/tor-dev/2013-August/005310.html [2]: https://lists.torproject.org/pipermail/tor-dev/2013-August/005311.html [3]: https://github.com/wfn/torsearch/blob/master/docs/onionoo_api.md

1 0

IRC meeting to discuss sponsor F progress on Tue September 3, 18:00 to 19:00 UTC in #tor-dev
by Karsten Loesing 05 Sep '13

05 Sep '13

Hi all, I'd like to schedule an IRC meeting to discuss what progress we made on sponsor F deliverables in August. Time and place are: Tue September 3, 18:00 to 19:00 UTC in #tor-dev That's in (exactly) 4 1/2 days from now (sorry for the short notice). The time in other timezones is: 11:00 in San Francisco 14:00 in Boston 20:00 in Berlin 21:00 in Athens 23:30 in New Delhi People who should attend are: Roger, Nick, George, David, Ximin, Nathan, Matt, Colin, Andrew, Tom, and anyone else who wants to attend. If your name is on that list and you're working on sponsor F stuff, but the date or time doesn't work for you, please email me your progress and I'll report it for you at the meeting. Thanks, Karsten

3 4

Traffic Obfuscation
by josef.winger＠email.de 05 Sep '13

05 Sep '13

Can a developer please explain to me why something like the following obfuscation of 'torified traffic' is exploitable? Suppose a scenario where a collective of authorities is able to observe large parts of the www. Then observing traffic correlation can unreveal a connection through the network. But why can't we just alter the pattern inside the network, such that there is no correlation between 'incomming' and 'outgoing' data anymore? Suppose I'm connected to a server and there is a lot of traffic between from the server to me. Through the TOR network, of course. Data is encrypted and the exploit measures the raw data stream pattern. But why not change the data stream inside the network? Suppose the server A is outside of the TOR network, i.e. it is not a hidden service. The data stream into our network then is out of our controll. Encrypted or not, we can't change it. Now it flows to node B (exit node) and inside TOR. Then node B streams the data to node C, than node C streams the data to node D and node D exits the stream to me. (simplification) Ok now node B got the data from the 'outside world'. Then B and C first make a handshake to define a shared key for a private encryption protocoll only valid for some time. Now node B does not stream the data to node C, but obfuscates it. That means if there are n packages it transforms them into m packages in some unpredictable way and each new packages gets a small amount of additional random-data. (The point is that the new stream will not look at all like the old one) Only node B nows the way to de-obfuscate this. But B and C did a handshake and using this encryption B shares with C how to de-obfuscate the data. Now C recovers the real data and then does another secret handshake with D. (separated from the shared secret with B of course) Then C obfuscates the data again and only D will know how to recover the original data. This repeats until I recive an obfuscatet stream and my client can recover the original data. ====== The point is, that patterns of the in-stream of the server A aren't correlated to what streams from D to me anymore. Hence an observer isn't able to see correlations anymore. Number, size and pattern of the stream packages are different. Above this one could try to add randomly zero-information streams between the network and its clients. That way an observer can't even be sure if a client recieves information at al... ====== Now can someone please downtalk this approach? best /jo

3 2

regarding control spec for hidden service descriptor
by Qingping Hou 04 Sep '13

04 Sep '13

Hi all, I am currently working on ticket 8510: https://trac.torproject.org/projects/tor/ticket/8510 However, I notice that in the control spec, there is no event related to hidden service descriptors. Should we add a new event in the spec? For example, we can have something like the following: HS_DESC SP REQUESTED SP OnionAddress SP DescptorID SP AuthType SP HsDir SP Any thoughts? Cheers, qp

4 10

Re: [tor-dev] Pluggable Transport Browser Bundle FTE Integration
by Arlo Breault 03 Sep '13

03 Sep '13

> On Sat, Aug 31, 2013 at 09:43:21PM -0400, Kevin P Dyer wrote: > > === Building/Testing Tor on Linux/OSX/Windows > > I'm trying to understand exactly how the current build/release process works > > for tor. In regards to the PT TBB it seems like there are a few resources > > [1,2,3]. However, is there a canonical documentation on how the release process > > works? I'm especially interested in what you guys are doing to produce builds > > on Windows. Are you using virtualization or do you have a few physical build > > machines? > > [1] https://gitweb.torproject.org/pluggable-transports/bundle.git > > [2] https://lists.torproject.org/pipermail/tor-dev/2013-June/005056.html > > [3] https://gitweb.torproject.org/builders/tor-browser-bundle.git > > > > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bu… You may have an easier time with MinGW than Cygwin. One way to get a compatible shell environment is to install Git Bash from, http://git-scm.com/downloads

1 0

Draft of proposal "Migrate HS identity keys to Ed25519"
by George Kadianakis 03 Sep '13

03 Sep '13

Greetz, I'm posting the draft of a proposal that specifies how to upgrade the identity keys of HSes (currently RSA-1024) to use Ed25519. This proposal is supposed to go along with a proposal that specifies how to hide HS descriptors from HSDirs. I'm going to post that second proposal in a few minutes. This proposal is incredibly drafty in the sense that I might have forgotten to specify things that need to be specified. On the other hand, "release early; release often" they say, so here it goes. Inlining: Filename: xxx-hs-ecc-id-keys.txt Title: Migrate HS identity keys to Ed25519 Author: George Kadianakis Created: 10 August 2013 Target: 0.2.5.x Status: Draft [More draft than Guiness.] 0. Overview: This proposal suggests the adoption of ECDSA keys as the long-term identity keys of Hidden Services. It also proposes the adoption of ECDSA keys for the per-introduction-point service key that was introduced in the v2 hidden service protocol. This proposal specifies the key generation procedure, paths where the keys should be stored, and flagday/migration procedures. XXX 1. Motivation: Hidden Services currently use an RSA-1024 keypair as their long-term identity keys. RSA-1024 is considered weak and should be replaced. Dan Bernstein's ed25519 ECDSA scheme provides 2^128 bits of security and better signature performance than RSA. Furthermore, the keysize is much smaller than the keys of RSA. 2. Related proposals This document is part of a proposal triptych. Upcoming sister proposals are: - "Stop HS address enumeration by HSDirs", where we propose an alternative HS descriptor format that will not allow HSDirs to learn the addresses of the Hidden Services it serves. - "Deployment strategy for recent HS proposals", where we specify how these proposals should be deployed without making people angry. 3. Changes to the current HS protocol 3.0. Overview of changes to the current HS protocol There are two places where RSA-1024 is used in the current HS: Longterm RSA-1024 "identity keys" are used by the HSes to authenticate themselves to clients. Multiple short-term RSA-1024 "service keys" are used by the HSes for each introduction point so that they don't reveal their identity key. This proposal specifies: a) The generation of the new ed25519 long-term identity keys and service keys (#KEYGEN) b) The new HS descriptor format (v3) that contains identity keys and service keys (#NEWDESC) c) A way for clients to upload and fetch v3 descriptors from HSDirs (#HSDIRV3) 3.1. Generation of ed25519 keypairs (#KEYGEN) 3.1.0. Generation of long-term ed25519 identity key Compliant Hidden Services need to generate a fresh ed25519 keypair and store the private key in: $HiddenServiceDirectory/hs_ed25519_privkey For deployment and migration reasons, the new ed25519 keys must be kept alongside with the old RSA-1024 keys. 3.1.1. Generation of short-term ed25519 service keys Hidden Services generate an ed25519 service key for each introduction point -- instead of the RSA-1024 key they currently generate. 3.2. New v3 Hidden Service descriptors 3.2.0. New v3 Hidden Service Descriptor Format (#NEWDESC) v2 Hidden Service descriptors contain both the long-term identity public key of the Hidden Service, and service keys for each introduction point. It also contains a signature of the descriptor. We need to change all these fields to carry our brand new ed25519 keys. For v3 Hidden Service descriptors, we keep the v2 format and perform the following changes: [*] The "permanent-key" field is replaced by "permanent-key-ed25519": "permanent-key-ed25519" NL an ed25519 public key [Exactly once] The public key of the hidden service which is required to verify the "descriptor-id" and the "signature-ed25519". In base64 format with terminating =s removed. [*] The "service-key" field is replaced by "service-key-ed25519': "service-key-ed25519" NL an ed25519 public key [Exactly once] The public key that can be used to encrypt messages to the hidden service. In base64 format with terminating =s removed. [*] The "signature" field is replaced by "signature-ed25519': "signature-ed25519" NL signature-string [At end, exactly once] A signature of all fields above with the private ed25519 key of the hidden service. In base64 format with terminating =s removed. 3.2.1. Uploading v3 HS descriptors to HSDirs (#HSDIRV3) A new type of Hidden Service Directory Server must be established which understands v3 Hidden Service descriptors. The Hidden Service follows the same publishing procedure as for v2 descriptors but instead of sending an HTTP 'POST' to "/tor/rendezvous2/publish", the HS sends the 'POST' request to "/tor/rendezvous3/publish". (This part of the proposal conflicts with the "Stop HS address enumeration by HSDirs" proposal.) 3.3. Fetching v3 HS descriptors from HSDirs (#HSDIRV3) When a client needs to fetch a v3 Hidden Service Descriptor from an HSDir, it follows the exact same procedure as for v2 descriptors but instead of sending an HTTP 'GET' to "/tor/rendezvous2/<z>", it sends an HTTP 'GET' to "/tor/rendezvous3/<z>". (This part of the proposal conflicts with the "Stop HS address enumeration by HSDirs" proposal) 3.4. Service keys and INTRODUCE cells In INTRODUCE cells, when v2 descriptors are used, PK_ID is defined as the hash of the service key. This means that we don't need to change the format of INTRODUCE cells since we can just use the hash of the ed25519 service key. # XXX will this cause problems?

3 5

Re: [tor-dev] Help with rebasing arma's n23-5 tor branch to current master
by Karsten Loesing 03 Sep '13

03 Sep '13

(Moving this discussion back to tor-dev@ with Charlie's permission.) On 8/6/13 4:10 AM, Charlie Belmer wrote: > Hey Karsten, > > I've been reading up on Shadow & Scallion to do some performance testing on > N23. Before I go too far down that road, I wanted to check in with you. > > Are there existing blueprints or best practices I should be following? Thanks for looking into this! So, I wouldn't call the following notes blueprints or best practices, but they are a description of what I would do to run Shadow simulations of your branch. First of all, be sure to read the Shadow wiki which has contains lots of interesting stuff: https://github.com/shadow/shadow/wiki The next step would be to get a vanilla Tor master (that your branch is based on) running in a minimal or tiny Shadow network. It's probably easiest to use a local Ubuntu VM for this. Or you can start an m1.large EC2 instance for the tiny network, or an even smaller instance for the minimal network. Once that simulation succeeds, you'll want to run another simulation using your branch, extract performance metrics using Shadow's analyze script, produce a PDF with graphs also using the analyze script, and then wonder if things are faster than before. The minimal or tiny networks are mostly there to learn that things are working as expected. You'll probably want larger networks for real performance results, but starting small to get everything working is probably easier. Sounds doable, I guess. However, here's where things start to get tricky: in order to simulate Tor master (or a branch based on it) in Shadow, you'll have to perform quite a few extra steps like compiling your own CMake and Clang/LLVM. Not really hard, but you need to get all pieces together before Shadow does what you want. I'm pasting detailed instructions that I used to simulate Steven's uTP branch in Shadow below as an example. Not everything applies to your branch, but it should give you the general idea. If something remains unclear, please feel free to ask! > Depending how it goes, I'd also look at re-trying adaptive 23 or tuning the > existing implementation. Neat! We should probably include Mashael in that discussion once you have a working and simulated n23-5 branch. Maybe she has ideas what parts need tweaking. I'm cc'ing her in this mail so she knows what you're working on. In any case, having a working rebased n23-5 branch is already a big step forward. Thanks! Karsten === START OF INSTRUCTIONS === Launch new EC2 instance running Ubuntu Server 13.04 64 bit Set ulimit -n to 25000, see Shadow wiki for details https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs… $ sudo apt-get update $ sudo apt-get upgrade Clone and test-compile modified libutp $ sudo apt-get install build-essential git automake libssl-dev libevent-dev $ mkdir src $ cd src/ $ git clone https://github.com/sjmurdoch/libutp $ cd libutp/ $ make $ cd ../ Clone and test-compile Tor branch utp $ git clone https://git.torproject.org/tor.git $ cd tor/ $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git $ git fetch sjm217 $ git checkout -b utp sjm217/utp $ ./autogen.sh $ LDFLAGS="-L/home/ubuntu/src/libutp" CFLAGS="-I/home/ubuntu/src/libutp" LIBS="-lutp -lrt" ./configure --disable-asciidoc $ make $ make distclean $ cd ../ Modify Tor to use uTP for all links (or "0 &&" for not using uTP for any link) diff --git a/src/or/channeltls.c b/src/or/channeltls.c index 0551b73..b7b36e1 100644 --- a/src/or/channeltls.c +++ b/src/or/channeltls.c @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, uint16_t port, /* Create a uTP connection */ tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, sizeof(sin)); tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0); - if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { + if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { log_info(LD_CHANNEL, "Trying uTP connection to %s", addr_str); tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const struct sockaddr*)&sin, Install CMake 2.8.10 $ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz $ tar xf cmake-2.8.10.2.tar.gz $ cd cmake-2.8.10.2/ $ ./configure $ make $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH $ cd ../ $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake Build Clang and LLVM from source $ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz $ tar xaf llvm-3.2.src.tar.gz $ tar xaf clang-3.2.src.tar.gz $ cp -R clang-3.2.src llvm-3.2.src/tools/clang $ cd llvm-3.2.src/ $ mkdir build $ cd build/ $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../. $ make $ make install $ export PATH=~/.local/bin/:$PATH $ cd ../../ $ which clang # result: /home/ubuntu/.local/bin//clang Install Shadow $ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7 python-matplotlib python-numpy python-scipy $ git clone https://github.com/shadow/shadow.git $ cd shadow $ ./setup dependencies $ ./setup build -g -i /home/ubuntu/src/libutp -i /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp --tor-prefix /home/ubuntu/src/tor --tor-lib utp $ ./setup install $ export PATH=~/.shadow/bin/:$PATH Simulate tiny network in Shadow $ cd resource/examples/scallion/ $ tar xf tiny-4GB-m1.large.tar.xz $ cd tiny-4GB-m1.large/ $ scallion Graph simulation results Add one line to beginning of analyze.py: import matplotlib; matplotlib.use('Agg') For additional traffic simulations: $ grep -v "\[traffic-" scallion.log > scallion-notraffic.log $ cd ~/src/ $ mkdir results $ cd results/ $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output vanilla ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for none of the links" === END OF INSTRUCTIONS ===

1 1

[RELEASE] Torsocks 2.0.0-rc2
by David Goulet 02 Sep '13

02 Sep '13

Greetings everyone! After a week or so, the release candidate 2 is now out after receiving various contributions for BSD and OS X support. A quick note. Please use the Github bug tracker for any issues and *not* trac.torproject.org. Until this code base is accepted as a potential replacement for the old version, please use the current repository issue tracker. Here is the ChangeLog: 2013-09-02 torsocks 2.0.0-rc2 * Fix: remove FAQ file from Makefile * Fix: remove out of date and inaccurate FAQ * Tests: add connection object unit test * Fix: Improve README file * Tests: add onion pool subsystem unit test * Use extern for tsocks_libc_* in torsocks.h * Define LIBC_SYSCALL_ for OS X * Make sure __darwin__ is defined * Fix: explicitly ignore fileno return value * Use AC_CHECK_FUNC rather than AC_LINK_IFELSE. * Find out if we really need libdl. * Define LIBC_SYSCALL_ for FreeBSD. * Use SYS_ from <sys/syscall.h>. * Include <sys/socket.h> for AF_INET*. * Use getsockname(2) for finding out socket address family Please continue to test, review and contribute it! :) Tarball: https://github.com/dgoulet/torsocks/archive/v2.0.0-rc2.tar.gz Big thanks to all contributors for that release! Cheers! David

1 0

Searchable metrics archive - Onionoo-like API available online for probing
by Kostas Jakeliunas 02 Sep '13

02 Sep '13

Hello, this accompanies my status report [1], and includes info how to query the searchable metrics archive for anyone curious. I also refer to the original (now semi-outdated) project proposal/document. [0] Only sending to tor-dev@for now. The Onionoo-like backend is listening on http://ts.mkj.lt:5555/ (backup URI where it's actually running (domain unrelated): ravinesmp.com:5555) This document details how it can be queried: https://github.com/wfn/torsearch/blob/master/docs/onionoo_api.md (It is, by design, an almost-subset (it does change some things as of now, though) of the Onionoo API [2].) Try not to DoS the thing, it is still fragile! :) (or, maybe do, but gently?) I will try to follow-up with a more detailed email, but this will have to do for now. Anyone curious should refer to the API doc. I'll soon add date range parameters, so it will become somewhat more useful. Examples: http://ts.mkj.lt:5555/details?search=mor http://ts.mkj.lt:5555/details?search=79. http://ts.mkj.lt:5555/statuses?lookup=9695DFC35FFEB861329B9F1AB04C46397020C… Please report any inconsistencies / errors / time-outs / anything that takes a few seconds or more to execute. I'm logging the queries (together with IP addresses for now - for shame!), so will be able to later correlate activity with database load, which will hopefully provide some realistic semi-benchmark-like data. [0]: http://kostas.mkj.lt/gsoc2013/gsoc2013.html [1]: https://lists.torproject.org/pipermail/tor-dev/2013-August/005310.html [2]: https://onionoo.torproject.org/ -- Kostas. 0x0e5dce45 @ pgp.mit.edu

2 3

Re: [tor-dev] Pluggable Transport Browser Bundle FTE Integration
by David Fifield 02 Sep '13

02 Sep '13

On Sat, Aug 31, 2013 at 09:43:21PM -0400, Kevin P Dyer wrote: > === Building/Testing Tor on Linux/OSX/Windows > I'm trying to understand exactly how the current build/release process works > for tor. In regards to the PT TBB it seems like there are a few resources > [1,2,3]. However, is there a canonical documentation on how the release process > works? I'm especially interested in what you guys are doing to produce builds > on Windows. Are you using virtualization or do you have a few physical build > machines? > [1] https://gitweb.torproject.org/pluggable-transports/bundle.git > [2] https://lists.torproject.org/pipermail/tor-dev/2013-June/005056.html > [3] https://gitweb.torproject.org/builders/tor-browser-bundle.git There is fragmented documentation on how releases work. I have been building the PT bundles, but I have nothing to do with the building of the vanilla bundles from which the PT bundles are derived. The PT bundles are built by unzipping the vanilla bundles, copying some programs inside and modifying the torrc, and zipping the bundles back up. The PT build is mostly driven by this makefile: https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/Ma… You should know that there are two varieties of the vanilla Tor Browser Bundle: the beta 2.0 series (e.g. https://blog.torproject.org/blog/new-tor-02416-rc-packages-and-updated-stab…) and the alpha 3.0 series (e.g. https://blog.torproject.org/blog/tor-browser-bundle-30alpha3-released) They differ much in their build process. We are using the 2.0 series to build the PT bundles, and there is no 3.0 version of the PT bundles. I would like to have the PT bundles built using the same deterministic process as is used for the 3.0 bundles (e.g. https://blog.torproject.org/blog/deterministic-builds-part-one-cyberwar-and…) See https://trac.torproject.org/projects/tor/ticket/9444 for the status of this project. Some further reading: https://trac.torproject.org/projects/tor/ticket/8019#comment:3 Much of the PT bundle build process is automated, but it still takes me two or three hours to boot VMs, download everything, verify signatures, build, sign, upload, and announce. I use VMs for the GNU/Linux and Windows builds. I use a physical Mac mini running OS X 10.8 for the Mac builds. (This is one reason I want to move the PT bundles to the 3.0 series--they cross-compile for all platforms and don't require a Windows VM or physical Mac.) The instructions for setting up the VMs and Mac are here: https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bu… https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bu… https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bu… > Prior to doing anything with FTE. I'd love to be able to create my own build > environment that produces that current obfs2+obfs3+flash_proxy bundles [6] > across all 4 OS/architecture configurations. > [6] https://www.torproject.org/docs/pluggable-transports.html.en Start with bundle-gnulinux.txt, as that's probably the easiest one. George told me that you have to use whatever the current Debian version number is rather than 7.0.0. It would help if you cna keep track of what parts of the instructions didn't work for you. David Fifield

1 0

Fraudulent PGP key with my email address (CEE1590D)
by Erinn Clark 02 Sep '13

02 Sep '13

Hello everyone, I discovered that there is a key out there (CEE1590D) associated with my Tor email address that is NOT me. I don't know who generated it, but I can think of many nefarious or incompetent reasons why they might have done it. This email is for two purposes: 1. To inform you that this is NOT MY KEY. Do not under any circumstances trust anything that may have ever been signed or encrypted with this key. I looked around and was unable to find anything, but nonetheless, it is out there and that is creepy. 2. If anyone on any of these lists has encountered this key anywhere -- the main fear being that it has been used to fraudulently sign packages of some kind -- can you please let me/us know ASAP? Tor Project official signatures are listed here: https://www.torproject.org/docs/signing-keys.html.en Consider that the canonical source for all signatures! Be suspicious of anything not listed there and let us know if you ever find anything. Thanks, The Real Erinn

2 1

Re: [tor-dev] Pluggable Transport Browser Bundle FTE Integration
by Kevin P Dyer 01 Sep '13

01 Sep '13

[Taking this discussion to tor-dev.] On Sun, Sep 1, 2013 at 6:32 AM, George Kadianakis <desnacked(a)riseup.net>wrote: > Kevin P Dyer <kpdyer(a)gmail.com> writes: > > > Hi George/David, > > > > Hi Kevin, > > > I spoke with Roger at USENIX. He said you're the pluggable transport (PT) > > gatekeepers. Please bear with me while I get up to speed. > > > > My goals: > > 1. I want Format-Transforming Encryption (FTE) [4] to be a "deployed" PT > in > > the PT TBB. > > 2. I want FTE to be integrated seamlessly with your existing deployment > > process. > > > > My initial roadblocks: > > > > === Building/Testing Tor on Linux/OSX/Windows > > I'm trying to understand exactly how the current build/release process > > works for tor. In regards to the PT TBB it seems like there are a few > > resources [1,2,3]. However, is there a canonical documentation on how the > > release process works? I'm especially interested in what you guys are > doing > > to produce builds on Windows. Are you using virtualization or do you > have a > > few physical build machines? > > > > Prior to doing anything with FTE. I'd love to be able to create my own > > build environment that produces that current obfs2+obfs3+flash_proxy > > bundles [6] across all 4 OS/architecture configurations. > > > > Building PTTBBs is mainly done by David these days. He has documented > his process here: > https://gitweb.torproject.org/pluggable-transports/bundle.git > (For example for Windows you would look here: > > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bu… > ) > > The release process is not standarized. David is doing PTTBB releases > in a best-effort manner. > Right. On first glance, looks like this process will increase in complexity (and utilize more of David's time) as the number of PTs increases. I need to better understand the build process, then. > > === Implementing Managed Mode in FTE > > I've implemented preliminary functionality for "managed" mode in FTE. > > However, I think I'm confused about the role of managed mode. > > > > Say I add "Bridge fte IP:port" to torrc. Is "IP:port" supposed to be a > tor > > bridge, or a server-side PT service? If it's the former, then it seems > that > > the PT is completely responsible for managing a list of its own PT > servers. > > If it is the latter, I can't figure out how to dynamically determine, via > > the "managed" environmental variables, how to capture user-entered > > "IP:port" information in Vidalia. > > > > It is the latter. <IP:port> points to the server-side PT service. It > does *not* point to the ORPort of the bridge (we don't care about it). > > > Concretely, if I have "Bridge fte IP1:port" and ""Bridge fte IP2:port" in > > my torrc, how does "IP1:port" and "IP2:port" get propagated to my PT via > > the managed interface? > > > > The <IP:port> is *not* passed to your transport using the managed > mode. Instead, <IP:port> is passed to your transport using the SOCKS > protocol. That is, when Tor wants to connect to the bridge, it does a > SOCKS handshake to your transport, and asks your transport to connect > to <IP:port>. Is this documented anywhere? > > === How do we invoke PTs? > > I had this discussion with Roger, but I don't see any open tickets or > clear > > discussion on this already. If we have N>1 PTs and at least one bridge > per > > PT, how do we select which PT (and which bridge associated with that PT) > to > > use? Determinism is bad because then only one PT is used. Booting up all > > PTs is bad, especially if (say) the PTs make network connections prior to > > any incoming SOCKS connections. Selecting a random PT is potentially bad, > > too, depending upon how hostile and persistent and stateful the adversary > > is. > > > > That's an interesting question. I'm not sure if the process of Tor > picking bridges is deterministic or not. I should test it out. David > might know. > > (A good scenario would be that Tor treats bridges like guards and > selects some at random to build circuits.) We should definitely try to flesh this out. > > We should probably chat about this. (Maybe you already have and I'm out > of > > the loop?) It is especially important as the number of PTs increases. > > > > I'm happy to take this discussion (or a subset of it) public, if you > think > > it'll help others. I just didn't want to spam tor-dev/tor-assistants with > > this initial email. > > > > Yes, let's take it public. > Feel free to CC tor-dev in your next reply. > Done. > BTW, on the topic of deploying your PT, have you seen: > https://lists.torproject.org/pipermail/tor-dev/2013-August/005231.html ? > > FTE seems to be missing out on the code quality front. > The Python code is quite complex and undocumented. There is also some > C++ code in the codebase that is also complex and undocumented. We > have decided that we won't ship C/C++ code in PTs, except if it's dead > easy to review or if it has gotten heavily scrutinized. Any chance > that the C++ code could be written in a memory-safe language? Roughly, FTE has an offline mode (building DFAs, needs to be done once) and an online mode (transporting data, deployed to everyone.) In terms of C++ code, there are ~300 line of C++ code used in online mode that were implemented for performance-critical algorithms. Implementing this code in Python will slow FTE down by at least an order of magnitude. I'll document why I made this decision. Alternatively, any suggestions for a memory-safe language that allows hooks for Python and affords the same performance as C++? In terms of reducing complexity of the Python code, and increasing code documentation, do you have concrete suggestions? It would be great if you could raise a few issues on FTE's github [7]. My time is limited and I would prefer to focus on the things you care about. Thanks, Kevin > [1] https://gitweb.torproject.org/pluggable-transports/bundle.git > > [2] https://lists.torproject.org/pipermail/tor-dev/2013-June/005056.html > > [3] https://gitweb.torproject.org/builders/tor-browser-bundle.git > > [4] https://kpdyer.com/fte/ > > [5] > > > https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/180-pluggabl… > > [6] https://www.torproject.org/docs/pluggable-transports.html.en > [7] https://github.com/redjack/FTE

1 0

Trac upgraded to 0.12.3
by Erinn Clark 01 Sep '13

01 Sep '13

Hello everyone! Weasel and I have upgraded https://trac.torproject.org to a newer version. Right now it has all of the functionality you were already accustomed to, but in the next few days it will get some more exciting features, like git integration. (I already verified that we are able to do it with our current git setup, but I will need to go through and poke at individual repos to make sure it works correctly.) Right now I'm just starting with a small list: arm, tor-browser-bundle, obfsproxy, stem, tor, torbrowser, and torbutton. (I tried bridgedb but trac freaks out because it doesn't like something about that repo; I will try to figure out what's wrong after I finish these others). If you want git integration for your repo(s), please file a ticket against the Trac component and I'll see what I can do! Once the git integration works, we'll demo a code review tool (http://trac-hacks.org/wiki/CodeReviewerPlugin) I'll email again once we're further along with both of these processes. In the meanwhile, let me know if you run into any issues with the new trac. Also, I would be remiss if I did not mention Mike's new favorite feature: a tag cloud. https://trac.torproject.org/projects/tor/tags Thanks, Erinn (& weasel)

2 2

Tor Control PHP library
by Kévin Dunglas 30 Aug '13

30 Aug '13

Hi, I've built a PHP library implementing the TOR control protocol: https://github.com/dunglas/php-torcontrol/ I've also built a Symfony framework bundle for this library: https://github.com/dunglas/DunglasTorControlBundle It allows to control a Tor server from a PHP script. Of course it's open source (MIT license). Can you add this library to the list ? -- Kévin Dunglas Freelancer http://dunglas.fr

2 2

Torsocks 2.0.0-rc1 release
by David Goulet 29 Aug '13

29 Aug '13

Greetings everyone! Two months ago I sent an email [1] to the tor-dev mailing list explaining some important issues with torsocks. I proposed myself to do a complete rewrite for the reasons detailed in the email. I've released today the first release candidate for a 2.0.0 version. https://github.com/dgoulet/torsocks Package: https://github.com/dgoulet/torsocks/archive/v2.0.0-rc1.tar.gz I'm asking the community for feedbacks, testing, code review, optimization design and/or anything useful to help improve the project (even correcting the mistakes or improving any comments). The TODO file in the repository contains some needed features but that are not critical to the project for a beta release. Hopefully, with your help, we will be able to replace the current version with a better one, more tested, more hardened and more portable!... and maybe with a more meaningful name :). What is really needed right now is OS X and BSD testers. I did my best to only use POSIX standard and making sure that libc calls are the same on all platforms but I was not able to test it on other OS other than Linux so they might some issues. There is *no* Windows support but again that would be amazing if someone could port it. Again, please contribute and help this very useful tool getting better! I encourage you all to use tor-dev mailing list for any development issues. I'm also on IRC in #tor-dev as "dgoulet". You can also find me on OFTC, freenode and indymedia servers. Big thanks to everyone! Using it is testing it! :) Cheers! David [1] https://lists.torproject.org/pipermail/tor-dev/2013-June/004959.html

4 7

Trac Ticket 1922
by Bao Nguyen 28 Aug '13

28 Aug '13

Providing a fix for #1922, I urge whoever has the authority to decide what folder tor is to parse. I saw /etc/torrc.d/ and /etc/tor.d/ or any others. I am going to make a patch. So which one is it?

1 0

HSDir hash ring modification
by Kang 28 Aug '13

28 Aug '13

Hello. I was reading about hidden services and a thought occurred to me regarding the hash ring used in choosing and determining the HSDirs for a hidden service. As far as I can tell the hash ring is more or less static since a relay's position is determined by their identity key, which doesn't change. I'm also under the impression that the hash ring is only used for calculation of HSDirs of hidden services. I don't have a particular method in mind, but it seems to me that you could use the "time-period" value that is used in calculation of the service's descriptor-id to shuffle the ring. This would cause the ring to be different for each hidden service, and also make its order change periodically. I imagine in particular it would make onion address enumeration attacks more difficult, since an attacker wouldn't just be able to "cast a net" over the ring for all services. Can anybody see any problems or false assumptions with this? Thanks

2 3

Trac Ticket 8291
by Bao Nguyen 27 Aug '13

27 Aug '13

Hi, I am one of the volunteer from Whonix project, a third party application. We want to use tor and modify it to parse the folder /etc/tor.d/ for configuration files. Please take a look at https://trac.torproject.org/projects/tor/ticket/8291 What is the process to get it approved and check in the file. I have modified the file src/or/config.c Who should I talk to? Thanks

3 4

[GSOC] Status report - Tor capabilities
by Cristian-Matei Toader 26 Aug '13

26 Aug '13

Hello tor-dev, Apologies for submitting the report later than planned, but I was on holiday towards the end of last week. I am currently waiting for a code review for the stage 2 of the seccomp filter, which will very likely include a relatively long list of change requests. As a reminder, at the moment there should be one filter which does both syscall and parameter filtering. As a brief update the following changes were made for the past 2 weeks: - various syscall filter changes for linux 32/64 bit - dealt with issues related to the open syscall, due to the fact that we use path filtering and external dependencies happen to use 'open' with strings unprotected by the filter (libevent, getaddrinfo, openssl) - received feedback and added socket filters, which support parameter filtering only for 64 bits, since on 32 they get multiplexed on one syscall (socketcall) - waiting for review on [1] in public branch [2] For stage 3, further privileges will be dropped for different sections of code (eg. the worker threads), and the project will support multiple filters. More details may be found here [1]. Looking forward to some feedback, if you happen to have any! References: [1] https://trac.torproject.org/projects/tor/ticket/9249 [2] https://github.com/cristiantoader/tor -gsoc-capabilities/tree/gsoc-cap-stage2

1 0