- tor-dev - lists.torproject.org

[OONI] Designing the OONI Backend (OONIB). RESTful API vs rsynch
by Arturo Filastò 19 Jul '12

19 Jul '12

I would like to follow up on the discussion we had in Florence on some design choices behind OONIB. In particular the most controversy was around using HTTP or rsync. Before discussion the pro and contra about one choice over the other it would be useful to frame what are exactly the requirements for OONIB. # What is OONIB OONIB is the backend of OONI. It will run mainly on one centralized machine and may in a later stage run distributed across multiple ones. Currently we have not though of how to make it scale to being distributed, so we will look at it as if it were running only on one central machine. It will be responsible for: * Reporting a) Collecting reports from OONIProbes. Such reports are the results of tests. b) Collecting reports from third party censorship measurement tools (e.x. Bismark, NeuBot, etc.) * Assistance in test running Certain OONI Tests require to have a backend component (e.x. b0wser). On OONIB we will have the serverside component that will assist us in running the test. Note: Certain tests require the server to make connections to the client. This means that the client will need to request the server to probe them. * Control Channel Signaling This is required for making some measurements to verify that the data received by the backend specific to the test matches with the one sent by the client. # What properties we would like it to have note: these are not ordered. * Efficient even over high latency networks. * Ease of integration for third party developers. * Expandable to support requirements of new tests we develop. * Anonymous * Secure # HTTP and rsych comparison note: I will not deal with the security aspects of OONIB. We will suppose to have an encrypted and authenticated transport (this can be TLS, Tor Hidden Services, etc.) ## Rsync: Pro: * It supports good compression algorithms * It's efficient and supports resume * It does integrity checking on the uploaded files Contra: * It's designed only for copying files, this means we can't implement any more advanced API like logic. [*] * It's not supported by many languages (for example in python we only have an implementation of the rsync algorithm, not of the protocol [1]) * It's not as commonly used by other application developers that have similar requirements. * Painful to do sanitization of the data sent by clients. * Does not allow bidirectional communication (Request-Response pattern) [*] I would like to be able to create a session ID for a specific test and be able to reference such test ID when interacting with the Test helpers. rsync is one way, I push data to the server, but the server cannot signal me back with some data. This largely impeeds it's usefulness as an API interface. ## HTTP: note: I am not necessarily talking only about HTTP, we could use any other protocol with similar properties (e.x. SPDY). I will discuss HTTP because it is the one that I am most familiar with, but don't Pro: * Industry standard for exposing APIs * Supported natively in most programming languages * Well understood protocol * Implementation of sanitization of passed data can be done more easily * Allows bidirectional communication * Good support in twisted (what we use as a language for OONIB) Contra: * Compression is not enabled by default (we can use gzip compression with HTTP 1.1), and no compression for headers. * No resume support (this can be implemented on top of HTTP, we could even implement the rsyc algorithm on top of HTTP). * No support for deltas (we can use rsych protocol over HTTP if we really need this). I feel like we are a bit comparing apples and oranges here and I don't see why we could not use rsync algorithm on top of HTTP. Anyways I would like to get some feedback as to what we should use for something that should have the above described properties. Thoughts? - Art. [1] https://github.com/isislovecruft/pyrsync

7 10

[GSoC] Vidalia Report - Week 8
by Feroze Naina 18 Jul '12

18 Jul '12

Last fortnight, I had got the add, remove and edit Hidden Service functionality working. I have been working on fixing a few UI issues - sizing, input sanity check and a browse button for selecting directory. I'm beginning work on the bandwidth scheduling plugin. I've been trying out the ways of doing it through relaybandwidthrate and will be starting on the UI. [1] https://github.com/feroze/vidalia-plugins/commits/hiddenservice

1 0

First-time tails/tor user feedback
by Andrew Lewman 18 Jul '12

18 Jul '12

I spent the past week in Sweden, attending the Stockholm Internet Forum, http://www.stockholminternetforum.se/, for part of it. I made a number of tails 0.10.2 usb sticks for people on request. I also asked a lot of people their impressions of Tor and Tails. I received a plethora of feedback. All 8 people are involved in the Internet Freedom policy, technology, or freedom of speech communities. They had very different levels of self-assessed technical skill. The 8 people represented 6 countries. tldr; 8 people were tested, 8 people had trouble doing simple things with tor browser in Tails. Issues 4-6 are directly related to Tor. And by asked, I mean, I stuck them in front of my laptop, put the usb stick in the computer, and asked them to browse to their favorite site. No one wanted to be video recorded, even if I offered to only record the screen and not audio. Everyone managed to power on the laptop and wait for Tails to boot. # First issue: Language selection The first issue was on the language selection screen. 4 of 8 people were confused why it was called "Debian Live System" and not "Tails Live System". 8 of 8 knew what language selection meant, but weren't sure how this mapped to Tails. # Second issue: wifi and tor browser The tor browser starts up before the wireless is configured. The tor browser then reports a proxy error. With some prompting, all 8 figured out the wifi and then didn't know what to do. Tor does keep trying to load, and takes forever because it needs to download the entire directory. Users have no feedback as to what's going on behind the scenes because vidalia is hidden. 8 of 8 waited patiently for something to happen on the screen. # Third issue: green onion 3 of 8 people saw the green onion appear in the menu bar up top. These three people hovered over it and saw the 'Connected to the Tor Network' message. No one knew to double-click on it to get a menu of other things to do. No one knew to right-click on it to get the drop-down menu. They were presented with the default check.torproject.org 'congratulations' page and then sat there. # Fourth issue: check.tpo is not helpful 8 of 8 people saw the default check.torproject.org site telling them 'congratulations. Your browser is configured to use tor.' 7 of 8 people asked 'where is my browser?' The one who didn't ask this question was already a firefox user and recognized the interface. 0 of 8 understood what the IP address message meant. Comments ranged from 'is that different than my current IP address?' to 'what's an ip address?' As an aside, when showing someone TBB on their own laptop, they saw the check.tpo site, and then went to Safari and started it up. When asked why they did this, the answer was 'safari is my browser. this says your browser is configured to use tor.' No one used the language selections at the bottom of check.tpo, nor even understood why they were there. # Fifth issue: exit relay congestion/failures 8 of 8 people tried to get to their own sites. 'I wonder what my site looks like when I'm anonymous' was the most common comment (5 of 8). For 6 of 8 people, their site didn't load at all, and tor browser reported their site was unreachable. All 6 then tried to go to google search in their own language; meaning google.es, google.se, etc. For 3 of those 6, this didn't work either. They gave up and assumed tor was broken or was censoring their destinations. I intervened, opened the vidalia network map, closed the circuit in question, and asked them to repeat their browsing. 5 of 6 were able to get to their sites now. The one that was not able to had the same exit relay as last time, Amunet1, in a new circuit and just couldn't get anywhere through it. After yet another new circuit, they could get through to everything. The user has no feedback as to why their site didn't work. And tor assumes everything is working fine. When asked "please find a video you like", they all went to youtube. Most of the videos they wanted to see resulted in 'This video is currently unavailable.' 8 of 8 assumed it was because youtube was blocking tor, not because the video is flash-required. 2 of 8 started randomly clicking videos suggested by youtube to see if any of them worked. Eventually, 2 of 8 got videos to work with youtube and were amazed it worked at all. # Sixth issue: no flash, no warning 2 of 8 people had flash apps on their website. 4 of 8 had ad banners that used flash. All were surprised at the red outline with a snake in it appearing instead of their flash apps. None understood what happened. After an explanation, one person suggested changing the red outline with snake to an actual message written inside, along the lines of 'this app blocked for your protection. click here to unblock it.' I explained why that wouldn't work (because there is no flash, java, silverlight plugins installed) and their answer was 'then do not show it at all'. Inside noscript, I unchecked the 'show placeholder..' option and had them browse again. they were happy. It seems if the user cannot do anything about the blocked apps, not showing them may be preferred. # Seventh issue: shutdown I asked all 8 to shutdown tails and let me know when they thought their data was safely no longer on the system. 1 of 8 figured out how to shutdown tails by clicking the big red button in the upper right corner. The rest hit the power button on the laptop. After rebooting, i showed them all they could just pull the usb drive to do it as well. As soon as tails started shutting down, they all assumed everything was safe and tried to power off the laptop. -- Andrew http://tpo.is/contact pgp 0x6B4D6475

5 5

Re: [tor-dev] Ports with hidden/rend services?
by Kevin Evans 17 Jul '12

17 Jul '12

D'oh! I hope this wasn't the wrong place to ask this. I couldn't seem to find anywhere else to ask this.

1 0

Ports with hidden/rend services?
by Kevin Evans 17 Jul '12

17 Jul '12

I am utterly dumbfounded by this (it's probably because I'm really tired), but in rendservice.c in the rend_service_t structure, there is a smartlist_t for the ports. How exactly are the [source and destination] ports stored in there? Thanks, Kevin

1 0

Clarification of prop 186
by Linus Nordberg 17 Jul '12

17 Jul '12

Hi, Can votes and consensuses have more than one "a" line? Prop 186 says, on one hand [...] votes should include a single "a" line for every relay that has an IPv6 address, to include the first IPv6 line in its descriptor. [...] The remaining or-address lines will turn into "a" lines in the microdescriptor. I interpret this to mean that there is at most one "a" line in a status vote or consensus. On the other hand it also says Clients that use microdescriptors should consider a node's addresses to be the address:port listed in the "r" line of a consensus, plus all "a" lines for that node in the consensus, [...] and the vote will include whichever set of "a" lines are given by the most authorities who voted for the descriptor digest that will be used for the router. (Is "vote" a typo? Should it read "consensus"?) This makes me think there can be more than one "a" line in both votes and consensuses. -- Linus

3 3

License for flash proxy software
by David Fifield 16 Jul '12

16 Jul '12

I feel there should be a LICENSE file in the flash proxy client packages currently at http://www.bamsoftware.com/dist/flashproxy/. What is the recommended practice? Right now the files carry no license. I would be happy with anything that is free software and convenient to the Tor Project. For example the 3-clause BSD used by obfsproxy is fine. I am pretty sure that I wrote all the code that is in use at this point. David Fifield

3 3

[GSoc] [Stegotorus] Progress report
by vmon 16 Jul '12

16 Jul '12

Hello everybody, Quick update on the progress of apache payload class for stegotorus: - https://github.com/vmon/payload_scraper/ is working now, checking apache document root dir, find all files usable as payload, compute their capacity and store them in a database. For now, I'm using the database in Stegotorus, but later the whole code is going to be part of Stegotorus and Stegotorus runs it periodically to keep the database updated. - I've written the basic functionality of apache_payload_server class, i.e. get_payload function (and anything else it needs to work). In theory, we should be able to call apache_payload_server::get_payload instead of get_payload, and Stegotorus would request the payload from apache (or as a matter of fact, any http server running on localhost). In practice, it still has compiling error which I'm working on and I expect them to be solved today. - I'm meeting Zack tomorrow to move my codes to his repo, so we have a unified repo from now on. Hopefully, I'll push the apache_payload_server on the new repo. - Afterward, I'm going to incorporate the infranet code http://nms.csail.mit.edu/infranet/ into the apache_payload_server to improve the quality of the steg. That is for now. Cheers, vmon

1 0

[GSoC] Stem stuff - Progress report
by Ravi Chandra Padmala 16 Jul '12

16 Jul '12

Hello. During the last week Damian merged my patches that implement loadconf & saveconf wrapper methods[1][2] in Stem. I am currently working on implementing the router descriptors (from the network status documents). I should have something ready for review by tuesday. Once that is done, I'll work on implementing the microdescriptor parser. I will also be getting my remaining controller class patches merged in parallel. 1. https://gitweb.torproject.org/stem.git/commit/6cc4e195452bc573e34f0b2de295b… 2. https://gitweb.torproject.org/stem.git/commit/e68adb7fdd465e1780bc4ee1243e0… -- neena

1 0

Minimum required ciphers for running Tor as both client and router.
by Gino Badouri 13 Jul '12

13 Jul '12

Hi there, First of all thanks for helping me last time with my mipsel build of Tor with statically linked OpenSSL. It's running fine and we're waiting for the last review of the bandwith-checking scripts before they go live. The second cluster we prepare will consist of low-powered PowerPC devices 250MHz/256MB Ram/8MB Flash. This time Tor will dynamically link to OpenSSL (libcrypto.so.1.0.0 and libssl.so.1.0.0) because there are also other programs depending on it. Because the devices have very limited flash-space the binaries and libraries have to be as small as possible. Tor 0.2.3.19-rc is only 550KB with only libevent staticly linked in (stripped and bz2 compressed). When run, it will be extracted to /tmp, executed and then deleted (since /tmp runs in RAM). It runs very well so far :) However the OpenSSL libraries are quite large for this system and they don't fit in at the moment. libcrypto.so.1.0.0 is 1.9MB and libssl.so.1.0.0 is 375KB (stripped). Our goal is to run Tor both as client and router and therefor I'd like to know the absolute minimum required ciphers for doing so. We've currently compiled OpenSSL 1.0.1 with: NM=powerpc-gnu-nm CC=powerpc-linux-gnu-gcc RANLIB=powerpc-linux-gnu-ranlib AR=powerpc-linux-gnu-ar \ ./Configure --prefix=/home/mastag/root --openssldir=/home/mastag/root/lib/ssl \ shared threads zlib enable-tlsext no-ssl2 no-dso no-idea no-mdc2 no-rc5 \ no-engines no-hw linux-ppc \ >From the OpenSSL documentation it seems that no-hw and no-engines leaves out support for hardware crypto engines so those are safe to set (our devices don't have them). Could anybody provide us with more "no-" options for ciphers we can skip? Thanks alot!

2 2