- tor-dev - lists.torproject.org

GSoC Status Report - EvilGenius
by Johannes Fürmann 12 Jul '13

12 Jul '13

Hi *, As the weekend approaches, it is time for my second status report on the censorship simulation project "EvilGenius". In the last two weeks I learned a lot about virtualization and the OONI project EvilGenius is being planned and written for. After fixing smoothing out some rough edges in the Vagrantfiles, I created the first 'censorship provider', i.e. a blueprint for a box that actually does censorship, in this case by untruthfully answer DNS Queries to example.com with its own IP address. As soon as this was done, Arturo and I talked about wrapping a nice CLI around it and came up with a plan, which i will explain briefly. basically, there are two core components to EvilGenius: - Censorship providers and - Measuremet Instruments. Both of those components are boxes that run in different configurations described by a .yml file. Our first goal is to generate vagrantfiles from those files and boot them up. During the weekend, I'm trying to tackle the Censorship provider class and see how it goes from there. You all have nice and relaxing weekends Johannes

1 0

'time_t' warnings on MSVC
by Gisle Vanem 12 Jul '13

12 Jul '13

I've been building tor myself for 2 years now. In order to reduce the number of warnings, I've added '-D_USE_32BIT_TIME_T' to my CFLAG. This has worked fine until I updated and built OpenSSL 4 days ago. It's 'ASN1_time_adj()' function and related 'OPENSSL_gmtime()' just doesn't work when given a 32-bit 'time_t'. (Not sure how/if they worked with a 32-bit time before). Maybe this could be detected by common/tortls.c before calling into OpenSSL? Well, tor builds and works okay w/o this define. But I wish the noise-level from "cl.exe -W3" could be reduced. They're probably harmless now. But what about after the year 2036? The warnings related to 'time_'t are: channeltls.c(1403) : warning C4244: 'function' : conversion from 'time_t' to 'long', possible loss of data channeltls.c(1404) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data directory.c(1711) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data directory.c(2924) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data dirserv.c(1795) : warning C4244: 'return' : conversion from 'const time_t' to 'long', possible loss of data dirserv.c(3092) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data geoip.c(1240) : warning C4244: 'initializing' : conversion from 'time_t' to 'long', possible loss of data hibernate.c(356) : warning C4244: 'return' : conversion from 'time_t' to 'long', possible loss of data networkstatus.c(754) : warning C4244: 'initializing' : conversion from 'time_t' to 'long', possible loss of data networkstatus.c(1299) : warning C4244: 'initializing' : conversion from 'time_t' to 'long', possible loss of data networkstatus.c(1329) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data networkstatus.c(1337) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data networkstatus.c(1846) : warning C4244: 'initializing' : conversion from 'time_t' to 'long', possible loss of data rephist.c(189) : warning C4244: '+=' : conversion from 'time_t' to 'unsigned long', possible loss of data rephist.c(192) : warning C4244: '+=' : conversion from 'time_t' to 'unsigned long', possible loss of data rephist.c(213) : warning C4244: '+=' : conversion from 'time_t' to 'unsigned long', possible loss of data rephist.c(227) : warning C4244: '+=' : conversion from 'time_t' to 'unsigned long', possible loss of data rephist.c(333) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data rephist.c(387) : warning C4244: 'initializing' : conversion from 'time_t' to 'long', possible loss of data rephist.c(497) : warning C4244: '+=' : conversion from 'time_t' to 'long', possible loss of data rephist.c(515) : warning C4244: '+=' : conversion from 'time_t' to 'long', possible loss of data rephist.c(517) : warning C4244: '+=' : conversion from 'time_t' to 'long', possible loss of data rephist.c(531) : warning C4244: 'initializing' : conversion from 'time_t' to 'long', possible loss of data rephist.c(535) : warning C4244: '+=' : conversion from 'time_t' to 'long', possible loss of data rephist.c(559) : warning C4244: 'return' : conversion from 'time_t' to 'long', possible loss of data rephist.c(1053) : warning C4244: 'initializing' : conversion from 'time_t' to 'long', possible loss of data routerlist.c(5038) : warning C4244: 'function' : conversion from 'time_t' to 'long', possible loss of data entrynodes.c(160) : warning C4244: '=' : conversion from 'time_t' to 'long', possible loss of data --gv

1 0

Stray ';'
by Gisle Vanem 11 Jul '13

11 Jul '13

There's an extra ';' in src/or/config.h that MSVC doesn't like so much. Patch: --- Git-latest\src\or\config.h 2013-05-28 07:32:46 +0000 +++ src\or\config.h 2013-05-28 07:39:42 +0000 @@ -109,7 +109,7 @@ char *transport_name; /* The name of the pluggable transport that should be used to connect to the bridge. */ char digest[DIGEST_LEN]; /* The bridge's identity key digest. */ - smartlist_t *socks_args;; /* SOCKS arguments for the pluggable + smartlist_t *socks_args; /* SOCKS arguments for the pluggable transport proxy. */ } bridge_line_t; ---gv

2 1

On status of Stegotorus on github vs SRI
by vmonmoonshine＠gmail.com 10 Jul '13

10 Jul '13

Hi Jeoren, Thanks for your comments or tor-dev IRC channel. I thought I may as well take my time and reply to them here so I can also include Zack, Vinod and others into the discussion. >I think the large chunks of commits without specific >goals/descriptions what the commit fixes will be timecostly to > integrate, I started working on Stegotorus as a GSoC student last summer and I have been more or less continuously working on it since then. I always kept the tor-dev list informed on my progress. So a quick search on [Stegotorus] should gives anybody a good idea on stuff I was working on. But I can summarize all my works in four categories: - Debug. - Writing more unit and behavior tests, including tester-proxy, webpage-tester and bad-dropper-proxy. - Integrating Stegotorus with a web server (e.g Apache Httpd) so the covers can be served by the web server. - Debugging the ack/transmit protocol and the packet transmission - reliability in general. In general, along the way, I tried to reconstruct the code into being more C++ and more object oriented (I'm continuously doing so). Obviously Stegotorus was designed to be as modular as possible. That is the main motivation behind "pluggable transport": the DPI got more intelligent? Plug the compromised module out, plug something else in with minimum effort. All of Stegotorus different modules are begging for OOP and inheritance. However, forcing the code into C had just made the steg folder unmanageable and un-expandable. The different steg modules (pdf, swf and js) have something like 70%-90% code in common but instead of being inherited from a common parent the code has been copied multiple times. It seems like an artificially generated example by OOP evangelists to convert people into OOP by showing how terrifying functional programming can be. >especially as the hidden upstream has diverged quite a >bit in some places (which one can nicely attribute to the wrong of >that 'model', but unfortunately not an easy way around that); Until recently that Roger sent an email to Vinod, I did not know that the upstream in SRI is making any progress (I'm not sure if even Roger (or anybody else on the Tor side for that matter) was fully aware of the amount of progress on the other side) . Last winter, Zack told me that I'll be the maintainer of the code (at least from Zack's prospective) in the sense I'd decide what patch to keep and what to discard. > nevertheless, I think the best plan to get those in there would be to > see when that code is out, and have a list of important > ... fixes/commits available and then at one point go over everything > and try to get them merged into that code base, will be big > job though to get that done, and would 'require' > that upstream takes them, which mostly depends on > available time I would say.... As Roger and other have mentioned multiple time, it is hard to fully reconcile the SRI model of development with the open source model. In that sense, we'll probably always have an open source version of Stegotorus and a SRI version of it (At least till SRI development is active). Nevertheless, my goal is to reconstruct the best piece of software possible out of all available code and I'll do my best to integrate all improvement into the open source version. > I think for that matter, trying to really have upstream have a > github version and accept patches would be a good way to avoid such > scenarios; hence also why we'll be pushing the acs code there into > the safdef repo, just the easiest way to make that happen if one > wants it; but first to finish that code and stop restructuring it > and adding new features etc.... then it will be pushed there, > otherwise not much use for people looking at it anyway. I do agree that the best thing is to wait for updated code to be published before going ahead to avoid re-invention of the wheel and waste of time. But I have a contract to add some features to Stegotorus with deadlines and I think Roger has asked Vinod for the code release on March 25 and it is not clear how longer it is going to take. Nonetheless, I'm going to ask Vinod if he can have a quick look at those features and decide which one are more likely to be affected by the recent development. For example when you told me about the ACK, I moved to the other parts. Thanks for your advice and updating me about the other side. Please keep me updated of any detail that you think will help with Stegotorus development. Cheers, Vmon

1 0

atlas.torproject.org question
by me＠rndm.de 08 Jul '13

08 Jul '13

Hi there, I rebuild atlas.torproject.org using Ember.js instead of Backbone. It looks like this right now: search page: http://i.imgur.com/GEM86mk.png details page: http://i.imgur.com/1UwpwlA.png I'd like to share the code and thought about putting it on github but I don't know if it's ok for you with all the Tor assets (tor logo). Would love to hear what you think about it. thank you

7 25

Torsocks development status
by David Goulet 07 Jul '13

07 Jul '13

Hi everyone, For those who don't know, I've been working on a new version of Torsocks in the last three weeks or so. https://lists.torproject.org/pipermail/tor-dev/2013-June/004959.html I just wanted to give a quick status report on the state of the development. The DNS resolution is working for domain name (PTR) and IPv4 address. Currently, Tor does not support IPv6 resolution but the torsocks code support it. Hidden service onion address resolution is also working using a "dead IP range" acting as cookie that is sent back to the user and mapped to the .onion address on the hijacked connect(). I've changed quite a bit the configuration file (torsocks.conf) to fit the style of tor (torrc). At this point, the tor address and port can be configured as well as the "dead IP range" mention above. More is coming but pretty simple for now. Logging is working, connection registry and thread safety as well. There is also a compat layer for mutexes and once I start porting the project to other *nix system (BSD, OS X, ...) probably more subsystem will be added to that compat layer. So, in a nutshell, some libc calls still need to be implemented, *moar* tests and other OS supports. I'm confident to have a beta version to present to the community in a couple of weeks (if nothing goes wrong). Feel free to browse the code, comment on it, contribute!, etc... https://github.com/dgoulet/torsocks/tree/rewrite Cheers! David

2 2

Vidalia resources no longer accepting translations
by Runa A. Sandvik 06 Jul '13

06 Jul '13

Hi everyone, As of today, the Vidalia resources on our Transifex page [1] will no longer be accepting translations. I feel translators should not work on resources which are currently not being maintained by a developer. When/if this does change, I will happily enable translations again. [1]: https://www.transifex.com/projects/p/torproject/ -- Runa A. Sandvik

1 0

Lead Automation Engineer Job Posting
by Mike Perry 06 Jul '13

06 Jul '13

Good news, everyone! Tor is looking for a QA/automation engineer. We want to deploy nightly builds and continuous integration for as many of our key software components and platform combinations as possible. Your job would be build and deploy the initial functional versions of a wide range of testing frameworks and continuous integration systems. This is a contract position. Candidates are expected to be capable of taking the lead in selecting, deploying, and maintaining multiple automation systems in several different programming languages. Candidates should also be capable of reproducing bugs and writing new reproduction test cases for one or more of the testing frameworks. Eventually, we hope to add additional staff to assist in this project, but to start, you will be expected to prioritize your own work such that the most important tasks get attention first, without letting any specific core component starve for attention. For more details, including information on how to apply, see the job posting: https://www.torproject.org/about/jobs-lead-automation.html.en -- Mike Perry

1 0

Re: [tor-dev] Migration of Tor Weather to Stem
by Damian Johnson 05 Jul '13

05 Jul '13

> Hi, > > My name is Sreenatha. I've made an attempt to migrate Tor Weather > from TorCtl to Stem as stated in my gsoc proposal(Module 2). > > Can you please take a look at the top 3 commits that I've made at my > github account? > Please suggest any additional changes that you think are necessary. > > Cheers, > Sreenatha > > P.S : I am lucyd@OFTC on IRC. Hi Sreenatha, glad that you've started in on this! We'd much prefer for development discussions to be on tor-dev@ so looping that in. This looks like a great start. I suspect there is a little confusion though around the socket object that the Controller uses. The Controller doesn't take a socket.socket instance, but rather a stem.socket.ControlSocket... https://stem.torproject.org/api/socket.html This comes up a couple places, most notably... https://github.com/lucyd/weather/commit/ff22e74c247bd61163a974329cd2feb2a12… I doubt that function presently works. Rather, it should be... def listen(): controller = Controller.from_port(port = config.control_port) controller.authenticate(config.authenticator) controller.add_event_listener(newconsensus_listener, EventType.NEWCONSENSUS) I'm not sure what 'config.authenticator' is so that part might be wrong. Also note that this creates a control socket then forgets about it (Weather never cleans it up). This isn't a bug you've introduced but rather what Weather already did. Not the end of the world (interpreter shutdown will clean it up, though possibly with a stacktrace about lingering threads). Have you gotten a Weather instance up and running to test your changes? Cheers! -Damian

2 2

IRC meeting to discuss sponsor F progress on Wed July 3, 16:00 to 17:00 UTC in #tor-dev
by Karsten Loesing 04 Jul '13

04 Jul '13

Hi all, I'd like to schedule an IRC meeting to discuss what progress we made on sponsor F deliverables in June. Suggested time and place are: Wed July 3, 16:00 to 17:00 UTC in #tor-dev That time in other timezones is: 9:00 in San Francisco 12:00 in Boston 18:00 in Berlin 19:00 in Athens 21:30 in New Delhi People who should attend are: Roger, Nick, Andrea, George, David, Sathya, Moritz, Linus, Andrew, Tom, and anyone else who wants to attend. If your name is on that list and the suggested date or time doesn't work for you, please let me know, either here or in private mail. An alternative date would be: Tue July 2, 16:00 to 17:00 UTC in #tor-dev I'm going to send out another reminder by the end of next week that will contain the definite date. Thanks, Karsten

3 5

Metrics Plans
by Damian Johnson 01 Jul '13

01 Jul '13

Hi Kostas. Now that we no longer need to worry about accidentally leaking GSoC selection we can talk more openly about your project. Below is an interchange between me and Karsten - thoughts? ---------- Forwarded message ---------- From: Karsten Loesing <karsten(a)torproject.org> Date: Thu, May 23, 2013 at 11:37 AM Subject: Re: Metrics Plans To: Damian Johnson <atagar(a)torproject.org> Cc: Tor Assistants <tor-assistants(a)lists.torproject.org> On 5/23/13 7:22 PM, Damian Johnson wrote: > Hi Karsten. I just finished reading over Kostas' proposal and while it > looks great, I'm not sure if I fully understand the plan. Few > clarifying questions... > > * What descriptor information will his backend contain? Complete > descriptor attributes (ie, all the attributes from the documents), or > only what we need? His proof of concept importer [1] only contains a > subset but that's, of course, not necessarily where we're going. > > If we're aiming for this to be the 'grand unifying backend' for > Onionoo, Exonerator, Relay Search, etc then it seems like we might as > well aim for it to be complete. But that naturally means more work > with schema updates as descriptors change... This GSoc idea started a year back as a searchable descriptor search application, totally unrelated to Onionoo. It was when I read Kostas' proposal that I started thinking about an integration with Onionoo. That's why the plan is still a bit vague. We should work together with Kostas very soon to clarify the plan. > * The present relay search renders raw router status entries. Does it > actually store the text of the router status entries within the > database? With the new relay search I suppose we'll be retrieving the > attributes rather than raw descriptor text, is that right? The present relay search and ExoneraTor store raw text of router status entries in their databases. But that doesn't mean that the new relay search needs to do that, too. > * Kostas' proposal includes both the backend importing/datastore and > also a Flask frontend for rendering the search results. In terms of > the present tools diagram [2] I suppose that would mean replacing > metrics-web-R and having a python counterpart of metrics-db-R (with > the aim of later deprecating the old metrics-db-R). Is that right? Not quite. We cannot replace metrics-db-R yet, because that's the tool that downloads relay descriptors for all other services. It needs to work really stable. Replacing metrics-db-R would be a different project. The good thing though is that metrics-db-R offers its files via rsync, so that's a very clean interface for services using its data. In terms of the tools diagram, Kostas would write a second tool in the "Process" column above Onionoo that would feed two replacement tools for metrics-web-R and metrics-web-E. His processing tool would use data from metrics-db-R and metrics-db-E. If his tool is supposed to replace more parts of Onionoo and not only replace relay search and ExoneraTor, it would use data from metrics-db-B and metrics-db-P, too. > Maybe we should focus on a 'grand unified backend' rather than > splitting Kostas' summer between both a backend and frontend? If he > could replace the backends of the majority of our metrics services > then that would greatly simplify the metrics ecosystem. I'm mostly interested in the back-end, too. But I think it won't be as much fun for Kostas if he can't also work on something that's visible to users. I don't know what he prefers though. In my imagination, here's how the tools diagram looks like by the end of summer: - Kostas has written an Onionoo-like back-end that allows searches for relays or bridges in our archives since 2007 and provides details for any point in the past. Maybe his tool will implement the existing Onionoo interface, so that Atlas and Compass can switch to using it instead of Onionoo. - We'll still keep using Onionoo for aggregating bandwidth and weights statistics per relay or bridge, but Kostas' tool would give out that data. - Thomas has written Visionion and replacements for metrics-web-N and metrics-web-U. You probably saw the long discussion on this list. This is a totally awesome project on its own, but it's sufficiently separate from Kostas' project (Kostas is only interested in single relays/bridges, whereas Thomas is only interested in aggregates). I'm aware that not all of this may happen in one summer. That's why I'm quite flexible about plans. There are quite a lot of missing puzzle pieces in the overall picture, people can start wherever they want and contribute something useful. > I was very, very tempted to start up a thread on tor-dev@ to discuss > this but couldn't figure out a way of doing so without letting Kostas > know that we're taking him on. If you can think of a graceful way of > including him or tor-dev@ then feel free. Let's wait four more days, if that's okay for you. Starting a new discussion there about this together with Kostas sounds like a fine plan. This will be an exciting summer! :) Best, Karsten > [1] https://github.com/wfn/torsearch/blob/master/tsweb/importer.py#L16 > [2] https://metrics.torproject.org/tools.html >

3 9

Run With Limited Capabilities - GSOC
by Cristian-Matei Toader 01 Jul '13

01 Jul '13

Hello, My name is Cristian Toader, and I feel very excited about designing and implementing a capabilities based sandbox for the central Tor project, as part of the GSOC program. ---- About myself: I have been a Linux enthusiast for almost 6 years and have first started using Tor around 3 years ago. I am currently studying in the UK. In approximately one month I will be graduating the Computer Science programme at the University of Surrey, and plan on pursuing a master's degree in Advanced Computer Science at the University of Cambridge for the following academic year. I have completed a placement year at Qualcomm (UK) LTD which involved implementing and testing security solutions for the Linux Android OS. These were based on cryptography and the TrustZone run-mode of the ARM processors. Most of the development during the placement year was performed in C, with some tests written in Java and C++ for the upper layers. ---- About the project: The project I will be working on as part of GSOC is based on the "Run With Limited Capabilities" proposal [1] mentored by Nick Mathewson (nickm) and Andrea Shepard (athena). The project is still in the planning stage. I will start working on an appropriate design as soon as I finish my last exams, which is the 3rd of June. As part of the project I will need to: - investigate research papers regarding capability based sandboxes - get familiar with the Tor code structure - decide on whether there should be different states starting from which the tor program only has a limited set of capabilities, depending on what syscalls it should be able to perform; or maybe have a more complex approach based on a trusted process representing a root of trust (with no network interactions) which controls the capabilities of the processes it launches - integrate an appropriate solution - develop and run tests for the project A constraint agreed with nickm, would be that once the program capabilities are set they should not be modifiable (otherwise a potential attacker could have the option of first enabling capabilities and then execute privileged code). Some additional details can be found in tickets #7005 [2], #5219 [3], and #5220 [4]. I will try to keep everyone updated. I am looking forward to advice and suggestions. If anyone needs to contact me, this is my primary email, my irc.oftc.net username is ctoader, and I am geographically located in GMT+2. Best regards, Cristian Toader. [1] https://www.torproject.org/getinvolved/volunteer.html.en#limitCapabilities [2] https://trac.torproject.org/projects/tor/ticket/7005 [3] https://trac.torproject.org/projects/tor/ticket/5219 [4] https://trac.torproject.org/projects/tor/ticket/5220

5 6

[GSOC] Status report - Tor capabilities
by Cristian-Matei Toader 01 Jul '13

01 Jul '13

Hello tor-dev, As a small reminder the purpose of the project is to create capabilities based sandboxing for Tor, which may only allow the program to execute a number of predefined syscalls. For the past 2 weeks: - I have consulted with Nick Mathewson (nickm) and agreed upon using seccomp2 [1], and more recently a library built on top of that called libseccomp [2]. - I have set up a public remote branch [3]. - We have agreed on a 3 step plan for the project: 1. General sandbox based on a single (permisive) filter which restricts tor to using a number of syscalls. 2. Add configuration option for step 1, if any parts were broken in phase 1 by adding capabilities, they can be re-enabled at the cost of security. 3. Figure out what functionality should be split into separate processes, based on our experience from step 1 and step 2. - So far I have implemented step 1 using both libseccomp and seccomp2 [3]. Step 1 was developed in such a way that nothing from tor should be broken at the moment; What this means is that sandboxing currently exists in the remote branch, but is fairly coarse and will need some fine tuning at a later stage such as only allowing specific files to be open, or allowing the exec syscall to be called with specific parameters. These days I will be adding command line support, which is basically step 2, which will be followed by a code review and merge in the master branch. [1] http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-precise.git;a=blob;f=Documenta… [2] http://sourceforge.net/projects/libseccomp/ [3] https://github.com/cristiantoader/tor-gsoc-capabilities

2 2

New flash proxy facilitator domain fp-facilitator.org
by David Fifield 30 Jun '13

30 Jun '13

I moved the flash proxy facilitator to a new domain, fp-facilitator.org. This is to get it away from bamsoftware.com, which also has a lot of unrelated stuff. The old facilitator name tor-facilitator.bamsoftware.com will continue to work (the DNS for both points to the same place). https://trac.torproject.org/projects/tor/ticket/7160 is the ticket. David Fifield

2 3

Status report - HTTP pluggable transport
by Chang Lan 30 Jun '13

30 Jun '13

Hi there, During the first two weeks of my GSoC project, I have implemented a HTTP CONNECT-based pluggable transport. In short, I use HTTP CONNECT semantics to establish a secure channel between the client and the bridge. Specifically, this is the scenario: 1. Connection establishment: +------------------------------+ |CONNECT bridge_ip:443 HTTP/1.0| |User-agent: blablabla | +------------------------------+ | | +----------+ | +----------+ +----------+ | client |----->| proxy |----->| bridge | +----------+ +----------+ | +----------+ | | +------------------------------+ | (Establish a connection) | +------------------------------+ 2. Data relay +------------------------------+ | (Encrypted Payload) | +------------------------------+ | | +----------+ | +----------+ +----------+ | client |<----->| proxy |<----->| bridge | +----------+ +----------+ | +----------+ | | +------------------------------+ | (Encrypted Payload) | +------------------------------+ I hope the diagrams above are self-explanatory. It is only my initial attempt to get familiar with HTTP protocol. Once I make sure it is working correctly under squid proxy I will upload it to the repository. The use of CONNECT method is restricted in many networks, so it is better to implement the HTTP transport using the usual HTTP methods as POST, GET, etc. In the next stage, I plan to implement a new HTTP transport based on BOSH[1]. There are many issues to care about (in order of priority): * bi-directional data transfer over HTTP * proxy cache * HTTP request/response encoding * encryption * scanning resistance Have a nice weekend! [1]: http://xmpp.org/extensions/xep-0124.html Best, Chang Sent from my mobile device. Sorry for the brevity.

3 2

Re: [tor-dev] Gitian builds in VirtualBox with Vagrant
by Arlo Breault 29 Jun '13

29 Jun '13

> This is rather exciting. Do you think that this method can be adapted to > build the pluggable transports bundles? Yes. I originally started working on this so I could submit a patch to include the tor-fw-helper in the TBB. But since the helper is only needed in the PT bundles, reworking their build process seems the more productive route. I'll see what I can do. > This is currently done > semi-manually, with a makefile that builds the pluggable transports, > unzips the bundle, copies in some files, and re-zips the bundle. > > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/Ma… > > We also have instructions for setting up a build VM from scratch--which > it sounds like is what Vagrant does--but I guess we would instead use > whatever is the build machine for the vanilla bundle. > > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/Ma… > > It would solve a lot of problems for us to have the PT bundles built at > the same place and time as the vanilla bundles.

3 2

Vidalia 2.0 - an complete rewrite
by Leo Unglaub 28 Jun '13

28 Jun '13

Hey friends, i want to inform you that i am working on a complete new control tool for Tor. I know some people use Vidalia for it, but i never liked the program and the people i showed vidalia often got confused. So i started to rewrite it from scratch and i am close to the release of the source code. (Currently i need to clean up some classes in there) I had a few things on my feature list for the new control tool and i want to share/discuss it with you. Because maybe some of my ideas are bad or maybe you have some additional ideas for features. A: "native work flow". Thats one of the main goals of my rewrite. The current Vidalia does things a little bit different than people are used to do it. So they get confused, because it does not behave "naturally". So instead of QT i used GTK and followed as always the GNOME HIG. (http://developer.gnome.org/hig-book/3.5/) B: "Multi configuration setup". What do i mean by that? Well, in my case for example i use Tor on my laptop and i am often in different places where i need different settings for tor. For example, if i am at home i can give tor all the bandwith i have, because at home i don't care. In my company on the other hand i cant do that. I am allowed to run Tor in the company, but with a bandwith limit of 2000kbit/s. (Yes, my company allows this ... well, i am the technical directory, so it was my call :) ) But i need a different Tor configuration there. Or if i am giving a talk and only have mobile internet connections i need every bit of bandwith for checking my emails so i want no server, only Tor client support. Currently i have multiple torrc files for that and always move them arround. That works for command line junkies like me, but even i am anoyed to do this multiple times a day. So i added the "multi configuration" support into the tool. This allows you to define multiple configurations and simply change them with a click from the gnome2 notification panel. C: "GUI for config settings". Well, some users are simply not ready to modify config files on there own. So i provide a simple gui (tab based) for all config options i found in the normal torrc file. Always with an explonation (with needs spellchecking, because my spelling sucks in german and is even worse in english :) ) so the user can deside what an option would to. D: "Notifications": My control tool uses the normal desktop notifications to inform the user about changes in the running tor application. For example if there is a problem building the circuit or something you get the notifyd message and can simply react to events without having the arm log open all the time. Thats just a quick overview of what i am duing right now. The source code will be released soon and is of course 100% open source. If you have some addiional ideas, feedback, wishes, ... i would like to hear them. Thanks and greetings Leo PS: I am Germam and very sorry for every spelling error i did above. -- Leo Unglaub Nachreihengasse 39 / Top 7 A-1170 Vienna Austria Phone: 0650 / 6575103 Website: http://www.leo-unglaub.net Twitter: http://twitter.com/LeoUnglaub Send with Mozilla Thunderbird on Linux (Debian)

9 24

Status Report - Steganography browser addon a GSOC 2013 project
by Hareesan 28 Jun '13

28 Jun '13

Hi all, In the first 4 weeks of my GSOC project steganography browser add-on, I have implemented the basic UI parts in first two weeks, and later I spent more time on web content downloading part. Up to now, I have finished downloading image and get them as bytes. Now when you right click on an image and click on test button, you will be getting the byte format of image which can be used with steganography algorithms to write messages. Since I'm mainly concentrating on features rather than algorithms now, my plan for next 2 week is to extend the file downloading for video and sound files. You can find the source code under git repository [1]. [1] https://github.com/rharishan/Steganography-Browser/ -- Hareesan

1 0

Re: [tor-dev] Torsocks development status
by David Goulet 27 Jun '13

27 Jun '13

Ian Goldberg: > On Wed, Jun 26, 2013 at 03:55:58PM -0400, David Goulet wrote: >> Hi everyone, >> >> For those who don't know, I've been working on a new version of Torsocks >> in the last three weeks or so. >> >> https://lists.torproject.org/pipermail/tor-dev/2013-June/004959.html >> >> I just wanted to give a quick status report on the state of the development. >> >> The DNS resolution is working for domain name (PTR) and IPv4 address. >> Currently, Tor does not support IPv6 resolution but the torsocks code >> support it. >> >> Hidden service onion address resolution is also working using a "dead IP >> range" acting as cookie that is sent back to the user and mapped to the >> .onion address on the hijacked connect(). >> >> I've changed quite a bit the configuration file (torsocks.conf) to fit >> the style of tor (torrc). At this point, the tor address and port can be >> configured as well as the "dead IP range" mention above. More is coming >> but pretty simple for now. >> >> Logging is working, connection registry and thread safety as well. There >> is also a compat layer for mutexes and once I start porting the project >> to other *nix system (BSD, OS X, ...) probably more subsystem will be >> added to that compat layer. >> >> So, in a nutshell, some libc calls still need to be implemented, *moar* >> tests and other OS supports. I'm confident to have a beta version to >> present to the community in a couple of weeks (if nothing goes wrong). >> >> Feel free to browse the code, comment on it, contribute!, etc... >> >> https://github.com/dgoulet/torsocks/tree/rewrite > > Are non-blocking sockets, select/poll/etc. (especially at connect() > time), and optimistic data on the to-do list? Yes! Good point I should have put the todo list. So yes, non block socket support. For optimistic data, it is kind of tricky. I can use it for DNS resolution without a problem because torsocks control the complete flow of data from opening a SOCKS5 connection to closing it after the DNS response is received however for actual real data (sendmsg, send, ...) a connect is needed before so it would means that a connect() call will return "yes OK socket connected" but where in fact it is not really true. So, when the first data are sent, there is a possibility that the Tor connections failed or even we block for an unknown amount of time during the send*/write() call. Now the question is, is this the kind of behavior that would be acceptable meaning basically lying to the caller at connect() and possibly blocking I/O calls and returning something like ECONNRESET or ENOTCONN if the Tor socks5 connection fails. This is *real* tricky especially with non blocking socket, if torsocks needs to do some possible blocking call for the SOCKS5 replies during an I/O call from the caller that is not suppose to block. Furthermore, having pending data that *might* come at any time on the connection from the SOCKS5 negotiation, the caller could put the file descriptor in poll() mode, wake up and try to receive the data but where in fact it's the socks5 reply... it's possible to handle that but it seems here a VERY intrusive behavior. Does optimistic data worth it here vis-a-vis the complexity of handling that it and high intrusiveness ? Cheers! David > > Thanks, > > - Ian

3 4

Status Report - Evil Genius
by Johannes Fürmann 27 Jun '13

27 Jun '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi *, The first two weeks of working on my GSoC project[0], which has been dubbed "EvilGenius", are nearly over. and I just wanted to share a bit of the experience I made during that time. EvilGenius is a tool that - much like Descartes' idea of an "evil genius", it "presents a complete illusion of an external world, including other minds, to Descartes' senses, where in fact there is no such external world in existence." (Source: Wikipedia), it distorts the guests' view of the real internet, in short, it creates a virtual environment that simulates censorship in order to verify OONI's nettest modules. So, after getting started with vagrant, the virtualization abstraction layer used for my project, i started simulating a virtual netwotrk and configuring the hosts to act as if they were connected to each other and the rest of the internet through a router that is part of the simulated environment. the "precise32" box is used as base image for all nodes, and I created scripts that configure the network and packed it up in a nice vagrantfile. Then I began deploying ooniprobe and oonibackend and executed the first successful tests. Next week, I will begin working on censoring environments. Have a nice weekend everyone! Johannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRzKDWAAoJEAgie5o3Q/JpUhAQAIt2WEpDsNyLVDuSdMBM0ETf q0sOxdx20Q6o77J93FJ4ZBJRzpBX4mlsE4t22Dl/TPgnHu3b+DG+dvVhfCBrQOqw wGUuA6/X/khCikAOo+f3T9fNtaHKw54phW1Tw9Yj3Gea7SBN5ZikfXKjIIHOT+lu n0xqWVzxAxqGOETy0byDznsdgc2OfaR2NkdxzCsLlZTLtE75h/EQ3U96ZCej8lw/ xmZZUD+q+4ZCE6bBKTebufQeh0eI/rUPgo0UX1rNLgUapfCs9oLVIF+8nwBKBU0j E2myEOUbXMCxtTURbHikRQKzGKlsXMCEqqSatik37Rc3VaeVdkxNiiyXFVRwzTr0 IwRTibzJ8hD+aqh9d9VDD6s7dIdRio42p+XzD8jswFltEyYcbBGeAFFiybdNeeE2 EFb7n7iHz7Gr1csezyma1fdqvvMHZaaeCMTqeRENQtBaazOnqDwrGJn83r/FC7QF m8i+3eVDm8suM7CzYvuYi8YSIhF7yMVerIBOcBSZ2A929SVKtQFT4by1V2Jc4pz+ 0ZJV4teS8xct0wjhpDLANBCXgnV07OO6Z6EgLhcWpJBWoZDONDaxikH7clmTlzdg w/9ZktL3Ac3PjhSdMjOI/TKqc68+CHQ7ln1mG+YtSimMjf5Abpx1wPgLLILmzFNS lpZ2vev2pW0DsCnvs946 =abNV -----END PGP SIGNATURE-----

1 0

Gitian builds in VirtualBox with Vagrant
by Arlo Breault 27 Jun '13

27 Jun '13

If anyone has VirtualBox and Vagrant installed, https://www.virtualbox.org/ http://www.vagrantup.com/ and wants to try the new TBB deterministic builds, this worked for me, > git clone -b vagrant https://github.com/arlolra/tor-browser-bundle.git > cd tor-browser-bundle/vagrant > make Mike, there's a few patches in there you may want, https://github.com/arlolra/tor-browser-bundle/compare/vagrant Or you can take the whole thing. Enjoy, Arlo

3 2

Damian's Status Report - June 2013
by Damian Johnson 27 Jun '13

27 Jun '13

Hi all. Without much ado here's my status report for June... * Stem: Migrated to the Mock module Our homemade mocking framework has served us well, but over time it taught me one very important lesson: writing a mocking framework is hard. On the surface it seems pretty simple: apply and revert a set of monkey patches. But how do you monkey patch class methods? What about alias imports like the os module? And god forbid you want to mock python's open() function. I'm finally taking a lesson from one of my coworkers and using a library for this. Python has several options but the most common is PyPI's mock module, which became part of the standard library in Python 3.3... http://www.voidspace.org.uk/python/mock/ https://gitweb.torproject.org/stem.git/commitdiff/101cf0b?hp=65846e8 ======================================== * Arm: Pruning the torTools utility Arm's torTools module was a wrapper around TorCtl that provided caching, thread safety, and a better API. Now that we're using stem it's obsolete, and my aim is to completely eliminate it from the codebase. This is easier said than done however. This month I pruned vast swaths of the module, reducing it from 2768 lines to 1020... before: https://gitweb.torproject.org/arm.git/blob/refs/heads/release:/src/util/tor… after: https://gitweb.torproject.org/arm.git/blob/HEAD:/src/util/torTools.py Doing this required some expansions on stem's part. Added functionality includes... * get_pid() and get_user() methods in the Controller * system functions for getting a process' start time and FreeBSD jail path * the system module's call() wasn't respecting exit codes ======================================== * Stem: Remote descriptor fetching Throughout the month Karsten and I have been bouncing ideas back and fourth concerning a stem API for remote descriptor fetching... https://trac.torproject.org/projects/tor/wiki/doc/stem#RemoteDescriptorFetc… https://lists.torproject.org/pipermail/tor-dev/2013-June/004957.html I have some s3krit ideas for improving this that will address both our use cases even better (spoiler: future style instances). We're coming up on a four day weekend so hopefully I'll be able to start implementing this soon! ======================================== * Stem: Expanded FAQ with answers to common Stack Overflow questions I took a pass over the tor related questions on Stack Overflow (http://stackoverflow.com/questions/tagged/tor) answering fifteen that concerned controller scripting. The vast majority of those unfortunately were some variation of 'how do I programmatically change my IP?' which I answered with a Stem FAQ entry... https://stem.torproject.org/faq.html#how-do-i-request-a-new-identity-from-t… The only question I thought was especially interesting went along the lines of 'How do I check the IPs of the exits I'm presently using?'. I answered this with a FAQ entry too... http://stackoverflow.com/questions/9777192/how-do-i-get-the-tor-exit-node-i… https://stem.torproject.org/faq.html#how-do-i-get-information-about-my-exits ======================================== ... and a handful of smaller tasks... * Investigated the descriptor version provided via tor's 'GETINFO ns/*' options. Contrary to to the spec it turns out these have been v3 all along, and stem now parses them as such. (#7953) * Our automated Jenkins test runs caught their first instance of tor regression. This concerned LOADCONF's behavior after merging a branch for ticket #6752 (#9122). * Handful of GSoC tasks (welcome/sorry emails after acceptance was announced, and org admin prep for the program's start) * Added msg_type argument to ControlMessage.from_str() (request by meejah, #8605) * Investigated cache consistency issue (thanks to Ravi, #7713) * Fixes for ONLINE target (patch by Jeremy, #8692) * Minor revisions to how consensus bandwidth-weights are validated (#6872) * Addressed an arm issue with certain TERMs (#9064) * Answered some questions from Sreenatha that came up while migrating Tor Weather to Stem (https://lists.torproject.org/pipermail/tor-dev/2013-June/005035.html) Cheers! -Damian

1 0

Re: [tor-dev] [tor-talk] Client simulation
by Karsten Loesing 27 Jun '13

27 Jun '13

(Sorry for cross-posting, but I think this is a topic for tor-dev@, not tor-talk@. If you agree, please reply on tor-dev@ only. tor-talk@ people can follow the thread here: https://lists.torproject.org/pipermail/tor-dev/2013-June/thread.html) On 6/6/13 7:32 PM, Norman Danner wrote: > I have two questions regarding a possible research project. > > First, the research question: can one use machine-learning techniques > to construct a model of Tor client behavior? Or in a more general form: > can one use <fill-in-the-blank> to construct a model of Tor client > behavior? A student of mine did some work on this over the last year, > and the results are encouraging, though not strong enough to do anything > with yet. > > Second, the meta-question: is it worthwhile to answer the first > question? It seems to me that if the answer to the first question is > "yes," then the solution could be used to (at least) provide better > simulations of Tor (e.g., via Shadow or ExperimenTor). This possibly > naive thought would imply that the answer to the second question is "yes." > > I'd be interested to hear responses to my second question, either > validating my naive thought or explaining why the first question isn't > worth answering. I'd accept responses to my first question, too, in > case this has already been done. Hi Norman, yes, it's worthwhile to answer this question! I can imagine how at least Shadow and the Tor path generator would benefit from better client models. User number estimates on the metrics website might benefit from them, too. I found two tickets where we asked similar questions before, and maybe there are more tickets like these: https://trac.torproject.org/projects/tor/ticket/2963 https://trac.torproject.org/projects/tor/ticket/6295 Some very early thoughts: - How do we make sure that we ask a representative set of people to instrument their clients and export data on their usage behavior? If we only ask people who read their favorite news site twice per day, our client model will be just that, but not representative for all Tor users. (Still, we would know more than we know now.) - Can we somehow aggregate usage information enough to make it safe for people to send actual usage reports to us? I could imagine having a torrc flag that is disabled by default and that, when enabled, writes sanitized usage information to disk. For this we need a very good idea what we're planning to do with the data, and we'll need to specify the aggregation approach in a tech report and get it reviewed by the community. Are your student's results available somewhere? Best, Karsten

2 5

XMPP Pluggable Transport
by Alex Eftimiades 25 Jun '13

25 Jun '13

I have been working on creating an XMPP pluggable transport for Tor for a couple of weeks now, and someone suggested I send an email to this mailing list for suggestions. At this point, my priorities are ordered as follows: 1. Speed--I can watch low quality youtube videos on a bad internet connection, but there is a lot of down time while it buffers (not an unbearable amount of time, but enough to be annoying). This has been a big problem for a while, and I could really use some suggestions on how to speed it up. 2. Full GTalk compatibility. It seems you need to be on a recipient's contact list to send a message to them (at least without supplying a resource). However, I currently use messages to send connection requests to JIDs with unknown resources. Then when they reply, their resource is saved in a table. This technique works fine for everything but GTalk, which of course is by far the fastest of the chat servers I have tested. 3. Error handling--I need to make sure I take everything into account. I do not have anything that can reestablish connections if one computer disconnects from the chat server and reconnects. In fact, if it acquires new resources (which is what happens on GTalk), the other computers will be stuck sending messages to a JID with the wrong resource. I am pretty sure I can come up with some ways of dealing with this. Right now, looking for potential problems is just as important as pointing out solutions to more obvious ones. 4. Security. I know security is important, but the nature of the program just does not seem to admit many security threats. There are really just three that I can see: - The server can be told to connect to any ip:port by a client. - The server and client exchange their Python interpreter's sys.maxsize values (which could lead to profiling a computer's OS). The former can be fixed with a whiltelist and the latter is not even necessary (I just have not gotten around to tweaking that out of the protocol yet). - DoS. This would be relatively easy to take into account by limiting the number of connections a bot is willing to accept. Then the chat server itself will limit the amount of data sent from any one JID (for better or worse). If you think you are able/willing to help, read on for a summery of the concept and protocol below. Even if you are not well versed in XMPP protocols (I myself am learning as I go), just helping to test the program would be a big help. Thanks, Alex Eftimiades A brief summery of how the program works can be found here: https://trac.torproject.org/projects/tor/ticket/9022#comment:42 , and is reproduced below: ---------------------------------------------<Summery of concept > --------------------------------------------------------------------------------------------------- When using hexchat, the data is sent to a local TCP socket running hexchat (call it hexchat1). Hexchat1 then reads the data (thereby stripping it of its TCP header) and passes it over a chat server to another hexchat program (call it hexchat2) that sends the data to the appropriate ip:port (giving it a new TCP header in the process). The client thinks it is sending the data to hexchat1, and the server thinks it is receiving data from hexchat2, but the data itself is never changed. It might be broken into smaller chunks or combined into bigger chunks, and it might be delivered at unpredictable rates, but it is never altered. ---------------------------------------------</Summery of concept > --------------------------------------------------------------------------------------------------- A more detailed overview of the protocol can be found in the protocol- spec here, though I will summarize it: ---------------------------------------------<Summery of protocol > --------------------------------------------------------------------------------------------------- When the program is started, it is told what JID(s) to use to connect to a chat server, and possibly the following optional parameters: 1. An ip:port to listen on. 2. A JID. Connection requests will be sent to this JID when a connection is is made to the ip:port specified in (1). 3. Another ip:port. This is the ip:port the bot connected from the JID in (2) should connect to. Note that any bot can connect to any ip:port that client asks for. This could be a 127.0.0.1 address, or a remote address (but will probably be the former). I plan to add a feature in which you can provide a list of acceptable ip:ports to connect to and refuse to connect to anything else (for security). However, that is not a priority at the moment. When a connection is made, the server and client exchange all JIDs they are using to connect to the chat server so they can each rotate through JIDs to send messages from and JIDs to send messages to. With the exception of initial connections, all messages are sent as IQ stanzas. When data is sent, it is base 64 encoded and sent with an ID number that is incremented each time a message is sent. When data is received, the id is checked against the id of the last message that resulted in writing data to a socket. The difference between these two ids is used as an index in a buffer. Then, messages are read from the buffer and sent to the appropriate socket. When a socket closes, a disconnect message is sent. Like when sending messages, an id stanza is included. When a disconnect message is received, the bot checks if there are any entries in the appropriate socket's message buffer. If not, it closes the socket and deletes it from the routing table. If there are messages in the buffer, the disconnect message is added to the buffer at the appropriate index as a string "disconnect" as though it were normal data. When a "disconnect" string is found in the buffer, the socket is closed and deleted from the bot's routing table. ---------------------------------------------</Summery of protocol > ---------------------------------------------------------------------------------------------------

2 2

Re: [tor-dev] Gitian builds in VirtualBox with Vagrant
by Arlo Breault 24 Jun '13

24 Jun '13

> So this is a wrapper to install+launch a VirtualBox Ubuntu VM to use > Gitian in LXC mode? Sounds good! > > Exactly. Just codifies what it took to get it running in a VM. > > Hopefully by Monday I will have tagged TBB 3.0a2, and we will begin > building it. If your build matches (or comes close), I will merge this > stuff for 3.0a3. > > Note that I am about to commit some changes to mkbundle-*.sh, so be > sure to rebase your github branch for your build (which you will also > need to do anyways to pick up the latest 'versions' file tags). > > Ok. I'll keep my eye out for 3.0a2

1 0