- tor-dev - lists.torproject.org

Tor in Google Summer of Code 2014
by Damian Johnson 27 Feb '14

27 Feb '14

Interested in coding on Tor and getting paid for it by Google? If you are a student, we have good news for you: We have been accepted as a mentoring organization for Google Summer of Code 2014! https://www.google-melange.com/gsoc/homepage/google/gsoc2014 https://www.google-melange.com/gsoc/org2/google/gsoc2014/tor Here are the facts: The summer of code gives you the opportunity to work on your own Tor-related coding project with one of the Tor developers as your mentor. You can apply for a coding project related to Tor itself or to one of its many supplemental projects... https://www.torproject.org/getinvolved/volunteer.html.en#Projects Your mentor will help you when you're stuck with your project and guide you in becoming part of the Tor community. Google pays you 5,000 USD for the three months of your project, so that you can focus on coding and don't have to worry about how to pay your bills. Did we catch your attention? These are your next steps: Go look at the Google Summer of Code FAQ [1] to make sure you are eligible to participate. Have a look at our ideas list [2] to see if one of those projects matches your interests. If there is no project on that list that you'd want to work on, read the documentation on our website [3] and make up your own project idea! Come to the tor-dev@ list [4] or #tor-dev on OFTC [5] and let us know about your project idea. Communication is essential to success in the summer of code, and we're unlikely to accept students we haven't heard from before reading their application. So really, come to the list or IRC channel and talk to us! Finally, write down your project idea using our template [6] and submit your application to Google before March 21 [7]. We are looking forward to discussing your project idea with you! [1] https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2… [2] https://www.torproject.org/about/gsoc.html.en#Ideas [3] https://www.torproject.org/docs/documentation.html.en#UpToSpeed [4] https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev/ [5] https://www.torproject.org/about/contact.html.en#irc [6] https://www.torproject.org/about/gsoc.html.en#Template [7] https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2…

1 0

Using MaxMind's GeoIP2 databases in tor, BridgeDB, metrics-*, Onionoo, etc.
by Karsten Loesing 26 Feb '14

26 Feb '14

Hi devs, you probably know that we use MaxMind's GeoIP database in various of our products (list may not be exhaustive): - tor: We ship little-t-tor with a geoip and a geoip6 file for clients to support excluding relays by country code and for relays to generate by-country statistics. - BridgeDB: I vaguely recall that the BridgeDB service uses GeoIP data to return only bridges that are not blocked in a user's country. Or maybe that was a feature yet to be implemented. - Onionoo: The Onionoo service uses MaxMind's city database to provide location information of relays. (It also uses MaxMind's ASN database to provide information on AS number and name.) - metrics-db: I'm planning to use GeoIP data to resolve bridge IP addresses to country codes in the bridge descriptor sanitizing process. - metrics-web: We have been using GeoIP data to provide statistics on relays by country. This is currently disabled because the implementation was eating too many resources, but I plan to put these statistics back. However, the GeoIP database that we currently use has a big shortcoming: it replaces valid country codes with A1 or A2 whenever MaxMind thinks that a relay is an "anonymizing proxy" or "satellite provider". That's why we currently repair their database by either automatically guessing what country code an A1 entry could have had [1, 2], or by manually looking it up in RIR delegation files [3, 4]. This is just a workaround. Also, I think BridgeDB doesn't repair its GeoIP database. Here's the good news: MaxMind now provides their databases in new formats which provide the A1/A2 information in *addition* to the correct country codes [5, 6]. We should switch! How do we switch? First option is to ship their binary database files and include their APIs [7] in our products. Looks there are APIs for C, Java, and Python, so all the languages we need for the tools listed above. Pros: we can kick out our parsing and lookup code. Cons: we need to check if their licenses are compatible, we have to kick out our parsing and lookup code and learn their APIs, and we add new dependencies. Another option is to write a new tool that parses their full databases and converts them into file formats we already support. (This would also allow us to provide a custom format with multiple database versions which would be pretty useful for metrics, see #6471.) Also, it looks like their license, Creative Commons Attribution-ShareAlike 3.0 Unported, allows converting their database to a different format. If we want to write such a tool, we have a few options: - We use their database specification [8] and write our own parser using a language of our choice (read: whoever writes it pretty much decides). We could skip the binary search tree part of their files and only process the contents. Whenever they change their format, we'll have to adapt. - We use their Python API [9] to build our parser, though it looks like that requires pip or easy_install and compiling their C API. I don't know enough about Python to assess what headaches that's going to cause. - We use their Java API [10] to build our parser, though we're probably forced to use Maven rather than Ant. I don't have much experience with Maven. Also, using Java probably makes me the default (and only) maintainer, which I'd want to avoid if possible. Thoughts? What other options did I miss, and what pros and cons that I overlooked? And is this something that people on this list would want to help with, once we agreed on one of the options? If so, please feel free to join the discussion now and maybe influence which path we're going to take. All the best, Karsten [1] https://gitweb.torproject.org/tor.git/blob/HEAD:/src/config/deanonymind.py [2] https://gitweb.torproject.org/onionoo.git/blob/HEAD:/geoip/deanonymind.py [3] https://gitweb.torproject.org/tor.git/blob/HEAD:/src/config/geoip-manual [4] https://gitweb.torproject.org/onionoo.git/blob/HEAD:/geoip/geoip-manual [5] http://dev.maxmind.com/geoip/geoip2/whats-new-in-geoip2/ [6] http://dev.maxmind.com/geoip/geoip2/geolite2/ [7] http://dev.maxmind.com/geoip/geoip2/downloadable/ [8] https://github.com/marklr/MaxMind-IPDB-perl/blob/master/docs/MaxMind-IPDB-s… [9] https://pypi.python.org/pypi/geoip2 [10] https://github.com/maxmind/MaxMind-DB-Reader-java

2 4

Tor Browser Weekly IRC meetings at 19:00 UTC Wednesdays
by Mike Perry 26 Feb '14

26 Feb '14

At the dev meting, we agreed to hold weekly TBB meetings in #tor-dev on IRC (irc.oftc.net) ideally either before or after the tor-core meetings. It looks like we're going to shoot for Wednesdays at 11:00am PST/19:00 UTC. If we can agree on this time, Nick has said that the tor-core meetings may be able to follow ours, pending approval from the rest of the Tor folks. For the meetings themselves, I think we should try the following format, inspired by Scrum, but less rigid. Everyone makes 0 or more statements on the following three topics, with time for responses from others: 1. What have you been working on last week that other TBB people should be aware of? 2. Will you be working on anything TBB related next week that should/could involve other people? 3. Do you need anything else from anyone else to help you make progress or to get something reviewed+merged? If so, what do you need? These meetings won't be used for progress metrics/evaluation, so don't feel the need to inflate things, explain already-solved snags in detail, discuss your various non-TBB/life distractions, or anything like that. The purpose here is strictly TBB coordination. In particular, if you've been working on stuff that either isn't TBB-related or is too premature to be worth discussing in detail yet, you can skip items 1 and 2 entirely, or make them extremely brief (ie "Worked on the updater"). Similarly, if you're blocked on stuff that doesn't require other people's help/input to solve, you can skip item 3. Don't feel obligated to tell us about issues that involving others won't really help. However, when discussion is necessary, we can and should deviate from the #1/#2/#3 pattern to discuss people's work/questions in some detail without plowing straight through to the next person immediately. After everyone has their turn to say what's they've been doing/what's needed, we can do a little bit of freeform discussion on whatever other issues seem pressing and who should be doing what next, discuss any future releases/deadlines, and discuss recent support issues if any support people are present. We will hard-stop at 20:00 UTC for the tor-core dev meeting to start (but hopefully will finish much earlier than that). If we end up consistently finishing early, we can make it later so as to run more fluidly into the tor-core meeting, for people who want to attend both. -- Mike Perry

2 2

GSoC Ideas
by Vighnesh Birodkar 26 Feb '14

26 Feb '14

Hello I am found a couple of ideas from the Ideas Page interesting . I was a GSoC student for SimpleCV last year. In the past I've programmed in C,C++,Java and Python . Following are my queries . 1. Search for Hidden Services . I apologize in advance if there is something obviously wrong with my idea. Dark Web consists of information that cannot be crawled because it doesn't appear as hyperlinks in other pages . But someone somewhere will always have access to this information, either by entering search queries , through subscriptions or logging in. What if we can index all the pages a browser visits ? Users can voluntarily install and enable or disable a plugin in their browsers . This plugin will index process ( and maybe index ) pages locally and upload it's data to servers which will hold the global index . 2. Develop a Censorship Analyzer Will this be a part of any existing tor projects ? What is a student required to do to be considered suitable for this ? Thanks Vighnesh

3 3

IRC meeting on Wed, Feb 26, 18:00 UTC to discuss next steps for Weather rewrite
by Karsten Loesing 25 Feb '14

25 Feb '14

Hi everyone, we're going to have an IRC meeting to discuss next steps for the Weather rewrite: Wednesday, February 26, at 18:00 UTC in #tor-dev on OFTC. http://www.timeanddate.com/worldclock/fixedtime.html?iso=20140226T18 Topics include reporting progress made since the last meeting two weeks ago, making plans for the next week, and possibly turning this project into a GSoC project. If people want to read up on what this is all about, here's the wiki page for the rewrite project: https://trac.torproject.org/projects/tor/wiki/doc/weather-in-2014 Talk to you on Wednesday! All the best, Karsten

1 0

Re: [tor-dev] Help hacking Mumble
by Matt 25 Feb '14

25 Feb '14

On Fri, 07 Feb 2014 09:54:35 +0400 meejah <meejah(a)meejah.ca> wrote: > What I would suggest: > > 1. change ConnectDialog so that it always uses not-yet-looked-up > QHostAddresses (perhaps only if a proxy is enabled). This still > won't help Tor, since they're sending UDP. > > 2. change Cert.cpp so it doesn't have to do a host-lookup. I sort-of > asked in my other email, but does this actually gain anything? > That is, does checking that the domain looks up "to something" really > accomplish anything useful? If the answer is "yes", then more > thinking; all I can think of is attempting a TCP/UDP connection > (again, with a not-yet-looked-up QHostAddress) to the server in > question (which will only work if there is a well-known port upon > which it should be listening to be considered valid). > I haven't actually confirmed that there is a DNS leak here. I've been focusing mostly on ConnectDialog, but mentioned Cert because it also had DNS-related code that I hadn't spent as much time with. > 3. suggest/document that Tor uses need to un-set the "show reachable" > (Settings::ShowReachable) option so that a server doesn't need a > valid ping-reply to show up in the list. > > Another option might be to factor out all the "ping" stuff to its own > class and simply not instatiate/use it when ShowReachable is off (or > have a separate option for pinging all the servers). > > Yet another different option might be to just have an option to turn > off the pinging of servers, leaving open the Cert.cpp question (which > one could also punt on by having an option). Then the game would be > to get Tor users to have the right options enabled ;) > I tried to do something like this in my branch; optionally turning off pings to Public and LAN servers was easy. Turning off pings to Favorite servers seemed a little trickier, especially if they are still to remain reachable. Thanks for all the pointers. A C++ developer volunteered to take a closer look at this issue. At this point, I think further discussion would most useful for all parties if it moved onto the bug ticket: https://github.com/mumble-voip/mumble/issues/1033

2 1

Weekly Tor dev meeting: Tuesday 25 Feb, 20:00 UTC
by Nick Mathewson 25 Feb '14

25 Feb '14

Hi, all! We're going to start trying to have a developer's meeting for working on the program "tor". (This won't cover all the other programs developed under the Tor umbrella.) My tentative plan is to do it on the #tor-dev IRC channel, unless the meeting turns out to be too disruptive for the rest of the channel or vice versa. We'll have a lot to do, and we won't get it all done: this will only go on for an hour or so. But we'll do another one next week. These meetings probably won't be very well run for a while. It will be 3pm EST and 12:00 noon PST. Unless I am mistaken, that's 20:00 UTC. but I've been travelling for a while, and adding five is hard. peace, -- Nick

5 4

Re: [tor-dev] GSOC Idea
by Devang Thakkar 25 Feb '14

25 Feb '14

Hi Damian, nice to know you. I wanted to suggest using Tor for web scraping, I mean, like creating a feature like Firebug/XPather are for Mozilla Firefox. That would easily help users to scrape easily and anonymously. Can this be considered as a valid idea for a GSOC project? Thanks, Devang

2 1

obfsproxy-0.2.6: Gentoo package
by George Kadianakis 24 Feb '14

24 Feb '14

Hey blueness, just wanted to say thanks for supporting obfsproxy-0.2.6 so quickly. I was about to send you an email, but I just noticed that obfsproxy-0.2.6 is already in portage. Just a small comment, the pkg_postinst() is a bit outdated since it references the old obfs2 transport (which we are starting to deprecate), and doesn't mention the new scramblesuit transport: http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/net-proxy/obfsproxy… Maybe you could steal the post-install hook from the Debian packages? You can find it here: https://packages.debian.org/sid/obfsproxy http://ftp.de.debian.org/debian/pool/main/o/obfsproxy/obfsproxy_0.2.6-2.deb… specifically in the file README.Debian . Thanks :)

1 0

Re: [tor-dev] obfsproxy dns transport
by George Kadianakis 24 Feb '14

24 Feb '14

> > > George, > > I'd like to write a dns transport... and it seems to me the > obfsproxy api isn't designed for non tcp transports... > Maybe we again make some changes to the obfsproxy api? > It would transport IP packets using a tun device... > we can route it to a socks endpoint and proxy from there. > > I think there is a whole class of obfuscated transports that could benefit > from an obfsproxy plugin interface for vpns... transports that use a tun > interface... > > I'll have to take a closer look at the obfsproxy api and think about this > some more. > I was just curious to see if you had any thoughts on the subject. > > There is this interesting code to learn about dns transport here: > https://code.google.com/p/dnscapy/ > it uses the scapy automatom api instead of twisted. Could easily be ported > to twisted I think. > > Feel free to forward your response to the tor-dev mailing list. > No hurry. No worry. > > Cheers! > > David > Hello David! I won't be able to answer thoroughly atm since I'm at the dev meeting. I will be back home soon and be able to answer more smoothly. In general, a DNS transport would be great! A well designed DNS transport would be a PITA to block and would also work in most networks (even through captive portals etc.). The short answer is that obfsproxy is indeed not designed to facilitate PTs like DNS transports. It will probably require a considerable refactor of obfsproxy to write a DNS pluggable transport. It will probably require so much refactoring that I'm not sure if it would make sense to merge the changes back upstream (because the changes might not be useful for other PTs). The main reason you want to use obfsproxy is so that you don't have to write your own SOCKS code, or your own Extended ORPort code, or your own PT-protocol code. That said, here are two other approaches you can take: a) You can "fork" obfsproxy, perform your changes, and then we can deploy your fork as a standalone PT. Since obfsproxy's architecture is designed to support multiple PTs you might have to perform considerable changes to obfsproxy; but that's fine, since you will be the only one using that fork of obfsproxy. A small problem with this approach is that we will have to deploy two obfsproxy codebases which will increase the size of the bundle. The good news is that the size of the obfsproxy codebase in the latest TBBs prepared by David is only 350Kb which is not too much even if doubled. b) You can write your own Python code and selectively steal code from obfsproxy. You will have to steal the code that does networking, Extended ORPort, environment variable parsing (pyptlib), SOCKS, etc. This might not be too hard. Alternatively you can use Go, and use David's goptlib library [0] which does both Extended ORPort and SOCKS. Also, before you start writing code, it might be worth looking at the pt-spec and seeing if it can support DNS PTs as it is. BTW, do you know of any censorship circumvention tools that use DNS as a covert channel? [0]: https://gitweb.torproject.org/pluggable-transports/goptlib.git

4 8