- tor-dev - lists.torproject.org

[GSoC '16] Exitmap project - Introduction and request for comments
by Mridul Malpotra 18 Mar '16

18 Mar '16

Hi everyone! I'm Mridul. I wish to apply for the Exitmap improvements project mentored by Dr. Philipp Winter for the Google Summer of Code 2016. My current IRC handle is mtyamantau. Contents -------- 1. Introduction - About myself and experience with Tor 2. Exitmap - Current progress and questions 3. GSoC - Rough proposal structure and questions 1. Introduction - About myself and experience with Tor ------------------------------------------------------ I'm Mridul Malpotra, currently in my senior year pursuing bachelors in Computer Science from IIIT Delhi, India. My interests primarily lie in computer networks and network security, specifically anonymous networks like Tor and I2P. Through my now 1-year long undergraduate thesis work under Dr. Sambuddho Chakravarty, I have had exposure to the Tor network, relevant literature and some related projects, which helped me better understand and appreciate the current research and development going on. My work involved manually setting up testbeds through testing Tor networks on our institute intranet as well as on PlanetLab (for those wondering, I had recommended Chutney and Shadow). The current private testing Tor network is running on a PlanetLab slice (iiitd_mridul2) with ~170 nodes globally and 3 directory authorities. I used the control protocol through the Stem library to help in multiple circuit creation and stream attachments for measuring performance of a software over Tor. I have also had experience with open source software, by contributing to the Non intrusive load-monitoring toolkit (NILMTK) which is based on Python and Pandas. While working there, I helped contribute code for additional features, fixed a few bugs and also worked with a few of Python's package management and documentation systems. Relevant links: github.com/nilmtk/nilmtk/commits?author=mridulmalpotra 2. Exitmap - Current progress and questions ------------------------------------------- I recently read about Exitmap in the 'Differential Treatment of Anonymous Users' paper by Khattak et. al. The use case for fast automated scanning through Exitmap to evaluate ~1000 exit nodes was really interesting. On top of that, it fitted my use case of testing a particular software's performance over Tor. Familiarizing myself with the source code, I think I understand the basic layout for how the scanner works and appreciate the modularity of task executions. I followed the project's progress on github and have read the 'Spoiled Onions' paper by Winter et. al. In the coming 2 days, I plan to tinker around more with the code, discuss concerns, issues and/or suggestions if any, and get myself properly familiarized with the codebase. I also have certain ideas regarding what modules could be added and improvements made, some of which I have mentioned in the next section. I will also be reading the tech report on Exitmap and would be grateful if you can recommend any other resource(s) that I should be referring to. Lastly, I had a few queries related to the project and/or paper and apologize for the naivety in the questions if any. a. How was the bifurcation between stand-alone and same-process modules decided? Are there any advantages to allow for multiple forked processes for specific modules? b. For testing active attacks, can there be modules developed keeping other cleartext protocols like SNMP and Telnet in mind? Alternatively, is there a way to determine what protocols are being used over Tor and their popularity? c. How is Exitmap being crowdsourced currently? I'm interested to know how data is being collected from volunteers running the scanner. 3. GSoC - Rough proposal structure and questions ------------------------------------------------ Here I am listing the possible objectives that my project will be focusing on. I request your feedback and comments on the chosen topics and their descriptions. 1. Achieve autonomous scanning in Exitmap with periodic scans that, based on a certain algorithm, fetches relay descriptors and automates various subtasks for consistent data collection and verification. The main challenges that I expect will be intelligently recognizing which tasks to automate and when, and making the entire background process execution efficient in resource consumption. 2. Emulating multiple user interaction in individual modules and in Exitmap overall to provide indistinguishability to Exitmap from regular users. I will try to explore libraries for this purpose like Splinter with Selenium or BeautifulSoup with Requests that help dynamically interaction with the web resource. The main challenges that I expect will be to scale this automated testing alongside the running asynchronous jobs and making the entire scans look like genuine user interactions. Any suggestions on better ways to do this will be helpful. 3. Making the codebase more robust by adding unit test cases. I plan on using either the plain unittest/unittest2 framework or nose/nose2/pytest tools or any other alternatives that I may find or be recommended. I plan to simultaneously write the unit test cases for new code added and improve upon the exiting testing programs. 4. (Optional) I read from the mail threads on the tor-dev mailing list that the code needs to be converted to be Python3 compatible. Would like your opinion on whether it is a viable option and if it is possible, would like to include this in my list of tasks. 5. (Optional) If I can spare time in the milestone timeline and if discussion leads to some clarity, I would like to add another module for more cleartext protocols that could be implemented like SNMP or Telnet. I am also looking at possible local to remote attacks that are active at the application layer and could be tested in Exitmap. I'll update if I find anything. Next, I am drafting a week-wise timeline of what I plan to do over the duration of ~3 months. I will be dividing task 1 and 2 before and after the mid-term, with the optional tasks done in either of the slots and testing done alongside for everything. This is to ensure incremental milestones that can be useful to the community as I develop. Also, as I gain clarity on my objectives, I plan to refine this timeline to a week-wise format with buffers placed accordingly and present it in 2 days time. Lastly, I would appreciate any suggestions, criticism or feedback on this proposal regarding content, volume or specificities. I am looking forward to contributing to the Tor Project organization and interacting with developers here. Thanking you, (long mail I know, sorry) Sincerely, -- Mridul Malpotra Undergrad @ IIIT-Delhi <https://www.iiitd.ac.in/> mridulmalpotra.github.io ============================= PGP keyID: 0xb716e33ab6d0a653 <http://pgp.mit.edu/pks/lookup?op=get&search=0xb716e33ab6d0a653>

3 2

"Not our bug" bugs
by Griffin Boyce 18 Mar '16

18 Mar '16

Hey all, There have been quite a few bug reports that discuss incompatibility with various Firefox extensions and with websites. In most cases, I can't replicate these bugs -- either because the extension in question has been patched, the website reported no longer exists, or the issue can't be replicated (which could be due to site updates and past Firefox incompatibility). Occasionally, the issue is real and still in effect, but isn't really a Tor bug (such as #7279, where a forum restricts logins by Tor users). We've all worked very hard to reduce overly-restrictive blacklist policies, but can't be everything for everyone. In these cases, I'd propose rejecting these bugs as either invalid or `not a bug`. These are all varying degrees of "not our bug" or "actually not a bug at all." Open to more thoughts on this. ~Griffin

2 1

Notes from the prop267 meeting 2016-03-17
by Linus Nordberg 18 Mar '16

18 Mar '16

Hi, Here's a summary of what happened in the prop267 meeting in #tor-dev yesterday with Nick, Tom, Sebastian and myself. Sebastian started by summarising the proposal nicely. Topics discussed: - for bootstrap reasons we can't mandate using tor for all communication - SCT's or not: using SCT's instead of inclusion proofs would lower the burden on operating a log and save bytes on the wire - pushback on suggestion of turning all relays into auditors, reason being increased (code) complexity - there are some differences between "MITM:ing tor" and "MITM:ing on the internet", one being that a tor mitm is the controlling of a majority of the dirauth keys, which in many cases is more persistent than an IP layer MITM - the "eventually you escape a MITM" hand waving is problematic - we need a story on how a perfectly tor-MITM:ed TAILS user get back to the real network - relays and clients must check consensus documents the same way - hard fail or not when consensus isn't shown to be in a log? - handling of changes in the set of dirauths Identified next steps: - add text about what needs to be and what should be anonymised, taking bootstrapping issues into account - decide on replacing SHA2-256 with SHA3-512 or SHA2-512||SHA3-512 (taking availability of implementations into account) - decide on using SCT's or not - analyse the "TAILS user under MITM" case - analyse "set of dirauths changing" What did I miss? Full logs can be found at http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-03-17-14.58.log.html

1 0

What does TorBulkExitList.py return?
by Linus Nordberg 17 Mar '16

17 Mar '16

Hi, I'm trying to figure out why a list from [TorBulkExitList.py] is so much larger than what is seen in [exit-addresses]. Point in case: Earlier today the list from TorBulkExitList.py contained 58% more addresses than exit-addresses: --8<---------------cut here---------------start------------->8--- $ curl-tor -q 'https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=8.8.8.8' | egrep -v ^# | wc -l 1554 $ curl-tor -q https://check.torproject.org/exit-addresses | egrep ^ExitNode | wc -l 985 --8<---------------cut here---------------end--------------->8--- Counting relays with the Exit flag in a consensus of roughly the same time gives 912: --8<---------------cut here---------------start------------->8--- $ curl-tor -q http://171.25.193.9:443/tor/status-vote/current/consensus | egrep '^s.*Exit' | wc -l 912 --8<---------------cut here---------------end--------------->8--- If I read the code [check.py][exitips.py] correctly, we include the following IP addresses in what is served from [TorBulkExitList.py]: - all routers in the current consensus - with exit addresses from TorDNSEL file(s) added - for which stem's exit_policy.is_exiting_allowed() returns true I haven't read any TorDNSEL code and don't know exactly what above mentioned TorDNSEL file(s) are but I think they include results from active testing of what address exit relays really use for exiting. [exit-addresses] seems to be from TorDNSEL too but holding more of a snapshot while the files mentioned above supposedly cover some period of time. Before I dig further, perhaps someone here already know why the numbers differ so? I wouldn't expect the churn of addresses used for exiting to be high enough to explain the difference. [TorBulkExitList.py] https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=8.8.8.8 [exit-addresses] https://check.torproject.org/exit-addresses [check.go] https://gitweb.torproject.org/check.git/tree/check.go [exitips.py] https://gitweb.torproject.org/check.git/tree/scripts/exitips.py Thanks, Linus

2 1

Revisiting prop224 cells
by George Kadianakis 16 Mar '16

16 Mar '16

Hello, I took a look at proposal 224 again, with the aim of revisiting the cell logic and format. Here are some matters that require discussion: 1) Should we keep backwards compability with old introduction and rendezvous points? Currently, proposal 224 actually tries to maintain backwards compatbility but at the cost of complicating the design. Specifically, to achieve backwards compatibility we need at least the following functionality: - Hidden services need to generate and publish an extra encryption key for each legacy introduction point (legacy enc-key). - Hidden services need a new cell subtype to register themselves to legacy introduction points (LEGACY_EST_INTRO). - Clients need a new cell to introduce themselves through legacy introduction points (LEGACY-INTRODUCE1) The above features are not extremely hard to implement, but because of their hacky backwards-compatible nature they do complicate the protocol and the code. Also, at some point when the network has upgraded we will have to rip this code off our codebase, otherwise it will just rot there. Also also, we will have to write non-trivial chutney tests to ensure the correctness of the backwards compatibility logic. Alternatively, we could choose to completely drop backwards compatibility with old introduction and rendezvous points. This means, that we will have to wait till a good part of the network has upgraded before we enable prop224 support for clients and services. This means that we will probably have to wait a whole release cycle (till the relay-side prop224 code becomes stable; is that 6 months?) before we can globally enable the client/HS prop224 functionality (although we can use the network ourselves for testing prop224 till then). What do you people think we should do here? Paradoxically, I'm currently thinking of _keeping_ the backwards compatibility design. Looking at the spec it seems like a medium difficulty engineering issue for us (maybe an 8% of the total prop224 task size), which sucks, but at least we don't have to worry about doing proper incremental deployment of prop224 on the network and worrying about release cycles. Also, as we move towards implementing prop224 cells, we can reevaluate our position here. I'm not confident about my position here, so feedback would be helpful. 2) I'd like to simplify the ESTABLISH_INTRO logic. Currently, ESTABLISH_INTRO seems like a needlessly _complex_ cell that is also _incomplete_. It's _complex_ because it takes 3 different forms depending on the value of its first byte. This complexity is caused partially by our backwards compability needs (see above), but also because we tried to cram the MAINT_INTRO message into this cell. It's _incomplete_ because it does not actually contain the "introduction point encryption key", so hidden services are forced to send the encryption key right after the initial ESTABLISH_INTRO cell using a second ESTABLISH_INTRO cell that is actually a MAINT_INTRO/UPDATE-KEYS-SUBCMD message. I have two suggestions here: * Let's include the intro point encryption key in the ESTABLISH_INTRO cell, so that hidden services can establish intro with a single cell (not for performance, but for simplicity). * Let's introduce a new cell type for MAINT_INTRO instead of cramming it into ESTABLISH_INTRO. Or at least, let's make it an extension of ESTABLISH_INTRO instead of using the first byte of the cell to get the cell subtype. What do you think? Also, this brings me to the next topic which is: 3) What is UPDATE-KEYS-SUBCMD good for? And why do intro points need to know the intro point encryption key? UPDATE-KEYS-SUBCMD seems to be the only use of MAINT_INTRO currently. It seems to be able to update the encryption keys of an introduction point circuit on the fly. But why does the introduction point need to know the encryption key in the first place? That key is only used by clients and hidden services to encrypt stuff end-to-end to each other. After discussing with dgoulet, the only reason I can think of is that so that the IP is aware of the encryption key, and if an incoming client Alice does not know the correct encryption key, then the IP can send it to her using an INTRODUCE_ACK message with [00 02] (and then Alice does not need to refetch the descriptor). But why would a client know the authentication key but not the encryption key? Do they have different rotation times? Why would the encryption key rotate before the authentication key? Maybe all these things are not necessary for now and we can just ditch UPDATE-KEYS-SUBCMD completely, assuming that both of those keys have the same rotation lifetime? And maybe even the IP does not need to know the encryption key at all? Am I missing something? Cheers!

3 2

Notes from network team meeting Wed 3/16/2016
by Isabela 16 Mar '16

16 Mar '16

Hello there, this is something new we are trying out. The goal is to improve communication between team members, since we are spread out through many different time zones and is hard to get everyone together at the same time for discussions and updates. == Notes from little t Tor team - Wed March 16th 13:30 UTC @ #tor-dev == Participants + updates: * nickm - trying to get 028 done, getting ready to help with 029 triage, fixing that old 027-authorities bug, and trying to make sure that all the sponsorS EOM items get done, special focus on module isolation right now * isabela - sent a few emails with exercise to start triaging for 029 / working on a proposal for our release process * dgoulet - working on prop#224 (#17238, dev branch ticket17238_029_01) / reviewed 028 tickets and cleaned up the "tor-hs" ticket queue also / plan on triaging for 029 by Friday then PlanetLibre * asn - worked mostly on prop224 - wrote a thread about design issues (https://lists.torproject.org/pipermail/tor-dev/2016-March/010534.html) / working on GSoC preparations and review queue is empty so can help review code if needed * special - nothing much to report, been busy with travel and auxiliary responsibilities :< * Sebastian - helped implement 'sponsor' field on Trac / has ~8hrs next week to work on something that needs help * karsten - tried to find 028 tickets that need review, found one that's actually mine and that needed work, worked on that (#18460) == Discussion topics and action items: Triage 029 - first triage exercise is to mark sponsor tickets as: * 'sponsorY-must' means "We have promised this to sponsorY." * 'sponsorY-can' means "We can bill this to sponsorY, but we haven't promised this specifically." We will be looking into sponsor S, U and R tickets. Isabela sent a spreadsheet via email for people to use and mark the tickets. We should get it done by Friday EOM so we can move on with another triage exercise. If you have any questions, please reach out to isabela via irc or email. Meeting time discussion: Is impossible to have something that will work perfectly for everyone. We discussed a few possible solutions that Nick will compile and email the list to continue the discussion with other members of the team. We also decided on sharing meeting notes beyond the meet bot log. Triagging 'must-fix-before-028-rc': During this discussion we also ended up deciding on creating a new state for the review process 'need-merge', so when a ticket is done with the review process goes to this state. This is the equivalent to 'needs-nick-review' state we discussed during the Dev Meeting discussion. Some of the action items from the discussion are listed below. Mostly for folks to review tickets so we can close them and move one with 028-rc. Action Items ------------ * everybody look at https://lists.torproject.org/pipermail/tor-dev/2016-March/010534.html * nickm and dgoulet see if they have time to hang out when dgoulet is around for libreplanet * isabela to write her thoughts on release-engineering this stuff, and making broken things easier to revert * somebody open a ticket for a network-team mailing list : #18563 [DONE] * nickm will send an email about meeting times. * if you think something is in must-fix-before-028-rc and we can let it slip, please comment on the ticket. [people worked on it after the meeting] * dgoulet, Sebastian, teor: please take another look at the ed25519 patch and see if I fixed the issues in #18318 and #17668 ? * let's see if we can get weasel (or somebody else) to review or test my fix for #18253 * nickm asks athena for ETA on bugfix. == Next meeting and other updates: * I guess next meeting will depend on the results of the meeting time poll - we will keep the list updated. * We are having a proposal discussion meeting tomorrow, Thursday March 17th at 1500UTC - prop#267 * We had a proposal discussion meeting today as well, Wednesday March 16yh, on prop#224. You can check the log here: http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-03-16-15.03.html -- PM at TorProject.org gpg fingerprint = 8F2A F9B6 D4A1 4D03 FDF1 B298 3224 4994 1506 4C7B @isa

1 0

Come work on Tor at Freedom of the Press Foundation!
by Garrett Robinson 16 Mar '16

16 Mar '16

Hello tor-devizens, Freedom of the Press Foundation is hosting a Mozilla-Ford Open Web Fellow for 2016. We have a lot of potential projects for our fellow to work on, depending on their skill set, but we have our fingers crossed for an applicant who is interested in working on Tor! SecureDrop, our flagship project, depends heavily on Tor and Tor Hidden Services to protect the communications of journalists and anonymous sources. If we end up accepting such an applicant, we would like to have them work with the Tor Project on issues that are pertinent to SecureDrop, particularly Next Generation Hidden Services and the proposed padding defenses for website fingerprinting attacks. The deadline for applications is *March 20th* (this Sunday). More info on our blog: https://freedom.press/blog/2016/01/apply-be-freedom-press-foundations-open-… Apply: https://advocacy.mozilla.org/open-web-fellows/ Let me know if you have any questions! You can reply to this email on the list, or email me directly. Cheers, Garrett

1 0

Interested in GSoC opportunity
by Deepankar Tyagi 16 Mar '16

16 Mar '16

Hi everyone, I am a computer science student at undergrad level, would love to contribute to Tor, currently in the process of drafting the proposal. This mail is an effort to seek guidance to define scope, goals and timeline more clearly. Currently interested in two ideas and plan to submit only one of these based on feedback and guidance from mentors. idea #1 Extending and Improving Krotor. This is continuation of my last year GSoC project in which libevent and Tor were ported to NaCl (a C/C++ sandbox environment on chrome), full detail can be seen at https://deepankar.io/gsoc and https://deepankar.io/krotor. It was later published on webstore and currently has close to 13k active users. A lot of improvement can be made to this project, viz supporting obfuscation, making it more user friendly, adding support for configuring exit gates etc, updating it to latest stable release. I would greatly appreciate any guidance on various matters for instance: making it more resistant to any privacy leaks such as dns leaks on webrtc and on windows due to no global policy of dns etc. I would be very grateful if anyone is willing to mentor for this. idea #2 Exitmap Improvements I got stuck when defining timeline for this project. I would appreciate any insights regarding what should be deliverable by mid-term of GSoC. Also this project will a involve lot of brainstorming, preferably with mentor, is there a possibility of weekly/every two weeks session for an hour or half hour with mentor on hangout/skype etc (at least in the beginning) What would be preferred means to get/see progress reports and preferred frequency of the same. Some hyperlinks: Github <https://github.com/codebuff> || LinkedIn <https://www.linkedin.com/in/codebuff> || KroTor <https://deepankar.io/krotor> Best Regards. *Deepankar Tyagi* *Projects & Resume : deepankar.io <https://deepankar.io>*

3 7

Prop259 simulator and results
by George Kadianakis 16 Mar '16

16 Mar '16

Hello there, I'm not sure what kind of statistics we get out of the current guard simulator. In general, we are interested in security and performance. For security we are trying to minimize our exposure to network. For performance, we want to minimize our downtime when our current guard becomes unreachable or after our network comes back up. Here are some concrete statistics that we could gather in the simulator: Security statistics: - Number of unique guards we connected to during the course of the simulation. - Time spent connected to lower priority guards while a primary guard was online. - Time spent connected to lower priority guards while a higher priority guard was online and the network was up. Performance statistics: - Time spent cycling through guards. - Time spent cycling through guards while network is up. - Time spent on dystopic mode. - Time spent on dystopic mode while the network was utopic. Is it possible to collect those statistics? I'm curious to learn how the current guard algorithm compares to the new prop259 on those aspects. What other stats are important here you think?

2 7

GSOC: Panopticlick
by - - 15 Mar '16

15 Mar '16

Hello! My name is Akito Ono. I'm a computer science student in Japan and very intrested in participating in GSoC this year. I read docs about projects and I was particularly intrested in Panopticlick project. And I have some questions about Panopticlick. Is this project based on Libre-Panopticlick( https://github.com/qqTYXn7/browserprint) or building from scratch? Is programming language used in project designated? Best regards!

4 5