- tor-dev - lists.torproject.org

Big performance improvement for meek-azure
by David Fifield 12 Apr '15

12 Apr '15

I found and fixed a big performance limitation in the meek-azure transport. If you tried meek-azure before, but it was too slow, give it another try! meek-azure has always seemed slower than the other two backends, and I recently spent the time to figure out why. It was a lack of persistent HTTP connections between the Azure host and the Tor bridge. Every single HTTP request was doing a complete TCP and TLS handshake. This of course increased latency, but the bottleneck was actually CPU on the bridge, as it tried to cope with all those TLS handshakes. Here is the commit that causes connections to be shared and reused: https://gitweb.torproject.org/pluggable-transports/meek.git/commit/?id=e06b… This issue did not affect meek-google because the App Engine URL fetch API reuses HTTP connections transparently. It did not affect meek-amazon because the CloudFront CDN does its own connection reuse. Usage on meek-azure has already increased in the past few days since I started testing the change. I attached a graph that shows the all-time bandwidth history of the bridge. https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… David Fifield

1 0

Re: [tor-dev] Draft of proposal "Direct Onion Services: Fast-but-not-hidden services"
by str4d 11 Apr '15

11 Apr '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 George Kadianakis wrote: > David Goulet <dgoulet(a)ev0ke.net> writes: > >> On 09 Apr (21:58:49), George Kadianakis wrote: >>> Hello, >>> >>> I inline a proposal for Direct Onion Services. It's heavily >>> based on the "encrypted services" proposal of Roger, but I >>> refined it, made it more proposaley and enabled a few more >>> optimizations. You can find the original piece here: >>> https://gitweb.torproject.org/torspec.git/tree/proposals/ideas/ >>> >>> xxx-encrypted-services.txt >>> >>> >>> >>> Feedback would be greatly appreciated. >> >> First thanks for that! Really useful stuff. >> >>> >>> Here are some specific points that I would like help with: >>> >>> - Should we require a specially compiled version of Tor to >>> enable this feature? Like we do for tor2web mode? In this >>> proposal, I didn't require any special compilation flags. I >>> don't have strong opinions here. >> >> IMO, I'm not sure this is useful. This feature requires prior >> knowledge of the HS operator to know what's a "Direct Onion >> Service" and decide that she wants that for her service. Thus, a >> torrc option seems enough. I don't see any reasons for this >> option that will limit deployment. >> > > Yes, I also feel the same. > > My main fear is sample torrc's that are being posted around the > net and people just copy-paste them into their box: someone might > not notice the DirectOnionServiceEnable option. Perhaps this could be mitigated by alerting the user the first time Tor is started with this option enabled. This is obviously harder to do if Tor is started as a service, but it should be possible for manually-started Tor or TBB. >> >> [snip] >> >>> >>> 3.2. One-hop circuits [ONEHOP] >>> >>> When in this mode, the onion service establishes 1-hop circuits >>> to introduction and rendezvous points. The onion service SHOULD >>> also ignore cannibalizable 3-hop circuits when creating such >>> circuits. >>> >>> This looks like this for the rendezvous point case: >>> >>> |direct onion service| -> RP <- client middle <- client guard >>> <- |client| >>> >>> This change allows greater speeds when establishing a hidden >>> service circuit and reduces round-trip times. As a side-effect, >>> busy onion services with this feature cause less damage to the >>> rest of the network, since their traffic gets passed around >>> less. >> >> Would it be possible to drop the rendezvous part where the client >> could simply connect to the IP and then the circuit back to the >> HS would be used? (Though that would require that the client >> knows the HS it's trying to reach is a Direct Onion Service, >> could be mention in the descriptor?...) >> > > I think that's indeed possible, but would require multiplexing > multiple client connections on a single IP circuit, which Tor > cannot do at the moment AFAIK. It is my understanding that it's not > an easy change, but I'm pretty sure that I2P and other projects are > doing it, so we could in theory take a look at how those guys do > it. I2P does have the ability to run "zero-hop" tunnels, yes - the router simply provides its own details for the tunnel entry point. But this probably isn't directly helpful for Tor because I2P is a packet-switched network, and it doesn't use separate rendezvous points for individual connections between Destinations. For high-traffic services that don't require anonymity, we generally recommend using one-hop tunnels instead of zero-hop. This is primarily to avoid connection limits being exceeded on the hosting router. The speed hit of having an additional hop in the path is generally outweighed by the benefit of only having e.g. 8 incoming connections for a particular service, instead of several thousand. It therefore also helps to mitigate DDoS attacks (although the biggest of services e.g. Facebook would likely have their own systems that would better handle this). str4d > _______________________________________________ tor-dev mailing > list tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVKGTIAAoJEBO17ljAn7PgYg8P/1gn8BleY9GyuceCXCvVicwk ROWeEahM/ujSzpJXqjTJMBjuBcXocan/RHbeTyrfjCGZhj4p/eUMKnXpMBpJcuKp oKEDXP3n14tnUxTnUT7vRl4ZYGmCZAiIKUqeqwAz2/1LxKYK+IDsxaurnbOA5uXV vuL2Q8WxbnhExEr+lvRX4zV2/5hGK7jj5R1Hz35qxZsJTjPUu3mRXxS5aiExUfzS kjxUA5uevKdE6/K0nQ4hJ7rY3ZkfcD8YIsW7CQsC0FfK8DXMaPew+tvUsUuegeDt nbIFRx8oGItFwDDwZIUWZED2g60YhrSc7Ig192amXYe42GtoPQC3n6ul0GqNM2cb m5AJAw5GGlxluprAwu3AcVl3/kYCLqN50bK/Lz3f2bBcYuUvcbNJ5GK6F4W/ye45 Pl2QmiiaGuZM5CKWNY4/kHc3sJaPfnFvbxdocqcIok3/AlnhVotQVDTl7bFklYOP qJ58oNJAU2UHY+uOX6c+8+d2G062AbWxo4iFlXOGguNSKnR+3w0fE+P+lMQjtAGj cE/7zj/DcFrj9TpVoZBAiDrEJfkWxywASoHhe1Y/eayzDg1D/jqfNNJ9/0YnPWPd tC/pnHPQ1/gc9mqIJzmW1c9tKdwzys1UI9UGNjdggVn80dnXYyE5xAwx5Ymh11NT VzHWkgab7+Shrp4dQL3a =B87T -----END PGP SIGNATURE-----

1 0

Next PT/Bridges Meeting: Wednesday 15 April 0200 UTC
by isis 10 Apr '15

10 Apr '15

Hello everyone, As per the votes for the new PT/Bridges meeting time, [0] we've settled on biweekly meetings in ircs://irc.oftc.net#tor-dev, on Wednesdays, at 0200 UTC. Please let me know if this is a severe inconvenience to anyone. The next meeting will be on Wed., 15 April, at 0200 UTC, which currently translates to the following (ridiculous, antediluvian, and nationalistic) timezones: - Tue., 14 April, 1900 PDT - Tue., 14 April, 2200 EDT - Wed., 15 April, 0400 CEST - Wed., 15 April, 0500 MSK - Wed., 15 April, 1100 JT See you there! [0]: https://lists.torproject.org/pipermail/tor-dev/2015-April/008601.html -- ♥Ⓐ isis agora lovecruft _________________________________________________________ OpenPGP: 4096R/0A6A58A14B5946ABDE18E207A3ADB67A2CDB8B35 Current Keys: https://blog.patternsinthevoid.net/isis.txt

1 0

Fwd: [guardian-dev] Orbot v15 RC 2
by Nathan Freitas 09 Apr '15

09 Apr '15

----- Original message ----- From: Nathan of Guardian <nathan(a)guardianproject.info> To: guardian-dev(a)lists.mayfirst.org Subject: [guardian-dev] Orbot v15 RC 2 Date: Thu, 09 Apr 2015 13:24:46 -0400 Orbot v15 RC2 is out addressing some critical and *odd* issues with copy and paste of bridges, VPN refresh on network switching, dumb bug fixes related to root and transproxy settings, and one or two other odds and ends. WARNING: If you are using the "email for bridges" feature on Android, and you copy and paste the bridge lines into the Bridges setting box, you might not notice that Android somehow removes some spaces in the string. Make sure to put a space back in between the server:port and the key portion of the string. If you scan the QR codes from bridges.torproject.org this shouldn't happen. APK: https://guardianproject.info/releases/Orbot-v15.0.0-RC-2.apk SIG: https://guardianproject.info/releases/Orbot-v15.0.0-RC-2.apk.asc Source tag: https://gitweb.torproject.org/orbot.git/tag/?id=15.0.0-RC-2 /** 15.0.0 RC 2 / 9-Apr-2015 / 60f19ca **/ d6c51bc Fixes for bridge setup, and root/shell interaction - If you paste bridge addresses from Gmail, you get some strange characters c39cdcb improve root access check for transproxy 7d8eea2 switch back to DNS on 10.0.0.1, update after VPN refresh 690a8c3 Improved handling of VPN and Tun2Socks on Network Switch 9974654 fix for setting root and transproxy preferences /** 15.0.0 RC 1 / 8-Apr-2015 / 280f69dfa10c38d880e98e71954526c68f1b3df5 **/ 776b7af use loopback address e6fe252 auto-restart Tor when config changes 384fe1c fix handling of network connectivity state management 02a42e4 update translated strings 628c9d8 update tor to 0.2.6.7 906ec7f v15-beta-2 small fixes for VPN d6eb1dc fixes for network switching with VPN enabled f37b935 modifications to bridge setup strings -- Nathan of Guardian nathan(a)guardianproject.info _______________________________________________ List info: https://lists.mayfirst.org/mailman/listinfo/guardian-dev To unsubscribe, email: guardian-dev-unsubscribe(a)lists.mayfirst.org

1 0

Re: [tor-dev] Tor Summer of Privacy [ Incorporating ruleset testing into https-everywhere release process ]
by yan 09 Apr '15

09 Apr '15

Oops, sending again from my torproject account. On 4/9/15 10:24 AM, yan wrote: > Hi Karan, that is great to hear. I've cc'ed Jacob, one of the current > maintainers of HTTPS Everywhere. > > -Yan > > On 4/8/15 11:40 PM, karan grover wrote: >> Hello there! >> I am Karan Grover. I am a 2nd year computer science student at IIIT >> Delhi. >> >> I am a part of my college's Cybersecurity Education and Research Center, >> which is the first of its kind in the country. I have worked primarily >> in the field of Privacy and Security of Online Social Media (PSOSM). >> >> I am interested in contributing to Tor as a part of Tor Summer of >> Privacy. The project I am interested in working on is *Incorporating >> ruleset testing into https-everywhere release process*. I have been >> understanding the codebase of *https-everywhere-checker, *and am now >> quite familiar with it. I would like to discuss further about what the >> project entails. >> >> I contribute to open source every now and then, but am now looking to be >> an active open source contributor and Tor seems to align with my >> interests. >> >> Looking forward to your reply. >> >> Thank you. >> >> >> >> _______________________________________________ >> tor-dev mailing list >> tor-dev(a)lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >>

1 0

Brainstorming ideas for controller features for improved testing; want feedback
by Nick Mathewson 09 Apr '15

09 Apr '15

Hi! I've got an end-of-month deliverable to flesh out as many good ideas here as I can, and I'd appreciate feedback on what kind of features it would be good to add to the controller protocol in order to better support testing. More ideas would be most welcome. Yes, some of these ideas are probably foolish or pointless or half-baked or useless or even dangerous; this is a brainstorming exercise, not a declaration of intent. The goal right now is to generate a lot of ideas and thoughts now, and to make decisions about what to build later. IDEAS ===== 1. Step-by-step hidden service connections Add the ability to create connections to hidden services step by step, to best What's necessary here is commands to: * Establish a rendezvous point on a given circuit. * Construct and send an introduce2 cell on a given circuit. * Realize that a rendezvous circuit has been constructed. 2. Send a single cell on a circuit (TESTING ONLY) For fuzzing and low-level testing purposes, it would be handy to be able to send a single cell on a tor circuit. This might be better to expose via a low-level modular API than via the control port. 3. Intercept cell by cell on a circuit (TESTING ONLY) For fuzzing, testing, and debugging purposes, it might be handy for a controller to be able to observe data cell by cell on a circuit of interest. This might be better to expose via a low-level modular API than via the control port. 4. Send a single cell on a connection. (TESTING ONLY) As 2, but for connections. Note that we might even, for testing, expose this at a sub-cell level. 5. Intercept all cells on a connection (TESTING ONLY) As 2, but for connections. 6. Plug-in to handle a relay or other command. Right now, all Tor's features need to be baked into Tor; it's not easy to write extensions. We could change that by having the controller able to intersect particular relay or extension commands and act accordingly. This could be used for prototyping new features, etc. 7. Force a given protocol on a given connection We could add a feature to restrict what protocols can be negotiated on a given connection we create. This could help us better test our protocols for interoperatbility. 8. Examine fine-grained connection detail. There are many data available for a given connection (such as fine-grained TLS information) that are not currently exposed on the GETINFO interface. We could make most of this available for testing, pending security analysis. 9. Examine cache in detail In the past we've seen crazy issues with our descriptor caching code. It might be good to expose for testing information about where exactly descriptors are stored, what attributes are set on them, and so on. We could also expose events for cache compaction and discarded expired descriptors. 10. Fetch literal documents Currently there's no way for a controller to ask Tor to download a given descriptor or microdescriptor or networkstatus. That could change. 11. OOM stats To resist out-of-memory attacks, we track our memory usage and kill off circuits as needed when memory gets low. We could expose the memory thresholds and current sizes via one or more controller commands. 12. Timeout values Tor has a truly huge variety of internal timers to ensure that given periodic events happen enough; we could expose those, and (TESTING ONLY) allow controllers to adjust them or trigger their corresponding events. 13. Detailed connection debugging info Current connection events expose only large-scale state changes in connections; we could instead expose every state transition at the cell-by-cell handshake level. 14. Detailed circuit debugging info As 13 but for circuits. 15. Halt main loop except for control layer. (TESTING ONLY) For inspection/debugging purposes, it might be clever to have Tor be able to freeze itself, except for the control layer, and let the controller inquire about information. This presents implementation challenges, and is probably not a great idea to do before a _big_ refactoring. 16. Service a single connection (TESTING ONLY) Currently controllers can disable circuit construction or stream attachment, and do them manually. We might also do this for connections, allowing a testing controller to trace what Tor does cell by cell on a single connection. 17. All rephist data There are many data about history and usage in rephist.c (which stands for 'reputation and history!'). We could expose them, to let us better test them. Some of this might be useful for Seth (previously arm) users. Spec: This would use GETINFO extensions, and probably some new events. 18. All ratelim data Sometimes our rate-limiting code can get wonky. It would be great to expose it to Tor controllers in order to help ensure it's behaving correctly. This would include send/receive windows and bw stuff. Spec: This would use GETINFO extensions, and probably some new events. 19. All accounting data As 17, but for hibernate.c, which performs bandwidth accounting. 20. All guard transitions Our guard node state logic is very complicated, and much in need of testing and refactoring. Exposing more state transitions and guard selection transitions to the controller might help. (We have a "GUARD" event now, but it is a bit out of sync with the main implementation) 21. All key transitions We could generate events every time we change keys, and (TESTING ONLY) allow a controller to time-out a key early. 22. Examine mux settings (TESTING ONLY) The circuitmux code that we use to decide which cell to send next is very complex; it would be good to expose its thinking and decisionmaking to a testing controller for better observation. 23. Examine pathbias settings Our pathbias code is also complex, and a bit flakier than the circuitmux code. We could do for it as with 22 above. 24. Examine cpuworker queues As of 0.2.6, we have a new cpuworker infrastructure that better sends data to worker threads, but not much visibility into how well it's working. Exposing some information about this to the controller could help us tune better. 25. All geoip data As 17, but for any geoip information we aren't currently exporting. 26. Replay detection For hidden service security (and maybe eventually for secure ntor Y reuse) we have to keep replay caches to prevent us from being tricked into handling the same value twice. We don't expose the load on these caches to the controller, however. And we could, to help us better tune them into using a good memory/error-rate trade-off. 27. Hidden service intropoint changes, desc changes, uploads Many hidden service transitions currently generate no events. We could at minimum generate events for changed inroduction points, changed hidden service descriptors, uploading our own HS descriptor. 28. Descriptor uploads. We have an event for when our descriptor has changed, but not for a successful upload for it. We could fix that. 29. Path generation logic -- expose, allow. Currently a controller's only visibility into path selection logic is in its outputs, and in the opportunity to replace path selection logic entirely. We could expose more details about the algorithm's operation in order to help better test our path selection. 30. All PT status information. Pluggable transport feedback is, at present, very coarse-grained. For testing we might expose more. 31. Crypto operation counts. We ought to keep count of our various cryptographic operations, and expose them to the controller. This would help us know where to spend our optimization efforts. 32. Forget cached information To better test our download logic, it would be helpful to have a way to drop items from our caches. IMPLEMENTATION NOTES: ===================== Not lightly does one list 31 controller improvement areas. If we're hoping to do these without too much programmer time and , we need to take a much more principled approach to implementing controller commands. For #8351 I worked on a branch called 'ticket8351' that has some code we could use here. COMPATIBILITY NOTES: ==================== Many of the features here are ones that we might not want to promise to support indefinitely; we should gate them behind a USEFEATURE command, and maybe place them in an annex of the control spec.

7 10

Tagged and moved tickets based on triages spreadsheet
by Nick Mathewson 09 Apr '15

09 Apr '15

Hi, all! Since Isabela is ill (get well soon, Isabela!), I've tried to translate the results of 0.2.7 triage onto the bugtracker. To ensure that everything is reversible, I've tagged items with: 027-triaged-1-in 027-triaged-1-out 027-triaged-1-deferrable. I've then moved all of the 027-triaged-1-out items that didn't already have code into 0.2.???. Perhaps the rest should follow them soon. I've likely messed something up at some point; not too much, I hope. Here are the items that have found their way into the 0.2.7 milestone that I did not see on the triage spreadsheet: https://trac.torproject.org/projects/tor/query?status=accepted&status=assig… I suspect this isn't our last triage round, and that once Isa is back in action (get well soon!) she'll have improvements to make. cheers, -- Nick

1 0

Tor Summer of Privacy [ Incorporating ruleset testing into https-everywhere release process ]
by karan grover 09 Apr '15

09 Apr '15

Hello there! I am Karan Grover. I am a 2nd year computer science student at IIIT Delhi. I am a part of my college's Cybersecurity Education and Research Center, which is the first of its kind in the country. I have worked primarily in the field of Privacy and Security of Online Social Media (PSOSM). I am interested in contributing to Tor as a part of Tor Summer of Privacy. The project I am interested in working on is *Incorporating ruleset testing into https-everywhere release process*. I have been understanding the codebase of *https-everywhere-checker, *and am now quite familiar with it. I would like to discuss further about what the project entails. I contribute to open source every now and then, but am now looking to be an active open source contributor and Tor seems to align with my interests. Looking forward to your reply. Thank you.

2 1

Orfox (Firefox for Android) - Recent Updates
by Amogh Pradeep 09 Apr '15

09 Apr '15

Hey guys, At the Valencia meet I got Orfox building again with the latest mozilla code and since then, I've been trying to get the project back on track. The current repo can be found at[0] After going through the source code for firefox for andoroid, these were the network related findings that I've made. The application uses maily 3 layers to work. 1) The core layer, written in C++. 2) A JS layer with some more functional code. 3) A Java layer that makes uses the Android API's and other code used for android. Out of the 3 layers, 2 of them make network calls. 1) The C++ layer. There is an advantage of this layer making the network calls as all the calls made by this layer are proxied. They obey the fennec settings for proxy that has currently been set at mozilla-central/mobile/android/app/mobile.js . Hence, we do not need to bother about setting proxy to these connections and calls. 2) The Java layer. The Java code, the code run by android also has many network calls but the problem is that these calls are not done obeying the tor proxy and hence they are harmful. List of places where these calls are made are: a) any class which imports org.apache.* (commons.net.ftp.*, http.*): build/mobile/sutagent/android/DoCommand.java build/mobile/sutagent/android/SUTAgentAndroid.java mobile/android/base/distribution/Distribution.java mobile/android/base/favicons/LoadFaviconTask.java mobile/android/base/tests/BaseRobocopTest.java b) The browser also seems to be using another library for network communications and that lies at mobile/android/thirdparty/ch/boye/httpclientandroidlib This library seems to have proxy support but the calls made to this library have not been proxied, a list of those are [1] NOTE: Not all these classes listed there make network calls, I will make a shorter list of this in the next few days. [0] https://github.com/amoghbl1/gecko-dev/tree/dev_orfox [1] https://dxr.mozilla.org/mozilla-central/search?q=ch.boye.httpclientandroidl… Feedback on what you think would be very useful and much appreciated. Best, amoghbl1

1 0

Aspirant for Tor Summer of Privacy ( for Improving Tails)
by Saket Sinha 08 Apr '15

08 Apr '15

Hi, I would like to participate to Tor Summer of Privacy by contributing to "Improve Tails by working on Debian". I have worked on Debian packaging and build systems and find this project a match for my technical skills. Kindly let me know how to go ahead with this. Regards, Saket Sinha

2 3