- tor-dev - lists.torproject.org

Stem Release 1.4
by Damian Johnson 13 May '15

13 May '15

Greetings wonderful carbon-based residents of the Internet. I'm pleased to announce the 1.4.0 release of Stem! What is Stem, you ask? For those who aren't familiar with it Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write applications similar to Nyx and Vidalia. https://stem.torproject.org/ So what's new in this release? ------------------------------------------------------------------------------ Ephemeral Hidden Services and Descriptors ------------------------------------------------------------------------------ Tor's 0.2.7.1 release is bringing with it new hidden service capabilities, most notably ADD_ONION and HSFETCH. Ephemeral hidden services let you easily operate a hidden service that never touches disk... https://stem.torproject.org/tutorials/over_the_river.html#ephemeral-hidden-… This latest Tor release also brought with it the ability to retrieve a hidden service's descriptor information. Stem knows how to parse, validate, and decrypt these documents... https://stem.torproject.org/tutorials/over_the_river.html#hidden-service-de… ------------------------------------------------------------------------------ Faster Descriptor Parsing ------------------------------------------------------------------------------ When reading descriptors without validation (which is the new default), documents are now lazily parsed. This provides a very substantial speedup depending on the document's type... * Server descriptors: 27% faster * Extrainfo descriptors: 71% faster * Microdescriptors: 43% faster * Consensus: 37% faster Prefer to keep validation? No problem! Just include 'validate = True' and you'll be good to go... https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#valida… ------------------------------------------------------------------------------ As always this is just the tip of the iceberg. For a full rundown on the myriad of improvements and fixes in this release see... https://stem.torproject.org/change_log.html#version-1-4 Cheers! -Damian

1 0

Notes on Donncha's Summer of Privacy project about scaling hidden services
by Nick Mathewson 12 May '15

12 May '15

Hi! Here are some notes on Donncha O'Cearbhaill's design proposal at https://gist.github.com/DonnchaC/03ad5cd0b8ead0ae9e30 . (David asked me to write these up in time for them to be useful.) You'll probably understand what I'm saying here a lot better if you read the link above. :) In general I think this is a solid proposal. Below are a few small questions and suggestions. 1. Getting the introduction points to the management service * Synchonization, and push vs pull. I think we've started to get statistics on the average longevity of a busy introduction point. If the answer is "they change fairly often" then we should look at the expected failure rate for trying a random introduction point at any given time, given a periodic polling interval. And if *that's* high, we should look for ways to ensure that the management service learns about changes in introduction points as soon as those changes occur. One possibility here would be to have the management service keep a connection open to each of the onion services. Alternatively, the management service could have a hidden service of its own, 2. Future-proofing for proposal 224 (rend-spec-ng.txt) Stealth authorization should no longer be needed in this case; anybody who doesn't know the onion address of a prop224 hidden service won't be able to find or decrypt its descriptor. But proposal 224 does require that the onion key be cross-certified by the introduction point keys. For that to work, we will need support on the onion-service side. They won't need the master private key; only the public key. We need to make sure that as much of this as possible works with the offline-private-keys design from proposal 224 as well. Proposal 224 does not yet describe how to make the controller commands and events here work for prop-224 hidden services. We should figure out what the requirements are there at some point. 3. Introduction point collision Sometimes two different onion services might wind up picking some introduction points in common. Should we care? 4. How many introduction points to expose? It's not clear to me what the ideal number of introduction points is to put in each master service descriptor. All of them? Two from each node? All possibilities here seem to have some risks. yrs, -- Nick

2 1

txtorcon v0.13.0
by meejah 10 May '15

10 May '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm pleased to announce txtorcon 0.13.0. This adds: * support basic and stealth hidden service authorization, and parse client_keys files. * 2x speedup for TorState parsing (mostly by lazy-parsing timestamps) * can now parse ~75000 microdescriptors/second per core of 3.4GHz Xeon E3 * launch_tor now doesn't use a temporary torrc (command-line options instead) * tons of pep8 cleanups * several improvements to hidden-service configuration from sambuddhabasu1 * populate valid signals from GETINFO signals/names from sambuddhabasu1 You can download the release from PyPI or GitHub (or of course "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/0.13.0 https://github.com/meejah/txtorcon/releases/tag/v0.13.0 Releases are also available from the hidden service: http://timaq4ygg2iegci7.onion/txtorcon-0.13.0.tar.gz http://timaq4ygg2iegci7.onion/txtorcon-0.13.0.tar.gz.asc You can verify the sha256sum of both by running the following 4 lines in a shell wherever you have the files downloaded: cat <<EOF | sha256sum --check 3218d0fa0c22f49eee9324a5862b2d53ef77d5cb8e555e2bcffc24070aaeca7d txtorcon-0.13.0.tar.gz de266cd1b35cc2d9f4600e510d9d3a5645771d36ce36a5888a2828594feb1ef0 txtorcon-0.13.0-py2-none-any.whl EOF thanks, meejah -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJVTw5SAAoJEMJgKAMSgGmn1NwIAKdBhIXgBv/0yjYHdyNxhVBp HZiluNh3sB8oRWJNuM1BY5IRuClEx05nAulKSN+ibRneHL8O7TGuAjO56HPAckn8 rtYYsUNSTjzTjGhDr4sVijLvx4lRLlzGykvMhRewB4/D/qcn9dx3vtX+7ZJ3YfSf 0/lp5XQ2n5nfHrYvfq8dUzTNmmKGx74mNQgAmQfM04Km2iWRVLMAPDZQiBCXixtF Gerkf0ybYI/lLCf/3WJOn8qCF++qQeTP49Hd2OjG8xD+Hl6dhsDBBf7lgGCLn/lw 3m+GEuBwN8TuqFogBzyChoLMkZuVe/CI2qDoHh3FsI6RJK0CqI/SOUyY0mMXdl4= =KiSA -----END PGP SIGNATURE-----

1 0

patch for trivial issue in onionoo
by firebrand＠ruggedinbox.com 09 May '15

09 May '15

Hi, I have recently begun to read the onionoo sources. I have submitted a small patch some time ago to fix a rather trivial issue. Could someone have a look? https://trac.torproject.org/projects/tor/ticket/15654 ** firebrand

2 1

Adding event listener for torbutton in in tor-browser
by Mohiuddin Ebna Kawsar 09 May '15

09 May '15

Hi, I'm trying to add an event listener to torbutton . at first my while initialize torbutton will send "setevents warn" through control port. and when any warning rise from OP it wil be catch by my event listener and show the warning through alert message". i'm novice in javascript. please help. Regards kawsar

1 0

Summary of meek's costs, April 2015
by David Fifield 07 May '15

07 May '15

Here's the summary of meek's CDN fees for April 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 -- total by CDN $3292.25 + $3792.79 + $0.00 = $7085.04 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… The number of simultaneous users increased hugely in April, from 2000 to around 5000. There are big increases around April 9 and April 15, which I conjecture were caused by a performance improvement in meek-azure and its coverage in TWN. meek-azure used to be hugely bottlenecked, but now it should handle as many users as the others. https://lists.torproject.org/pipermail/tor-dev/2015-April/008637.html https://blog.torproject.org/blog/tor-weekly-news-%E2%80%94-april-15th-2015 We moved over 10 TB in April. Because of meek's extra layer of indirection, it doesn't correctly compute per-country user statistics (the server sees the IP address of the CDN, not of the actual user). Now that a nontrivial fraction of bridge users use meek, a nontrivial fraction of bridge users are being miscounted. Keep that in mind when you look at the per-country bridge user graphs--if users are connecting through meek, they don't count towards that country's total. Here is a ticket that aims to fix the issue by passing the client IP through the CDN. https://trac.torproject.org/projects/tor/ticket/13171 If you want to help reduce costs, you can 1. Use meek-azure; it's still covered through a grant for the next four months. 2. Set up your own App Engine or CDN account. Then you can pay for your own usage (it might even be free depending on how much you use). Here are instructions on how to set up your own: https://gitweb.torproject.org/pluggable-transports/meek.git/tree/appengine/… https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure Then you will have to enter a bridge line manually. Follow the instructions at https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtochangethefrontd… but instead of changing the "front=" part, change the "url=" part. For example, bridge meek 0.0.2.0:1 url=https://<myappname>.appspot.com/ front=www.google.com == App Engine a.k.a. meek-google == Both bandwidth and instance hours were up about 19%. Here is how the Google costs broke down: 6304 GB $756.59 2597 instance hours $129.84 Compared to the previous month: 5316 GB $637.93 2173 instance hours $108.64 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Both usage and cost were down a bit from the previous month. Asia Pacific (Singapore) 114M requests $136.79 884 GB $117.92 Asia Pacific (Sydney) 443K requests $0.55 3 GB $0.34 Asia Pacific (Tokyo) 51M requests $61.03 305 GB $39.32 EU (Ireland) 173M requests $208.17 1598 GB $127.21 South America (Sao Paulo) 5M requests $9.91 23 GB $5.31 US East (Northern Virginia) 46M requests $45.56 418 GB $33.26 -- total 389M requests $462.01 3231 GB $323.36 https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… == Azure a.k.a. meek-azure == meek-azure increased a lot because of improved performance this month. https://onionoo.torproject.org/bandwidth?fingerprint=AA033EEB61601B2B7312D8… Estimated bandwidth use this month: 2015-04 1982 GB Compared to last month: 2015-03 737 GB https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… David Fifield Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html

6 14

exitmap feature requests?
by nusenu 06 May '15

06 May '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Philipp, do you consider feature requests via [1] or would you recommend forking and implementing it oneself? thanks, nusenu [1] https://github.com/NullHypothesis/exitmap/issues -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVSlpSAAoJEFv7XvVCELh0lCQQALCIDN+o7o5rubDUP7+O2J01 pI1ezUWqRk1BkKXYOPhB17gJke/5XzYCQkA/VW8FLmjf52WnucFg25c7TJ6hbRFd T73jfQvIAoNm938c/ZRHND8xOCOxaO4oa+sd5+oPNDYCfEg3gHfFxHtjgdvUzF+P Y2CqbO271drrFzkWk0ILSoRJihmZjSZHJl21gvfmeDKRsyycig/yKip/Lyq9n8QF O9rneAsVXtT72uCKnjd7rAqVwetTgqiNKQMrOG4hZ61BoGgrwYueWpTVN9wSNNOO 755TnSjZw+/rjYJxIQngbhdFSeFOX2MvE2gaUsut4b1tUa23QSQ5bLhPVEw4P6u2 Hm9lo06RCPNoYe16Hr+ff2rLMK/UU2LfoYkBZk2f0FM7LcYzEOXs6mmeUREBoJw8 V6m+O1orczjwppF9aPBLXihUr8FZQyuhIYMPpwYnMdrtysiFAKvOusOxr7svmM/w nQvPrEy3SuiyyoQa8VGZt0DejeUfyXwhkndbTC/goxKf44pWGDVUB15fyUIKh17l 9vNVdMbCmmRVHa8Mto3RmjlNRG9c3gqsmaMteAJohXGAbe8qoSMqmRF095HYjlhb MD5UPiuNIoAqOBjC2FMTYnLT2bmrRmcQD0i0dUEtK6wg5w11M3pSbk+jR/7AD5uZ gPjSfXUM718ly0avzegm =+LN2 -----END PGP SIGNATURE-----

2 1

Tor + Apache Traffic Server w/ SOCKS - works now!
by CJ Ess 05 May '15

05 May '15

So I've been looking for a long time for something modern to sit between my browser and Tor -- something modern, capable, and efficient (i.e. doesn't fork every connection). Years ago Yahoo got some proxy software from an acquisition, a few years later they made it open source as Apache Traffic Server ( http://trafficserver.apache.org/) and today its the backbone of Yahoo's infrastructure. They have a number of full time engineers that work on it full time, they use it in production, and they are implementing cutting edge features like IPv6, SPDY, and HTTP/2 support. SOCKS is was one of the legacy features of Apache Traffic Server. However, it hasn't been maintained. If you build from git right now you'll find SOCKS support completely broken at least four ways (a couple bad asserts, wrong byte order, and an uninitialized field). They took the documentation on the SOCKS feature out a while ago but never got around to removing the code. Since it was there I spent some time over the weekend and fixed it. There are still some issues around SOCKS still but it works well enough that you can surf though tor with it. If there is interest in it here I'd be happy to put together a how-to for Linux and MacOS to get it built and configured. I'd also like to encourage people to make some noise - Yahoo does have SOCKS servers internally but they don't test using Traffic Server with them because they don't think anyone uses the feature (and they are right, there is no way the code works for anyone in the present state). But if there was interest then maybe they'd keep the code fresh going forward. I'm including a copy of the patch with this e-mail just to get it out. You can pull their git repository (https://github.com/apache/trafficserver) and apply it to the master master branch.

3 3

Re: [tor-dev] working back to socks_request_t
by teor 05 May '15

05 May '15

> Date: Mon, 4 May 2015 16:07:20 -0400 > From: CJ Ess <zxcvbn4038(a)gmail.com> > > Thanks for going into so much detail, you've given me a lot to think about. > The real solution is probably the one that nobody wants to take on - having > an application HTTP port that could take direct input from HTTP aware stuff > and utilize a richer set of information then SOCKS allows for. I've spent a > couple evenings looking to see if I could take the code stuff from the > dirport and use it for that purpose. I need to spend another couple > evenings and and go back and look at the SOCKS4 stuff, I've just recently > realized that the state machine for that is closer to an http > request/response. CJ, Pluggable transports are designed to arbitrarily change how tor makes outgoing connections. The existing transports are used for obfuscation, but they could be used for dynamic proxy authentication as well. (Some do take authentication arguments.) You'd just have to work out which user to authenticate each tor connection with - which is a difficult question of policy. This arises because tor only makes a small number of long-term connections to a few guard nodes, and multiplexes multiple streams over these connections. teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7

1 0

Re: [tor-dev] working back to socks_request_t
by teor 04 May '15

04 May '15

> Date: Sun, 3 May 2015 12:34:51 -0400 > From: CJ Ess <zxcvbn4038(a)gmail.com> > > On Sun, May 3, 2015 at 11:06 AM, teor <teor2345(a)gmail.com> wrote: > >>> Date: Sun, 3 May 2015 02:50:46 -0400 >>> From: CJ Ess <zxcvbn4038(a)gmail.com> >>> >>> So I'm doing a bit of an experiment, the idea being that if you have a >>> group of tor users sharing common infrastructure then its a slightly >>> different situation then one lone user, and you wantto emphasize that >>> resources should not be shared, caching should be minimal and >>> non-persistent, you need to keep usage from standing out, etc. … >>> So plan B is everyone involved runs their socks speaking browser on their >>> desktop/laptop, everyone runs a tor client on the same device as their >>> browser, we use the HTTPProxy/HTTPSProxy feature of the clients to navigate >>> the firewall, everyone uses their own credentials instead of having one ID >>> draw attention for high utilization, and the presence of >>> the Proxy-Authorization header takes care of any caching/session sharing >>> issues along the way. >>> >>> To make that work, the one question I have for tor-dev is if its possible >>> Here: >>> >>> https://github.com/torproject/tor/blob/24f170a11f59e26dec3a24d076b749c8acc7… >>> >>> To work back to the socks_req, so that I can pass through the username and >>> password to the upstream proxy instead of the one global username/password? >> >> Hi CJ, >> >> It sounds like you're looking for one of the HTTP(S)ProxyAuthenticator >> options - you can configure a different username and password in the torrc >> file on each client's desktop/laptop. >> >> … >> >> If these options aren't what you're looking for, can you explain what you >> want done with the SOCKS request in a bit more detail? > > So underlying idea in this case is to pass thru the proxy credentials from > the browser, so they don't have to be had coded in plain text in the tor > config - you exit the browser and the credential goes away (or maybe its > encrypted in the browser password manager), if you change your password you > don't have to go update the tor config and bounce it, if its a shared > device you don't have to reconfigure and possibly leave your credentials > around, etc. > > That sounds good conceptually, but to implement I somehow need to work back > from the connection_t passed into the function I mentioned to something > that has the socks info (circuit? associated edge connection?) I tried to > trace it though and nothing jumped out at me but maybe there is some type > casting happening and I'm missing it. The other option would be to pass the > info down through extra arguments or copied into extra field members. > Either way I'm speculating there might be a really simple way to do this > and worth the time writing up the question. If I get it working I'd be > happy to send in a patch to the this list. Hi CJ, There isn't a one-to-one correspondence between web browser SOCKS connections and Tor connections: 1. Each SOCKS connection results in a stream, and tor multiplexes multiple streams over multiple circuits through the same connection to the first hop in the path. 2. tor also initiates streams, such as directory fetches, without any corresponding SOCKS connection from the browser. 3. tor pre-emptively launches circuits, then uses them for client connections once a SOCKS request is received. 4. tor sometimes cannibalizes existing circuits by adding a hop to them. tor makes a connection to 1-3 guard nodes, and those connections carry the majority of the web browser SOCKS traffic. They are authenticated at tor launch, before any browser makes a request, and I'm pretty sure that's it for authentication from then on. If you could get authentication passthrough working, you'd still run into the following compatibility issues: 5. Tor Browser uses the input SOCKS username (I think) to manage stream-based isolation for different tabs, based on the URL. This would prevent you using Tor Browser with your modified version of tor, leaving you with a browser with fewer anonymity protections. 6. Some pluggable transports use the output proxy username to pass arguments. This would prevent you using pluggable transports with your modified version of tor. And the following anonymity / usage issues: 7. If each SOCKS connection has a corresponding circuit, and you identify the user who initiated the circuit, you make it easier to de-anonymise the user behind the circuit. 8. Given that tor can multiplex multiple users over each circuit, who is responsible for bandwidth over-use? If you're still keen on continuing, the closest that the socks authentication details get to where you need them is in conn->socks_request in circuit_get_open_circ_or_launch(). The call stack from then on is: circuit_launch_by_extend_info() circuit_establish_circuit() circuit_handle_first_hop() channel_connect_for_circuit() channel_connect() channel_tls_connect() connection_or_connect() Which initialises the or_connection_t struct "conn", then makes the connection to the proxy (if set). You could add the username and password to the conn data structure here. When the connection has finished connecting, the callback is: connection_or_finished_connecting() Which calls: connection_proxy_connect() Which could then use the username and password from the conn data structure, if they exist - otherwise, it could use the default set in the config. I still think you'll get the default authentication most of the time, because of tor's pre-emptive guard connections. When you reply, can you avoid top-posting? Some mail clients top-post by default. But it's easier to follow the conversation if replies come after the initial email. Feel free to reply in the middle of the email as well. teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7

2 1