- tor-dev - lists.torproject.org

0.2.8 triage at dev meeting!
by Isabela 25 Sep '15

25 Sep '15

Hello everyone! We are planing on doing a triage session for 0.2.8 release in Berlin! Below is some guidance of what you can do to prepare for that, also what you can do if you are planning on being there but want to help build the release. Who should participate? Core Tor developers, obfuscation developers, Tor Browser developers, Hidden Services developers, and of course anyone else To help with the triage we are asking everyone to add the following information to tickets you think should be part of 0.2.8 * keyword field: 0.2.8.x-triage * Size field: an estimation of how long you think it will take to get it done (not counting review time, just execution): * small < 1 day * medium < 1 week * large > 1 week * Priority field: * blocker * critical * major * normal * minor * trivial * Sponsor field: if it's related to a sponsor deliverable For 0.2.7 triage we did a few rounds until we ended with a good chunk of work for the release. The release is coming out this month and here is some numbers related to it if you would like to use as a reference: https://storm.torproject.org/shared/TLgTuwGNswrIbfhj9NUctPKj6iG9V7KGHpwy7yz… I will send more updates on how this first triage session will be done in Berlin and how we will get remote folks to participate. Also, this does not mean what we do in Berlin is final, things can be refined after too. Cheers, Isabela -- PM at TorProject.org gpg fingerprint = 8F2A F9B6 D4A1 4D03 FDF1 B298 3224 4994 1506 4C7B @isa

4 4

Desired exit node diversity
by Virgil Griffith 24 Sep '15

24 Sep '15

Let's try a simple special case. In an idealized Tor network, what would the distribution of exit nodes look like? * Would each exit node have the same bandwidth? Or would there instead be only one exit node per AS? * Would the number of exit nodes constitute exactly 1/3 of all Tor nodes? Would the total exit node bandwidth constitute 1/3 of all Tor bandwidth? * Would their distribution of AS numbers match the distribution of AS numbers of Tor clients? The distribution of Internet users? Exit nodes seem a nice place to start concretizing what's meant when we say we want relay diversity. Comments immensely appreciated because as-is I don't know the answers to these questions. -V

8 11

[Measurement Team] Next IRC meeting happens on Thursday, Sep 24, 14:00 UTC in #tor-dev
by Karsten Loesing 23 Sep '15

23 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Thursday, September 24, 2015, 14:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150924T14 This will be the last IRC meeting before the Berlin dev meeting. Maybe we'll hold an in-person team meeting next week at the same time. We'll resume the IRC meeting format the week after. Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWArn+AAoJEJD5dJfVqbCrm/YIAMTJvTAPpmbg0B85By/GfzLL mtTRB108vCLl+mkAOOkefo3o0u7RXcGlWeuSIUsqfoS4AcNpKVTF8dMrOyxCm80c nIOcwC8QX9gHGNXJXKIXaDCSeZ5MRcSLa2gLpllc/M844LbrfILIVQiF9AKafEiM DhJ7Hw41E9dNma1Qfm8LQXYwd9Sp1DLjRIFuGs3hOurZH9b2H+9Z9S+8M4D0I/hM FDyDo11MbRHEWcombVAX6IdkCUBjcZnjOUIu3tvca3pvf67or3gsRKI1GCLIRwan /Ff5TxDPkltTQwmU2E4xLpuI09baKcRpdiInkrsg1E6JfH1tT2VESJxNYb+1STM= =QMdU -----END PGP SIGNATURE-----

1 0

Looking for resources
by TOR USER 21 Sep '15

21 Sep '15

Hello, I was going through the Tor proposals which have now been closed. I have been looking for discussion regarding those proposals (here - https://gitweb.torproject.org/torspec.git) Could you link me to some discussion threads because I couldn't find much on them? Thank you

2 1

Onionoo as_number / as_name data
by Gareth Llewellyn 19 Sep '15

19 Sep '15

Hello, How often does Onionoo update its ASN information? Relays 518FF8708698E1DA09C823C36D35DF89A2CAD956, F92B3CB9BBE0CB22409843FB1AE4DBCD5EFAC835 & 707C1B61AC72227B34487B56D04BAA3BA1179CE8 are relatively new and whilst their IPs recently received updated GeoIP information (from Unassigned to UK) the ASN (AS28715) and ISP name aren't showing up. Gareth

2 1

Summary of meek's costs, August 2015
by David Fifield 18 Sep '15

18 Sep '15

Here's the summary of meek's CDN fees for August 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 -- total by CDN $6154.49 + $7165.85 + $154.89 = $13475.23 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… This month there was a big rebound as a temporary outage (not related to censorship) with meek-azure was resolved. The user count is now almost as high as it has ever been, at about 5000 simultaneous users. This month is the first time we have actual usage numbers from Azure. They told us what the cost would have been, but did not actually charge us. This month (September) is the last month on the free grant we have for Azure. After that, we will probably have to throttle Azure just like the other backends. If you want to help make domain fronting more efficient and future-proof, a good place to direct your attention is to this ticket, which aims to start using domain fronting as a cheap, low-volume interface to BridgeDB, instead of as a bulk data transfer channel: https://bugs.torproject.org/16650 If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2920 GB $350.38 6128 instance hours $306.38 Compared to the previous month: 2934 GB $352.11 7598 instance hours $379.90 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 150M requests $180.30 1191 GB $157.13 Asia Pacific (Sydney) 1M requests $2.26 7 GB $0.98 Asia Pacific (Tokyo) 28M requests $34.17 177 GB $22.80 EU (Ireland) 174M requests $209.36 905 GB $70.40 South America (Sao Paulo) 7M requests $16.87 49 GB $11.68 US East (Northern Virginia) 75M requests $75.07 490 GB $38.60 -- total 437M requests $518.03 2821 GB $301.59 https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… == Azure a.k.a. meek-azure == This is the first month where we have actual (not estimated) usage numbers, as a result of the new kind of account we're using. Unfortunately (also as a result of the account transition), meek-azure was broken until August 14, so these number really only represent half a month. Zone 1 1062 GB $92.40 Zone 2 453 GB $62.49 -- total 1515 GB $154.89 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html https://lists.torproject.org/pipermail/tor-dev/2015-August/009213.html

1 0

Re: [tor-dev] Proposal: Single onion services
by tordev123＠Safe-mail.net 18 Sep '15

18 Sep '15

Doesn't your proposal imply that you are turning all relays into exit-nodes lite? The last relay in the path will know what service you are connecting to (at least if that service is hosted with a unique relay), right? Have you considered all the implications?

7 7

Re: [tor-dev] Towards a new version of the PT spec...
by Adam Pritchard 18 Sep '15

18 Sep '15

At Psiphon we often discuss (and get asked about) using Tor's pluggable transports directly. The cost/benefit balance hasn't yet been in favour of doing this, but if there's discussion of a big PT revamp... maybe Psiphon should indicate how the cost side of the balance could come down for us. We're obviously not a priority for what Tor does with PTs, but there's surely no harm in us mentioning our wishlist. So... What would be best for us is if PTs were written in Go and implemented the net.Conn[1] interface. We have had good results with the composability of net.Conn implementations: an obfuscated SSH net.Conn on top of a meek net.Conn[2] on top of a upstream proxy net.Conn[3] on top of a TCPConn net.Conn[4]. Layering in a PT net.Conn would be sane and clean and reasonably easy. Conversely: Python is difficult-to-impossible due to runtime distribution. Separate executables are difficult-to-impossible due to Android PIE requirements and distribution size bloat. Anyway, if this is of any interest we can discuss it further. (Note: Probably Lantern people are reading this too. And probably they would benefit from the same things we would, since their architecture is similar to ours.) [1]: https://golang.org/pkg/net/#Conn [2]: https://github.com/Psiphon-Labs/psiphon-tunnel-core/blob/master/psiphon/mee… [3]: https://github.com/Psiphon-Labs/psiphon-tunnel-core/tree/master/psiphon/ups… [4]: https://golang.org/pkg/net/#TCPConn Adam Pritchard Psiphon Inc. > Date: Mon, 7 Sep 2015 22:45:52 +0000 > From: Yawning Angel <yawning(a)schwanenlied.me> > To: tor-dev(a)lists.torproject.org > Subject: [tor-dev] Towards a new version of the PT spec... > Message-ID: <20150907224552.6959bcbc(a)schwanenlied.me> > Content-Type: text/plain; charset="us-ascii" > > So, we currently have a Pluggable Transport (PT) spec, and it kind-of > sort-of works (The documentation is a mess that I'm working on > cleaning up, but it's an orthogonal issue for how well it works). > > There are a number of problems with the current PT spec that require > breaking backward compatibility to fix, so eventually I would like to > do so. > > I'm soliciting input on what people would also like to see in a > (currently hypothetical) PT spec 2.0 beyond what I already have in mind: > > MUST haves: > * Support dual stack Bridges correctly (Multiple server endpoints per > transport) > * Increase the argument space beyond 510 bytes (Prop. #227). > * Mandatory ExtORPort support (currently optional, but metrics are > good). > * Centralized logging by the calling process (Probably via stderr). > * AF_UNIX support where sensible for better sandboxing. > > MIGHT haves: > * Rename the env vars to not start with "TOR_PT". Some people claim > that this is a good idea (I think it is stupid and cosmetic). > * Ability to force at least clients to stop network activity without > tearing the PT down. > * Deprecate SOCKS4a, and make SOCKS5 mandatory for clients. > > UNLIKELY: > * Specify an interface for where fork()/exec() isn't possible (iOS). > I don't think this is makes sense because it is probably too > platform/caller specific. > * Allow operating both as a client and a server simultaneously. I > don't see a problem with running 2 copies of something for this > use case. > > I probably missed some things. If people have strong opinions about > this, do reply, otherwise I *will* design something that I like, which > will not include what other people want. > > Regards, > > -- > Yawning Angel -- Adam Pritchard Psiphon Inc.

3 2

How long does it take for 25/50/75% of relays to update to a new Tor version?
by Karsten Loesing 18 Sep '15

18 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Sebastian and list, Sebastian again suggested a fun task for the latest measurement team 1-1-1 task exchange round [0] that I picked up. He asked the following question: When a new Tor version is released, how long does it take for 25, 50, 75% of all relays to update? How often do relays update in general? Is there an effect when for example Debian updates its packages? It would help me to get an idea what I need to answer these questions :-) "update" means you're running, then you go down, then within a few minutes you're back up again with a changed version Here's what I came up with: I suggest we use two data sets as input for this task. The first data set consists of archived consensuses, available at: https://collector.torproject.org/archive/relay-descriptors/consensuses/ We'll need "valid-after" and "server-versions" from the header and "v" and "w" lines from all contained entries. Example: valid-after 2015-09-18 08:00:00 server-versions 0.2.4.23,0.2.4.24,0.2.4.25,0.2.4.26,0.2.4.27,0.2.5.8-rc,0.2.5.9-rc,0.2.5.10,0.2.5.11,0.2.5.12,0.2.6.5-rc,0.2.6.6,0.2.6.7,0.2.6.8,0.2.6.9,0.2.6.10,0.2.7.1-alpha,0.2.7.2-alpha v Tor 0.2.7.2-alpha-dev w Bandwidth=450 v Tor 0.2.6.10 w Bandwidth=23 [...] We can use those lines to *count* the number of running relays with a given version string, and we can sum up the *fraction* of consensus weights of relays running a given version string. Both can be useful, though I suspect the latter to be more meaningful. We should aggregate these consensus parts into daily statistics to make them easier to handle. We can do that by keeping counters per (date, version) tuple and per (date) for a) number of relays and b) consensus weight, and then we divide the (date, version) numbers by the (date) numbers to obtain averages. We'd also determine whether a version is recommended, whether it's the latest recommended version in its series, and whether it's the latest recommended stable version. The result would be a .csv file like the following (example data!): date,version,recommended,series,stable,count,fraction 2015-09-18,0.2.6.10,TRUE,TRUE,TRUE,0.387,0.451 2015-09-18,0.2.6.9,TRUE,FALSE,FALSE,0.063,0.0102 2015-09-18,0.2.6.8,TRUE,FALSE,FALSE,0.021,0.001 2015-09-18,0.2.6.7,TRUE,FALSE,FALSE,0.016,0.009 Read this as: "On 2015-09-18, version 0.2.6.10, which was both recommended and the latest in its series, was run by 38.7% of relays by count or 45.1% by consensus weight." (again, example data!) You could then draw graphs similar to https://metrics.torproject.org/versions.html, but with a percent scale on y. Though I'm not sure how readable those graphs would be with a few dozen lines in them. Still, they might be useful to explore the data set. The second data set that would be useful here is events related to versions. The following events come to mind: - tagged as version in Git (use a command similar to [1]), - tagged as alpha, beta, rc, or stable (look at version string), - first/last recommended by directory authorities for relays (parse from consensuses), - last recommended as newest version in a series or newest stable version (also parse from consensuses), or - first released in Debian stable (unclear how to obtain those dates). A possible data format would be: date,version,event 2015-07-12,0.2.6.10,git_tag 2015-06-10,0.2.6.9,git_tag 2015-05-19,0.2.6.8,git_tag 2015-04-06,0.2.6.7,git_tag For exploratory purposes, you could add these as colored vertical lines to the graphs above. Of course, adding more elements to a probably already overloaded graph doesn't exactly make it easier to understand. But if you limit the x axis on just a month or two, it might be useful. So, regarding your original question: "How long does it take for 25/50/75% of relays to update to a new Tor version?" I'm not sure whether that's really the best question to ask/answer here. Some versions will never be deployed on 75%, 50%, or even 25% of relays, because a new version was released shortly after. But that doesn't mean that the previous version was bad. This question also weights all versions the same, though in reality some releases are more important than others. If you get one data point for each version and then aggregate them by taking the average, what exactly does that tell you? I think that question could be rephrased to: "How long does it take for 25/50/75% of relays to update to a new Tor version *series*?" And I think that's something you could answer with the first data set above. But there are other interesting questions you could answer with the two data sets. For example, - What fraction of relays (by number or consensus weight) is running (un-)recommended versions? - What fraction of relays (by number of consensus weight) is (not) running the latest version in a series or latest stable version? Those could again be answered by a graph that uses the first data set, dates on x and fractions on y, and possibly events from the second data set as vertical lines. Explore, explore. There, the hour is over ("It took an hour to write, I thought it would take an hour to read.", Fry, Futurama). Hope this is useful in any way. And maybe others have good/better ideas for you as well? If you come up with some interesting answers, please post them here. All the best, Karsten [0] 1-1-1 task exchange: you get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do for you within 1 week (review a document, write some analysis code, fix a small bug, etc.; better come prepared to get the most out of this; give 1, take 1) [1] `git log --tags --simplify-by-decoration --pretty="format:%ci %d" | grep "tag: tor" | less` -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV++DHAAoJEJD5dJfVqbCr+sYH/1nViWWJgie+BD6Mo+9pC+g9 qEfMQ+NBvYWr3/SFcn/zWjydgZf+YPVXcVsOA1B1YcLX2s8OY1cWM5t5cDCstvfh /vOYmsDw1nddBh8VQxggYPkTLfEWRRSB0IeMiFWWUdVVKkcepOxgCfXMWJlBEAgG 8EtarcTkxSl1YMrurH3iuL8PrHGRiGf+mEKOZOMFtpmkSii/bWpO7BOAEyOXC7Ib EX2OouhCvae5X3Lyp2CnlQZLSrRGtAFV8sQUw5gcQb5/nouk38M7/ayrkBW/H3Ng qM9k0Y9EHAs1I1rn1ZY5lfGMKZ0WR+1NNCQ3mdsBSF5Qah01i1By7uk2FiH0e4U= =H6/Z -----END PGP SIGNATURE-----

1 0

List Administrivia [...encrypted onion services...]
by grarpamp 17 Sep '15

17 Sep '15

fabio, sky, et al... Please stop top quoting massive amounts of useless text that everyone's already seen before. It's seriously annoying to peoples workflow trying to figure out what youre replying to and if youve inlined other stuff below. And it needlessly bloats peoples mailboxes and thus search results. Trim the original to relavent bits and write your reply below that in context. Thanks, users everywhere.

1 0