- tor-dev - lists.torproject.org

Simplifying load balancing by removing Guard+Exit?
by Mike Perry 28 Sep '15

28 Sep '15

Hey all, I am playing with the load balancing equations in 3.8.3 in dir-spec.txt (https://gitweb.torproject.org/torspec.git/tree//dir-spec.txt#n2454) in order to account for padding, hidden/onion services, and directory traffic, and I've noticed that the fact that nodes can be both Exits and Guards not only vastly complicates the current equations, but will make any attempt at accounting for this additional overhead nearly impossible. If you've previously tried to understand the current load balancing equations, or wondered why they have 3 major sub-cases and roughly 9 sub-cases, this is entirely to account for Guard+Exit nodes, and to handle relative scarcity of pure Guard and pure Exit nodes in various conditions. Removing the Guard+Exit ('D') nodes makes all of those cases condense to one single, simple set of solutions: ⎧ E + G + M E + G + M 2⋅E - M - G 2⋅G - M - E ⎫ ⎨Wee: ─────────, Wgg: ─────────, Wme: ───────────, Wmg: ────────────⎬ ⎩ 3⋅E 3⋅G 3⋅E 3⋅G ⎭ No missing constraints, no sub-cases, no boundary conditions. Easy to reason about. Easy to intuitively convince yourself that the solutions are correct. What I want to do is add two variables to the consensus that allow us to specify expected (or measured) overhead at Guard nodes (G_o) and at Middle nodes (M_o). These two variables would represent the sum of padding, directory overhead, and hidden service traffic experienced by Guard and Middle nodes, respectively. If we ignore Guard+Exit nodes, this makes the load balancing equations become: (1-G_o)*Wgg*G == (1-M_o)*(M + Wme*E + Wmg*G) # Guard bw == middle bw (1-G_o)*Wgg*G == Wee*E # Guard bw == exit bw Wmg*G + Wgg*G == G # Guards can be middles or Guards Wme*E + Wee*E == E # Exits can be middles or Exits These solutions are a bit longer so I won't paste them here, but they are still reasonable if we ditch the Guard+Exit idea. They can be simplified even further by using a similar approach as the GuardFraction idea from Proposal 236 (Section 1.3): We can simply adjust each Guard's bandwidth by G_o either before solving the equations, or by adjusting Wgg and Wmg after the solution, then we only need to include the (1-M_o) factor in the equations themselves. We can't really do this simplification if we keep Guard+Exit, because of the interplay between Exit and Guard weights in the system of equations and cases. Additionally, if we keep Guard+Exit, the solutions literally consume an entire page of text to represent symbolically, and the sub-cases become even more complicated to evaluate and check for proper bounds and correctness. In fact, George has already run into such edge cases while trying to implement Proposal 236 (See https://trac.torproject.org/projects/tor/ticket/16255) Moreover, I've been thinking about the surveillance incentives for Guard+Exit nodes, and I actually think it is a net risk to the network for nodes to be both Guards and Exits. It's easy to find an excuse to watch an Exit node. If anything bad ever happens from an Exit, you may be able to convince someone to let you keep an eye on it** "just in case". A Guard-only node shouldn't cause anybody any problems, though, and should be much harder to justify monitoring. So this means that Guard+Exit nodes allow an adversary to justify monitoring large portions of both the entry and exit traffic of the network without any strange/technical/esoteric arguments about the need to perform complicated statistical attacks with questionable accuracy and high collateral damage, and instead force them to justify their monitoring another way. I also hear from large node operators that it is still the Exit flag that causes their nodes to really become overloaded, so if that's actually true, then it's likely that Exit nodes make poor Guards anyway, especially if Exit bandwidth is scarce (which it often is). So, what do people think? Does anyone violently disagree with removing the Guard flag from Guard+Exit nodes? ** I used to run an Exit node, and my lawyer at the time was told exactly this at one point during a police visit about a nonsensical bomb threat to some 4chan equivalent. He was specifically told "We will be monitoring this node from now on" by one of the visiting officers. I was unable to ever discover what that actually meant, and the EFF unfortunately did not feel it was worth pursuing to help me find out. It may have been a scare tactic, but it could just as easily have not been, and for all I know they may have been actually able to convince a judge to let them do this. -- Mike Perry

3 2

Two new Onionoo versions 2.4 and 2.5 add new "effective_family" and "measured" fields
by Karsten Loesing 27 Sep '15

27 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi devs, I was reminded by Harmony that I said a while back [0] that I would announce new Onionoo versions on this list and not only in private message to Harmony for including them in the next TWN issue. Oops. So, version 2.4 added a new "effective_family" field to details documents, listing all the relays with which the relay in question is in an effective, mutual family relationship. The main goal here is to make it easier to detect misconfigured relay families. This can be relay operators or friendly people watching over the Tor network and reminding relay operators to fix their configurations. [1] The new version 2.5 that I deployed this week adds the optional "measured" field to details documents. The main idea behind this new field is that relay operators and Tor network debuggers can now figure out easily whether a relay is affected by not being measured by a sufficient number of bandwidth authorities and as a result has lower usage than it could handle. This field is not yet displayed by the Onionoo clients Atlas or Globe, but it's accessible via Onionoo's API [2]. More details are on ticket #16020 [3]. All the best, Karsten [0] onionoo-announce@ posting from April 27, 2015 [1] https://lists.torproject.org/pipermail/tor-news/2015-August/000109.html [2] https://onionoo.torproject.org/protocol.html#details [3] https://trac.torproject.org/projects/tor/ticket/16020 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV1HYeAAoJEJD5dJfVqbCrFJAIAIQYIC0CiSx6GJJikQ6CTxgw +AljFxABfnLtHu58EtBLx1jRJo0u9fStCMA/YFWLiq3FmEhCb5YvoL4zwa92PwlN bIoEricgq3sXjOm6TxJ75/6OBeH8AVIsah6qxHFkVfCPwjmt0bsDyaE7VqlCv2tN tAunIZcDyhsStGOOErr5uc9FHgPquvb66Iy7YFwnJxioMRE3yp4GBTDv78WyUEuR TZOWBh/cbJYrwSu5b+vU3C+t9eAI9ErQ8BDYlwy0C6zgJO+paSUkxWkCvuI3nv2l e/Ol6M40c9syZEb5jexn3K3NTOpJCKKhjXoXtP7UHXz8ZSdhZw2gxi31ZXYrSkQ= =/rwg -----END PGP SIGNATURE-----

3 3

Tor on iOS 9
by Conrad Kramer 27 Sep '15

27 Sep '15

Hi all! I'm new to the list, my name is Conrad. As some of you may or may not have heard, Apple's latest version of iOS enables functionality that allows for a system-wide Tor client. A few people in the community, myself included, started working on a project to get this client into the App Store. I wrote up a small document (attached) that explains the project in detail, so that you can learn about it if you're interested! I'm very excited to get Tor into the hands of many more users. Conrad

1 0

0.2.8 triage at dev meeting!
by Isabela 25 Sep '15

25 Sep '15

Hello everyone! We are planing on doing a triage session for 0.2.8 release in Berlin! Below is some guidance of what you can do to prepare for that, also what you can do if you are planning on being there but want to help build the release. Who should participate? Core Tor developers, obfuscation developers, Tor Browser developers, Hidden Services developers, and of course anyone else To help with the triage we are asking everyone to add the following information to tickets you think should be part of 0.2.8 * keyword field: 0.2.8.x-triage * Size field: an estimation of how long you think it will take to get it done (not counting review time, just execution): * small < 1 day * medium < 1 week * large > 1 week * Priority field: * blocker * critical * major * normal * minor * trivial * Sponsor field: if it's related to a sponsor deliverable For 0.2.7 triage we did a few rounds until we ended with a good chunk of work for the release. The release is coming out this month and here is some numbers related to it if you would like to use as a reference: https://storm.torproject.org/shared/TLgTuwGNswrIbfhj9NUctPKj6iG9V7KGHpwy7yz… I will send more updates on how this first triage session will be done in Berlin and how we will get remote folks to participate. Also, this does not mean what we do in Berlin is final, things can be refined after too. Cheers, Isabela -- PM at TorProject.org gpg fingerprint = 8F2A F9B6 D4A1 4D03 FDF1 B298 3224 4994 1506 4C7B @isa

4 4

Desired exit node diversity
by Virgil Griffith 24 Sep '15

24 Sep '15

Let's try a simple special case. In an idealized Tor network, what would the distribution of exit nodes look like? * Would each exit node have the same bandwidth? Or would there instead be only one exit node per AS? * Would the number of exit nodes constitute exactly 1/3 of all Tor nodes? Would the total exit node bandwidth constitute 1/3 of all Tor bandwidth? * Would their distribution of AS numbers match the distribution of AS numbers of Tor clients? The distribution of Internet users? Exit nodes seem a nice place to start concretizing what's meant when we say we want relay diversity. Comments immensely appreciated because as-is I don't know the answers to these questions. -V

8 11

[Measurement Team] Next IRC meeting happens on Thursday, Sep 24, 14:00 UTC in #tor-dev
by Karsten Loesing 23 Sep '15

23 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Thursday, September 24, 2015, 14:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150924T14 This will be the last IRC meeting before the Berlin dev meeting. Maybe we'll hold an in-person team meeting next week at the same time. We'll resume the IRC meeting format the week after. Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWArn+AAoJEJD5dJfVqbCrm/YIAMTJvTAPpmbg0B85By/GfzLL mtTRB108vCLl+mkAOOkefo3o0u7RXcGlWeuSIUsqfoS4AcNpKVTF8dMrOyxCm80c nIOcwC8QX9gHGNXJXKIXaDCSeZ5MRcSLa2gLpllc/M844LbrfILIVQiF9AKafEiM DhJ7Hw41E9dNma1Qfm8LQXYwd9Sp1DLjRIFuGs3hOurZH9b2H+9Z9S+8M4D0I/hM FDyDo11MbRHEWcombVAX6IdkCUBjcZnjOUIu3tvca3pvf67or3gsRKI1GCLIRwan /Ff5TxDPkltTQwmU2E4xLpuI09baKcRpdiInkrsg1E6JfH1tT2VESJxNYb+1STM= =QMdU -----END PGP SIGNATURE-----

1 0

Looking for resources
by TOR USER 22 Sep '15

22 Sep '15

Hello, I was going through the Tor proposals which have now been closed. I have been looking for discussion regarding those proposals (here - https://gitweb.torproject.org/torspec.git) Could you link me to some discussion threads because I couldn't find much on them? Thank you

2 1

Onionoo as_number / as_name data
by Gareth Llewellyn 19 Sep '15

19 Sep '15

Hello, How often does Onionoo update its ASN information? Relays 518FF8708698E1DA09C823C36D35DF89A2CAD956, F92B3CB9BBE0CB22409843FB1AE4DBCD5EFAC835 & 707C1B61AC72227B34487B56D04BAA3BA1179CE8 are relatively new and whilst their IPs recently received updated GeoIP information (from Unassigned to UK) the ASN (AS28715) and ISP name aren't showing up. Gareth

2 1

Summary of meek's costs, August 2015
by David Fifield 18 Sep '15

18 Sep '15

Here's the summary of meek's CDN fees for August 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 -- total by CDN $6154.49 + $7165.85 + $154.89 = $13475.23 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… This month there was a big rebound as a temporary outage (not related to censorship) with meek-azure was resolved. The user count is now almost as high as it has ever been, at about 5000 simultaneous users. This month is the first time we have actual usage numbers from Azure. They told us what the cost would have been, but did not actually charge us. This month (September) is the last month on the free grant we have for Azure. After that, we will probably have to throttle Azure just like the other backends. If you want to help make domain fronting more efficient and future-proof, a good place to direct your attention is to this ticket, which aims to start using domain fronting as a cheap, low-volume interface to BridgeDB, instead of as a bulk data transfer channel: https://bugs.torproject.org/16650 If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2920 GB $350.38 6128 instance hours $306.38 Compared to the previous month: 2934 GB $352.11 7598 instance hours $379.90 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 150M requests $180.30 1191 GB $157.13 Asia Pacific (Sydney) 1M requests $2.26 7 GB $0.98 Asia Pacific (Tokyo) 28M requests $34.17 177 GB $22.80 EU (Ireland) 174M requests $209.36 905 GB $70.40 South America (Sao Paulo) 7M requests $16.87 49 GB $11.68 US East (Northern Virginia) 75M requests $75.07 490 GB $38.60 -- total 437M requests $518.03 2821 GB $301.59 https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… == Azure a.k.a. meek-azure == This is the first month where we have actual (not estimated) usage numbers, as a result of the new kind of account we're using. Unfortunately (also as a result of the account transition), meek-azure was broken until August 14, so these number really only represent half a month. Zone 1 1062 GB $92.40 Zone 2 453 GB $62.49 -- total 1515 GB $154.89 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html https://lists.torproject.org/pipermail/tor-dev/2015-August/009213.html

1 0

Re: [tor-dev] Proposal: Single onion services
by tordev123＠Safe-mail.net 18 Sep '15

18 Sep '15

Doesn't your proposal imply that you are turning all relays into exit-nodes lite? The last relay in the path will know what service you are connecting to (at least if that service is hosted with a unique relay), right? Have you considered all the implications?

7 7