November 2015 - tor-dev - lists.torproject.org

Scaling Tor Metrics
by Karsten Loesing 01 Dec '15

01 Dec '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello devs, the Tor Metrics website [0] claims to be "the primary place to learn interesting facts about the Tor network" and invites its visitors who "come across something that is missing" to contact the website authors about it. That's a bold statement I put there! :) Yet, there's considerable product backlog with possible enhancements [1] that doesn't seem to ever become shorter. Even worse, it can be expected that the backlog will refill quickly once the community notices that feature requests are suddenly considered. The main reason for this unfortunate situation is that Tor Metrics contains many moving parts, including some heavy database lifting that takes place below the surface, that all want to be maintained. Adding more parts just makes the whole thing even more likely to break. At the same time, knowing about the situation that Tor Metrics has become almost closed to contributions is painful. This posting shall discuss possible solutions. The goal is to let Tor Metrics grow in a healthy fashion that encourages contributions from the community. These solutions are not mutually exclusive, and the best solution may use parts of more than one solution sketched out here. 1 Make Tor Metrics better and bigger, internally The obvious solution is that the maintainers of Tor Metrics could just work harder to overcome the problems stated above. Let's think this through. 1.1 Add more development resources If only the current Tor Metrics maintainers had more time to devote to cleaning up existing parts and to add new parts, that would solve our problem. They could refactor parts that are hard to maintain, and they could work off the serious backlog that has piled up. Of course, this means dropping or handing over responsibilities for other products, and it may mean finding (and paying) new developers to help maintain Tor Metrics. It's unclear whether anything like this would fit into Tor's budget, and whether these changed priorities would make users of tools that had to be dropped or handed over unhappy. 1.2 Rewrite internal parts of Tor Metrics to encourage external contributions Most of Tor Metrics would have run 10 or 15 years ago with only minor modifications. It's not necessarily a bad thing to use established technologies. But maybe, if we rewrite it using modern data-processing, web, and visualization frameworks, it becomes more attractive to other developers to contribute code and help maintain existing (well, then rewritten) code. The result would be a larger Tor Metrics website that is easier to maintain and hopefully maintained by more people. It's unclear how realistic this plan is, though, and it requires attention by Tor Metrics maintainers to bring it enough into shape for external contributors to get involved. 2 Add more ways to contribute to Tor Metrics externally It may be possible to further grow Tor Metrics without adding more code to it, hence not making it any harder to maintain. However, if code to generate visualizations is run elsewhere, there's a certain risk that results are not perceived as trustworthy as if that code were run as part of Metrics. This is primarily a problem of setting user expectations right. We could add different ways for contributing to Tor Metrics, depending on the level of commitment that contributors are willing to make. Possible new ways (in addition to filing a Trac ticket, which is already possible, though not very effective) are: 2.1 Accept contribution of static data or static graphs Somebody might contribute data (in a tarball, download link, etc.) or a static graph (static as in "doesn't break, ever", not "static HTML with a tiny amount of JavaScript that will surely never break"). The Tor Metrics team reviews that and puts it on the Tor Metrics website, together with a short description, author information, license, etc. There are plenty of visualizations on Trac and on the mailing lists, so we'll have to define criteria what we add and what not, and we'll need a good process for making that happen. 2.2 Link to external websites Somebody might write a website that visualizes Tor network data. The Tor Metrics team reviews the idea behind it, but not necessarily look at its code, and adds an external link to Tor Metrics. It becomes obvious that the authors remain responsible for their visualization, so there's no risk involved for Tor Metrics, but users may not trust it as much, because it doesn't have the Tor Metrics label. Note that we're already doing this approach by linking to the visualizations showing "Tor users as percentage of larger Internet population" [2] and "Data flow in the Tor network" [3]. Also note that we could as well have hosted the former directly on Tor Metrics with appropriate attribution, because it's a static image. This is not the case with the latter. 2.3 Run an externally developed website as if it were part of Tor Metrics Let's imagine that somebody produces a visualization of Tor network data and would like to make it part of Tor Metrics but without limiting themselves to the technology used by Tor Metrics. We could let them write their visualization as website and integrate it into Tor Metrics after reviewing its code. Technically, part of this integration would be to "redress" the website by applying the Tor Metrics design (which has lots of room for improvement, but let's just say the result will look as seamlessly integrated into Tor Metrics as the "Network bubble graphs" [4]). Another part would probably be to rewrite web requests, so that users still think they're talking to https://metrics.torproject.org/, but really they're talking to another webserver behind that. Regarding hosting and maintenance, in theory, the website could be hosted by the original creators, but that effectively means that the Tor Metrics team gives up part of the control about what's on the Tor Metrics website. The creators of the external website could change parts or add new parts that wouldn't be reviewed by Tor Metrics developers, but they would be perceived as part of Metrics, which seems bad. The Tor Metrics team could run the externally developed website on a separate host or on the same host as Tor Metrics. We could imagine variants where the original creator stays around to fix any issues as they come up, or we could imagine that they donate their visualization that the Tor Metrics people will then maintain. We could even imagine that the Tor Metrics maintainers some day decide to integrate the originally external website into Tor Metrics proper, but that would not be required for this model to work. All these ideas require writing down guidelines, criteria, and processes. In particular, they require more thoughts and input from other people who are not currently involved in Tor Metrics maintenance and who can be expected more objective. And once these ideas are implemented, we'll need more Tor Metrics maintainer than just one. What are your thoughts? All the best, Karsten [0] https://metrics.torproject.org/ [1] https://trac.torproject.org/projects/tor/query?status=!closed&component=Met… [2] https://metrics.torproject.org/oxford-anonymous-internet.html [3] https://metrics.torproject.org/uncharted-data-flow.html [4] https://metrics.torproject.org/bubbles.html -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWVdmJAAoJEJD5dJfVqbCrmlEH/j6IjRNEkXzRVJBIBcFMKIwR eAA958Zg+DCzKPHI6Y7KZ/jGCHMP21r+YGIFevbJV4LDos9D2G0RP681/5PK7/dW if4Pz0xhl/LqbhLoOSqU2wGJG+GdWgjuTxO1TBMPUhYK71lwJsZ0cUzbNee7iAqJ zjdl3W3o4XJma0ZjZFH/gVVPbemvQHftrO8d0v7L2gFHfXmJg0kEwSZ4lJW0hfOx wIizEadFSx9/7CrINvbcHbyck0N+DRSfRYyMNSjpyFbnYI7HjYk7/+jvLWALEYZT 0MZbuL/zl/PFpkTDNY0jzE5fPaKjtS2pEoZn85Wn1+0kCjeFLE/Hvulvn6ZFxsA= =n5p1 -----END PGP SIGNATURE-----

6 5

[Applications Team] Meeting next Tuesday 19:00 UTC
by Georg Koppen 01 Dec '15

01 Dec '15

Hi, just a reminder that the applications team is having its monthly meeting next week on Tuesday, December 1 at 19:00 UTC in #tor-project on irc.oftc.net. There is no proposed topic for this meeting yet. Please come up with suggestions if you feel there is something we should discuss. See you there, Georg

2 1

Summary of meek's costs, October 2015
by David Fifield 29 Nov '15

29 Nov '15

Here's the summary of meek's CDN fees for October 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 September 2015 $617.08 + $710.75 + $490.58 = $1818.41 October 2015 $672.01 + $110.72 + $300.64 = $1083.37 -- total by CDN $7443.58 + $7987.32 + $946.11 = $16377.01 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… In October we had about 4,000 simultaneous users all month. There was an unfortunate outage of meek-amazon (not the result of censorship, just operations failure). Between 30 September and 9 October the bridge had an expired HTTPS certificate. [tor-talk] Outage of meek-amazon https://lists.torproject.org/pipermail/tor-talk/2015-October/039231.html https://lists.torproject.org/pipermail/tor-talk/2015-October/039234.html And then, as a side effect of installing a new certificate, the bridge's fingerprint changed, which caused Tor Browser to refuse to connect. It used to be that we didn't include fingerprints for the meek bridges, but now we do, so we didn't anticipate this error and didn't notice it quickly. Update the meek-amazon fingerprint to B9E7141C594AF25699E0079C1F0146F409495296 https://trac.torproject.org/projects/tor/ticket/17473 [tor-talk] Changed fingerprint for meek-amazon bridge (attn support) https://lists.torproject.org/pipermail/tor-talk/2015-November/039397.html Interestingly, the meek-amazon bridge still had about 400 simultaneous users (not as much as normal) during the time when the fingerprint didn't match. I would have expected it to go almost to zero. Maybe it's people using an old version of Tor Browser (from before March 2015) or some non–Tor Browser installation. Our grant for meek-azure ran out and now it costs money. Accordingly I've rate-limited it to limit costs. I set it to 1.1 MB/s on 2 October and to 0.8 MB/s on 30 October. [tor-talk] meek-azure now rate-limited https://lists.torproject.org/pipermail/tor-talk/2015-October/039169.html meek-google was also out for a few days around 30 October because I messed up the app upload and pointed it to the wrong bridge. If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2842 GB $341.06 6619 instance hours $330.95 Compared to the previous month: 2871 GB $344.53 5451 instance hours $272.55 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Usage of meek-amazon was quite low this month because of bridge outages having to do with an expired HTTPS certificate and a changed relay fingerprint. Asia Pacific (Singapore) 6M requests $8.24 36 GB $5.05 Asia Pacific (Sydney) 635K requests $0.79 1 GB $0.21 Asia Pacific (Tokyo) 1M requests $1.71 5 GB $0.71 EU (Ireland) 39M requests $46.81 130 GB $9.97 South America (Sao Paulo) 2M requests $4.98 5 GB $1.23 US East (Northern Virginia) 24M requests $24.36 86 GB $6.64 -- total 74M requests $86.89 266 GB $23.81 https://globe.torproject.org/#/bridge/F4AD82B2032EDEF6C02C5A529C42CFAFE5165… (Note new fingerprint.) == Azure a.k.a. meek-azure == Zone 1 1652 GB $202.51 Zone 2 586 GB $98.13 -- total 2238 GB $300.64 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html https://lists.torproject.org/pipermail/tor-dev/2015-August/009213.html https://lists.torproject.org/pipermail/tor-dev/2015-September/009533.html https://lists.torproject.org/pipermail/tor-dev/2015-October/009672.html

2 2

does renewing ed25519 signing key hurt if done to often?
by nusenu 28 Nov '15

28 Nov '15

Hi, I'm wondering if it does any bad if ed25519 online keys are renewed (to) often (i.e. on every ansible-playbook run and the admin runs it 3 times in a row to get it to complete on all hosts). If key renewal causes all existing connections to die I'll try to workaround the problem and make tor --keygen idempotent (replace the key only if its expiry changed).

1 1

tor weather warning tor ops about expiring signing keys?
by nusenu 28 Nov '15

28 Nov '15

Hi, I'm wondering if a service like a future tor weather could have an additional check to warn relay ops about key expiry: (something like "Email me when the router's signing key is about to expire") Do relays disclose the fact that they are run via OfflineMasterKey 1? Do dir auths/tor clients see when a currently used signing key is about to expire? thanks

1 0

documentation for new offline master key functionality (--keygen is undocumented)
by nusenu 28 Nov '15

28 Nov '15

Hi, since tor 0.2.7.5 is apparently not very far [1] from being released I was wondering whether there is any documentation about the new offline master key functionality? (or is this undocumented because it is not considered for general use yet?) tor v0.2.7.4-rc's manual has the following: " SigningKeyLifetime N days|weeks|months For how long should each Ed25519 signing key be valid? Tor uses a permanent master identity key that can be kept offline, and periodically generates new "signing" keys that it uses online. This option configures their lifetime. (Default: 30 days) OfflineMasterKey 0|1 If non-zero, the Tor relay will never generate or load its master secret key. Instead, you’ll have to use "tor --keygen" to manage the master secret key. (Default: 0) " but doesn't say anything about --keygen itself [2]. The 0.2.7.x mentions also a '--newpass' option that I wasn't able to find in the manpage: " - Add a new OfflineMasterKey option to tell Tor never to try loading or generating a secret Ed25519 identity key. You can use this in combination with tor --keygen to manage offline and/or encrypted Ed25519 keys. Implements ticket 16944. - Add a --newpass option to allow changing or removing the passphrase of an encrypted key with tor --keygen. Implements part of ticket 16769. - On receiving a HUP signal, check to see whether the Ed25519 signing key has changed, and reload it if so. Closes ticket 16790. " Can a tor operator use one offline master key for several relays (that are running at the same time) or is one master key required for every relay? (I assume the latter) How does the process of renewing the signing keys look like? According to the logs I assume simple run tor --keygen again and copy ed25519_signing_cert + ed25519_signing_secret_key to the relay's /keys folder the logs say: "It looks like I need to generate and sign a new medium-term signing key, because you asked me to make one with --keygen. To do that, I need to load the permanent master identity key." Does a tor operator has to SIGHUP a running tor instance after copying the new signing keys to the appropriate folder or will tor attempt to reload that file as soon as this signing key expires? How can a tor relay op display a given signing key's expiry date? Does using the offline master key functionality imply that the relay will only have an ed25519 and no RSA key? Is the offline master key limited to ed25519 keys and useless while using ed25519 + RSA keys at the same time? (because the RSA key is not offline?) thanks! [1] https://lists.torproject.org/pipermail/tor-consensus-health/2015-November/0… [2] https://trac.torproject.org/projects/tor/ticket/17583

5 13

tor ignores --SigningKeyLifetime when keys exist
by nusenu 28 Nov '15

28 Nov '15

(thread split from [1]) s7r wrote: > - - when you run tor --orport [...] just to generate the keys in a > non-interactive way, include a PublishServerDescriptor 0 in the > command as well, send the log to /dev/null and terminate the process > immediately. The descriptor will have to be published by the Tor > process actually running the relay. If the master id private key is > not encrypted, --keygen should be able to renew the medium term > signing key in a non-interactive way. But it's not a big deal if you > decide to do it with tor --orport [...] if it's easier for you this way. Turns out my workaround to generate keys without a passphrase non-interactively is not working entirely in every case since tor apparently ignores --SigningKeyLifetime (when used without --keygen) when keys exist: Signing keys are not (re)generated according to the (new) SigningKeyLifetime parameter (signing key/cert remains unchanged). reproducer: mkdir tdata tor --PublishServerDescriptor 0 --orport 1234 --datadirectory tdata --list-fingerprint --quiet (new signing key with default expiry created) attempt to change (reduce) expiry: tor --PublishServerDescriptor 0 --orport 1234 --datadirectory tdata --SigningKeyLifetime "1 week" --list-fingerprint --quiet expected result: key lifetime is reduced to 7 days actual result: key lifetime is not changed (remains at 1 month) (invoking tor with --keygen causes the expected lifetime but can not be run non-interactively if keys do not exist) So I reopened [2]. [1] https://lists.torproject.org/pipermail/tor-dev/2015-November/009959.html [2] https://trac.torproject.org/projects/tor/ticket/17127

2 4

Special handling of .onion domains in Chrome/Firefox (post-IETF-standarization)
by George Kadianakis 27 Nov '15

27 Nov '15

Hello, as you might know, the IETF recently decided to formally recognize .onion names as special-use domain names [0]. This means that normal browsers like Chrome and Firefox can now handle onion domains in a special manner since they know that they only correspond to Tor. How would we like those browsers to treat onions? For starters, those browsers should refuse to connect to onion domains entirely. Onions don't work on normal browsers anyway, and also this will reduce the onion leakage through the DNS system [1]. An extra measure would be to persuade those browser vendors to display some sort of message to poor people who click onions using their normal browser. For example they could display: Oops, seems like you visited an onion link. You need a special anonymous browser for this: www.torproject.org What else could we do here? And is there anyone who can lobby for the right behavior? :) Of course, we all know that that inevitably those browsers will need to bundle Tor, if they want to visit the actually secure onion Internet. But let's give them a bit more time till they realize this :) Cheers! [0]: https://blog.torproject.org/blog/landmark-hidden-services-onion-names-reser… https://www.rfc-editor.org/rfc/rfc7686.txt https://www.iana.org/assignments/special-use-domain-names/special-use-domai… [1]: https://www.petsymposium.org/2014/papers/Thomas.pdf

4 3

help
by Juan Berner 20 Nov '15

20 Nov '15

2015-11-20 17:24 GMT+01:00 <tor-dev-request(a)lists.torproject.org>: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Re: Shared random value calculation edge cases (proposal 250) > (George Kadianakis) > 2. Re: Shared random value calculation edge cases (proposal 250) > (David Goulet) > 3. Re: Shared random value calculation edge cases (proposal 250) > (David Goulet) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 20 Nov 2015 16:06:59 +0200 > From: George Kadianakis <desnacked(a)riseup.net> > To: s7r <s7r(a)sky-ip.org> > Cc: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Shared random value calculation edge cases > (proposal 250) > Message-ID: <87fv0069z0.fsf(a)riseup.net> > Content-Type: text/plain > > s7r <s7r(a)sky-ip.org> writes: > > > Hello, > > > > <snip> > > > > The idea of adding flags in the votes so each dirauth can advertise if > > it is participating (has an opinion for the <current> SR or not) is > > great and helps us build more defenses, probably make it easier in the > > future too if we decide to change anything. > > > > What if the consensus for SR calculation would define majority based > > on dirauths actually participating (and advertising so with a flag in > > the vote). Also, the participating or not participating flag should be > > used per vote/consensus and split into: > > > > a) we know current SR value for today so we vote it > > or > > we know previous SR value and we know for sure if we should follow the > > disaster protocol or not (in case we are about to vote at 01:00 UTC). > > so > > We participate in the vote for <current SR>. > > > > b) we are able to participate in this protocol run which will > > calculate the SR value for next day (after 00:00 UTC) so we send our > > commits/reveals. > > > > This is useful in case we are a dirauth that joined at 00:30 UTC and > > we couldn't get the _latest_ consensus (to find out if the 00:00 UTC > > consensus was created, and if not, previous SR value so we can follow > > the disaster procedure) we will not have an opinion for the <current> > > SR value at 01:00 UTC, but we can start participating in the protocol > > run for the next day - send our commit values. Once we decided on a > > <current> SR value for that day we save it and vote normally next time. > > > > So, if we have 5 dirauths running/signing consensus in total, out of > > which only 4 participate in the shared randomness protocol, the 4 > > participating ones should be able to create a valid consensus > > themselves with the insurance that the 5th one won't break consensus. > > > > One way to do this is: the dirauth which is not participating will > > take the SR value voted by the majority of the participating dirauths > > and include that in its consensus and sign. We need at least 3 > > dirauths agreeing on a SR value in order to accept it. > > > > Is this crazy? It shouldn't open the door new attacks, since this > > doesn't allow a single actor to game it, only the majority could game it. > > > > Thanks for the suggestions. > > Let me try to suggest a procedure here based on your ideas and some other > ideas. > > [Notation: SRV = shared random value] > > The goal here is to minimize the edge cases during SRV calculation and > disaster > SRV calculation. The edge cases here appear because there is no clear view > on > whether other dirauths know the current or previous SRVs, or whether the > SRV for > this period was ever created. The disaster recovery scenario is especially > annoying here. > > Here are some edge cases for example: > > * Dirauth boots up at 02:00UTC and the 01:00UTC consensus does not contain > any > SR information (maybe because not enough SR-enabled dirauths > participated at > that time). > > Should the dirauth do the disaster recovery procedure, or just play it > cool > and put no SR information on the consensus? If it has to do disaster > recovery, > then what previous SRV does it use (the 01:00UTC consensus did not > contain > such info)? > > This type of edge case is my main concern, since with dirauths upgrading > and > going offline at random times, it's likely that we will eventually > create a > consensus without SR info in the middle of the protocol run. > > * Dirauth boots up at 23:55UTC without having a previous consensus. It is > supposed to vote and form a 00:00UTC consensus without knowing any > previous > SRVs. How does it figure out whether all the other dirauths are also > bootstrapping, or whether the other dirauths actually know the previous > SRVs? > > Here are some prerequisites for the logic I'm going to suggest. The two > first > suggestions are useful in any case, I think: > > - First of all, we treat consensuses as trusted, so dirauths MUST learn > previous/current SRVs they didn't know about from any consensus they > fetch. > > - We are also going to assume that we have some sort of "SR flag" on votes > to > denote whether the dirauth participates in the protocol. > > - We are introducing another 'status' for shared random values (see > [SRVOTE]). > > Specifically, if a dirauth witnessed a 00:00UTC consensus failing to be > created, or it did not contain any SRV information, then it sets the > status of > "shared-rand-current-value" in its votes to "none". > > Now we are going to use the SRV lines in the votes as an indicator on how > the > consensus creation should play out. > > > ================================================================================ > > Here is some logic for the consensus at 00:00UTC: > > if majority of votes have disabled the SR flag: > then don't write anything SR-related to consensus > exit > > If the majority of votes contain the previous SRV value: > then calculate SRV as detailed in section [SRCALC] > else: > then calculate SRV as in [SRCALC] but with previous_SRV set > to 0 > > And here is some logic for all the other consensuses (to figure out when > dirauths should perform the disaster procedure): > > if majority of votes have disabled the SR flag: > then don't write anything SR-related to consensus > exit > > if we know the current SRV: > then write it on the consensus > exit > > if the majority of votes have the current SRV status as "none" _AND_ > those votes also contain the previous SRV value: > then do the disaster SRV procedure > else: > then don't write anything SR-related to consensus > > > ================================================================================ > > This _might_ work for fixing a good bunch of edge cases. But is it far too > complex? > > Should we just assume that these things will never happen on the real > network > and avoid baking additional complexity? What do you think? :/ > > > > > ------------------------------ > > Message: 2 > Date: Fri, 20 Nov 2015 10:19:07 -0500 > From: David Goulet <dgoulet(a)ev0ke.net> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Shared random value calculation edge cases > (proposal 250) > Message-ID: <20151120151907.GD10269@raoul> > Content-Type: text/plain; charset="us-ascii" > > On 20 Nov (16:06:59), George Kadianakis wrote: > > s7r <s7r(a)sky-ip.org> writes: > > > > > Hello, > > > > > > <snip> > > > > > > The idea of adding flags in the votes so each dirauth can advertise if > > > it is participating (has an opinion for the <current> SR or not) is > > > great and helps us build more defenses, probably make it easier in the > > > future too if we decide to change anything. > > > > > > What if the consensus for SR calculation would define majority based > > > on dirauths actually participating (and advertising so with a flag in > > > the vote). Also, the participating or not participating flag should be > > > used per vote/consensus and split into: > > > > > > a) we know current SR value for today so we vote it > > > or > > > we know previous SR value and we know for sure if we should follow the > > > disaster protocol or not (in case we are about to vote at 01:00 UTC). > > > so > > > We participate in the vote for <current SR>. > > > > > > b) we are able to participate in this protocol run which will > > > calculate the SR value for next day (after 00:00 UTC) so we send our > > > commits/reveals. > > > > > > This is useful in case we are a dirauth that joined at 00:30 UTC and > > > we couldn't get the _latest_ consensus (to find out if the 00:00 UTC > > > consensus was created, and if not, previous SR value so we can follow > > > the disaster procedure) we will not have an opinion for the <current> > > > SR value at 01:00 UTC, but we can start participating in the protocol > > > run for the next day - send our commit values. Once we decided on a > > > <current> SR value for that day we save it and vote normally next time. > > > > > > So, if we have 5 dirauths running/signing consensus in total, out of > > > which only 4 participate in the shared randomness protocol, the 4 > > > participating ones should be able to create a valid consensus > > > themselves with the insurance that the 5th one won't break consensus. > > > > > > One way to do this is: the dirauth which is not participating will > > > take the SR value voted by the majority of the participating dirauths > > > and include that in its consensus and sign. We need at least 3 > > > dirauths agreeing on a SR value in order to accept it. > > > > > > Is this crazy? It shouldn't open the door new attacks, since this > > > doesn't allow a single actor to game it, only the majority could game > it. > > > > > > > Thanks for the suggestions. > > > > Let me try to suggest a procedure here based on your ideas and some > other ideas. > > > > [Notation: SRV = shared random value] > > > > The goal here is to minimize the edge cases during SRV calculation and > disaster > > SRV calculation. The edge cases here appear because there is no clear > view on > > whether other dirauths know the current or previous SRVs, or whether the > SRV for > > this period was ever created. The disaster recovery scenario is > especially > > annoying here. > > > > Here are some edge cases for example: > > > > * Dirauth boots up at 02:00UTC and the 01:00UTC consensus does not > contain any > > SR information (maybe because not enough SR-enabled dirauths > participated at > > that time). > > > > Should the dirauth do the disaster recovery procedure, or just play it > cool > > and put no SR information on the consensus? If it has to do disaster > recovery, > > then what previous SRV does it use (the 01:00UTC consensus did not > contain > > such info)? > > > > This type of edge case is my main concern, since with dirauths > upgrading and > > going offline at random times, it's likely that we will eventually > create a > > consensus without SR info in the middle of the protocol run. > > > > * Dirauth boots up at 23:55UTC without having a previous consensus. It is > > supposed to vote and form a 00:00UTC consensus without knowing any > previous > > SRVs. How does it figure out whether all the other dirauths are also > > bootstrapping, or whether the other dirauths actually know the > previous SRVs? > > > > Here are some prerequisites for the logic I'm going to suggest. The two > first > > suggestions are useful in any case, I think: > > > > - First of all, we treat consensuses as trusted, so dirauths MUST learn > > previous/current SRVs they didn't know about from any consensus they > fetch. > > Yes ++. > > A dirauth booting up should always try to learn the current and previous > SRV from the latest consensus (< 24h) and update the disk state > accordingly _even_ if the disk state has SRV values in there. We should > trust the consensus voted by majority before our disk state imo. > > > > > - We are also going to assume that we have some sort of "SR flag" on > votes to > > denote whether the dirauth participates in the protocol. > > The requirements for having that flag "on" or "off" eludes me a bit. I > assume that this SR flag is "on" when a dirauth thinks its able to > compute a valid SRV that is I have enough commits. We can't make "having > the previous SRV" a requirement else we will never bootstrap properly > ever since in the first runs, previous SRV is alway 0. > > > > > - We are introducing another 'status' for shared random values (see > [SRVOTE]). > > > > Specifically, if a dirauth witnessed a 00:00UTC consensus failing to be > > created, or it did not contain any SRV information, then it sets the > status of > > "shared-rand-current-value" in its votes to "none". > > Hrm... I'm wondering here if "none" status is instead the same of having > the dirauth putting the disaster value in the vote? > > I would argue that a dirauth should _always_ try to compute a SR value > at any time (even when booting). Between having no line because we > couldn't compute due to lack of commits or a line that is the disaster, > I think the former simplifies things that is "We always have a SRV value > at 00:00 from a dirauth". > > Here is what I propose: > > 1) Like you said, we always try to learn SRV values from previous > consensus. If we can't get one from a consensus or our state, compute > disaster at 00:00. > > 2) We do _not_ use a SR flag for vote, a dirauth always tries to compute > a current SRV value at 00:00. If it's 13:00, it simply doesn't put > anything in the vote since no SRV data. > > The big issue I see here is when a dirauth does _not_ have the previous > SRV. Without it, both disaster and current SRV computation will end up > not matching the majority. So let's explore this binary state: > > - I have the previous SRV (either I can get it from my state or > consensus). > > This seems a non issue, dirauth will use it, carry it in the vote and > compute the current SRV if enough commits have been seen. At 00:00, if > majority wasn't reached for the current SRV value, we have a consensus > without it which is a valid use case but we at least have the previous > one in the consensus flagged as non-fresh so it can be used for > disaster or SRV computation at the next protocol run. > > - I don't have a previous SRV (couldn't get it in the latest consensus > and I have _no_ disk state). > > For instance, if this happens at 23:30, a dirauth won't have the > chance to participate in the SR protocol because it didn't see any > commits to compute SRV value thus disaster mode engage. > > If that dirauth was the 5th one out of 9 (meaning the majority > breaker), we end up with no SRV values at all at 00:00 (not even the > previous SRV since only 4/9 had it). So a new protocol run starts, > we'll end up with a valid current SRV in 24 hours and both values in > 48 hours. Essentially, we end up back in bootstrap mode. > > IMPORTANT note here: I think a dirauth MUST NOT keep any SRV values > that aren't in the 00:00 consensus. > > Question is, how big of a deal it is to have this issue where if we > don't reach majority because a dirauth rebooted and for some magical > reason didn't get the latest consensus to at least get the previous > SRV, we return in bootstrap? > > Cheers! > David > > > > > Now we are going to use the SRV lines in the votes as an indicator on > how the > > consensus creation should play out. > > > > > ================================================================================ > > > > Here is some logic for the consensus at 00:00UTC: > > > > if majority of votes have disabled the SR flag: > > then don't write anything SR-related to consensus > > exit > > > > If the majority of votes contain the previous SRV value: > > then calculate SRV as detailed in section [SRCALC] > > else: > > then calculate SRV as in [SRCALC] but with previous_SRV > set to 0 > > > > And here is some logic for all the other consensuses (to figure out when > > dirauths should perform the disaster procedure): > > > > if majority of votes have disabled the SR flag: > > then don't write anything SR-related to consensus > > exit > > > > if we know the current SRV: > > then write it on the consensus > > exit > > > > if the majority of votes have the current SRV status as "none" _AND_ > > those votes also contain the previous SRV value: > > then do the disaster SRV procedure > > else: > > then don't write anything SR-related to consensus > > > > > ================================================================================ > > > > This _might_ work for fixing a good bunch of edge cases. But is it far > too complex? > > > > Should we just assume that these things will never happen on the real > network > > and avoid baking additional complexity? What do you think? :/ > > > > > > _______________________________________________ > > tor-dev mailing list > > tor-dev(a)lists.torproject.org > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >

1 0

DoS resistance for Next-Generation Onion Services
by teor 20 Nov '15

20 Nov '15

Hi all, I think we can make next-generation onion (hidden) services (proposal #224) more resilient against certain kinds of DoS / client discovery attacks, by using a different blinded public key for each HSDir. Attack Summary: Once a malicious HSDir receives a descriptor, it can locate other HSDirs with that descriptor's blinded public key. It doesn't need to know the onion service address or decrypt the descriptor. Each descriptor contains the blinded public key: > The 'certificate' field contains a certificate in the format from proposal 220, with the short-term ed25519 descriptor-signing key signed by the blinded public key. It must contain a ed25519-signing-key extension containing the blinded public key. (All quotes are from https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.t… <https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.t…> ) This allows an attacker to target onion services for DoS, as long as they don't care which onion service they're targeting. The effects of this attack could be: * if the malicious HSDir also stops serving the descriptor, it denies service to some onion service, * if the malicious HSDir continues to serve the descriptor, it becomes the only source of descriptors for some onion service. I think this enables traffic confirmation attacks on some onion service or its clients (by bringing the sole descriptor source up and down). Client descriptor downloads could also be subject to traffic fingerprinting or slow (byte-at-a-time) delivery. Attack Details: Let's assume an attacker who: * doesn't know any onion service addresses * runs a malicious HSDir that collects blinded public keys * can check other HSDirs for descriptors based on blinded public keys * wants to DoS some onion services, but doesn't care which ones This attacker then: 1. chooses an arbitrary blinded public key from their HSDir's collection 2. calculates possible HSDirs for the blinded public key 3. checks each of the possible HSDirs to see if they actually have a descriptor with that blinded public key 4. runs a DoS on each HSDir with that blinded public key (this is the resource-intensive step) A refinement of the attack is to: A. find the HSDirs for each blinded public key uploaded to the malicious HSDir (steps 1-3) B. find the blinded public keys with the most vulnerable HSDirs (perhaps using the maximum consensus weight among all 5) C. DoS the blinded public keys with the most vulnerable HSDirs (step 4) I think this is a plausible attack that could be run repeatedly until the attacker happens to hit a high-value onion service, or a high enough number of onion services. The targeted onion services would change blinded keys and HSDirs with the next shared random period, but they'd be down for up to 24 hours. Mitigation Summary: If each onion service uploads a descriptor with a different blinded public key to each HSDir, a malicious HSDir is unable to locate other HSDirs serving the same onion service. Even with this change, an attacker who knows an onion service's address can still find all its HSDirs. They can set up a malicious HSDir and wait unit it happens to get chosen as the HSDir for that service. But this targeted attack requires more resources and time than the unknown target attack I describe above. The unknown target attack can also consider hundreds of blinded public keys, and target the ones with the weakest HSDirs. The targeted attack is unlikely to host a particular descriptor on a malicious HSDir, and also simultaneously find that all the other HSDirs for that descriptor are vulnerable to a DoS. Alternative Mitigations: The current keyblinding nonce is based on the period number and start time. This allows the blinded keys to be generated in advance. But the drawback is that the blinded key is the same for each HSDir. > There is a master keypair (sk, pk). Given the keypair and a nonce n, there is a derivation function that gives a new blinded keypair (sk_n, pk_n). This keypair can be used for signing. Given only the public key and the nonce, there is a function that gives pk_n. > … > > We propose the following scheme for key blinding, based on Ed25519. > > (This is an ECC group, so remember that scalar multiplication is the > trapdoor function, and it's defined in terms of iterated point > addition. See the Ed25519 paper [Reference ED25519-REFS] for a fairly > clear writeup.) > > Let the basepoint be written as B. Assume B has prime order l, so > lB=0. Let a master keypair be written as (a,A), where a is the private > key and A is the public key (A=aB). > > To derive the key for a nonce N and an optional secret s, compute the > blinding factor h as H(A | s, B, N), and let: > > private key for the period: a' = h a > public key for the period: A' = h A = (ha)B > > Generating a signature of M: given a deterministic random-looking r (see EdDSA paper), take R=rB, S=r+hash(R,A',M)ah mod l. Send signature (R,S) and public key A'. > > Verifying the signature: Check whether SB = R+hash(R,A',M)A'. > > (If the signature is valid, > SB = (r + hash(R,A',M)ah)B > = rB + (hash(R,A',M)ah)B > = R + hash(R,A',M)A' ) > … > (To use this with Tor, set N = INT_8(period-number) | INT_8(Start of period in seconds since epoch).) Mitigation #1: Mitigation #1 generates a blinded key for every replica. (This means the attacker can only target a single replica of the descriptor, which they can likely do anyway, because the "spread_store" HSDirs are in order on the hash ring.) But it looks like this can't be done in advance, because the number of replicas is a consensus parameter. > The following consensus parameters control where a hidden service descriptor is stored; > hsdir_n_replicas = an integer in range [1,16] with default value 2. > hsdir_spread_fetch = an integer in range [1,128] with default value 3. > hsdir_spread_store = an integer in range [1,128] with default value 3. > hsdir_spread_accept = an integer in range [1,128] with default value 8 But we can instead generate a blinded key for every replica mod default_hsdir_n_replicas. Then, if the number of replicas in the consensus increases, the attack can only target 1/default_hsdir_n_replicas of the replicas. It simply doesn't know where the other replicas are, because they have different blinded keys. The spec change for Mitigation #1 would look like: > (To use this with Tor, create a key, blinded_public_key(advance_replicanum) for each N = INT_8(period-number) | INT_8(Start of period in seconds since epoch) | INT_8(advance_replicanum), where advance_replicanum takes every value from 1 to default_hsdir_n_replicas. Each key is used for the replicas where replicanum mod default_hsdir_n_replicas is equal to advance_replicanum.) The blinded key is then used by both the service and clients to determine the descriptor's hashring index: > To determine where a given hidden service descriptor will be stored in a given period, after the blinded public key for that period is derived, the uploading or downloading party calculate > for replicanum in 1...hsdir_n_replicas: > hs_index(replicanum) = H("store-at-idx" | > blinded_public_key(replicanum mod default_hsdir_n_replicas) | > INT_8(replicanum) | > INT_8(periodnum) ) For Mitigation #1, including advance_replicanum in the keyblinding does NOT remove the need for replicanum in the hs_index(replicanum) hash. (advance_replicanum could be lower than replicanum if the hsdir_n_replicas consensus parameter is increased from the default.) There's also a slight clarification to the spread upload wording: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors for the corresponding replica's blinded signing key to the first hsdir_spread_store nodes whose indices immediately follow hs_index(replicanum). In summary, by creating a blinded public key for each replica, Mitigation #1 restricts the attack described to a single descriptor replica per malicious HSDir. This significantly increases the resources or knowledge required for the attack. Mitigation #2: Mitigation #2 generates a blinded key for every spread. (This means the attacker can only target a single spread for the descriptor.) Spread is specified like this in the existing proposal: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors to the first hsdir_spread_store nodes whose indices immediately follow hs_index(replicanum). A similar process applies for advance generation of blinded public keys: > (To use this with Tor, create a key blinded_public_key(advance_spreadnum) for each N = INT_8(period-number) | INT_8(Start of period in seconds since epoch) | INT_8(advance_spreadnum), where advance_spreadnum takes every value from 1 to default_hsdir_spread_store. Each key is used for the spreads where spreadnum mod default_hsdir_spread_store is equal to advance_spreadnum.) Mitigation #2 changes the way the spread indexes are positioned on the hashring. Because the blinded key for each advance_spreadnum is different, the spreads are distributed around the hashring, rather than appearing on it sequentially. To maintain this distribution if the consensus value of hsdir_spread_store is increased, we have two options: * spreadnum can be included in the descriptor index hash (Mitigation #2A), or * the spread position need only be incremented once for each advance_spreadnum spreads (Mitigation #2B). Mitigation #2A: spreadnum can be included in the descriptor index hash: > To determine where a given hidden service descriptor will be stored in a given period, after the blinded public key for that period is derived, the uploading or downloading party calculate > for replicanum in 1...hsdir_n_replicas: > for spreadnum in 1...hsdir_spread_store: > hs_index(replicanum, spreadnum) = H("store-at-idx" | > blinded_public_key(spreadnum mod default_hsdir_spread_store) | > INT_8(replicanum) | > INT_8(spreadnum) | > INT_8(periodnum) ) There's also a slight clarification to the spread upload wording: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors for the corresponding spread's blinded signing key to the first node whose index immediately follows hs_index(replicanum, spreadnum). This uploads to hsdir_spread_store nodes per replica. Mitigation #2B: The spread position need only be incremented once for each advance_spreadnum spreads: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors to hsdir_spread_store nodes per replica as follows: > the descriptor for the corresponding spreadnum's blinded signing key is uploaded to the first node whose index immediately follows hs_index(replicanum, spreadnum). If any spreads would upload to the same node, they instead upload to the second and subsequent nodes whose indices immediately follows the hs_index. But spreadnum is NOT included in the descriptor index hash: > To determine where a given hidden service descriptor will be stored in a given period, after the blinded public key for that period is derived, the uploading or downloading party calculate > for replicanum in 1...hsdir_n_replicas: > hs_index(replicanum, spreadnum) = H("store-at-idx" | > blinded_public_key(spreadnum mod default_hsdir_spread_store) | > INT_8(replicanum) | > INT_8(periodnum) ) In summary, by creating a blinded public key for each replica, Mitigation #2 (both variants) restricts the attack described to a single descriptor spread per malicious HSDir. This significantly increases the resources or knowledge required for the attack. Mitigation #3: Alternately, we could double blind each public key: once when it is generated from the master key in advance, and again just before it is uploaded to the HSDir. This double-blinding should use two nonces, P and Q: * P is the advance-blinding nonce based on the period * Q is the post-blinding nonce based on the replica and/or spread (I considered and rejected a design where Q is based on the fingerprint of the HSDir. This would lead to blinding with an externally-supplied fingerprint, potentially chosen by an attacker to reveal information on blinding.) The blinding would look something like this, where H(Q, H(P, …)) replaces H(N, …): > There is a master keypair (sk, pk). Given the keypair and a nonce n, there is a derivation function that gives a new blinded keypair (sk_n, pk_n). This keypair can be used for signing. Given only the public key and the nonce, there is a function that gives pk_n. > … > > We propose the following scheme for key blinding, based on Ed25519. > > (This is an ECC group, so remember that scalar multiplication is the trapdoor function, and it's defined in terms of iterated point addition. See the Ed25519 paper [Reference ED25519-REFS] for a fairly clear writeup.) > > Let the basepoint be written as B. Assume B has prime order l, so lB=0. Let a master keypair be written as (a,A), where a is the private key and A is the public key (A=aB). To derive the key for a advance-blinding nonce P, an optional secret s, and a post-blinding nonce Q: > compute the advance-blinding factor h as H(A | s, B, P), and let: > > advance-blinded private key for the period: a' = h a > advance-blinded public key for the period: A' = h A = (ha)B > > compute the post-blinding factor j as H(A' | 0, B, Q), and let: > > post-blinded private key for the period: a'' = j a' = j (ha) > post-blinded public key for the period: A'' = j A' = j ((ha)B) > > Generating a signature of M: given a deterministic random-looking r (see EdDSA paper), take R=rB, S=r+hash(R,A'',M)ahj mod l. Send signature (R,S) and public key A''. > > Verifying the signature: Check whether SB = R+hash(R,A'',M)A''. > > (If the signature is valid, > SB = (r + hash(R,A'',M)ahj)B > = rB + (hash(R,A'',M)ahj)B > = R + hash(R,A'',M)A'' ) > ... > (To use this with Tor, set P = INT_8(period-number) | INT_8(Start of period in seconds since epoch) and Q = INT_8(replicanum) and/or INT_8(spreadnum). Other Considerations: Combining Mitigations: Mitigations #1 and #2 (either variant) can be combined, but this means generating 6 blinded keys per period, rather than 1. I don't think the extra benefit gained from implementing both outweighs the complexity involved. Issue with Per-Spread Keys: One drawback of Mitigations #2 and #3 (different blinding keys for each spread) is that the client may need to request multiple blinded keys from each HSDir due to relays dropping out of the hash ring. For example: 1. An onion service uploads descriptors with keys A, B, C to spread servers a, b, c respectively 2. Spread server a goes down 3. the client wants the descriptor for the onion service, and tries spread servers b, c, d The client doesn't know how many servers have gone down. Therefore, it will need to ask for A, B, C from the first server (actually b, which has B); B, C from the second server (actually c, which has C); and C from the third server (actually d, which has nothing). I don't like the extra load involved in this, and I wonder about the potential for information leaks to either the HSDir, or an observer of the client's connection. (For example, each of the traffic patterns above is distinct, so an observer of the client could tell which spread number it used for the descriptor.) Security Proofs: The modified security proof in Mitigation #3 would need to check out for me to be comfortable with this option. Conclusion: Can we consider Mitigation #1: creating a different blinded public key for each replica? This would double the number of keys we generate in advance, one for each replica. Alternately, we could use Mitigation #3: double-blinding each key with the period in advance, then the replica number just before upload. But we'd need to check the modified security proof carefully. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

3 4