- tor-dev - lists.torproject.org

[Applications Team] Meeting next Tuesday 19:00 UTC
by Georg Koppen 01 Dec '15

01 Dec '15

Hi, just a reminder that the applications team is having its monthly meeting next week on Tuesday, December 1 at 19:00 UTC in #tor-project on irc.oftc.net. There is no proposed topic for this meeting yet. Please come up with suggestions if you feel there is something we should discuss. See you there, Georg

2 1

BridgeDB 0.3.3 is released
by isis 01 Dec '15

01 Dec '15

Hello! I released BridgeDB 0.3.3 a few weeks ago, and deployed it on the production server. However, I completely forgot to email the list to notify you all of the changes. Oops, sorry! For those who are curious, BridgeDB-0.3.3 brought about the following changes (and, as always, the current changelog is available at https://gitweb.torproject.org/bridgedb.git/tree/CHANGELOG) Changes in version 0.3.3 - 2015-10-25 * FIXES #12029 https://bugs.torproject.org/12029 BridgeDB now has an API for creating Bridge Distributors. See the bridgedb.distribute module, or its developer documentation at https://pythonhosted.org/bridgedb/bridgedb.distribute.html. * FIXES PART OF #12506 https://bugs.torproject.org/12506 BridgeDB's two Distributors (HTTPS and Email) are now entirely modularised and self-contained within separate subdirectories in the source code. This is the first step to redesigning these Distributors into their own separate processes, which will allow the Distributors to remain functional while BridgeDB is reparsing bridge descriptors. * FIXES #15968 https://bugs.torproject.org/15968 BridgeDB now sends a Content-Security-Policy header which explicitly allows Javascript, images, CSS, and fonts, from https://bridges.torproject.org. All other types of content are forbidden, including: - embedding https://bridges.torproject.org within <iframe>, <embed>, or <object>, and attempting to source additional resources into its embedded context - inline Javascript, including Javascript within SVG files - inline CSS - externally hosted fonts - inline SVG, e.g. via the HTML5 <svg> tag - any and all connections made via Javascript XMLHttpRequests, WebSockets, sendBeacon(), and Web Workers - plugins - applets BridgeDB's Content-Security-Policy does not yet make use of certain newer, lesser supported, Content-Security-Policy v2.0 directives, such as "reflected-xss" and "frame-ancestors", but may someday. * FIXES #16273 https://bugs.torproject.org/16273 Several links to Tor Project gitweb URLs within the developer documentation were outdated in that they still used the old gitweb URL format. These are now updated. Thanks to David Fifield for the bug report and patches. * FIXES #16330 https://bugs.torproject.org/16330 BridgeDB can now handle bridge-server-descriptors with extra-info-digest fields which have two values, as well as both bridge-server-descriptors and bridge-extrainfo descriptors which contain Ed25519 key material and signatures. See Tor proposals #220 and #228 for more information on the changes to these descriptors. Note that BridgeDB can now parse this information, but does not yet make use of any Ed25519 cryptographic material within bridge descriptors. https://gitweb.torproject.org/torspec.git/tree/proposals/220-ecc-id-keys.txt https://gitweb.torproject.org/torspec.git/tree/proposals/228-cross-certific… Thanks to Atagar for patching Stem. * FIXES #16616 https://bugs.torproject.org/16616 The HSDir flag can now be included within bridge-networkstatus documents. BridgeDB now has unittests which guarantee that its parsers safely ignore this flag, as well as any flags unknown to BridgeDB which may appear in the future. Thanks to Roger Dingledine for alerting me about the change. * FIXES #16649 https://bugs.torproject.org/16649 Mobile users, and other users with small screen pixel ratios, will find that the UI of BridgeDB's HTTPS Distributor has greatly increased in usability and readability. And includes the following general changes: * FIXES an error when requesting the non-HTML version of the bridges page (e.g. https://bridges.torproject.org/bridges?format=plain) * REMOVES the `bridgedb test` commandline option. BridgeDB's tests can be run via `python setup.py test` or `make test` (or `make coverage` for generating HTML test coverage statistics). * CHANGES the HTTPS Distributor to HTML-encode Bridge Lines. Previously, a malicious Pluggable Transport Bridge could include in its PT arguments something like "evil=<script>[…]</script>" and if such a Bridge were to be distributed to a user, that user's web browser would execute the script (if Javacript was enabled). Other characters, including non-ASCII, control characters, double quotes, and backslashes, are also sanitised from Bridge Lines. Thanks to Robert Ransom for the patches. * CHANGES BridgeDB's module/package version numbers to be compliant with PEP440. * CHANGES the layout of BridgeDB's source code directories. Rather than storing BridgeDB's source in "lib/bridgedb/", it is now kept in "bridgedb/". Similarly, the directory containing BridgeDB's tests has been moved from "lib/bridgedb/test/" to "test/", which means that the tests are no longer installed when running `python setup.py install` or `make install`. * ADDS several improvements to the developer documentation at https://pythonhosted.org/bridgedb. * UPDATE English (en_US) translations. * UPDATE English (en) translations. * ADD Serbian (sr) translations. Thanks to obj.petit.a, Ivan Radeljic, and Milenko Doder. * UPDATE Arabic (ar) translations. Thanks to A. Hassan, debo debo, KACIMI LAMINE, and Nudroid A. * UPDATE Catalan (ca) translations. Thanks to laia_. * UPDATE Czech (cs) translations. Thanks to Tomas Palik and Vlastimil Burián. * UPDATE Danish (da) translations. Thanks to Mogelbjerg. * UPDATE German (de) translations. Thanks to jschfr, Junge Limba, and Toralf Förster. * UPDATE English (en_GB) translations. Thanks to Andi Chandler. * UPDATE Farsi (fa) translations. Thanks to some awesome anonymous person for helping out. * UPDATE Finish (fi) translations. Thanks to Riku Viitanen. * UPDATE French (fr) translations. Thanks to elouann, Trans-fr, and Towinet. * UPDATE French (fr_CA) translations. Thanks to Trans-fr. * UPDATE Croatian (hr_HR) translations. Thanks to some awesome anonymous person for helping out. * UPDATE Hungarian (hu) translations. Thanks to some awesome anonymous person for helping out. * UPDATE Indonesian (id) translations. Thanks to Anthony Santana, Astryd Viandila Dahlan, cholif yulian, constantius damar wicaksono, Dwi Cahyono, L1Nus, km242saya, and Zamani Karmana. * UPDATE Italian (it) translations. Thanks to Random_R. * UPDATE Japanese (ja) translations. Thanks to ABE Tsunehiko. * UPDATE Latvian (lv) translations. Thanks to Ojārs Balcers. * UPDATE Norwegian Bokmål (nb) translations. Thanks to Erik Matson and Kristian Andre Henriksen. * UPDATE Dutch (nl) translations. Thanks to Mart3000. * UPDATE Polish (pl) translations. Thanks to Karol Obartuch. * UPDATE Portuguese (pt) translations. Thanks to Bruno D. Rodrigues and MMSRS. * UPDATE Brazillian Portuguese (pt_BR) translations. Thanks to Communia. * UPDATE Romanian (ro) translations. Thanks to Ana, axel_89, and Di N. * UPDATE Russian (ru) translations. Thanks to Ivan. * UPDATE Slovak (sk_SK) translations. Thanks to StefanH. * UPDATE Albanian (sq) translations. Thanks to some awesome unknown anonymous person who didn't add their name to the list of translators. * UPDATE Swedish (sv) translations. Thanks to Peter Michanek. * UPDATE Turkish (tr) translations. Thanks to Bullgeschichte and Fomas. * UPDATE Ukranian (uk) translations. Thanks to Yasha. * UPDATE Chinese Mandarin (zh_CN) translations. Thanks to khi. * UPDATE Taiwanese Mandarin (zh_TW) translations. Thanks to x4r. -- ♥Ⓐ isis agora lovecruft _________________________________________________________ OpenPGP: 4096R/0A6A58A14B5946ABDE18E207A3ADB67A2CDB8B35 Current Keys: https://blog.patternsinthevoid.net/isis.txt

1 0

Summary of meek's costs, October 2015
by David Fifield 29 Nov '15

29 Nov '15

Here's the summary of meek's CDN fees for October 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 September 2015 $617.08 + $710.75 + $490.58 = $1818.41 October 2015 $672.01 + $110.72 + $300.64 = $1083.37 -- total by CDN $7443.58 + $7987.32 + $946.11 = $16377.01 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… In October we had about 4,000 simultaneous users all month. There was an unfortunate outage of meek-amazon (not the result of censorship, just operations failure). Between 30 September and 9 October the bridge had an expired HTTPS certificate. [tor-talk] Outage of meek-amazon https://lists.torproject.org/pipermail/tor-talk/2015-October/039231.html https://lists.torproject.org/pipermail/tor-talk/2015-October/039234.html And then, as a side effect of installing a new certificate, the bridge's fingerprint changed, which caused Tor Browser to refuse to connect. It used to be that we didn't include fingerprints for the meek bridges, but now we do, so we didn't anticipate this error and didn't notice it quickly. Update the meek-amazon fingerprint to B9E7141C594AF25699E0079C1F0146F409495296 https://trac.torproject.org/projects/tor/ticket/17473 [tor-talk] Changed fingerprint for meek-amazon bridge (attn support) https://lists.torproject.org/pipermail/tor-talk/2015-November/039397.html Interestingly, the meek-amazon bridge still had about 400 simultaneous users (not as much as normal) during the time when the fingerprint didn't match. I would have expected it to go almost to zero. Maybe it's people using an old version of Tor Browser (from before March 2015) or some non–Tor Browser installation. Our grant for meek-azure ran out and now it costs money. Accordingly I've rate-limited it to limit costs. I set it to 1.1 MB/s on 2 October and to 0.8 MB/s on 30 October. [tor-talk] meek-azure now rate-limited https://lists.torproject.org/pipermail/tor-talk/2015-October/039169.html meek-google was also out for a few days around 30 October because I messed up the app upload and pointed it to the wrong bridge. If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2842 GB $341.06 6619 instance hours $330.95 Compared to the previous month: 2871 GB $344.53 5451 instance hours $272.55 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Usage of meek-amazon was quite low this month because of bridge outages having to do with an expired HTTPS certificate and a changed relay fingerprint. Asia Pacific (Singapore) 6M requests $8.24 36 GB $5.05 Asia Pacific (Sydney) 635K requests $0.79 1 GB $0.21 Asia Pacific (Tokyo) 1M requests $1.71 5 GB $0.71 EU (Ireland) 39M requests $46.81 130 GB $9.97 South America (Sao Paulo) 2M requests $4.98 5 GB $1.23 US East (Northern Virginia) 24M requests $24.36 86 GB $6.64 -- total 74M requests $86.89 266 GB $23.81 https://globe.torproject.org/#/bridge/F4AD82B2032EDEF6C02C5A529C42CFAFE5165… (Note new fingerprint.) == Azure a.k.a. meek-azure == Zone 1 1652 GB $202.51 Zone 2 586 GB $98.13 -- total 2238 GB $300.64 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html https://lists.torproject.org/pipermail/tor-dev/2015-August/009213.html https://lists.torproject.org/pipermail/tor-dev/2015-September/009533.html https://lists.torproject.org/pipermail/tor-dev/2015-October/009672.html

2 2

does renewing ed25519 signing key hurt if done to often?
by nusenu 28 Nov '15

28 Nov '15

Hi, I'm wondering if it does any bad if ed25519 online keys are renewed (to) often (i.e. on every ansible-playbook run and the admin runs it 3 times in a row to get it to complete on all hosts). If key renewal causes all existing connections to die I'll try to workaround the problem and make tor --keygen idempotent (replace the key only if its expiry changed).

1 1

tor weather warning tor ops about expiring signing keys?
by nusenu 28 Nov '15

28 Nov '15

Hi, I'm wondering if a service like a future tor weather could have an additional check to warn relay ops about key expiry: (something like "Email me when the router's signing key is about to expire") Do relays disclose the fact that they are run via OfflineMasterKey 1? Do dir auths/tor clients see when a currently used signing key is about to expire? thanks

1 0

documentation for new offline master key functionality (--keygen is undocumented)
by nusenu 28 Nov '15

28 Nov '15

Hi, since tor 0.2.7.5 is apparently not very far [1] from being released I was wondering whether there is any documentation about the new offline master key functionality? (or is this undocumented because it is not considered for general use yet?) tor v0.2.7.4-rc's manual has the following: " SigningKeyLifetime N days|weeks|months For how long should each Ed25519 signing key be valid? Tor uses a permanent master identity key that can be kept offline, and periodically generates new "signing" keys that it uses online. This option configures their lifetime. (Default: 30 days) OfflineMasterKey 0|1 If non-zero, the Tor relay will never generate or load its master secret key. Instead, you’ll have to use "tor --keygen" to manage the master secret key. (Default: 0) " but doesn't say anything about --keygen itself [2]. The 0.2.7.x mentions also a '--newpass' option that I wasn't able to find in the manpage: " - Add a new OfflineMasterKey option to tell Tor never to try loading or generating a secret Ed25519 identity key. You can use this in combination with tor --keygen to manage offline and/or encrypted Ed25519 keys. Implements ticket 16944. - Add a --newpass option to allow changing or removing the passphrase of an encrypted key with tor --keygen. Implements part of ticket 16769. - On receiving a HUP signal, check to see whether the Ed25519 signing key has changed, and reload it if so. Closes ticket 16790. " Can a tor operator use one offline master key for several relays (that are running at the same time) or is one master key required for every relay? (I assume the latter) How does the process of renewing the signing keys look like? According to the logs I assume simple run tor --keygen again and copy ed25519_signing_cert + ed25519_signing_secret_key to the relay's /keys folder the logs say: "It looks like I need to generate and sign a new medium-term signing key, because you asked me to make one with --keygen. To do that, I need to load the permanent master identity key." Does a tor operator has to SIGHUP a running tor instance after copying the new signing keys to the appropriate folder or will tor attempt to reload that file as soon as this signing key expires? How can a tor relay op display a given signing key's expiry date? Does using the offline master key functionality imply that the relay will only have an ed25519 and no RSA key? Is the offline master key limited to ed25519 keys and useless while using ed25519 + RSA keys at the same time? (because the RSA key is not offline?) thanks! [1] https://lists.torproject.org/pipermail/tor-consensus-health/2015-November/0… [2] https://trac.torproject.org/projects/tor/ticket/17583

5 13

tor ignores --SigningKeyLifetime when keys exist
by nusenu 28 Nov '15

28 Nov '15

(thread split from [1]) s7r wrote: > - - when you run tor --orport [...] just to generate the keys in a > non-interactive way, include a PublishServerDescriptor 0 in the > command as well, send the log to /dev/null and terminate the process > immediately. The descriptor will have to be published by the Tor > process actually running the relay. If the master id private key is > not encrypted, --keygen should be able to renew the medium term > signing key in a non-interactive way. But it's not a big deal if you > decide to do it with tor --orport [...] if it's easier for you this way. Turns out my workaround to generate keys without a passphrase non-interactively is not working entirely in every case since tor apparently ignores --SigningKeyLifetime (when used without --keygen) when keys exist: Signing keys are not (re)generated according to the (new) SigningKeyLifetime parameter (signing key/cert remains unchanged). reproducer: mkdir tdata tor --PublishServerDescriptor 0 --orport 1234 --datadirectory tdata --list-fingerprint --quiet (new signing key with default expiry created) attempt to change (reduce) expiry: tor --PublishServerDescriptor 0 --orport 1234 --datadirectory tdata --SigningKeyLifetime "1 week" --list-fingerprint --quiet expected result: key lifetime is reduced to 7 days actual result: key lifetime is not changed (remains at 1 month) (invoking tor with --keygen causes the expected lifetime but can not be run non-interactively if keys do not exist) So I reopened [2]. [1] https://lists.torproject.org/pipermail/tor-dev/2015-November/009959.html [2] https://trac.torproject.org/projects/tor/ticket/17127

2 4

Special handling of .onion domains in Chrome/Firefox (post-IETF-standarization)
by George Kadianakis 27 Nov '15

27 Nov '15

Hello, as you might know, the IETF recently decided to formally recognize .onion names as special-use domain names [0]. This means that normal browsers like Chrome and Firefox can now handle onion domains in a special manner since they know that they only correspond to Tor. How would we like those browsers to treat onions? For starters, those browsers should refuse to connect to onion domains entirely. Onions don't work on normal browsers anyway, and also this will reduce the onion leakage through the DNS system [1]. An extra measure would be to persuade those browser vendors to display some sort of message to poor people who click onions using their normal browser. For example they could display: Oops, seems like you visited an onion link. You need a special anonymous browser for this: www.torproject.org What else could we do here? And is there anyone who can lobby for the right behavior? :) Of course, we all know that that inevitably those browsers will need to bundle Tor, if they want to visit the actually secure onion Internet. But let's give them a bit more time till they realize this :) Cheers! [0]: https://blog.torproject.org/blog/landmark-hidden-services-onion-names-reser… https://www.rfc-editor.org/rfc/rfc7686.txt https://www.iana.org/assignments/special-use-domain-names/special-use-domai… [1]: https://www.petsymposium.org/2014/papers/Thomas.pdf

4 3

help
by Juan Berner 20 Nov '15

20 Nov '15

2015-11-20 17:24 GMT+01:00 <tor-dev-request(a)lists.torproject.org>: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Re: Shared random value calculation edge cases (proposal 250) > (George Kadianakis) > 2. Re: Shared random value calculation edge cases (proposal 250) > (David Goulet) > 3. Re: Shared random value calculation edge cases (proposal 250) > (David Goulet) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 20 Nov 2015 16:06:59 +0200 > From: George Kadianakis <desnacked(a)riseup.net> > To: s7r <s7r(a)sky-ip.org> > Cc: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Shared random value calculation edge cases > (proposal 250) > Message-ID: <87fv0069z0.fsf(a)riseup.net> > Content-Type: text/plain > > s7r <s7r(a)sky-ip.org> writes: > > > Hello, > > > > <snip> > > > > The idea of adding flags in the votes so each dirauth can advertise if > > it is participating (has an opinion for the <current> SR or not) is > > great and helps us build more defenses, probably make it easier in the > > future too if we decide to change anything. > > > > What if the consensus for SR calculation would define majority based > > on dirauths actually participating (and advertising so with a flag in > > the vote). Also, the participating or not participating flag should be > > used per vote/consensus and split into: > > > > a) we know current SR value for today so we vote it > > or > > we know previous SR value and we know for sure if we should follow the > > disaster protocol or not (in case we are about to vote at 01:00 UTC). > > so > > We participate in the vote for <current SR>. > > > > b) we are able to participate in this protocol run which will > > calculate the SR value for next day (after 00:00 UTC) so we send our > > commits/reveals. > > > > This is useful in case we are a dirauth that joined at 00:30 UTC and > > we couldn't get the _latest_ consensus (to find out if the 00:00 UTC > > consensus was created, and if not, previous SR value so we can follow > > the disaster procedure) we will not have an opinion for the <current> > > SR value at 01:00 UTC, but we can start participating in the protocol > > run for the next day - send our commit values. Once we decided on a > > <current> SR value for that day we save it and vote normally next time. > > > > So, if we have 5 dirauths running/signing consensus in total, out of > > which only 4 participate in the shared randomness protocol, the 4 > > participating ones should be able to create a valid consensus > > themselves with the insurance that the 5th one won't break consensus. > > > > One way to do this is: the dirauth which is not participating will > > take the SR value voted by the majority of the participating dirauths > > and include that in its consensus and sign. We need at least 3 > > dirauths agreeing on a SR value in order to accept it. > > > > Is this crazy? It shouldn't open the door new attacks, since this > > doesn't allow a single actor to game it, only the majority could game it. > > > > Thanks for the suggestions. > > Let me try to suggest a procedure here based on your ideas and some other > ideas. > > [Notation: SRV = shared random value] > > The goal here is to minimize the edge cases during SRV calculation and > disaster > SRV calculation. The edge cases here appear because there is no clear view > on > whether other dirauths know the current or previous SRVs, or whether the > SRV for > this period was ever created. The disaster recovery scenario is especially > annoying here. > > Here are some edge cases for example: > > * Dirauth boots up at 02:00UTC and the 01:00UTC consensus does not contain > any > SR information (maybe because not enough SR-enabled dirauths > participated at > that time). > > Should the dirauth do the disaster recovery procedure, or just play it > cool > and put no SR information on the consensus? If it has to do disaster > recovery, > then what previous SRV does it use (the 01:00UTC consensus did not > contain > such info)? > > This type of edge case is my main concern, since with dirauths upgrading > and > going offline at random times, it's likely that we will eventually > create a > consensus without SR info in the middle of the protocol run. > > * Dirauth boots up at 23:55UTC without having a previous consensus. It is > supposed to vote and form a 00:00UTC consensus without knowing any > previous > SRVs. How does it figure out whether all the other dirauths are also > bootstrapping, or whether the other dirauths actually know the previous > SRVs? > > Here are some prerequisites for the logic I'm going to suggest. The two > first > suggestions are useful in any case, I think: > > - First of all, we treat consensuses as trusted, so dirauths MUST learn > previous/current SRVs they didn't know about from any consensus they > fetch. > > - We are also going to assume that we have some sort of "SR flag" on votes > to > denote whether the dirauth participates in the protocol. > > - We are introducing another 'status' for shared random values (see > [SRVOTE]). > > Specifically, if a dirauth witnessed a 00:00UTC consensus failing to be > created, or it did not contain any SRV information, then it sets the > status of > "shared-rand-current-value" in its votes to "none". > > Now we are going to use the SRV lines in the votes as an indicator on how > the > consensus creation should play out. > > > ================================================================================ > > Here is some logic for the consensus at 00:00UTC: > > if majority of votes have disabled the SR flag: > then don't write anything SR-related to consensus > exit > > If the majority of votes contain the previous SRV value: > then calculate SRV as detailed in section [SRCALC] > else: > then calculate SRV as in [SRCALC] but with previous_SRV set > to 0 > > And here is some logic for all the other consensuses (to figure out when > dirauths should perform the disaster procedure): > > if majority of votes have disabled the SR flag: > then don't write anything SR-related to consensus > exit > > if we know the current SRV: > then write it on the consensus > exit > > if the majority of votes have the current SRV status as "none" _AND_ > those votes also contain the previous SRV value: > then do the disaster SRV procedure > else: > then don't write anything SR-related to consensus > > > ================================================================================ > > This _might_ work for fixing a good bunch of edge cases. But is it far too > complex? > > Should we just assume that these things will never happen on the real > network > and avoid baking additional complexity? What do you think? :/ > > > > > ------------------------------ > > Message: 2 > Date: Fri, 20 Nov 2015 10:19:07 -0500 > From: David Goulet <dgoulet(a)ev0ke.net> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Shared random value calculation edge cases > (proposal 250) > Message-ID: <20151120151907.GD10269@raoul> > Content-Type: text/plain; charset="us-ascii" > > On 20 Nov (16:06:59), George Kadianakis wrote: > > s7r <s7r(a)sky-ip.org> writes: > > > > > Hello, > > > > > > <snip> > > > > > > The idea of adding flags in the votes so each dirauth can advertise if > > > it is participating (has an opinion for the <current> SR or not) is > > > great and helps us build more defenses, probably make it easier in the > > > future too if we decide to change anything. > > > > > > What if the consensus for SR calculation would define majority based > > > on dirauths actually participating (and advertising so with a flag in > > > the vote). Also, the participating or not participating flag should be > > > used per vote/consensus and split into: > > > > > > a) we know current SR value for today so we vote it > > > or > > > we know previous SR value and we know for sure if we should follow the > > > disaster protocol or not (in case we are about to vote at 01:00 UTC). > > > so > > > We participate in the vote for <current SR>. > > > > > > b) we are able to participate in this protocol run which will > > > calculate the SR value for next day (after 00:00 UTC) so we send our > > > commits/reveals. > > > > > > This is useful in case we are a dirauth that joined at 00:30 UTC and > > > we couldn't get the _latest_ consensus (to find out if the 00:00 UTC > > > consensus was created, and if not, previous SR value so we can follow > > > the disaster procedure) we will not have an opinion for the <current> > > > SR value at 01:00 UTC, but we can start participating in the protocol > > > run for the next day - send our commit values. Once we decided on a > > > <current> SR value for that day we save it and vote normally next time. > > > > > > So, if we have 5 dirauths running/signing consensus in total, out of > > > which only 4 participate in the shared randomness protocol, the 4 > > > participating ones should be able to create a valid consensus > > > themselves with the insurance that the 5th one won't break consensus. > > > > > > One way to do this is: the dirauth which is not participating will > > > take the SR value voted by the majority of the participating dirauths > > > and include that in its consensus and sign. We need at least 3 > > > dirauths agreeing on a SR value in order to accept it. > > > > > > Is this crazy? It shouldn't open the door new attacks, since this > > > doesn't allow a single actor to game it, only the majority could game > it. > > > > > > > Thanks for the suggestions. > > > > Let me try to suggest a procedure here based on your ideas and some > other ideas. > > > > [Notation: SRV = shared random value] > > > > The goal here is to minimize the edge cases during SRV calculation and > disaster > > SRV calculation. The edge cases here appear because there is no clear > view on > > whether other dirauths know the current or previous SRVs, or whether the > SRV for > > this period was ever created. The disaster recovery scenario is > especially > > annoying here. > > > > Here are some edge cases for example: > > > > * Dirauth boots up at 02:00UTC and the 01:00UTC consensus does not > contain any > > SR information (maybe because not enough SR-enabled dirauths > participated at > > that time). > > > > Should the dirauth do the disaster recovery procedure, or just play it > cool > > and put no SR information on the consensus? If it has to do disaster > recovery, > > then what previous SRV does it use (the 01:00UTC consensus did not > contain > > such info)? > > > > This type of edge case is my main concern, since with dirauths > upgrading and > > going offline at random times, it's likely that we will eventually > create a > > consensus without SR info in the middle of the protocol run. > > > > * Dirauth boots up at 23:55UTC without having a previous consensus. It is > > supposed to vote and form a 00:00UTC consensus without knowing any > previous > > SRVs. How does it figure out whether all the other dirauths are also > > bootstrapping, or whether the other dirauths actually know the > previous SRVs? > > > > Here are some prerequisites for the logic I'm going to suggest. The two > first > > suggestions are useful in any case, I think: > > > > - First of all, we treat consensuses as trusted, so dirauths MUST learn > > previous/current SRVs they didn't know about from any consensus they > fetch. > > Yes ++. > > A dirauth booting up should always try to learn the current and previous > SRV from the latest consensus (< 24h) and update the disk state > accordingly _even_ if the disk state has SRV values in there. We should > trust the consensus voted by majority before our disk state imo. > > > > > - We are also going to assume that we have some sort of "SR flag" on > votes to > > denote whether the dirauth participates in the protocol. > > The requirements for having that flag "on" or "off" eludes me a bit. I > assume that this SR flag is "on" when a dirauth thinks its able to > compute a valid SRV that is I have enough commits. We can't make "having > the previous SRV" a requirement else we will never bootstrap properly > ever since in the first runs, previous SRV is alway 0. > > > > > - We are introducing another 'status' for shared random values (see > [SRVOTE]). > > > > Specifically, if a dirauth witnessed a 00:00UTC consensus failing to be > > created, or it did not contain any SRV information, then it sets the > status of > > "shared-rand-current-value" in its votes to "none". > > Hrm... I'm wondering here if "none" status is instead the same of having > the dirauth putting the disaster value in the vote? > > I would argue that a dirauth should _always_ try to compute a SR value > at any time (even when booting). Between having no line because we > couldn't compute due to lack of commits or a line that is the disaster, > I think the former simplifies things that is "We always have a SRV value > at 00:00 from a dirauth". > > Here is what I propose: > > 1) Like you said, we always try to learn SRV values from previous > consensus. If we can't get one from a consensus or our state, compute > disaster at 00:00. > > 2) We do _not_ use a SR flag for vote, a dirauth always tries to compute > a current SRV value at 00:00. If it's 13:00, it simply doesn't put > anything in the vote since no SRV data. > > The big issue I see here is when a dirauth does _not_ have the previous > SRV. Without it, both disaster and current SRV computation will end up > not matching the majority. So let's explore this binary state: > > - I have the previous SRV (either I can get it from my state or > consensus). > > This seems a non issue, dirauth will use it, carry it in the vote and > compute the current SRV if enough commits have been seen. At 00:00, if > majority wasn't reached for the current SRV value, we have a consensus > without it which is a valid use case but we at least have the previous > one in the consensus flagged as non-fresh so it can be used for > disaster or SRV computation at the next protocol run. > > - I don't have a previous SRV (couldn't get it in the latest consensus > and I have _no_ disk state). > > For instance, if this happens at 23:30, a dirauth won't have the > chance to participate in the SR protocol because it didn't see any > commits to compute SRV value thus disaster mode engage. > > If that dirauth was the 5th one out of 9 (meaning the majority > breaker), we end up with no SRV values at all at 00:00 (not even the > previous SRV since only 4/9 had it). So a new protocol run starts, > we'll end up with a valid current SRV in 24 hours and both values in > 48 hours. Essentially, we end up back in bootstrap mode. > > IMPORTANT note here: I think a dirauth MUST NOT keep any SRV values > that aren't in the 00:00 consensus. > > Question is, how big of a deal it is to have this issue where if we > don't reach majority because a dirauth rebooted and for some magical > reason didn't get the latest consensus to at least get the previous > SRV, we return in bootstrap? > > Cheers! > David > > > > > Now we are going to use the SRV lines in the votes as an indicator on > how the > > consensus creation should play out. > > > > > ================================================================================ > > > > Here is some logic for the consensus at 00:00UTC: > > > > if majority of votes have disabled the SR flag: > > then don't write anything SR-related to consensus > > exit > > > > If the majority of votes contain the previous SRV value: > > then calculate SRV as detailed in section [SRCALC] > > else: > > then calculate SRV as in [SRCALC] but with previous_SRV > set to 0 > > > > And here is some logic for all the other consensuses (to figure out when > > dirauths should perform the disaster procedure): > > > > if majority of votes have disabled the SR flag: > > then don't write anything SR-related to consensus > > exit > > > > if we know the current SRV: > > then write it on the consensus > > exit > > > > if the majority of votes have the current SRV status as "none" _AND_ > > those votes also contain the previous SRV value: > > then do the disaster SRV procedure > > else: > > then don't write anything SR-related to consensus > > > > > ================================================================================ > > > > This _might_ work for fixing a good bunch of edge cases. But is it far > too complex? > > > > Should we just assume that these things will never happen on the real > network > > and avoid baking additional complexity? What do you think? :/ > > > > > > _______________________________________________ > > tor-dev mailing list > > tor-dev(a)lists.torproject.org > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >

1 0

DoS resistance for Next-Generation Onion Services
by teor 20 Nov '15

20 Nov '15

Hi all, I think we can make next-generation onion (hidden) services (proposal #224) more resilient against certain kinds of DoS / client discovery attacks, by using a different blinded public key for each HSDir. Attack Summary: Once a malicious HSDir receives a descriptor, it can locate other HSDirs with that descriptor's blinded public key. It doesn't need to know the onion service address or decrypt the descriptor. Each descriptor contains the blinded public key: > The 'certificate' field contains a certificate in the format from proposal 220, with the short-term ed25519 descriptor-signing key signed by the blinded public key. It must contain a ed25519-signing-key extension containing the blinded public key. (All quotes are from https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.t… <https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.t…> ) This allows an attacker to target onion services for DoS, as long as they don't care which onion service they're targeting. The effects of this attack could be: * if the malicious HSDir also stops serving the descriptor, it denies service to some onion service, * if the malicious HSDir continues to serve the descriptor, it becomes the only source of descriptors for some onion service. I think this enables traffic confirmation attacks on some onion service or its clients (by bringing the sole descriptor source up and down). Client descriptor downloads could also be subject to traffic fingerprinting or slow (byte-at-a-time) delivery. Attack Details: Let's assume an attacker who: * doesn't know any onion service addresses * runs a malicious HSDir that collects blinded public keys * can check other HSDirs for descriptors based on blinded public keys * wants to DoS some onion services, but doesn't care which ones This attacker then: 1. chooses an arbitrary blinded public key from their HSDir's collection 2. calculates possible HSDirs for the blinded public key 3. checks each of the possible HSDirs to see if they actually have a descriptor with that blinded public key 4. runs a DoS on each HSDir with that blinded public key (this is the resource-intensive step) A refinement of the attack is to: A. find the HSDirs for each blinded public key uploaded to the malicious HSDir (steps 1-3) B. find the blinded public keys with the most vulnerable HSDirs (perhaps using the maximum consensus weight among all 5) C. DoS the blinded public keys with the most vulnerable HSDirs (step 4) I think this is a plausible attack that could be run repeatedly until the attacker happens to hit a high-value onion service, or a high enough number of onion services. The targeted onion services would change blinded keys and HSDirs with the next shared random period, but they'd be down for up to 24 hours. Mitigation Summary: If each onion service uploads a descriptor with a different blinded public key to each HSDir, a malicious HSDir is unable to locate other HSDirs serving the same onion service. Even with this change, an attacker who knows an onion service's address can still find all its HSDirs. They can set up a malicious HSDir and wait unit it happens to get chosen as the HSDir for that service. But this targeted attack requires more resources and time than the unknown target attack I describe above. The unknown target attack can also consider hundreds of blinded public keys, and target the ones with the weakest HSDirs. The targeted attack is unlikely to host a particular descriptor on a malicious HSDir, and also simultaneously find that all the other HSDirs for that descriptor are vulnerable to a DoS. Alternative Mitigations: The current keyblinding nonce is based on the period number and start time. This allows the blinded keys to be generated in advance. But the drawback is that the blinded key is the same for each HSDir. > There is a master keypair (sk, pk). Given the keypair and a nonce n, there is a derivation function that gives a new blinded keypair (sk_n, pk_n). This keypair can be used for signing. Given only the public key and the nonce, there is a function that gives pk_n. > … > > We propose the following scheme for key blinding, based on Ed25519. > > (This is an ECC group, so remember that scalar multiplication is the > trapdoor function, and it's defined in terms of iterated point > addition. See the Ed25519 paper [Reference ED25519-REFS] for a fairly > clear writeup.) > > Let the basepoint be written as B. Assume B has prime order l, so > lB=0. Let a master keypair be written as (a,A), where a is the private > key and A is the public key (A=aB). > > To derive the key for a nonce N and an optional secret s, compute the > blinding factor h as H(A | s, B, N), and let: > > private key for the period: a' = h a > public key for the period: A' = h A = (ha)B > > Generating a signature of M: given a deterministic random-looking r (see EdDSA paper), take R=rB, S=r+hash(R,A',M)ah mod l. Send signature (R,S) and public key A'. > > Verifying the signature: Check whether SB = R+hash(R,A',M)A'. > > (If the signature is valid, > SB = (r + hash(R,A',M)ah)B > = rB + (hash(R,A',M)ah)B > = R + hash(R,A',M)A' ) > … > (To use this with Tor, set N = INT_8(period-number) | INT_8(Start of period in seconds since epoch).) Mitigation #1: Mitigation #1 generates a blinded key for every replica. (This means the attacker can only target a single replica of the descriptor, which they can likely do anyway, because the "spread_store" HSDirs are in order on the hash ring.) But it looks like this can't be done in advance, because the number of replicas is a consensus parameter. > The following consensus parameters control where a hidden service descriptor is stored; > hsdir_n_replicas = an integer in range [1,16] with default value 2. > hsdir_spread_fetch = an integer in range [1,128] with default value 3. > hsdir_spread_store = an integer in range [1,128] with default value 3. > hsdir_spread_accept = an integer in range [1,128] with default value 8 But we can instead generate a blinded key for every replica mod default_hsdir_n_replicas. Then, if the number of replicas in the consensus increases, the attack can only target 1/default_hsdir_n_replicas of the replicas. It simply doesn't know where the other replicas are, because they have different blinded keys. The spec change for Mitigation #1 would look like: > (To use this with Tor, create a key, blinded_public_key(advance_replicanum) for each N = INT_8(period-number) | INT_8(Start of period in seconds since epoch) | INT_8(advance_replicanum), where advance_replicanum takes every value from 1 to default_hsdir_n_replicas. Each key is used for the replicas where replicanum mod default_hsdir_n_replicas is equal to advance_replicanum.) The blinded key is then used by both the service and clients to determine the descriptor's hashring index: > To determine where a given hidden service descriptor will be stored in a given period, after the blinded public key for that period is derived, the uploading or downloading party calculate > for replicanum in 1...hsdir_n_replicas: > hs_index(replicanum) = H("store-at-idx" | > blinded_public_key(replicanum mod default_hsdir_n_replicas) | > INT_8(replicanum) | > INT_8(periodnum) ) For Mitigation #1, including advance_replicanum in the keyblinding does NOT remove the need for replicanum in the hs_index(replicanum) hash. (advance_replicanum could be lower than replicanum if the hsdir_n_replicas consensus parameter is increased from the default.) There's also a slight clarification to the spread upload wording: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors for the corresponding replica's blinded signing key to the first hsdir_spread_store nodes whose indices immediately follow hs_index(replicanum). In summary, by creating a blinded public key for each replica, Mitigation #1 restricts the attack described to a single descriptor replica per malicious HSDir. This significantly increases the resources or knowledge required for the attack. Mitigation #2: Mitigation #2 generates a blinded key for every spread. (This means the attacker can only target a single spread for the descriptor.) Spread is specified like this in the existing proposal: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors to the first hsdir_spread_store nodes whose indices immediately follow hs_index(replicanum). A similar process applies for advance generation of blinded public keys: > (To use this with Tor, create a key blinded_public_key(advance_spreadnum) for each N = INT_8(period-number) | INT_8(Start of period in seconds since epoch) | INT_8(advance_spreadnum), where advance_spreadnum takes every value from 1 to default_hsdir_spread_store. Each key is used for the spreads where spreadnum mod default_hsdir_spread_store is equal to advance_spreadnum.) Mitigation #2 changes the way the spread indexes are positioned on the hashring. Because the blinded key for each advance_spreadnum is different, the spreads are distributed around the hashring, rather than appearing on it sequentially. To maintain this distribution if the consensus value of hsdir_spread_store is increased, we have two options: * spreadnum can be included in the descriptor index hash (Mitigation #2A), or * the spread position need only be incremented once for each advance_spreadnum spreads (Mitigation #2B). Mitigation #2A: spreadnum can be included in the descriptor index hash: > To determine where a given hidden service descriptor will be stored in a given period, after the blinded public key for that period is derived, the uploading or downloading party calculate > for replicanum in 1...hsdir_n_replicas: > for spreadnum in 1...hsdir_spread_store: > hs_index(replicanum, spreadnum) = H("store-at-idx" | > blinded_public_key(spreadnum mod default_hsdir_spread_store) | > INT_8(replicanum) | > INT_8(spreadnum) | > INT_8(periodnum) ) There's also a slight clarification to the spread upload wording: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors for the corresponding spread's blinded signing key to the first node whose index immediately follows hs_index(replicanum, spreadnum). This uploads to hsdir_spread_store nodes per replica. Mitigation #2B: The spread position need only be incremented once for each advance_spreadnum spreads: > Finally, for replicanum in 1...hsdir_n_replicas, the hidden service host uploads descriptors to hsdir_spread_store nodes per replica as follows: > the descriptor for the corresponding spreadnum's blinded signing key is uploaded to the first node whose index immediately follows hs_index(replicanum, spreadnum). If any spreads would upload to the same node, they instead upload to the second and subsequent nodes whose indices immediately follows the hs_index. But spreadnum is NOT included in the descriptor index hash: > To determine where a given hidden service descriptor will be stored in a given period, after the blinded public key for that period is derived, the uploading or downloading party calculate > for replicanum in 1...hsdir_n_replicas: > hs_index(replicanum, spreadnum) = H("store-at-idx" | > blinded_public_key(spreadnum mod default_hsdir_spread_store) | > INT_8(replicanum) | > INT_8(periodnum) ) In summary, by creating a blinded public key for each replica, Mitigation #2 (both variants) restricts the attack described to a single descriptor spread per malicious HSDir. This significantly increases the resources or knowledge required for the attack. Mitigation #3: Alternately, we could double blind each public key: once when it is generated from the master key in advance, and again just before it is uploaded to the HSDir. This double-blinding should use two nonces, P and Q: * P is the advance-blinding nonce based on the period * Q is the post-blinding nonce based on the replica and/or spread (I considered and rejected a design where Q is based on the fingerprint of the HSDir. This would lead to blinding with an externally-supplied fingerprint, potentially chosen by an attacker to reveal information on blinding.) The blinding would look something like this, where H(Q, H(P, …)) replaces H(N, …): > There is a master keypair (sk, pk). Given the keypair and a nonce n, there is a derivation function that gives a new blinded keypair (sk_n, pk_n). This keypair can be used for signing. Given only the public key and the nonce, there is a function that gives pk_n. > … > > We propose the following scheme for key blinding, based on Ed25519. > > (This is an ECC group, so remember that scalar multiplication is the trapdoor function, and it's defined in terms of iterated point addition. See the Ed25519 paper [Reference ED25519-REFS] for a fairly clear writeup.) > > Let the basepoint be written as B. Assume B has prime order l, so lB=0. Let a master keypair be written as (a,A), where a is the private key and A is the public key (A=aB). To derive the key for a advance-blinding nonce P, an optional secret s, and a post-blinding nonce Q: > compute the advance-blinding factor h as H(A | s, B, P), and let: > > advance-blinded private key for the period: a' = h a > advance-blinded public key for the period: A' = h A = (ha)B > > compute the post-blinding factor j as H(A' | 0, B, Q), and let: > > post-blinded private key for the period: a'' = j a' = j (ha) > post-blinded public key for the period: A'' = j A' = j ((ha)B) > > Generating a signature of M: given a deterministic random-looking r (see EdDSA paper), take R=rB, S=r+hash(R,A'',M)ahj mod l. Send signature (R,S) and public key A''. > > Verifying the signature: Check whether SB = R+hash(R,A'',M)A''. > > (If the signature is valid, > SB = (r + hash(R,A'',M)ahj)B > = rB + (hash(R,A'',M)ahj)B > = R + hash(R,A'',M)A'' ) > ... > (To use this with Tor, set P = INT_8(period-number) | INT_8(Start of period in seconds since epoch) and Q = INT_8(replicanum) and/or INT_8(spreadnum). Other Considerations: Combining Mitigations: Mitigations #1 and #2 (either variant) can be combined, but this means generating 6 blinded keys per period, rather than 1. I don't think the extra benefit gained from implementing both outweighs the complexity involved. Issue with Per-Spread Keys: One drawback of Mitigations #2 and #3 (different blinding keys for each spread) is that the client may need to request multiple blinded keys from each HSDir due to relays dropping out of the hash ring. For example: 1. An onion service uploads descriptors with keys A, B, C to spread servers a, b, c respectively 2. Spread server a goes down 3. the client wants the descriptor for the onion service, and tries spread servers b, c, d The client doesn't know how many servers have gone down. Therefore, it will need to ask for A, B, C from the first server (actually b, which has B); B, C from the second server (actually c, which has C); and C from the third server (actually d, which has nothing). I don't like the extra load involved in this, and I wonder about the potential for information leaks to either the HSDir, or an observer of the client's connection. (For example, each of the traffic patterns above is distinct, so an observer of the client could tell which spread number it used for the descriptor.) Security Proofs: The modified security proof in Mitigation #3 would need to check out for me to be comfortable with this option. Conclusion: Can we consider Mitigation #1: creating a different blinded public key for each replica? This would double the number of keys we generate in advance, one for each replica. Alternately, we could use Mitigation #3: double-blinding each key with the period in advance, then the replica number just before upload. But we'd need to check the modified security proof carefully. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

3 4