- tor-dev - lists.torproject.org

BridgeDB 0.3.4 is released
by isis 01 Dec '15

01 Dec '15

Hey, I've just tagged and released BridgeDB 0.3.4. For those interested, here are the changes (c.f. https://gitweb.torproject.org/bridgedb.git/tree/CHANGELOG) Changes in version 0.3.4 - 0.3.5 - 2015-11-30 * FIXES #14685 https://bugs.torproject.org/14685 This disables distribution of obfs2 bridges. This pluggable transport has known distiguishers which allow adversaries to identify client connections to obfs2 bridges, which in turn allows these connections to be blocked/censored. With numerous obfs3 and obfs4 bridges both readily available, users should not be presented with an easily-configurable choice that is known to be unsafe for the majority of users. And includes the following general changes: * ADDS error pages to BridgeDB's web interface, to provide friendlier explanations for downtime, missing pages, and internal server errors. For example: https://bridges.torproject.org/404 -- ♥Ⓐ isis agora lovecruft _________________________________________________________ OpenPGP: 4096R/0A6A58A14B5946ABDE18E207A3ADB67A2CDB8B35 Current Keys: https://blog.patternsinthevoid.net/isis.txt

1 0

[Applications Team] Meeting next Tuesday 19:00 UTC
by Georg Koppen 01 Dec '15

01 Dec '15

Hi, just a reminder that the applications team is having its monthly meeting next week on Tuesday, December 1 at 19:00 UTC in #tor-project on irc.oftc.net. There is no proposed topic for this meeting yet. Please come up with suggestions if you feel there is something we should discuss. See you there, Georg

2 1

BridgeDB 0.3.3 is released
by isis 01 Dec '15

01 Dec '15

Hello! I released BridgeDB 0.3.3 a few weeks ago, and deployed it on the production server. However, I completely forgot to email the list to notify you all of the changes. Oops, sorry! For those who are curious, BridgeDB-0.3.3 brought about the following changes (and, as always, the current changelog is available at https://gitweb.torproject.org/bridgedb.git/tree/CHANGELOG) Changes in version 0.3.3 - 2015-10-25 * FIXES #12029 https://bugs.torproject.org/12029 BridgeDB now has an API for creating Bridge Distributors. See the bridgedb.distribute module, or its developer documentation at https://pythonhosted.org/bridgedb/bridgedb.distribute.html. * FIXES PART OF #12506 https://bugs.torproject.org/12506 BridgeDB's two Distributors (HTTPS and Email) are now entirely modularised and self-contained within separate subdirectories in the source code. This is the first step to redesigning these Distributors into their own separate processes, which will allow the Distributors to remain functional while BridgeDB is reparsing bridge descriptors. * FIXES #15968 https://bugs.torproject.org/15968 BridgeDB now sends a Content-Security-Policy header which explicitly allows Javascript, images, CSS, and fonts, from https://bridges.torproject.org. All other types of content are forbidden, including: - embedding https://bridges.torproject.org within <iframe>, <embed>, or <object>, and attempting to source additional resources into its embedded context - inline Javascript, including Javascript within SVG files - inline CSS - externally hosted fonts - inline SVG, e.g. via the HTML5 <svg> tag - any and all connections made via Javascript XMLHttpRequests, WebSockets, sendBeacon(), and Web Workers - plugins - applets BridgeDB's Content-Security-Policy does not yet make use of certain newer, lesser supported, Content-Security-Policy v2.0 directives, such as "reflected-xss" and "frame-ancestors", but may someday. * FIXES #16273 https://bugs.torproject.org/16273 Several links to Tor Project gitweb URLs within the developer documentation were outdated in that they still used the old gitweb URL format. These are now updated. Thanks to David Fifield for the bug report and patches. * FIXES #16330 https://bugs.torproject.org/16330 BridgeDB can now handle bridge-server-descriptors with extra-info-digest fields which have two values, as well as both bridge-server-descriptors and bridge-extrainfo descriptors which contain Ed25519 key material and signatures. See Tor proposals #220 and #228 for more information on the changes to these descriptors. Note that BridgeDB can now parse this information, but does not yet make use of any Ed25519 cryptographic material within bridge descriptors. https://gitweb.torproject.org/torspec.git/tree/proposals/220-ecc-id-keys.txt https://gitweb.torproject.org/torspec.git/tree/proposals/228-cross-certific… Thanks to Atagar for patching Stem. * FIXES #16616 https://bugs.torproject.org/16616 The HSDir flag can now be included within bridge-networkstatus documents. BridgeDB now has unittests which guarantee that its parsers safely ignore this flag, as well as any flags unknown to BridgeDB which may appear in the future. Thanks to Roger Dingledine for alerting me about the change. * FIXES #16649 https://bugs.torproject.org/16649 Mobile users, and other users with small screen pixel ratios, will find that the UI of BridgeDB's HTTPS Distributor has greatly increased in usability and readability. And includes the following general changes: * FIXES an error when requesting the non-HTML version of the bridges page (e.g. https://bridges.torproject.org/bridges?format=plain) * REMOVES the `bridgedb test` commandline option. BridgeDB's tests can be run via `python setup.py test` or `make test` (or `make coverage` for generating HTML test coverage statistics). * CHANGES the HTTPS Distributor to HTML-encode Bridge Lines. Previously, a malicious Pluggable Transport Bridge could include in its PT arguments something like "evil=<script>[…]</script>" and if such a Bridge were to be distributed to a user, that user's web browser would execute the script (if Javacript was enabled). Other characters, including non-ASCII, control characters, double quotes, and backslashes, are also sanitised from Bridge Lines. Thanks to Robert Ransom for the patches. * CHANGES BridgeDB's module/package version numbers to be compliant with PEP440. * CHANGES the layout of BridgeDB's source code directories. Rather than storing BridgeDB's source in "lib/bridgedb/", it is now kept in "bridgedb/". Similarly, the directory containing BridgeDB's tests has been moved from "lib/bridgedb/test/" to "test/", which means that the tests are no longer installed when running `python setup.py install` or `make install`. * ADDS several improvements to the developer documentation at https://pythonhosted.org/bridgedb. * UPDATE English (en_US) translations. * UPDATE English (en) translations. * ADD Serbian (sr) translations. Thanks to obj.petit.a, Ivan Radeljic, and Milenko Doder. * UPDATE Arabic (ar) translations. Thanks to A. Hassan, debo debo, KACIMI LAMINE, and Nudroid A. * UPDATE Catalan (ca) translations. Thanks to laia_. * UPDATE Czech (cs) translations. Thanks to Tomas Palik and Vlastimil Burián. * UPDATE Danish (da) translations. Thanks to Mogelbjerg. * UPDATE German (de) translations. Thanks to jschfr, Junge Limba, and Toralf Förster. * UPDATE English (en_GB) translations. Thanks to Andi Chandler. * UPDATE Farsi (fa) translations. Thanks to some awesome anonymous person for helping out. * UPDATE Finish (fi) translations. Thanks to Riku Viitanen. * UPDATE French (fr) translations. Thanks to elouann, Trans-fr, and Towinet. * UPDATE French (fr_CA) translations. Thanks to Trans-fr. * UPDATE Croatian (hr_HR) translations. Thanks to some awesome anonymous person for helping out. * UPDATE Hungarian (hu) translations. Thanks to some awesome anonymous person for helping out. * UPDATE Indonesian (id) translations. Thanks to Anthony Santana, Astryd Viandila Dahlan, cholif yulian, constantius damar wicaksono, Dwi Cahyono, L1Nus, km242saya, and Zamani Karmana. * UPDATE Italian (it) translations. Thanks to Random_R. * UPDATE Japanese (ja) translations. Thanks to ABE Tsunehiko. * UPDATE Latvian (lv) translations. Thanks to Ojārs Balcers. * UPDATE Norwegian Bokmål (nb) translations. Thanks to Erik Matson and Kristian Andre Henriksen. * UPDATE Dutch (nl) translations. Thanks to Mart3000. * UPDATE Polish (pl) translations. Thanks to Karol Obartuch. * UPDATE Portuguese (pt) translations. Thanks to Bruno D. Rodrigues and MMSRS. * UPDATE Brazillian Portuguese (pt_BR) translations. Thanks to Communia. * UPDATE Romanian (ro) translations. Thanks to Ana, axel_89, and Di N. * UPDATE Russian (ru) translations. Thanks to Ivan. * UPDATE Slovak (sk_SK) translations. Thanks to StefanH. * UPDATE Albanian (sq) translations. Thanks to some awesome unknown anonymous person who didn't add their name to the list of translators. * UPDATE Swedish (sv) translations. Thanks to Peter Michanek. * UPDATE Turkish (tr) translations. Thanks to Bullgeschichte and Fomas. * UPDATE Ukranian (uk) translations. Thanks to Yasha. * UPDATE Chinese Mandarin (zh_CN) translations. Thanks to khi. * UPDATE Taiwanese Mandarin (zh_TW) translations. Thanks to x4r. -- ♥Ⓐ isis agora lovecruft _________________________________________________________ OpenPGP: 4096R/0A6A58A14B5946ABDE18E207A3ADB67A2CDB8B35 Current Keys: https://blog.patternsinthevoid.net/isis.txt

1 0

Summary of meek's costs, October 2015
by David Fifield 29 Nov '15

29 Nov '15

Here's the summary of meek's CDN fees for October 2015. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 January 2015 $464.37 + $669.02 + $0.00 = $1133.39 February 2015 $650.53 + $604.83 + $0.00 = $1255.36 March 2015 $690.29 + $815.68 + $0.00 = $1505.97 April 2015 $886.43 + $785.37 + $0.00 = $1671.80 May 2015 $871.64 + $896.39 + $0.00 = $1768.03 June 2015 $601.83 + $820.00 + $0.00 = $1421.83 July 2015 $732.01 + $837.08 + $0.00 = $1569.09 August 2015 $656.76 + $819.59 + $154.89 = $1631.24 September 2015 $617.08 + $710.75 + $490.58 = $1818.41 October 2015 $672.01 + $110.72 + $300.64 = $1083.37 -- total by CDN $7443.58 + $7987.32 + $946.11 = $16377.01 grand total https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… In October we had about 4,000 simultaneous users all month. There was an unfortunate outage of meek-amazon (not the result of censorship, just operations failure). Between 30 September and 9 October the bridge had an expired HTTPS certificate. [tor-talk] Outage of meek-amazon https://lists.torproject.org/pipermail/tor-talk/2015-October/039231.html https://lists.torproject.org/pipermail/tor-talk/2015-October/039234.html And then, as a side effect of installing a new certificate, the bridge's fingerprint changed, which caused Tor Browser to refuse to connect. It used to be that we didn't include fingerprints for the meek bridges, but now we do, so we didn't anticipate this error and didn't notice it quickly. Update the meek-amazon fingerprint to B9E7141C594AF25699E0079C1F0146F409495296 https://trac.torproject.org/projects/tor/ticket/17473 [tor-talk] Changed fingerprint for meek-amazon bridge (attn support) https://lists.torproject.org/pipermail/tor-talk/2015-November/039397.html Interestingly, the meek-amazon bridge still had about 400 simultaneous users (not as much as normal) during the time when the fingerprint didn't match. I would have expected it to go almost to zero. Maybe it's people using an old version of Tor Browser (from before March 2015) or some non–Tor Browser installation. Our grant for meek-azure ran out and now it costs money. Accordingly I've rate-limited it to limit costs. I set it to 1.1 MB/s on 2 October and to 0.8 MB/s on 30 October. [tor-talk] meek-azure now rate-limited https://lists.torproject.org/pipermail/tor-talk/2015-October/039169.html meek-google was also out for a few days around 30 October because I messed up the app upload and pointed it to the wrong bridge. If you want to set up your own bridge and CDN instance without rate limiting, I can help you do it. Here are some docs to look at: https://trac.torproject.org/projects/tor/wiki/doc/meek#Howtorunameek-server… https://trac.torproject.org/projects/tor/wiki/doc/meek#GoogleAppEngine https://trac.torproject.org/projects/tor/wiki/doc/meek#AmazonCloudFront https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure == App Engine a.k.a. meek-google == Here is how the Google costs broke down: 2842 GB $341.06 6619 instance hours $330.95 Compared to the previous month: 2871 GB $344.53 5451 instance hours $272.55 https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… == Amazon a.k.a. meek-amazon == Usage of meek-amazon was quite low this month because of bridge outages having to do with an expired HTTPS certificate and a changed relay fingerprint. Asia Pacific (Singapore) 6M requests $8.24 36 GB $5.05 Asia Pacific (Sydney) 635K requests $0.79 1 GB $0.21 Asia Pacific (Tokyo) 1M requests $1.71 5 GB $0.71 EU (Ireland) 39M requests $46.81 130 GB $9.97 South America (Sao Paulo) 2M requests $4.98 5 GB $1.23 US East (Northern Virginia) 24M requests $24.36 86 GB $6.64 -- total 74M requests $86.89 266 GB $23.81 https://globe.torproject.org/#/bridge/F4AD82B2032EDEF6C02C5A529C42CFAFE5165… (Note new fingerprint.) == Azure a.k.a. meek-azure == Zone 1 1652 GB $202.51 Zone 2 586 GB $98.13 -- total 2238 GB $300.64 https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… Earlier reports in this series: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html https://lists.torproject.org/pipermail/tor-dev/2015-January/008082.html https://lists.torproject.org/pipermail/tor-dev/2015-February/008235.html https://lists.torproject.org/pipermail/tor-dev/2015-March/008427.html https://lists.torproject.org/pipermail/tor-dev/2015-April/008596.html https://lists.torproject.org/pipermail/tor-dev/2015-May/008767.html https://lists.torproject.org/pipermail/tor-dev/2015-June/008932.html https://lists.torproject.org/pipermail/tor-dev/2015-July/009030.html https://lists.torproject.org/pipermail/tor-dev/2015-August/009213.html https://lists.torproject.org/pipermail/tor-dev/2015-September/009533.html https://lists.torproject.org/pipermail/tor-dev/2015-October/009672.html

2 2

does renewing ed25519 signing key hurt if done to often?
by nusenu 29 Nov '15

29 Nov '15

Hi, I'm wondering if it does any bad if ed25519 online keys are renewed (to) often (i.e. on every ansible-playbook run and the admin runs it 3 times in a row to get it to complete on all hosts). If key renewal causes all existing connections to die I'll try to workaround the problem and make tor --keygen idempotent (replace the key only if its expiry changed).

1 1

tor weather warning tor ops about expiring signing keys?
by nusenu 29 Nov '15

29 Nov '15

Hi, I'm wondering if a service like a future tor weather could have an additional check to warn relay ops about key expiry: (something like "Email me when the router's signing key is about to expire") Do relays disclose the fact that they are run via OfflineMasterKey 1? Do dir auths/tor clients see when a currently used signing key is about to expire? thanks

1 0

documentation for new offline master key functionality (--keygen is undocumented)
by nusenu 29 Nov '15

29 Nov '15

Hi, since tor 0.2.7.5 is apparently not very far [1] from being released I was wondering whether there is any documentation about the new offline master key functionality? (or is this undocumented because it is not considered for general use yet?) tor v0.2.7.4-rc's manual has the following: " SigningKeyLifetime N days|weeks|months For how long should each Ed25519 signing key be valid? Tor uses a permanent master identity key that can be kept offline, and periodically generates new "signing" keys that it uses online. This option configures their lifetime. (Default: 30 days) OfflineMasterKey 0|1 If non-zero, the Tor relay will never generate or load its master secret key. Instead, you’ll have to use "tor --keygen" to manage the master secret key. (Default: 0) " but doesn't say anything about --keygen itself [2]. The 0.2.7.x mentions also a '--newpass' option that I wasn't able to find in the manpage: " - Add a new OfflineMasterKey option to tell Tor never to try loading or generating a secret Ed25519 identity key. You can use this in combination with tor --keygen to manage offline and/or encrypted Ed25519 keys. Implements ticket 16944. - Add a --newpass option to allow changing or removing the passphrase of an encrypted key with tor --keygen. Implements part of ticket 16769. - On receiving a HUP signal, check to see whether the Ed25519 signing key has changed, and reload it if so. Closes ticket 16790. " Can a tor operator use one offline master key for several relays (that are running at the same time) or is one master key required for every relay? (I assume the latter) How does the process of renewing the signing keys look like? According to the logs I assume simple run tor --keygen again and copy ed25519_signing_cert + ed25519_signing_secret_key to the relay's /keys folder the logs say: "It looks like I need to generate and sign a new medium-term signing key, because you asked me to make one with --keygen. To do that, I need to load the permanent master identity key." Does a tor operator has to SIGHUP a running tor instance after copying the new signing keys to the appropriate folder or will tor attempt to reload that file as soon as this signing key expires? How can a tor relay op display a given signing key's expiry date? Does using the offline master key functionality imply that the relay will only have an ed25519 and no RSA key? Is the offline master key limited to ed25519 keys and useless while using ed25519 + RSA keys at the same time? (because the RSA key is not offline?) thanks! [1] https://lists.torproject.org/pipermail/tor-consensus-health/2015-November/0… [2] https://trac.torproject.org/projects/tor/ticket/17583

5 13

tor ignores --SigningKeyLifetime when keys exist
by nusenu 28 Nov '15

28 Nov '15

(thread split from [1]) s7r wrote: > - - when you run tor --orport [...] just to generate the keys in a > non-interactive way, include a PublishServerDescriptor 0 in the > command as well, send the log to /dev/null and terminate the process > immediately. The descriptor will have to be published by the Tor > process actually running the relay. If the master id private key is > not encrypted, --keygen should be able to renew the medium term > signing key in a non-interactive way. But it's not a big deal if you > decide to do it with tor --orport [...] if it's easier for you this way. Turns out my workaround to generate keys without a passphrase non-interactively is not working entirely in every case since tor apparently ignores --SigningKeyLifetime (when used without --keygen) when keys exist: Signing keys are not (re)generated according to the (new) SigningKeyLifetime parameter (signing key/cert remains unchanged). reproducer: mkdir tdata tor --PublishServerDescriptor 0 --orport 1234 --datadirectory tdata --list-fingerprint --quiet (new signing key with default expiry created) attempt to change (reduce) expiry: tor --PublishServerDescriptor 0 --orport 1234 --datadirectory tdata --SigningKeyLifetime "1 week" --list-fingerprint --quiet expected result: key lifetime is reduced to 7 days actual result: key lifetime is not changed (remains at 1 month) (invoking tor with --keygen causes the expected lifetime but can not be run non-interactively if keys do not exist) So I reopened [2]. [1] https://lists.torproject.org/pipermail/tor-dev/2015-November/009959.html [2] https://trac.torproject.org/projects/tor/ticket/17127

2 4

Special handling of .onion domains in Chrome/Firefox (post-IETF-standarization)
by George Kadianakis 27 Nov '15

27 Nov '15

Hello, as you might know, the IETF recently decided to formally recognize .onion names as special-use domain names [0]. This means that normal browsers like Chrome and Firefox can now handle onion domains in a special manner since they know that they only correspond to Tor. How would we like those browsers to treat onions? For starters, those browsers should refuse to connect to onion domains entirely. Onions don't work on normal browsers anyway, and also this will reduce the onion leakage through the DNS system [1]. An extra measure would be to persuade those browser vendors to display some sort of message to poor people who click onions using their normal browser. For example they could display: Oops, seems like you visited an onion link. You need a special anonymous browser for this: www.torproject.org What else could we do here? And is there anyone who can lobby for the right behavior? :) Of course, we all know that that inevitably those browsers will need to bundle Tor, if they want to visit the actually secure onion Internet. But let's give them a bit more time till they realize this :) Cheers! [0]: https://blog.torproject.org/blog/landmark-hidden-services-onion-names-reser… https://www.rfc-editor.org/rfc/rfc7686.txt https://www.iana.org/assignments/special-use-domain-names/special-use-domai… [1]: https://www.petsymposium.org/2014/papers/Thomas.pdf

4 3

help
by Juan Berner 20 Nov '15

20 Nov '15

2015-11-20 17:24 GMT+01:00 <tor-dev-request(a)lists.torproject.org>: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Re: Shared random value calculation edge cases (proposal 250) > (George Kadianakis) > 2. Re: Shared random value calculation edge cases (proposal 250) > (David Goulet) > 3. Re: Shared random value calculation edge cases (proposal 250) > (David Goulet) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 20 Nov 2015 16:06:59 +0200 > From: George Kadianakis <desnacked(a)riseup.net> > To: s7r <s7r(a)sky-ip.org> > Cc: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Shared random value calculation edge cases > (proposal 250) > Message-ID: <87fv0069z0.fsf(a)riseup.net> > Content-Type: text/plain > > s7r <s7r(a)sky-ip.org> writes: > > > Hello, > > > > <snip> > > > > The idea of adding flags in the votes so each dirauth can advertise if > > it is participating (has an opinion for the <current> SR or not) is > > great and helps us build more defenses, probably make it easier in the > > future too if we decide to change anything. > > > > What if the consensus for SR calculation would define majority based > > on dirauths actually participating (and advertising so with a flag in > > the vote). Also, the participating or not participating flag should be > > used per vote/consensus and split into: > > > > a) we know current SR value for today so we vote it > > or > > we know previous SR value and we know for sure if we should follow the > > disaster protocol or not (in case we are about to vote at 01:00 UTC). > > so > > We participate in the vote for <current SR>. > > > > b) we are able to participate in this protocol run which will > > calculate the SR value for next day (after 00:00 UTC) so we send our > > commits/reveals. > > > > This is useful in case we are a dirauth that joined at 00:30 UTC and > > we couldn't get the _latest_ consensus (to find out if the 00:00 UTC > > consensus was created, and if not, previous SR value so we can follow > > the disaster procedure) we will not have an opinion for the <current> > > SR value at 01:00 UTC, but we can start participating in the protocol > > run for the next day - send our commit values. Once we decided on a > > <current> SR value for that day we save it and vote normally next time. > > > > So, if we have 5 dirauths running/signing consensus in total, out of > > which only 4 participate in the shared randomness protocol, the 4 > > participating ones should be able to create a valid consensus > > themselves with the insurance that the 5th one won't break consensus. > > > > One way to do this is: the dirauth which is not participating will > > take the SR value voted by the majority of the participating dirauths > > and include that in its consensus and sign. We need at least 3 > > dirauths agreeing on a SR value in order to accept it. > > > > Is this crazy? It shouldn't open the door new attacks, since this > > doesn't allow a single actor to game it, only the majority could game it. > > > > Thanks for the suggestions. > > Let me try to suggest a procedure here based on your ideas and some other > ideas. > > [Notation: SRV = shared random value] > > The goal here is to minimize the edge cases during SRV calculation and > disaster > SRV calculation. The edge cases here appear because there is no clear view > on > whether other dirauths know the current or previous SRVs, or whether the > SRV for > this period was ever created. The disaster recovery scenario is especially > annoying here. > > Here are some edge cases for example: > > * Dirauth boots up at 02:00UTC and the 01:00UTC consensus does not contain > any > SR information (maybe because not enough SR-enabled dirauths > participated at > that time). > > Should the dirauth do the disaster recovery procedure, or just play it > cool > and put no SR information on the consensus? If it has to do disaster > recovery, > then what previous SRV does it use (the 01:00UTC consensus did not > contain > such info)? > > This type of edge case is my main concern, since with dirauths upgrading > and > going offline at random times, it's likely that we will eventually > create a > consensus without SR info in the middle of the protocol run. > > * Dirauth boots up at 23:55UTC without having a previous consensus. It is > supposed to vote and form a 00:00UTC consensus without knowing any > previous > SRVs. How does it figure out whether all the other dirauths are also > bootstrapping, or whether the other dirauths actually know the previous > SRVs? > > Here are some prerequisites for the logic I'm going to suggest. The two > first > suggestions are useful in any case, I think: > > - First of all, we treat consensuses as trusted, so dirauths MUST learn > previous/current SRVs they didn't know about from any consensus they > fetch. > > - We are also going to assume that we have some sort of "SR flag" on votes > to > denote whether the dirauth participates in the protocol. > > - We are introducing another 'status' for shared random values (see > [SRVOTE]). > > Specifically, if a dirauth witnessed a 00:00UTC consensus failing to be > created, or it did not contain any SRV information, then it sets the > status of > "shared-rand-current-value" in its votes to "none". > > Now we are going to use the SRV lines in the votes as an indicator on how > the > consensus creation should play out. > > > ================================================================================ > > Here is some logic for the consensus at 00:00UTC: > > if majority of votes have disabled the SR flag: > then don't write anything SR-related to consensus > exit > > If the majority of votes contain the previous SRV value: > then calculate SRV as detailed in section [SRCALC] > else: > then calculate SRV as in [SRCALC] but with previous_SRV set > to 0 > > And here is some logic for all the other consensuses (to figure out when > dirauths should perform the disaster procedure): > > if majority of votes have disabled the SR flag: > then don't write anything SR-related to consensus > exit > > if we know the current SRV: > then write it on the consensus > exit > > if the majority of votes have the current SRV status as "none" _AND_ > those votes also contain the previous SRV value: > then do the disaster SRV procedure > else: > then don't write anything SR-related to consensus > > > ================================================================================ > > This _might_ work for fixing a good bunch of edge cases. But is it far too > complex? > > Should we just assume that these things will never happen on the real > network > and avoid baking additional complexity? What do you think? :/ > > > > > ------------------------------ > > Message: 2 > Date: Fri, 20 Nov 2015 10:19:07 -0500 > From: David Goulet <dgoulet(a)ev0ke.net> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Shared random value calculation edge cases > (proposal 250) > Message-ID: <20151120151907.GD10269@raoul> > Content-Type: text/plain; charset="us-ascii" > > On 20 Nov (16:06:59), George Kadianakis wrote: > > s7r <s7r(a)sky-ip.org> writes: > > > > > Hello, > > > > > > <snip> > > > > > > The idea of adding flags in the votes so each dirauth can advertise if > > > it is participating (has an opinion for the <current> SR or not) is > > > great and helps us build more defenses, probably make it easier in the > > > future too if we decide to change anything. > > > > > > What if the consensus for SR calculation would define majority based > > > on dirauths actually participating (and advertising so with a flag in > > > the vote). Also, the participating or not participating flag should be > > > used per vote/consensus and split into: > > > > > > a) we know current SR value for today so we vote it > > > or > > > we know previous SR value and we know for sure if we should follow the > > > disaster protocol or not (in case we are about to vote at 01:00 UTC). > > > so > > > We participate in the vote for <current SR>. > > > > > > b) we are able to participate in this protocol run which will > > > calculate the SR value for next day (after 00:00 UTC) so we send our > > > commits/reveals. > > > > > > This is useful in case we are a dirauth that joined at 00:30 UTC and > > > we couldn't get the _latest_ consensus (to find out if the 00:00 UTC > > > consensus was created, and if not, previous SR value so we can follow > > > the disaster procedure) we will not have an opinion for the <current> > > > SR value at 01:00 UTC, but we can start participating in the protocol > > > run for the next day - send our commit values. Once we decided on a > > > <current> SR value for that day we save it and vote normally next time. > > > > > > So, if we have 5 dirauths running/signing consensus in total, out of > > > which only 4 participate in the shared randomness protocol, the 4 > > > participating ones should be able to create a valid consensus > > > themselves with the insurance that the 5th one won't break consensus. > > > > > > One way to do this is: the dirauth which is not participating will > > > take the SR value voted by the majority of the participating dirauths > > > and include that in its consensus and sign. We need at least 3 > > > dirauths agreeing on a SR value in order to accept it. > > > > > > Is this crazy? It shouldn't open the door new attacks, since this > > > doesn't allow a single actor to game it, only the majority could game > it. > > > > > > > Thanks for the suggestions. > > > > Let me try to suggest a procedure here based on your ideas and some > other ideas. > > > > [Notation: SRV = shared random value] > > > > The goal here is to minimize the edge cases during SRV calculation and > disaster > > SRV calculation. The edge cases here appear because there is no clear > view on > > whether other dirauths know the current or previous SRVs, or whether the > SRV for > > this period was ever created. The disaster recovery scenario is > especially > > annoying here. > > > > Here are some edge cases for example: > > > > * Dirauth boots up at 02:00UTC and the 01:00UTC consensus does not > contain any > > SR information (maybe because not enough SR-enabled dirauths > participated at > > that time). > > > > Should the dirauth do the disaster recovery procedure, or just play it > cool > > and put no SR information on the consensus? If it has to do disaster > recovery, > > then what previous SRV does it use (the 01:00UTC consensus did not > contain > > such info)? > > > > This type of edge case is my main concern, since with dirauths > upgrading and > > going offline at random times, it's likely that we will eventually > create a > > consensus without SR info in the middle of the protocol run. > > > > * Dirauth boots up at 23:55UTC without having a previous consensus. It is > > supposed to vote and form a 00:00UTC consensus without knowing any > previous > > SRVs. How does it figure out whether all the other dirauths are also > > bootstrapping, or whether the other dirauths actually know the > previous SRVs? > > > > Here are some prerequisites for the logic I'm going to suggest. The two > first > > suggestions are useful in any case, I think: > > > > - First of all, we treat consensuses as trusted, so dirauths MUST learn > > previous/current SRVs they didn't know about from any consensus they > fetch. > > Yes ++. > > A dirauth booting up should always try to learn the current and previous > SRV from the latest consensus (< 24h) and update the disk state > accordingly _even_ if the disk state has SRV values in there. We should > trust the consensus voted by majority before our disk state imo. > > > > > - We are also going to assume that we have some sort of "SR flag" on > votes to > > denote whether the dirauth participates in the protocol. > > The requirements for having that flag "on" or "off" eludes me a bit. I > assume that this SR flag is "on" when a dirauth thinks its able to > compute a valid SRV that is I have enough commits. We can't make "having > the previous SRV" a requirement else we will never bootstrap properly > ever since in the first runs, previous SRV is alway 0. > > > > > - We are introducing another 'status' for shared random values (see > [SRVOTE]). > > > > Specifically, if a dirauth witnessed a 00:00UTC consensus failing to be > > created, or it did not contain any SRV information, then it sets the > status of > > "shared-rand-current-value" in its votes to "none". > > Hrm... I'm wondering here if "none" status is instead the same of having > the dirauth putting the disaster value in the vote? > > I would argue that a dirauth should _always_ try to compute a SR value > at any time (even when booting). Between having no line because we > couldn't compute due to lack of commits or a line that is the disaster, > I think the former simplifies things that is "We always have a SRV value > at 00:00 from a dirauth". > > Here is what I propose: > > 1) Like you said, we always try to learn SRV values from previous > consensus. If we can't get one from a consensus or our state, compute > disaster at 00:00. > > 2) We do _not_ use a SR flag for vote, a dirauth always tries to compute > a current SRV value at 00:00. If it's 13:00, it simply doesn't put > anything in the vote since no SRV data. > > The big issue I see here is when a dirauth does _not_ have the previous > SRV. Without it, both disaster and current SRV computation will end up > not matching the majority. So let's explore this binary state: > > - I have the previous SRV (either I can get it from my state or > consensus). > > This seems a non issue, dirauth will use it, carry it in the vote and > compute the current SRV if enough commits have been seen. At 00:00, if > majority wasn't reached for the current SRV value, we have a consensus > without it which is a valid use case but we at least have the previous > one in the consensus flagged as non-fresh so it can be used for > disaster or SRV computation at the next protocol run. > > - I don't have a previous SRV (couldn't get it in the latest consensus > and I have _no_ disk state). > > For instance, if this happens at 23:30, a dirauth won't have the > chance to participate in the SR protocol because it didn't see any > commits to compute SRV value thus disaster mode engage. > > If that dirauth was the 5th one out of 9 (meaning the majority > breaker), we end up with no SRV values at all at 00:00 (not even the > previous SRV since only 4/9 had it). So a new protocol run starts, > we'll end up with a valid current SRV in 24 hours and both values in > 48 hours. Essentially, we end up back in bootstrap mode. > > IMPORTANT note here: I think a dirauth MUST NOT keep any SRV values > that aren't in the 00:00 consensus. > > Question is, how big of a deal it is to have this issue where if we > don't reach majority because a dirauth rebooted and for some magical > reason didn't get the latest consensus to at least get the previous > SRV, we return in bootstrap? > > Cheers! > David > > > > > Now we are going to use the SRV lines in the votes as an indicator on > how the > > consensus creation should play out. > > > > > ================================================================================ > > > > Here is some logic for the consensus at 00:00UTC: > > > > if majority of votes have disabled the SR flag: > > then don't write anything SR-related to consensus > > exit > > > > If the majority of votes contain the previous SRV value: > > then calculate SRV as detailed in section [SRCALC] > > else: > > then calculate SRV as in [SRCALC] but with previous_SRV > set to 0 > > > > And here is some logic for all the other consensuses (to figure out when > > dirauths should perform the disaster procedure): > > > > if majority of votes have disabled the SR flag: > > then don't write anything SR-related to consensus > > exit > > > > if we know the current SRV: > > then write it on the consensus > > exit > > > > if the majority of votes have the current SRV status as "none" _AND_ > > those votes also contain the previous SRV value: > > then do the disaster SRV procedure > > else: > > then don't write anything SR-related to consensus > > > > > ================================================================================ > > > > This _might_ work for fixing a good bunch of edge cases. But is it far > too complex? > > > > Should we just assume that these things will never happen on the real > network > > and avoid baking additional complexity? What do you think? :/ > > > > > > _______________________________________________ > > tor-dev mailing list > > tor-dev(a)lists.torproject.org > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >

1 0