- tor-dev - lists.torproject.org

Draft proposal: Tor Consensus Transparency
by Linus Nordberg 22 Jul '14

22 Jul '14

Hi Tor devs, It's surprisingly hard to work on Tor during the Tor developer meetings! My apologies for not publishing this text until now, despite my repeated ranting about the subject the last few days. Well, here it is, in an early draft version. Thank you all who've listened patiently and given valuable feedback. I welcome more feedback from the list. Thanks in advance. --8<---------------cut here---------------start------------->8--- Filename: xxx-tor-consensus-transparency.txt Title: Tor Consensus Transparency Author: Linus Nordberg Created: 2014-06-28 Status: Draft 0. Introduction WARNING!!! EARLY DRAFT -- MISSING IMPORTANT BITS AND PIECES! This document describes how to provide and use public, append-only, untrusted logs containing Tor consensus documents, much like what Certificate Transparency [RFC6962] does for X.509 certificates. Tor relays and clients can then refuse using a consensus not present in logs of their choosing. WARNING!!! EARLY DRAFT -- MISSING IMPORTANT BITS AND PIECES! 1. Overview Using a public, append-only, untrusted log like the history tree described in [CrosbyWallach], Tor clients and relays verify that consensus documents are present in one or more logs before using them. Consensus-users, i.e. Tor clients and relays, expect to receive one or more "proof of inclusions" with new consensus documents. A proof of inclusion is a hash sum representing the tree head of a log, signed by the logs private key, and an audit path listing the nodes in the tree needed to recreate the tree head. Consensus-users are configured to use one or more logs by listing a log address and a public key for each log. This is used to verify that a given consensus document is present in a given log. Anyone can submit a properly formatted and signed consensus document to a log and get a signed proof of inclusion in return. Directory authorities should do this and include the proofs when serving consensus documents. Directory caches and consensus-users receiving a consensus not including a proof of inclusion submit the document and use the proof they receive in return. Auditing log behaviour and monitoring the contents of logs is performed in cooperation between the Tor network and external services. Directory caches act as log auditors with help from Tor clients gossiping about what they see. Directory authorities are good candidates for monitoring log content since they know what documents they have issued. Anybody can run both an auditor and a monitor though, which is an important property of the proposed system. Summary of proposed changes to Tor: - Directory authorities start submitting newly created consensuses to at least one public log. - Tor clients and relays receiving a consensus not accompanied by a proof of inclusion start submitting that to at least one public log. - Consensus-users start rejecting consensuses accompanied by an invalid proof of inclusion. - A new cell type LOG_GOSSIP is defined, for clients and relays to exchange information about tree heads seen and their validity. - Consensus-users send LOG_GOSSIP cells with seen tree heads to relays. - Relays validate tree heads received in LOG_GOSSIP cells (section 3.2.2) and send results to consensus-users in LOG_GOSSIP cells. 2. Motivation Popping five boxes or factoring five RSA keys should not be ruled out as a possible attack against a subset of the Tor network. An attacker controlling a majority of the directory authorities signing keys can, using man-in-the-middle or man-on-the-side attacks, serve consensus documents listing relays under their control. If mounted on a small subset of the network, the chance of detection is probably low. This proposal increases the cost for such an attack by raising the chances of it to be detected. The complexity of the proposed solution is motivated by the value of the decentralisation given. Anybody can run their own log and use it. Anybody can audit any existing logs and verify their correct behaviour. This empowers people outside the group of Tor directory authority operators and the people who trust them on a personal basis. 3. Design Communication with logs is done over http(s) similar to what [RFC6962] defines. This proposal does not use [[the TLS data structures]] but instead structures based on [[FIXME]]. Parameters for POSTs and all responses are encoded as name/value pairs in JSON objects [RFC4627]. Definitions: - Log id: The SHA-256 hash of the log's public key, to be treated as an opaque byte string identifying the log. 3.1 Consensus submission Logs accept consensus submissions from anyone as long as the consensus is signed by a majority of the Tor directory authorities of the Tor network that the log is logging. [[TODO: Move most of this to "specification" section?]] Consensus documents are POST:ed to well-known URL https://<log server>/tct/v1/add Input: consensus: A consensus status document as defined in [dir-spec] section 3.4.1. Output: id: The log id, base64 encoded. tree_size: The size of the tree, in entries, in decimal. timestamp: The timestamp, in decimal. sha256_root_hash: The Merkle Tree Hash of a tree including the submitted entry, in base64. tree_head_signature: A TreeHeadSignature ([RFC6962] section 3.5) for the above data. audit_path: An array of base64-encoded Merkle tree nodes proving the inclusion of the submitted entry in the tree denoted by sha256_root_hash (see [RFC6962] section 2.1.1). The output is what we call a proof of inclusion. The tree_head_signature is signed with the private key of the log. 3.2 Consensus verification 3.2.1 Log entry membership Calculate a tree head from the hash of the received consensus and the audit path in proof. Verify that it's identical to the tree head in the proof. This can easily be done by consensus-users for each received consensus. We now know that the consensus is part of a tree which the log claims to be The Tree. Whether this tree is the same tree that everybody else see is unknown at this point. 3.2.2 Append-only property of the log Ask the log for a consistency proof between the received tree head and a previously known good tree head. The known good head can be the empty tree. [[TODO add text about how to deal with received heads that are older than the last known good tree.]] Communication with logs is done over http(s) [[as described in [RFC6962] section 4 -- TODO specify protocol and encoding]]. [[description of consistency verification goes here]] Tor relays may do this for tree heads received in LOG_GOSSIP cells and communicate results in the same cells. [[TODO: Do this synchronously or asynchronously?]] Relays cache results to minimise the need for communication with log servers and calculations. We now know that the received tree is a superset of the known good tree. 3.3 Log auditing A log auditor verifies that the log presents the same view to all its clients and its append-only property, i.e. that no entries once accepted by the log are ever changed or removed. [[TODO describe the Tor networks role in auditing a bit more than what's mentioned in 3.2.2]] 3.4 Log monitoring A log monitor verifies that the contents of the log is consistent with the rules of the Tor network, notably that all entries are properly formed and signed Tor consensus documents. Note that there can be more than one valid consensus documents for a given point in time. One reason for this is that the number of signatures can differ due to consensus voting timing details. [[Are there more?]] [[TODO expand on monitoring strategies -- even if this is not part of proposed extensions to the Tor network it's good for understanding]] 3.5 Consensus-user behaviour Keep an on-disk cache of consensus documents. Mark them as being in on of three states: LOG_STATE_UNKNOWN -- don't know whether it's present in enough logs or not LOG_STATE_LOGGED -- have seen good proof(s) of inclusion LOG_STATE_LOGGED_GOOD -- confident about the tree head representing a good tree Newly arrived consensus documents start in LOG_STATE_UNKNOWN or LOG_STATE_LOGGED depending on whether they are accompanied by enough proofs or not. There are two possible state transitions: - LOG_STATE_UNKNOWN --> LOG_STATE_LOGGED: Seen enough proofs of inclusion verifying correctly according to section 3.2.1. The number of good proofs needed is a policy setting in the configuration of the consensus-user. - LOG_STATE_LOGGED --> LOG_STATE_LOGGED_GOOD: Seen enough gossiping to know that the tree head in the proof belongs to a known log. Consensuses in state LOG_STATE_UNKNOWN are not used but are instead submitted to one or more logs. This may take the consensus to LOG_STATE_LOGGED. Consensuses in state LOG_STATE_LOGGED are used despite not being fully verified with regard to logging. LOG_GOSSIP cells with the tree heads from received proofs are being sent to relays for further verified. Clients send to all relays that they have a circuit to to. Relays send to three random relays that they have a circuit to. 3.6 Relay behaviour when acting as an auditor TODO 3.7 Notable differences from Certificate Transparency - The data logged is "strictly time-stamped", i.e. ordered. - Much shorter lifetime of logged data -- a day rather than a year. Is the effects of this difference of importance only for "one-shot attacks"? - Directory authorities have consensus about what they're signing -- there are no "web sites knowing better". - Submitters are not in the same hurry as CA:s and can wait minutes rather than seconds for a proof of inclusion. 4. Security implications TODO 5. Specification TODO ? Compatibility ? Implementation ? Performance and scalability notes A. Open issues - handle all consensus flavours (i.e. microdescriptor consensuses) - don't use "consensus verification" since that's misleading - maybe add hash function agility, i.e. don't fixate SHA-256 (but see CT discussion about why not and TODO summarize it here) - add a blurb about the values of publishing logs as Tor hidden services - should relays gossip amongst each others too? - discuss compromise of log keys - add 'version' and 'extensions' fields to the submission response? - maybe log votes as well B. Acknowledgements This proposal leans heavily on [RFC6962]. Some definitions are copied verbatim from that document. Valuable feedback has been received from Ben Laurie and Karsten Loesing. C. References [CrosbyWallach] http://static.usenix.org/event/sec09/tech/full_papers/crosby.pdf [dir-spec] https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt [RFC4627] https://tools.ietf.org/html/rfc4627 [RFC6962] https://tools.ietf.org/html/rfc6962 --8<---------------cut here---------------end--------------->8---

3 2

Re: [tor-dev] Proposal for improving social incentives for relay operators
by George Kadianakis 22 Jul '14

22 Jul '14

Am 2014-06-10 02:26, schrieb Virgil Griffith: > For a while I've been seeking to grow the Tor network in both size and > goodput. Towards this end, I've explored various avenues such as > increasing user-awareness via tor2web. More recently, I've been exploring > financial incentives like TorCoin. > > Not wanting to strictly limit ourselves to financial incentives, I began > reading the literature on incentivizing volunteers. The most relevant > papers I found are: > > * http://www-2.rotman.utoronto.ca/facbios/file/LMS2_ManSci-Paper-Final.pdf > * http://pareto.uab.es/~prey/gneezy_254.pdf > * https://dl.dropboxusercontent.com/u/3308162/Slonim%202013.pdf > > The most relevant of these papers (Lacetera 2013) cites the major > motivations for volunteer labor are: "pure altruism, warm glow, self-image, > and reputation". Upon reading this I realized TorCoin's technical > interestingness had blinded me to much easier to leverage motivations of > "warm glow" and "reputation". > > I propose the following system for harnessing "warm glow" and "reputation" > for Tor relay operators. I am willing to fund this in its entirety. > > I propose establishing a subdomain on torproject.org giving each Tor relay > operator (hereafter affectionately called "Torati") his/her own page using > the information her machines provide to the Tor Directory Consensus. The > fields to show on her "Torati profile page" would be things like: > ContactInfo, PGP fingerprint, list of server nicknames, date the Directory > Authorities first saw her contact info, etc. You can also imagine a > receiving special "special recognition stars" for operating an exit or Here are some ideas on various achievements/badges that could be awarded to relay operators (in no particular order): - You earned a flag (fast/hsdir/stable/named/exit)! - Your uptime is over 9000 (seconds/days/months)! - You are part of a family! - You have the contact field set! - With PGP! - With a BTC address! - You are part of an organization (torservers/etc.)! - You are in an exotic location! (not many relays there) - You have IPv6 enabled! - You are running an experimental Tor version! - You are a bridge with many clients from China! - You are an obfsbridge! You can imagine more special achievements, like: - You were around during event X! (arab spring, etc.) - You were the top guard during event X! - You were the first big relay ever in Lithuania! - You were in the top 10 exits. - You survived an hour of anarchy! (awarded when the authorities fail to establish a consensus for an hour) etc. You can also imagine recurring (monthly) achievements, and more one-time achievements that tend to really motivate people in games. Also, some kind of point system and levels might be fun. To be honest, I don't know if such a thing would motivate people to start running relays, but it might motivate people whole already run relays to run even better ones (or try to hack the system just to collect achievements).

2 1

help with minimal Tor build for Shadow
by Rob Jansen 22 Jul '14

22 Jul '14

Hi, Shadow uses CMake and a custom build process to build the Tor source files into a shadow-plugin (using Clang/LLVM and a custom pass). However, because my CMake build file is not smart enough to know how to turn orconfig.h.in into orconfig.h, I still have to run Tor through autotools first (and then build again with Clang). I’m hoping to eliminate at least parts of the Tor configure/make build process. Shadow for sure needs orconfig.h to exist, and I presume ./autogen.sh and ./configure is the best way to generate that. Does anyone know of a faster approach? (I’d like to avoid adding the rules to my CMakeLists.txt file, as it will be hard to keep up with Tor’s upstream changes to its Makefile.in.) However, still missing after the configure is at least or_sha1.i, and possibly other files. Presumably these are generated when running make (after configure). My custom Clang build fails without them. Is there a special make target in Tor that generates these “helper” files without building the entire Tor binary? If not, should/can there be? All the best, Rob

1 1

Typos in 224-rend-spec-ng.txt
by Tim 21 Jul '14

21 Jul '14

Hi All, I've found a few typos in the first few sections of 224-rend-spec-ng.txt at https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/224-rend-spe… I tried finding an existing bug or wiki entry on trac to log these against, but I couldn't find anything relevant with: https://trac.torproject.org/projects/tor/search?q=proposal+264 The typos are: (I read from the start to Section 2.2.3) Line 392, S 1.3, [IMD:AC]: 'knowledge od the contents' (knowledge *of* the contents) Line 618, S 2.2.1, [TIME-PERIODS]: 'a single set of hidden service directory' (*directories*) S 2.2.3. Where to publish a service descriptor: Line 702 : 'hsdir_n_replicas' should be 'n_replicas' to be consistent with lines 710-711: 'where n_replicas is determined by the consensus parameter "hsdir_n_replicas"' Line 733: "An HSDir should rejects a descriptor" (should *reject*) Hope this helps. Tim

3 2

Re: [tor-dev] [tor-relays] Hidden service policies
by grarpamp 21 Jul '14

21 Jul '14

On Sun, Jul 20, 2014 at 9:57 PM, Thomas White <thomaswhite(a)riseup.net> wrote: > Mike Hearn, > Simple. If you start filtering anything at all, regardless of what it > is ... then I will > block any connection of your relays to mine > ... > Freedom isn't free unless it is > totally free and a selective reading policy through Tor is not just a > bad idea as stated below, I find it outright insulting to me and > everyone else who cares about the free and open internet. The fact > somebody has the audacity to come to a project like Tor and propose > blacklisting mechanisms is jaw-dropping. > ... > As I recall, you are also the person who raised the idea of coin > tinting or a similar concept in the bitcoin community to identify > "suspect" coins and that backfired spectacularly on you. Yes, that is the person. Though the term is known as 'taint'. One of many discussions from that suggestion is here: https://bitcointalk.org/index.php?topic=333824.0 > so while you are reading this, let me know if you run any relays so I > can avoid them. router riker 207.12.89.16 9001 0 0 fingerprint 8657 6CF6 AA84 496F 62C0 5AFE 9F26 8962 A5F0 B2BD contact Mike Hearn <mike(a)plan99.net> accept *:8333 reject *:* Normally I would thank exits for passing BTC traffic, but now I'm unsure of this one (and a few others), especially given that's the only exit policy of the above node. To identify anon (Tor) coins for marking and tracking?

2 1

Crippled intro points
by Andrea Shepard 20 Jul '14

20 Jul '14

A recent thread has advocated crippling relays to selectively refuse to act as intro points for hidden services. Unfortunately, with the current HS design it would be implementable. I state my opposition to adding any such censorship feature now, and look forward to an improved HS protocol that will make it impossible. Also, note that it would be possible to patch Tor to do this without any official protocol changes, but this would have the effect of leaving the targeted HSes no way to discover intro points that will accept them other than trial and error. This strikes me as something that would comprise an attack on the network and be good grounds for a !reject. -- Andrea Shepard <andrea(a)torproject.org> PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5

1 0

ANN: txtorcon 0.10.1
by meejah 20 Jul '14

20 Jul '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, A new minor release of txtorcon exists, fixing a couple bugs introduced along with the endpoints feature in 0.10.0: * issue #78: Add tox tests and fix for Twisted 12.0.0 (and prior), as this is what Debian squeeze ships * issue #77: properly expand relative and tilde paths for hiddenServiceDir via endpoints * fix bug incorrectly issuing RuntimeError in brief window of time on event-listeners You can download the release from PyPI or GitHub (or of course "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/0.10.1 https://github.com/meejah/txtorcon/releases/tag/v0.10.1 sha256sum reports: 33f04523329b14accb2054b81c5da887c28b402c797ba895dc1ee58824e107f1 txtorcon-0.10.1-py2-none-any.whl 7a6e8fab71fd05c223d866b60b998cf308661ef1fc87d94e06c3b51f4ada4a6a txtorcon-0.10.1.tar.gz thanks, meejah -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEbBAEBAgAGBQJTzD25AAoJEMJgKAMSgGmnB8IH+MnRyCCKMgPJ2GjV01K+PVaU UgvUXhaqLdQUbViKeYy7KNst9d8Dm3ejHr21kedE8oeY/ztHlLdvtJtgWWvTOrTI qqh4wWfBVHeibSgsEzwNVdeJ3MtERPWuCrIkIWnathbVjfnkRW+cLPOtJUWtSi/d pdN4ZC+K1jBvSeHPCDhf8sXSqdzsOxXOWn/9SUaa70c7kMrbxjMxO1Jw/miqftMq /wo0vyXn4EeEdURa9hYFpUqgUbUFl6C1KiELNeHWtwGbGWku17bNgQn5HH7uY4/6 RODYBaAT327062m81ig7zQ6MmxKcngIG4Ic3qvJYeLerO/EnnNvzSs3Q+EGQeg== =nsEx -----END PGP SIGNATURE-----

1 0

[GSoC] Consensus diffs - Fourth report
by Daniel Martí 20 Jul '14

20 Jul '14

Hello everyone, Apologies for the delay, I'm a couple of days late. This is the fourth status report of my Google Summer of Code project, which is to implement consensus diffs for Tor. My mentors - Sebastian and Nick - and myself usually hold meetings on IRC on wednesday at 16h UTC. On the first week I finished implementing the consensus diff headers as described in the updated proposal #140 [1], including full SHA256 hashes of each of the two consensuses involved: the base one and the resulting one. I also finished the tests for that bit. Moreover, I thought it would make sense to make sure that the diff generation process never takes too long. In theory it should always be linear in time given any two realistic consensus files, since it would navigate the router entries and take advantage of them being sorted alphabetically. But if no matching router entries are found between the two consensuses, the algorithm would run in quadratic time. Running in quadratic time over two sets of 25K lines takes about 20 seconds on my laptop, which is pretty bad. So I made the algorithm error if 10K lines are found on both sides without any matching router entry id. Which makes perfect sense for any pair of consensuses - even if you take consensuses months or years apart, it would never occur that they shared so few router ids. And if they did, generating a diff between them would make no sense. Unfortunately, I wasn't able to do much else during that week since I was at RMLL running a Free Your Android workshop for the whole week. And just when I got home on Saturday, my laptop broke down, which kept me busy for a few days as well. Nevertheless I am still on schedule. What I must do now is start merging my code into a Tor feature branch, so that I can write the bits of code that will glue everything together. To do that, I've begun rewriting my git history so that it is simpler to understand and read. If you have a look at my new repo on github [2], you'll notice that the number of commits went down from 122 to 95, and quite a few were rewritten as well. Open to suggestions about how to further improve it. By the time the next report is due - August the 1st - I should have completed the integration of my code into Tor. Then comes writing some more tests and testing Tor with the new feature in place. [1] https://gitweb.torproject.org/torspec.git/blob_plain/refs/heads/master:/pro… [2] https://github.com/mvdan/tor/branches -- Daniel Martí - mvdan(a)mvdan.cc - http://mvdan.cc/ PGP: A9DA 13CD F7A1 4ACD D3DE E530 F4CA FFDB 4348 041C

1 0

hidden service involvement in external program
by Tomas Bortoli 20 Jul '14

20 Jul '14

Hi all, I want to develop a program that connects to a hidden server through tor. This is my problem: how can i establish a connection to the server from the client? I read the hidden service protocol, but i need a usable implementation of it. So i was looking in stem but seems it doesn't provide this functionality, i'm wrong? Anyone have an idea of how can i achieve a solution?? any library already implemented? Thanks!

3 2

Hidden services mailing list
by Michael Rogers 20 Jul '14

20 Jul '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi all, Several projects use Tor hidden services for peer-to-peer communication, including TorChat, TORFone, Ricochet, Ploggy, OnionShare, Invisible.im, Ciboulette and Briar. I suspect many of us are running into similar issues because the hidden service protocol and implementation were designed for hidden services that behave like servers rather than clients, i.e. that have long uptime and stable network connections. I've created a mailing list to discuss these issues. If you're interested in participating, you can join the list here: https://fulpool.org/cgi-bin/mailman/listinfo/hidden-services Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJTy9lOAAoJEBEET9GfxSfMZMIH/jHmO6/RlLUDSaEcxtgSsO1W rlxW9oPCmwoFg258+1FRAVmfpqmUxmr0RTMiG4yb2Go23mM7HmnNVJz0mRXJsMxA 1Bu9cEUlROYDL1SEMoc8Apt6ymZUTOxknYIrXRsNJlfEPCmte9Vf8i8xjZJna6cr QpL+/1C/NmD8synteIJvHTCu1pa4MMHlLfjawSJzavaP2dMK/IVz12XDrdBx8zQU GNUt/FanZ5Cc8AUQHsd/+eC2ey24yg3iMH/yhgaU1zdG05T2wcyXjOVe70cHo6Lk Z3aaXaXg9tyISXuTCQXNjdrurwv9MlTjwvOZt6sGbOOvxHKh0S0N9X8txfEusJ8= =uOE9 -----END PGP SIGNATURE-----

1 0