- tor-dev - lists.torproject.org

[Discussion] 5 hops rendezvous circuit design
by Qingping Hou 14 Feb '14

14 Feb '14

Hi all, We are a university research group and following is a proposal for possible optimization on Tor's rendezvous circuit design. It's not written as a formal Tor proposal yet because I want to have a discussion with the community first, get some feedbacks and make sure it does not introduce new vulnerabilities. So please feel free to comment if you have any thoughts or find any holes :) PS: a txt version of the proposal is attached in this mail as well. ------------------------------------------------------ The changes described in this document is focused on speeding up hidden service while at least keeping the same level of anonymity as current version provides. Naming conventions: HSv3: current version of hidden service protocol HS5h: hidden service protocol that uses 5 hops HS: hidden service client: The onion proxy that tries to communicate with HS HSDir: hidden service directory RP: rendezvous point 1. The 5 hops rendezvous circuit The setup on HS side is the same as HSv3. HS picks couple introduction points and uploads the descriptor to HSDir. The main difference is how to pick the RP. In HS5h, Instead of letting the client pick whatever node it wants, both client and HS use negotiation protocol to pick a RP collaboratively. Following is a high level description for the rendezvous circuit setup process in HS5h: (0) client fetches descriptor for a hidden service. (1) client connects to introduction point. (2) since client and HS are connected via introduction point, they can negotiate a random number using this channel. (For more details, see [RAND_NEGO]) (3) both client and HS maps that random number to a random onion router using the same scheme, so they end up with the same node. This is the candidate RP. (4) both client and HS create a 3 hops circuit using RP as last hop. (5) RP joins the circuit originates from HS to the circuit originates from client. (6) now client and HS are connected. Because their original circuits share the same endpoint(the RP), the length of the path is 5 hops. Improvement: (1) Reduction in circuit setup time. Both the client and HS only build 3 hops circuit now. While In current protocol(HSv3), HS needs to build a 4 hops circuit. (2) A shorter circuit means less transmission delay. It uses only one more hop compared to HSv3's Tor2Web mode. Also the circuit length for HS5h's Tor2Web mode can be further reduced to 3 hops. (3) Less setup time on client side for the first connection. In HSv3, a client has to first setup a 3 hops circuit to RP before sending INTRODUCTION1 cell to the introduction point. While in HS5h, this process is removed. Possible drawback: (1) An extra step is needed: random number negotiation. This result ins more communication between two nodes and Introduction point needs to remember states for the negotiation. We think this negotiation step will not introduce a noticeable delay to the whole process. Firstly, it reuses the connection to introduction point for both sides so it requires no extra circuits build up. Secondly, the bottle neck is circuit setup, cell/stream transmission delay is actually pretty low. 2. Negotiate random numbers between two nodes [RAND_NEGO] H(a) here means digest of message a. (1) client generates a random number R_a and sends the digest H(R_a) to HS (2) HS remembers H(R_a), generates a random number R_h and sends it back to the client (3) client receives R_h and sends back R_a to HS (4) HS checks R_a against H(R_a) to make sure it's not generated after client receives R_h. (5) Now both client and HS compute a shared random number using R_a and R_c. (for instance, simply add up these two number) The negotiation protocol can be very simple here because only two parties are involved and the random number does not need to be remained secret. Note that at step 2), if HS is able to recover R_a from H(R_a), it can take control over R_c. So to mitigate this, we can use a variant of Diffie-Hellman handshake: (1) client generates a public key g^x and sends the digest H(g^x) to HS (2) HS remembers H(g^x), generates a public key g^y and sends it back to the client (3) client receives g^y and sends back g^x to HS (4) HS checks g^x against H(g^x) to make sure it's not generated after client receives g^y. (5) Now both client and HS compute a shared random number: R_c = g^(xy) 3. Rough anonymity analysis for HSv3 and HS5h 3.1. Assumptions, attack rules and constraints In this section, we assume 3 hops being the minimal circuit length to keep the origin node of a circuit anonymous from the destination. Two simple attack rules are used to remove nodes from a circuit: 1. If node A is picked by the attacker, then he can pick himself as node A. 2. If the identity of node A is known by the attacker, then he can connect directly to it. To defense against these attack rules, following constraints need to be enforced for a circuit: 1. No one except the circuit originator can have total control over choice of all nodes on the circuit. Note that it's OK even if the originator does not have total control. 2. Identity for all nodes on the circuit, except the last hop, must remain secret. i.e. only known by circuit originator. 3.2. Why 6 hops is the minimal circuit length for HSv3 First let's begin with an ASCII graph for HSv3 rendezvous circuit: Client -> (a1) -> (a2) -> (RP) <- (b3) <- (b2) <- (b1) <- HS Client chooses 3 ORs: a1, a2, RP. HS chooses the other 3 ORs: b3, b2, b1. Now consider HS as the malicious side. In order to get closer to client, the best it can do is to connect directly to RP, which reduces the circuit length to 3 hops: Client -> (a1) -> (a2) -> (RP) <- HS If client is the malicious side, the best it can do is to choose itself as RP, which also reduces the circuit length to 3 hops: Client <- (b3) <- (b2) <- (b1) <- HS Taking any node from the original 6 hops circuit will result in 2 hops circuit in either case. As noted above, 3 hops is assumed to be the minimal circuit length. 3.3. Why HS5h only needs 5 hops HS5h makes use of the fact that the last hop of a circuit is different from all previous hops. Since last hop will be in direct contact with the destination host, its identity does not need to be remain secret. We only have to make sure the last hop cannot be determined by any malicious host at will. Otherwise the malicious side can just pick himself as last hop and thus shorten the circuit. This is where hop negotiation come into play. A negotiated hop is guaranteed to be a random node and cannot be determined by anyone. So it can be used as the last hop for both client and HS. Since the last hop for both sides overlaps, it's effectively acting as a rendezvous point. In comparison, in HSv3, the rendezvous point is picked and controlled by the client. Therefore it can not be used as the last hop for HSs' rendezvous circuit. Again, following is an ASCII graph for a HS5h rendezvous circuit: Client -> (a1) -> (a2) -> (RP) <- (b2) <- (b1) <- HS Now if the client is malicious, it can only connects directly to the RP, i.e. pick the second last hop as himself. Therefore HS can still maintain a 3 hops distance from the malicious client: Client -> (RP) <- (b2) <- (b1) <- HS While in HSv3, the client can pick himself as the RP. And because this is a symmetric scheme, the same argument goes for malicious HS: Client -> (a1) -> (a2) -> (RP) <- HS Either cases, we maintained a 3 hops circuit. 3.4. Summary We took 3 as the minimal circuit length because that's the default setup in Tor. The analysis should work for any number. To generalize it, if N is the minimal circuit length, then: 1) without node negotiation, the minimal rendezvous circuit length is 2N. 2) with node negotiation, the minimal rendezvous circuit length is 2N-1. 4. Other possible speed optimization under HS5h 4.1 Optimistic rendezvous mode Similar to optimistic_data for AP connections, we can implement optimistic_rendezvous mode under HS5h. The basic idea is to optimistically assume any picked candidate RP will be willing to act as a RP. Since now we need to maintain state in introduction point to help with node negotiation, we can use it to communicate with HS when establishing the rendezvous circuit. So both client and HS can launch circuits to newly negotiated RP simultaneously. If this candidate RP refuses to be a rendezvous point to one side, the other side can be notified via introduction point. Then client and HS can go through the negotiation process again to pick a new candidate RP. This mode can also be implemented under HSv3 as well. In HSv3, wait for RENDEZVOUS_ESTABLISHED cell and sending INTRODUCE1 cell has to be a sequential operation. With optimistic_rendezvous mode, this can be done in parallel. NOTE: we need to test RENDEZVOUS_ESTABLISHED rate in real tor network. If the rate is low, then this mode will slow down the setup process instead. To make this mode work even better, we can have all OR explicitly mark whether it is willing to be a rendezvous point in it's descriptor. This should greatly increase the success rate for receiving RENDEZVOUS_ESTABLISHED. 5. Hazards a) How to design the scheme for mapping a random number to the same node between client and server? This will be trivia if it's easy to synchronize node list between two onion proxy. The problem is: how much overhead will be caused by synchronizing node list (or consensus). b) How much overhead will be introduced by maintaining state for node negotiation? Since in HS5h, both introduction point and hidden service need to maintain state for node negotiation. To do so, the client can send something like a negotiation identifier in the initial introduction request. Then for introduction point, it can mark the circuit from the client with this ID. For HS, it can hash the negotiation state structure into a hash table with ID as the key. Theoretically, this overhead should be low, because the "negotiation ID" is just like rendezvous cookie.

8 16

Tor Weather: Running background script
by Abhiram Chintangal 14 Feb '14

14 Feb '14

Karsten, From last meetings discussion, I tried a few things to figure out why the script, "listener" (invoked by django command runlistener) fails to do its job : 1. make the event listener listen to a different type 2. Add "UseMicroDescriptors 0" to tor client config 3. Enable debug log for the tor process The script gets invoked from the django command and then after some log messages it calmly dies. The debug log from tor has a details about the brief activity on its control port [1]. Let me know what you think. Thanks! Abhiram Chintnagal [1]: https://gist.github.com/achintangal/9002896

1 0

Re: [tor-dev] [Discussion] 5 ^H 3 hops rendezvous circuit design
by str4d 14 Feb '14

14 Feb '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 02/13/2014 11:43 AM, Zack Weinberg wrote: > (Four hops is what I2P uses, with two chosen entirely by the client > and two entirely by the server; but there appears to be nothing to > guarantee that a malicious peer can't connect directly to its > counterparty's two-hop chain, sacrificing some of its own anonymity > but getting closerfor a while now to the counterparty. I did just > argue that that shouldn't matter, though...) Just FTR, since 0.9.7.1 I2P defaults to 3 hops for "client" tunnels (so default of 6 hops end-to-end, and 12 hops round-trip). And yes, there is nothing stopping a malicious peer building zero-hop tunnels (acting as its own Outbound Endpoint/Inbound Gateway), just as honest clients or servers can choose to use zero-hop tunnels (e.g. a website or IRC network that requires no anonymity of its own, but wants its users to benefit). str4d > > zw > > > > _______________________________________________ tor-dev mailing > list tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) iQEcBAEBCgAGBQJS/f/RAAoJENXeOJaUpGWyUB8H/RI45Bc7TXc7y2SN0Tj1Sabz Wwjqb4tb4fhcFV7C6biJEFg6sKR1GPnNAa6AsadoMREJ/KD64t9eoii1g38x/UdN GxwHLk/h7zJCTRxMR0W89S7nIGimzp/I12urLXwYvvIoKjbytX1a9e3TkZpBmOvS D1JvAhVeoMa2dUfjT4jZH83kGQ3PlWvYPOvyv5CiSgoE6KHXWn2pYfNiOQF8mzWU oDU0TaNMsATjX/y1bwPAdf6w+DoKOw2xmpVyutY5+CYRC9BrvqAyV5WWwVvOTZHz gTIYVToLGN79PBsMDT0nJoxvm58a734qR9XANo5fDegkJIWKY+Ac/cBwadq25A0= =1wAz -----END PGP SIGNATURE-----

1 0

Release Candidate: TBB 3.6 + fteproxy 0.2.6
by Kevin P Dyer 13 Feb '14

13 Feb '14

Hi all, Please see [2] for a patch that integrates fteproxy into dcf's 3.6-beta branch [3] of the TBB. This integration targets version v0.2.6 of fteproxy [1]. asn/mikeperry - What do you need from me to merge this patch to master? -Kevin [1] https://github.com/kpdyer/fteproxy/releases [2] https://trac.torproject.org/projects/tor/ticket/10362#comment:11 [3] https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git

3 4

ScrambleSuit issue in old bridges
by George Kadianakis 12 Feb '14

12 Feb '14

(Forwarding this mail to tor-dev, since I accidentally sent it to tor-assistants) Hey Philipp, another thing we should think about is what should happen when ScrambleSuit is run in bridges that don't support server-side PT parameters. In the past this was not a problem, because ScrambleSuit would not run at all in those older versions of Tor. However, after we introduced the automatic generation of scramblesuit fallback passwords, scramblesuit will happily startup, generate a fallback password, and pass it to Tor. The problem is that Tor will simply ignore the fallback password (see parse_smethod_line() in tor-0.2.4.x), but still pass the scramblesuit bridge to BridgeDB. So if I'm not mistaken, BridgeDB might see some scramblesuit bridges without a password. Such bridges are pretty much useless, since clients can't connect to them. What should we do about this? One solution might involve scramblesuit somehow recognizing the version of Tor, and refusing to start if Tor doesn't support server-side bridge parameters.

1 0

A threshold signature-based proposal for a shared RNG
by Nicholas Hopper 11 Feb '14

11 Feb '14

A Threshold Signature-based Shared RNG 0. Overview This proposal is for a design of a shared RNG to address ticket #8244 - "The HSDirs for a hidden service should not be predictable indefinitely into the future," /without/ requiring byzantine agreement protocols at each RNG period. The basic idea of this proposal is to have directory authorities use a deterministic threshold signature scheme to sign Hash(time-period) in each consensus document. This is done simply by having each authority contribute a publicly-verifiable share of the signature during the voting process. As long as at least ceiling(n/2) authorities are honest, the signature can be re-constructed by each client. Since the signature is hard to compute if you don't control at least ceiling(n/2) authorities, its inclusion in the input to a hash function (e.g. for use in constructing the HS hash ring) will make the output of the hash function pseudorandom. (In the strong cryptographic sense that unless you can break the signature scheme, you can't distinguish the hash from a truly random string of the same length) 1. Notation and such This protocol can work over a group G, using a subgroup generated by an element B of prime order p. We'll use Curve25519 [Curve25519] for concreteness, but any group where the Computational Diffie-Hellman problem is difficult would do. We'll denote multiplication of scalars (e.g. mod p) by *, e.g. a*b; we'll denote multiplication of a group element P by scalar a by a.P. We will make use of Shamir threshold secret sharing, described, e.g. in Handbook of Applied Cryptography [HAC], over the integers mod p. For a subset S = {P_1,...,P_t} of size t, we'll denote the lagrange multipliers for S by m_1,...m_t (eg. if each P_i has share s_i of secret x, then m_1*s_1 + m_2*s_2 + ... + m_t*s_t = x.) We assume that each authority S_i has a publicly known verification key VK_i and signing key sk_i for a digital signature scheme (Sign, Verify). We model network communications as point-to-point with bounded delay and assume at most t-1 authorities are compromised. (Without these assumptions, the consensus voting protocol is also broken) We'll need a hash function H that can output elements of the subgroup generated by B with unknown discrete logarithms. For Curve25519, it should work to compute H(x) by computing SHA256(x), and mapping this 256-bit string to the curve as in the Curve25519 paper. A security proof would model this H as a random oracle. We'll need a distributed threshold key generation algorithm which works as follows: Input: Parties S_1,...,S_n Output: to all players: group public key P = x.B. player public key shares s_1.B, ..., s_n.B to each player P_i: secret key s_i) such that the secret keys s_i are (n,t) secret shares of the (unknown) secret key x. We discuss choices to implement this protocol in section [DKG] We will also require a noninteractive zero-knowledge "signature of knowledge" that can be bound to a message m, to prove that two points P, Q have a common discrete logarithm to bases B and R respectively. We denote this proof by SK { s : s.B = P && s.R = Q } (m) and discuss its implementation in section [SKDH]. 2. Overall Protocol description [CONSENSUS-RANDOM] The protocol requires that periodically the authorities run the distributed key generation procedure to produce the threshold public key P = s.B, and public key shares P_1 = s_1.B, ... , P_n = s_n.B. This can be done infrequently, since its only purpose is to limit key exposure. For the time period starting at time T, the authorities generate the shared random value as follows: Step 1. [Period Base] Each authority (and anyone else interested) generates the "time period base point" R in G, by computing R = H("tor-hs-rand-base-point " || period valid-after time) Step 2. [Signature Share] Each authority S_i uses its secret share s_i and signing key sk_i to generate its signature share SHARE_i as follows: SHARE_i = R || Q_i || PROOF_i || Sign_i(R || Q_i || PROOF_i) Q_i = s_i.R PROOF_i = SK { s_i : s_i.B == P_i && s_i.R == Q_i }(S_i || T) Each SHARE_i is appended by S_i to the network consensus document for time period T in a separate section. Step 3. [Validation] Tor clients locally validate SHARE_i from server S_i for time period T as follows: i. Verifythe signature using verification key vk_i. ii. Check that R = H("tor-hs-rand-base-point " || T). iii. Verify PROOF_i for points (P_i, Q_i) with bases (B,R) . SHARE_i is considered valid if and only if all three checks succeed. Step 4. [Aggregation] If at least t = ceiling(n/2) valid shares appear in the consensus, Tor clients locally compute the randomizer string RAND for time period T as follows: i. Let (S[j], Q[j]) for j = 1,...,t denote the identities (S_i) and signature shares (Q_i) associated with t valid SHARE_i values. ii. Compute the Lagrange multipliers m[j] for x points S[1]...S[t] iii. Compute RAND = m[1].Q[1] + m[2].Q[2] + ... + m[t].Q[t] Notice that RAND = m[1]*s[1].R + ... + m[t]*s[t].R = x.R If there are not enough valid shares, the protocol fails; this indicates that a majority of the directory authorities are faulty or compromised. If we wanted a fail-open solution, possibilities include hashing the list of valid SHARE values, or taking the hash of the previous consensus RAND value. [XXX - other options or opinions?] 3. Signature of knowledge scheme [SKDH] Given values (R,S), bases (P,Q), and exponent s such that R=s.P, S = s.Q, we use the "Fiat-Shamir Heuristic" with the proof of equality of discrete logarithms due to Chaum-Pedersen [CP92, Sec. 3.2] to generate SK { s : s.P == R && s.Q == S } (m) As follows: i. Choose random integer a in [0,p-1] ii. Compute: U = a.P V = a.Q c = Hash(U || V || m) z = a + s*c (mod p) iii. Output (U,V,z) The proof is verified by computing cc = Hash(U || V || m) and checking that z.P == U + cc.R z.Q == V + cc.S 4. Distributed Key Generation [DKG] We'll need a protocol satisfying the following specs: Input: Parties S_1,...,S_n Output: to all players: group public key P = x.B. player public key shares s_1.B, ..., s_n.B to each player P_i: secret key s_i such that the secret keys s_i are (n,t) secret shares of the (unknown) secret key x. There are several options for such a protocol, depending on what we want to assume about the network over which the authorities will run the protocol (Presumably, the Internet): - If we assume "weak synchrony" (messages are eventually delivered without unbounded delay), the protocol of Kate and Goldberg [KG09] can tolerate floor((n-1)/3) corruptions and still run to completion. - If we assume bounded delay but allow "rushing" adversaries and adaptive corruptions, the protocol of Garay et al [GKKZ11] can be used to realize the broadcast channel required for Pedersen's protocol [Ped91] while tolerating up to t-1 = floor(n/2) corruptions. - If we assume bounded delay and static corruptions, the protocol of Dolev and Strong [DS83] can be used to realize the broadcast channel required for Pedersen's protocol and tolerate up to t-1 corruptions. I'm not an expert on byzantine agreement protocols, so it is possible that there exist other, more easy to implement/manage options as well. 5. References [CP92] David Chaum, Torben Pryds Pedersen. "Wallet Databases with Observers", CRYPTO 92, pp 89-105. http://link.springer.com/chapter/10.1007/3-540-48071-4_7 [CURVE25519] Daniel J. Bernstein. "Curve25519: new Diffie-Hellman speed records", PKC 2006, pp 207-228. http://cr.yp.to/papers.html#curve25519 [DS83] D. Dolev and H. Strong. "Authenticated algorithms for Byzantine agreement", SIAM J. Computing 12(4):656-666, 1983. http://www.cse.huji.ac.il/~dolev/pubs/authenticated.pdf [GKKZ11] Juan Garay, Jonathan Katz, Ranjit Kumaresan, Hong-Sheng Zhou. "Adaptively Secure Broadcast, Revisited", PODC 11, pp 179-186. http://chess.cs.umd.edu/~jkatz/papers/asb-final.pdf [HAC] Alfred Manezes, Paul van Oorschot, Scott Vanstone. "Handbook of Applied Cryptography", CRC Press, 1996. http://cacr.uwaterloo.ca/hac/ [KG09] Aniket Kate, Ian Goldberg. "Distributed Key Generation for the Internet", 29th International Conference on Distributed Computing Systems, June 2009. https://cs.uwaterloo.ca/~iang/pubs/DKG.pdf [Ped91] Torben Pryds Pedersen. "Non-interactive and information-theoretic secure verifiable secret sharing," CRYPTO91. http://www.cs.huji.ac.il/~ns/Papers/pederson91.pdf -- ------------------------------------------------------------------------ Nicholas Hopper Associate Professor, Computer Science & Engineering, University of Minnesota Visiting Research Director, The Tor Project ------------------------------------------------------------------------

3 7

Looking up bridges in Globe et al. by fingerprint
by Karsten Loesing 11 Feb '14

11 Feb '14

Hi bridge address distribution people, hi bridge status website developers, hi bridge operators, here's a problem that recently came up when Christian and I discussed making Globe a JavaScript-free website. The current Globe uses client-side JavaScript to detect if the user entered a 40-character hex fingerprint. In that case, it runs the input through SHA-1 and sends only the digest to the Onionoo service. That's why Onionoo supports four cases and does not support a fifth case (number 3 below): 1. original relay fingerprint: return the matching relay, because it might have been a different client than Globe that sent the request and did not run the fingerprint through SHA-1; 2. SHA-1 of relay fingerprint: return the matching relay, too, because this could have been Globe running the original relay fingerprint through SHA-1; 3. original bridge fingerprint: don't return anything, because Onionoo should not see original bridge fingerprints at all; 4. SHA-1 of bridge fingerprint: return the matching bridge, because not all clients are like Globe and might not run fingerprints through SHA-1; and 5. SHA-1 of SHA-1 of bridge fingerprint: return the matching bridge that the user was looking for when they put in the SHA-1 fingerprint which was then converted to the SHA-1 of the SHA-1. That's what the current Globe can do, because it uses JavaScript on the client side. But the new server-side Globe cannot do that! And I don't have a good solution yet. Here are some ideas, each of them having pros (-) and cons (+): 1. Globe runs full fingerprints through SHA-1 on the server. - The Globe server learns a lot of bridge fingerprints, which makes it a juicy target, because bridge fingerprints can (in the future?) be used to fetch bridge descriptors containing bridge addresses. + This variant is really easy to implement. 2. Globe does not run full fingerprints through SHA-1 on the server and instead forwards everything to Onionoo. - Not only the Globe server, but also the Onionoo server learns bridge fingerprints. + We teach users that they shouldn't copy their original bridge fingerprint into the Globe search window. For example, we could tell them the single line of Python they need to run their fingerprint through SHA-1. While they might have "spoiled" their first bridge, they'll know for their other bridges. - Nobody will understand the above. 3. Tor prints out the SHA-1 fingerprint to its logs or to a hashed-fingerprint file. + Bridge users who know to read the fingerprint file can as well read the hashed-fingerprint file or the logs. We could even add a link "How do I search for my bridge" to the Globe page and explain to look out for that file or read the logs. - Only very few people will find that file or click the link. We'll have to do either 1 or 2 in addition to 3. 4. Sanitized bridge descriptors contain original fingerprints. + It will be a lot easier to search for bridges if it works the same way as searching for relays. + The sanitizing process will be easier. - We lose the option to enable bridge clients to update their bridge descriptors by sending the bridge fingerprint to the bridge authority. - This is going to keep me busy for a while for updating the various metrics code that expects SHA-1 fingerprints rather than original fingerprints. I can do that though. What do you think? All the best, Karsten

3 5

obfsproxy & openvpn
by George Kadianakis 10 Feb '14

10 Feb '14

Hey Jurre, till we fix #9221 (the issue of obfsproxy not having a SOCKS5 listener), you might be able to work around this problem by doing what kheops did here: https://github.com/kheops2713/telecomix-openvpn/commit/83984d822290f3445adf… Basically he is using the external mode of obfsproxy, where he makes the client forward traffic to a specific IP/port instead of starting up a SOCKS listener. Might work...

2 1

Weather Rewrite Update
by Abhiram Chintangal 07 Feb '14

07 Feb '14

Hello Guys, I have made some small changes to the existing dajngo app, It would be great if you guys can provide some feedback. Since we might have different settings for testing and production and so on , I made a separate module for it [1]. For now, I check for the debug flag in the updaters module and disable email sending. Here is a link to the new commit [2] Thanks! [1] http://www.rdegges.com/the-perfect-django-settings-file/ [2] https://github.com/achintangal/weather/commit/68035eeea9d26c7f64efd9dbe5484… -- Abhiram Chintangal

1 0

Your 'Relay Web Status Panel' GSoC idea (was: r26589: {website} Adding 'Relay Web Status Panel' Adding a project I'm interes (website/trunk/getinvolved/en))
by Karsten Loesing 07 Feb '14

07 Feb '14

Hi Damian, this sounds like a fun project and a great GSoC project idea! Some quick feedback: Have you considered suggesting this relay web status panel for a possible Torouter? I could imagine making this the primary interface for a little Torouter box, so that people don't have to ssh into it or install a desktop application to connect to it. Which brings up two my questions: why not make this a "relay web status and *control* panel"? Are the security risks really that much higher compared to using arm? And if you distrust the web server part of this design, how do you prevent an attacker from breaking into the web server and abusing the controller connection to reconfigure the relay? And question number two: why not make this a "relay and *client* web status and control panel"? In the Torouter case, people might want to use their tor only as a client to route all their connections through tor. Think of a little box that provides wifi access to nearby strangers but tunnels everything through the local tor client to avoid legal trouble. Or, if strangers need anonymity, the guy running the box could configure it as (private) bridge and mention the address on its local website for strangers to use with their own tor client. I'm not at all suggesting to include all this in the GSoC project. But maybe these ideas could be mentioned or kept in mind? Looking forward to seeing this happen! All the best, Karsten On 05/02/14 18:01, Damian Johnson wrote: > Author: atagar > Date: 2014-02-05 17:01:31 +0000 (Wed, 05 Feb 2014) > New Revision: 26589 > > Modified: > website/trunk/getinvolved/en/volunteer.wml > Log: > Adding 'Relay Web Status Panel' > > Adding a project I'm interested in mentoring this summer. > > > > Modified: website/trunk/getinvolved/en/volunteer.wml > =================================================================== > --- website/trunk/getinvolved/en/volunteer.wml 2014-02-05 03:18:48 UTC (rev 26588) > +++ website/trunk/getinvolved/en/volunteer.wml 2014-02-05 17:01:31 UTC (rev 26589) > @@ -629,6 +629,7 @@ > > Project Ideas: > <a href="#txtorcon-stemIntegration">Txtorcon/Stem Integration</a> > + <a href="#relayWebPanel">Relay Web Status Panel </a> > > > <a id="project-txtorcon"></a> > @@ -917,6 +918,53 @@ > bonus points if it's Twisted. > </li> > > + <a id="relayWebPanel"></a> > + <li> > + Relay Web Status Panel > + > + Effort Level: Medium > + > + Skill Level: Medium > + > + Likely Mentors: Damian (atagar) > + > + Relay operators presently have a couple options for monitoring the status > + of their relay: <a > + href="https://www.torproject.org/getinvolved/volunteer.html.en#project-vidalia">Vidalia</a> > + which is a gui and <a href="https://www.atagar.com/arm/">arm</a> which uses > + curses. This project would be to make a new kind of monitor specifically > + for relay operators that provides a status dashboard site on localhost. > + > + > + The interface will likely <a > + href="https://www.atagar.com/arm/screenshots.php">borrow heavily from > + arm</a>, except of course in areas where we can improve upon it. Two > + important design constraints is that a localhost controller provides a > + bigger attack surface than guis or curses, so we should be a little more > + wary of what it does. This should be a read-only controller (ie, you can't > + *do* anything to the relay) and by default not surface any sensitive > + information (such as arm's connection panel). > + > + > + This project will likely include two parts: an AJAX site and a localhost > + daemon to fulfill those requests. <a > + href="https://stem.torproject.org/">Stem</a> is the backend of arm, and can > + be used to get everything you see in arm's interface (making it a natural > + choice for the daemon). That said, this project might entail some Stem > + improvements if we run across any gaps. > + > + > + Applicants should be familiar with Python, JavaScript, and learn about > + <a href="https://stem.torproject.org/">Stem</a>. As part of your > + application for this project please make both mockups of the interface and > + a proof of concept demo application using JS to surface something with > + Stem. <a > + href="https://trac.torproject.org/projects/tor/wiki/doc/stem/bugs">Involvement > + with Stem development</a> during the application process is also a big > + plus. > + > + </li> > + > <a id="httpsImpersonation"></a> > <li> > HTTPS Server Impersonation > > _______________________________________________ > tor-commits mailing list > tor-commits(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-commits >

3 2

[PATCH] Remove the trailing slash.
by Nikita Karetnikov 07 Feb '14

07 Feb '14

If you apply the attached patch atop of this one [1], you should be able to see the following. $ dist/build/runtests/runtests Cases: 21 Tried: 21 Errors: 0 Failures: 0 (The tests only check encoders/decoders, so you don’t need to have networking enabled.) Is there anything else I can do? [1] https://trac.torproject.org/projects/tor/attachment/ticket/9419/tordnsel.ca…

1 0

Re: [tor-dev] Help hacking Mumble
by Matt 07 Feb '14

07 Feb '14

On Thu, 30 Jan 2014 22:01:18 +0000 Matt <matt(a)pagan.io> wrote: Here is one of the Mumble developers' take on the issue: > I don't know where @zanethomas's post went, but I just want to clarify > what needs to be done here, and what's wrong. > > By default, Qt's proxy support can route hostname resolution through > the proxy if a hostname is used when creating the socket. > > However, in Mumble, we use QHostAddress to resolve addresses ahead of > time in a lot of places (for example, the server list), which causes > the DNS requests to not go through the proxy. > > We need some kind of abstraction on top of QHostAddress that allows us > to route the hostname lookups via the user's selected proxy. That, or > we need to upstream a Qt patch that allows QHostAddress to resolve > through the application's QNetworkProxy. > > ...Something along those lines. Source: https://github.com/mumble-voip/mumble/issues/1033

2 1

Web Relay Status Dashboard Brainstorming
by Damian Johnson 06 Feb '14

06 Feb '14

Hi Karsten, thanks for the feedback! Updating the subject so it's clearer to others that this is a brainstorming thread for a web relay status dashboard. For people just jumping in on this thread, we're discussing a possible GSoC project for a new controller interface... https://www.torproject.org/getinvolved/volunteer.html.en#relayWebPanel > this sounds like a fun project and a great GSoC project idea! > > Some quick feedback: Have you considered suggesting this relay web status > panel for a possible Torouter? I could imagine making this the primary > interface for a little Torouter box, so that people don't have to ssh into > it or install a desktop application to connect to it. Great point. Added a note regarding it... https://lists.torproject.org/pipermail/tor-commits/2014-February/068398.html > Which brings up two my questions: why not make this a "relay web status and > *control* panel"? Are the security risks really that much higher compared > to using arm? And if you distrust the web server part of this design, how > do you prevent an attacker from breaking into the web server and abusing the > controller connection to reconfigure the relay? Certainly could. I'd like for the daemon that services the AJAX requests to be customizable via a local config. By default it would only allow read-only requests for non-sensitive data (ie, not connections). However, the operator could easily allow those to make the panel more capable. The reason that I'm more wary of a web panel than a local application like arm is... * Network facing applications like web browsers and mail clients tend to be a more expose attack surface. To do something malicious with arm an attacker would need local filesystem access for the authentication cookie or control socket (at which point they could just connect to the control port directly - arm isn't adding to the risk). With a web panel, however, all they need to do is trick the web browser into issuing AJAX requests to localhost. * The daemon that services AJAX requests is essentially providing a method to bypass Tor's control port authentication. The daemon authenticates to Tor, but the daemon's users don't necessarily have access to the filesystem. > And question number two: why not make this a "relay and *client* web status > and control panel"? In the Torouter case, people might want to use their > tor only as a client to route all their connections through tor. Think of a > little box that provides wifi access to nearby strangers but tunnels > everything through the local tor client to avoid legal trouble. Or, if > strangers need anonymity, the guy running the box could configure it as > (private) bridge and mention the address on its local website for strangers > to use with their own tor client. > > I'm not at all suggesting to include all this in the GSoC project. But > maybe these ideas could be mentioned or kept in mind? Certainly could. If this project really takes off then I'd love to see it expand to cover additional use cases. I'm just starting with relays to give the project an initial scope. > Looking forward to seeing this happen! Me too! I hope we get great applicants for this and the txtorcon/stem integration projects. Cheers! -Damian

1 0

Status Report - Torsocks 2.x
by David Goulet 05 Feb '14

05 Feb '14

Greetings everyone! After Fosdem 2014, I had the chance to talk with a lot of people about Torsocks and the current status. It appears it's actually quite used! (more than I thought). I've already received multiple contributions after a quick announcement of the project during my talk. This email is to provide a quick status report on what's going on with this effort [1]. In a nutshell, unfortunately, it has been almost two months since the last commit but free time is coming back to me thus expect *MORE* activity. Nick did a *very* nice review of the code base [2] that I finally answered recently [3] but yet to fix stuff upstream. Feel free to jump in the discussion to help or even contribute. Yawning and I had a discussion on #tor-dev about the use of a bitmap instead of an hash table to store connection pointers indexed by file descriptor. After looking in the BSD and Linux code base, we ended up seeing that both OSes allocate the lowest FD everytime a new socket is created which makes thing much more easier to manage a bitmap. Apart from a performance stand point of view, it appears that there is no compelling reasons to move the connection registry from an HT (current state) to a bitmap. Feel free to speak up here if you see any reasons! Finally, Luke Gallagher will continue to contribute tests to the project so big thanks! [4] After the Tor dev. meeting this month, hopefully things will be close to ready for a full on release, I'll most probably write a blog post about this effort on the why it has been rewritten, the current state/limitation and upcoming features. Cheers! David [1] https://github.com/dgoulet/torsocks [2] https://lists.torproject.org/pipermail/tor-dev/2013-December/005886.html [3] https://lists.torproject.org/pipermail/tor-dev/2014-January/006101.html [4] https://lists.torproject.org/pipermail/tor-dev/2014-January/006136.html

1 0

Exitmap Patches
by Damian Johnson 03 Feb '14

03 Feb '14

Hi Philipp. For kicks and giggles I decided to dig into Exitmap a bit this weekend. Patches are available from... https://git.torproject.org/user/atagar/exitmap.git The commits give a pretty good description of the changes but here's a quick summary... * Changes the codebase to be PEP8 compliant. This was the largest (and most time consuming) change. [1] * Adds a run_tests.py executable that runs pyflakes and pep8 checks. [2] * Uses cookie auth for the tor instance we spawn. * Rewrites the get_exits() function. * Swaps some of the threads we use to be daemons. All in all nothing terribly impactful. Cheers! -Damian [1] https://gitweb.torproject.org/user/atagar/exitmap.git/commitdiff/57e0be3?hp… [2] https://gitweb.torproject.org/user/atagar/exitmap.git/blob/HEAD:/run_tests.…

2 1

Help hacking Mumble
by Matt 03 Feb '14

03 Feb '14

If you're into C++ or Qt, we could use some help. Mumble[0] is a free software VoIP application that various people have had success using with Tor. One problem using Mumble with Tor is that Mumble doesn't abide by the native proxy settings when looking up Mumble servers[1]. When using a Tor router device or Torsocks, the DNS requests are just dropped. When using the native proxy settings, the DNS requests still get sent out to the local ISP, not through the proxy. The desired outcome is that if a SOCKS5 proxy is set, DNS requests get sent through the SOCKS5 proxy. The proxy is set as g.s.ptProxyType, and gets set when main.cpp calls NetworkConfig::SetupProxy(), which in turn uses QNetworkProxy::setApplicationProxy. The window that makes these DNS requests is ConnectDialog.cpp, but you'll want to look at Cert.cpp as well. I submitted a pull request[2] that allows users to completely turn off automatic UDP and DNS traffic sent out to the big list of public Mumble Servers. This does not stop the automatic DNS traffic that gets sent to servers on the Favorite servers list though, which is where any private servers you might want to connect to would be. Getting this fixed would be great for Tor because it would make VoIP over Tor easier and safer for people on all platforms. Please help! [0]: http://mumble.info [1]: https://github.com/mumble-voip/mumble/issues/1033 [2]: https://github.com/mumble-voip/mumble/pull/1128

3 2

Re: [tor-dev] Help hacking Mumble
by Matt 03 Feb '14

03 Feb '14

On Fri, 31 Jan 2014 22:17:54 +0000 Andrew F <andrewfriedman101(a)gmail.com> wrote: > Matt, have you gotten any response? > On IRC, Yawning provided me some useful recommendations for further investigation. Any progress I make on this issue will be slow-going though, and getting a fresh set of eyes on this issue would helpful indeed.

1 0

New GPG key for Runa A. Sandvik
by Runa A. Sandvik 02 Feb '14

02 Feb '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi everyone, Tomorrow is my first day as a staff technologist at the Center for Democracy and Technology in Washington, D.C. I will continue to contribute to the Tor Project in one way or another. This new key will be used to sign email from me going forward. The new key is attached and is also available on the keyservers (with a signature from my old key) at: http://pgp.mit.edu/pks/lookup?op=get&search=0x094D46E2898DB6D5 Here's the fingerprint and current subkey information for reference: pub 4096R/898DB6D5 2014-02-02 Key fingerprint = BEBA DB46 6778 5EF9 B00E D141 094D 46E2 898D B6D5 uid Runa A. Sandvik (offline long term identity key) <runa.sandvik(a)gmail.com> uid Runa A. Sandvik (offline long term identity key) <runa(a)cdt.org> uid Runa A. Sandvik (offline long term identity key) <runa(a)torproject.org> sub 3072R/AC8C6CF6 2014-02-02 [expires: 2015-02-02] sub 3072R/FC71640D 2014-02-02 [expires: 2015-02-02] - -- Runa A. Sandvik -----BEGIN PGP SIGNATURE----- iQGcBAEBCgAGBQJS7qvsAAoJECgKOvb8cWQNT1UL/joI4P1++EIUPmudw179o+j3 4FxEV32snfYNwW9NTaQNwGODglu7EMZRQ4V1UhytXUb9P7taT0eDgiJxPIvC8Okw xZ7jmk5UnV+R37rMLuhcX1Ifwg3Z+tG0qAwJAbFhO/zZX3yTx+ZJoQBjZ4pX5SFQ qE5G7IeLUrWEAIZfm6mLp3M/kh/+8+e5MPsxrdywo9iLi3Y2IB9mO5VVb94qA5Ul l0ETWilusa9PNRD70jtWhQL6EjPS1tkKVbMFk04naDLr+74gwFAta38tJcnZtPId jZXhqpfqcxD0muaEwyvGtV2qfAxbDuDPDxVJofMeK5S5GESOe4AqLyrkMf4MPz4Z ifKFguKpYAiA4Hz9+v4AoZPhX1Zu33AV/4gfE/Q4ZHq3DtqzEkcyzvvEw9kL0pk+ lJC7+R58dJiJ7pGedG9ZCyJugyp6pujiD34RppPL90JxnP5s3V11VkBiv5ZEnB+k kvBq94TkyFD8bUncFhmh4ilkloPDOrqMcfYGl1VjGg== =KZvl -----END PGP SIGNATURE-----

1 0

Pluggable transports meeting tomorrow (Friday 31st of January 2014)
by George Kadianakis 30 Jan '14

30 Jan '14

Greetings humans, this is an email to remind you that the regular biweekly pluggable transports meeting is going to happen tomorrow. Place is the #tor-dev IRC channel in OFTC. Time is 17:00 UTC. Cheers!

1 0

Damian's Status Report - January 2014
by Damian Johnson 30 Jan '14

30 Jan '14

Hi all. This month I decided to switch from a depth-first to breadth-first approach for overhauling arm. Changes included... * Made arm PEP8 compliant. This took around a week during which sed and I were close friends. Like Stem, both pyflakes and pep8 checks are now included in arm's test runs. * Completely dropped tor_tools from the codebase. This is a neat milestone as tor_tools was a 2,700 line wrapper around TorCtl providing a friendlier API, thread safety, caching, etc. Stem provides all of these capabilities. Initially migration was simply switching tor_tools from TorCtl to Stem, but now arm works directly with a Stem controller allowing us to drop the tor_tools module completely. * Finished arm's new tracker module which provides tor resource usage, connections, and application names. It's not just much nicer code, but now has unit tests! Why am I sinking all this effort into arm? After all, it presently does pretty much what we want, right? Well, yes. And I suspect user's won't see many obvious changes in the next arm release. Rather, my present focus on arm is partly for code quality but more importantly to make sure Stem is ready for prime time. Arm and Vidalia are our two most complicated and demanding controller applications, and making a Stem-based arm release is a surprisingly good way of making sure Stem has all the pieces we need for tasks like a next-gen Tor GUI or a localhost status panel for relay operators. As I've rewritten arm Stem has improved as well. Stem changes this month included... * Added a port_usage() function for getting a description of a port's common usage. This nicely complements our other connection utils... >>> from stem.util.connection import port_usage >>> port_usage(80) 'HTTP' >>> port_usage(22) 'SSH' >>> port_usage(9418) 'Git' * Couple new Controller methods to help with NEWNYM handling: is_newnym_available() and get_newnym_wait(). * Pyflakes and pep8 are python modules so why shell out to them? We now use their APIs instead reduced the runtime of Stem's static checks by 60%. * Greatly reduced the verbosity of stem's test output. You can still get the previous, more detailed results with the '--verbose' argument. Last but not least, non-development tasks from this month included... * Finally moved doctor to its new home on cappadocicum. (#10413) * Submitted my Tor Ecosystem talk to LinuxFest Northwest... http://linuxfestnorthwest.org/2014/sessions/tor-ecosystem-developers-guide-… * Tickets for Iceland. Looking forward to seeing everyone there! Cheers! -Damian

1 0

Am I looking at the right spec for pluggable transports?
by Ox Cart 30 Jan '14

30 Jan '14

I'm interested in writing a pluggable transport for Tor. Is this spec a good place to start? https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/180-pluggabl… Cheers, Ox ----------------------------------------------------------------------------------------- "I love people who harness themselves, an ox to a heavy cart, who pull like water buffalo, with massive patience, who strain in the mud and the muck to move things forward, who do what has to be done, again and again." - Marge Piercy

4 5

Key revocation in Next Generation Hidden Services
by George Kadianakis 29 Jan '14

29 Jan '14

To achieve offline key storage in the new HS design, hidden service are using three layers of keys: (Skip the next three paragraphs if you know this stuff) Each hidden service has a "long-term master identity key". This is the key that is encoded in its onion address. Using the long-term identity key, the hidden service generates "ephemeral blinded signing" keys according to #8106. This key lasts for a short period of time (probably for a day or so; not decided yet). When the hidden service needs to make a new HS descriptor, it generates a "descriptor signing keypair" and signs it with the blinded signing key. It then includes in the descriptor the public part of the descriptor signing key as well as its signature by the blinded signing key. It finally signs the descriptor with the private part of the descriptor signing key. As described in the spec [0] this allows the HS to generate offline a bunch of blinded signing keys and descriptor signing keys, copy them to the online HS host, and let the host use those keys for the next few days or so. As a result, an attacker that owns the online HS host only gets access to the keys for a few days ahead. The attacker doesn't get access to the long-term identity keys. Of course, this doesn't solve the problem of how Hidden Services can revoke compromised keys. Having the attacker impersonate the HS even for a few days is not acceptable. Unfortunately, PKI revocation solutions are always messy and don't work really well (look at SSL's OCSP and CRLs). The question becomes how the legitimate Hidden Service can inform a client that its keys got compromised. A client that connects to a Hidden Service first fetches the consensus from the directory system, then fetches the HS descriptor from the HSDirs and finally connects to the HS. This probably means that the client should be informed of the compromise either by the directory system or by the HSDirs. If we wanted to use the directory system, we could imagine some sort of CRL scheme where hidden services notify the authorities about compromised keys, then the authorities pass the CRL to the directory servers, and the directory servers finally give the CRL to HS clients. This system might be possible theoretically, but it poses many engineering and security questions: * Are users supposed to fetch the CRL everytime they connect to a HS? This might be dangerous identifying behavior. * What happens if the CRL gets too big because the adversary fills it with fake revocations? The directory system can't handle big documents so there will need to be some sort of size limit (and old entries will need to be resetted frequently). * The list of compromised keys suddenly becomes public information. This might not be a good idea. On the other hand, if we wanted to use the HSDirs, we could imagine the HS sending some sort of revocation message to the responsible HSDirs so that they stop serving descriptors with compromised keys. Unfortunately, this scheme treats HSDirs as trusted parties, since they can simply ignore the revocation and continue passing the evil descriptor to clients. We could decrease the chance of this happening by implementing #8244 and also having the clients use multiple HSDirs to fetch descriptors. This will force the attacker to corrupt multiple HSDirs for the attack to succeed. Still the solution is not very elegant. What else should we consider? For example (crazy ideas ahead) could we solve this problem more elegantly by adding more layers of keys? Or maybe by adding yet another network entity which handles revocations [like in Dan Boneh et al. "A Method for Fast Revocation of Public Key Certificates and Security Capabilities." paper (be aware of crazy crypto)]? Or maybe we could look into certificate transparency; although transparency for centralized PKIs (like SSL-CA) sounds like a good idea, transparency for decentralized PKI systems with complex privacy threat models might not be a good idea. [0]: https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/224-rend-spe…

2 1

Fwd: [OpenITP Dev] Python & Javascript volunteer?
by Griffin Boyce 29 Jan '14

29 Jan '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 OpenITP & TA3M have had a python dev ask if STEM or txtorcon need assistance =) For more details, talk to Sandy (SandraOrdonez(a)openitp.org) ~Griffin - -------- Original Message -------- Subject: [OpenITP Dev] Do u needa Python & Javascript volunteer? Date: Wed, 29 Jan 2014 10:45:46 -0500 From: Sandra <SandraOrdonez(a)openitp.org> Organization: OpenITP To: dev(a)lists.openitp.org I've started asking at TA3M if people are interested in volunteering to shot me an email with qualifications. Do any of your projects need someone like this "....You said I should shoot you an email since I was one of the women in New York and also to get involved volunteering. I would like to help with development for one of the projects - I am currently programming in Python and JavaScript - I do Django development, and I have some background in security. I saw on the site that stem and txtorcon are in Python - do either of those groups need assistance? Also feel free to ask if you guys need help with set up or organizing local events. Let me know, I'll make intros :) _______________________________________________ Dev mailing list Dev(a)lists.openitp.org https://lists.openitp.org/mailman/listinfo/dev -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iQEcBAEBAgAGBQJS6SLCAAoJEAPPSgqzx5pjUUEH/1ybIOkHrUxnkBs0Sjmu4TP9 O/zsPLFFnGFOvDFrQCsv8nErNTgeSivpTT9I2+4FeIpDu4VEEdwnyCOe7qpAZE63 sef/zX2oZaMPAlM8Fbyjz1jUCUXgo1SsSFGpEGK35WiQi1O3Xa5kbBRn05oc7jvU Qspjd9wf/C6A23usK2Jq3XA7/sJf8oCXl+f6zBXTb+yhXJX+ZqunwYb6Xks2De3U pzTaOuVChcmlfFGTwPKND1io4Bhx/coi3+ANg2fNvHkyTLZRLHly7TkjRhc34lPs 9hCeMsfjoPHqFNu1zUSZ+qTmcmMtSiY/SaG8Kmkpz5UeCgkCWJR23VksTiXAiP4= =PWtb -----END PGP SIGNATURE-----

2 1

Tor-Weather-Rewrite Progress
by Abhiram Chintangal 29 Jan '14

29 Jan '14

Hello Guys, Here is the progress from my side. I have written a simple python helper module for Onionoo communication, mostly inspired by "stemUtil" module from the current weather implementation. For now, it checks for new documents with Onionoo and exposes the data through some simple functions. I guess once it done, I should be able to test it with the django app. Let me know if you have any questions. Thanks! -- Abhiram Chintangal

2 1

Making microdescriptor tarballs available on metrics.tpo
by Karsten Loesing 29 Jan '14

29 Jan '14

Hi Damian, hi devs, I'm planning to make microdescriptor tarballs available on the metrics website that contain both microdescriptor consensuses and microdescriptors. Some background: Recent Tor clients don't download the network status consensus and full server descriptors anymore, but instead download the microdescriptor consensus and the microdescriptors referenced from it. We didn't provide these formats on the metrics website yet, because they are derived from the formats we already provide and don't contain anything novel. But having the new formats will, for example, make it easier for developers to analyze the directory protocol and for researchers to understand what information is available to clients to make path selection decisions. If you need more background, see #2785 and search for "microdesc" in dir-spec.txt. Here's a sample tarball: https://people.torproject.org/~karsten/microdescs-2014-01.tar.bz2 Damian, can you try to parse these descriptors using stem, to see if the descriptor annotations are correct and if stem can parse them without issues? If all goes well, microdescriptor tarballs will start to be available on the metrics website before the end of the month. All the best, Karsten

2 3