- tor-dev - lists.torproject.org

Re: [tor-dev] [tor-reports] Sebastian's August 2015
by ilv 11 Sep '15

11 Sep '15

On 06/09/15 13:01, Sebastian Hahn wrote: > Hi there, > Hi Sebastian, > Next up is more of the same, especially focusing on website tickets > and preparing the community team's dev meeting contributions. Maybe we could have a session at the dev meeting to talk about the website (content, structure, translations, etc.). What do you think? (cc'ing @tor-dev in case more people are interested) Saludos. --ilv

2 1

[PATCH] Update prop#188 to match implementation from #7144.
by Isis Lovecruft 10 Sep '15

10 Sep '15

* ADD a new section detailing loose-source routed circuits, including: - How the circuit should appear to the OP, the bridge, and the bridge guard, - How the hop(s) in the path is/are chosen, - How the first hop is handled and how circuit extension is handled, in a generalised sense (i.e. not specific to a bridge using bridge guards), - Which cell types are used and how they are chosen, - When the original create cell is answered (in relation to when cells are sent to the other additional hops), - What should be done when relay cells are received when the additional hops in the loose-source routed circuit are not fully constructed, - How the circuit crypto and cell digests for incoming/outgoing relay cells works (again, in a generalised sense, i.e. not specific to bridges using bridge guards), - How and whether a given relay cell is treated as "recognized", - Under what circumstances (based on whether a stream is attached and which relay command was sent) should cause a node which uses loose-source routed circuits to drop a relay cell or mark a circuit for close, and, - When and what statistics should be gathered for loose-source routed circuits. * UPDATE and clarify the example loose-source routed circuit construction (the original one from the proposal which was specific to a client using a bridge that uses bridge guards). * CHANGE the "Make it harder to tell clients from bridges" section which described the tradeoffs between using CREATE versus CREATE_FAST cells for the first additional hop (i.e. the bridge guard). Those concerns is no longer entirely relevant, since, in the meantime, we've introduced the NTor handshake and it's extremely likely that both the client and the bridge will use CREATE2 for all their hops. * ADD a new section on enumerating bridges via the behaviours inherent to bridge reachability testing. * REMOVE open question (from the very end of the proposal) regarding whether the standard guard selection algorithms are adequate. They are. (Even if they are convoluted and overly-complicated.) --- proposals/188-bridge-guards.txt | 458 ++++++++++++++++++++++++++++++++-------- 1 file changed, 369 insertions(+), 89 deletions(-) diff --git a/proposals/188-bridge-guards.txt b/proposals/188-bridge-guards.txt index 5a5a005..c4d354f 100644 --- a/proposals/188-bridge-guards.txt +++ b/proposals/188-bridge-guards.txt @@ -1,7 +1,8 @@ Filename: 188-bridge-guards.txt Title: Bridge Guards and other anti-enumeration defenses -Author: Nick Mathewson +Author: Nick Mathewson, Isis Lovecruft Created: 14 Oct 2011 +Modified: 10 Sep 2015 Status: Open 1. Overview @@ -76,111 +77,317 @@ Status: Open remove, and remove a layer of encryption on incoming cells on that circuit corresponding to the encryption that the guard will add. -3.1. An example +3.1. Loose-Source Routed Circuit Construction - This example doesn't add anything to the design above, but has some - interesting inline notes. + Alice, an OP, is using a bridge, Bob, and she has chosen the + following path through the network: - - Alice has connected to her bridge Bob, and built a circuit - through Bob, with the negotiated forward and reverse keys KB_f - and KB_r. + Alice -> Bob -> Charlie -> Deidra - - Alice then wants to extend the circuit to node Charlie. She - makes a hybrid-encrypted onionskin, encrypted to Charlie's - public key, containing her chosen g^x value. She puts this in - an extend cell: "Extend (Charlie's address) (Charlie's OR - Port) (Onionskin) (Charlie's ID)". She encrypts this with - KB_f and sends it as a RELAY_EARLY cell to Bob. + However, Bob has decided to take advantage of the loose-source + routing circuit characteristic (for example, in order to use a bridge + guard), and Bob has chosen N additional loose-source routed hop(s), + through which he will transparently relays cells. - - Bob receives the RELAY_EARLY cell, and decrypts it with KB_f. - He then sees that it's an extend cell for him. + NOTE: For the purposes of bridge guards, N is always 1. However, for + completion's sake, the following details of the circuit construction + are generalized to include N > 1. Additionally, the following steps + should hold for a hop at any position in Alice's circuit that has + decided to take advantage of the loose-source routing feature, not + only for bridge ORs. - So far, this is exactly the same as the current procedure that - Alice and Bob would follow. Now we diverge: + From Alice's perspective, her circuit path matches the one diagrammed + above. However, the overall path of the circuit is: - - Instead of connecting to Charlie directly, Bob makes sure that - he is connected to his guard, Guillaume. Bob uses a - CREATE_FAST cell (or a CREATE cell, but see 4.1 below) to open a - circuit to Guillaume. Now Bob and Guillaume share keys KG_f - and KG_b. + Alice -> Bob -> Guillaume -> Charlie -> Deidra - - Now Bob encrypts the Extend cell body with KG_f and sends it - as a RELAY_EARLY cell to Guillaume. + From Bob's perspective, the circuit's path is: - - Guillaume receives it, decrypts it with KG_f, and sees: - "Extend (Charlie's address) (Charlie's OR Port) (Onionskin) - (Charlie's ID)". Guillaume acts accordingly: creating a - connection to Charlie if he doesn't have one, ensuring that - the ID is as expected, and then sending the onionskin in a - create cell on that connection. + Alice -> Bob -> Guillaume -> Charlie -> UNKNOWN - Note that Guillaume is behaving exactly as a regular node - would upon receiving an Extend cell. + Interestingly, because Bob's behaviour towards Guillaume and choices + of cell types is that of a normal OP, Guillaume's perspective of the + circuit's path is: - - Now the handshake finishes. Charlie receives the onionskin - and sends Guillaume "CREATED g^y,KH". Guillaume sends Bob - "E(KG_r, EXTENDED g^y KH)". (Charlie and Guillaume are still - running as regular Tor nodes do today). + Bob -> Guillaume -> Charlie -> UNKNOWN - - With this extend cell, and with all future relay cells - received on this circuit, Bob first decrypts the cell with - KG_r, then re-encrypts it with KB_r, then passes it to Alice. - When Alice receives the cell, it will be just as she would - have received if Bob had extended to Charlie directly. + That is, to Guillaume, Bob appears (for the most part) to be a + normally connecting client. (See §4.1 for more detailed analysis.) - - With all future outgoing cells that he receives from Alice, - Bob first decrypts the cell with KA_f, and if the cell does - not have Bob as its destination, Bob encrypts it with KG_f - before passing it to Guillaume. +3.1.1. Detailed Steps of Loose-Source Routed Circuit Construction - Note that this design does not require that our stream cipher - operations be commutative, even though they are. + 1. Connection from OP - Note also that this design requires no change in behavior from any - node other than Bob the bridge. + Alice has connected to Bob, and she has sent to Bob either a + CREATE/CREATE_FAST or CREATE2 cell. - Finally, observe that even though the circuit is one hop longer - than it would be otherwise, no relay's count of permissible - RELAY_EARLY cells falls lower than it otherwise would. This is - because the extra hop that Bob adds is done with a CREATE_FAST - cell, and so he does not need to send any RELAY_EARLY cells not - originated by Alice. + 2. Loose-Source Path Selection -4. Other ideas and alternative designs + In anticipation of Alice's first RELAY_EARLY cell (which will + contain an EXTEND cell to Alice's next hop), Bob begins + constructing a loose-source routed circuit. To do so, Bob chooses + N additional hop(s): - In addition to the design above, there are more ways to try to - prevent enumeration. + 2.a. For the first additional hop, H_1, Bob chooses a suitable + entry guard node, Guillaume, using the same algorithm as OPs. + See "§5 Guard nodes" of path-spec.txt for additional + information on the selection algorithm. -4.1. Make it harder to tell clients from bridges + 2.b. Each additional hop, [H_2, ..., H_N], is chosen at random + from a list of suitable, non-excluded ORs. - Right now, there are multiple ways for the node after a bridge to - distinguish a circuit extended through the bridge from one - originating at the bridge. (This lets the node after the bridge - tell that a bridge is talking to it.) + 3. Loose-Source Routed Circuit Extension and Cell Types - One of the giveaways here is that the first hop in a circuit is - created with CREATE_FAST cells, but all subsequent hops are created - with CREATE cells. In the above design, it's no longer quite so - simple to tell, since all of the circuits that extend through a - bridge now reach its guards through CREATE_FAST cells, whether the - bridge originated them or not. + Bob now follows the same procedure as OPs use to complete the key + exchanges with his chosen additional hop(s). - (If we adopt a faster circuit extension algorithm -- for example, - Goldberg, Stebila, and Ustaoglu's design instantiated over - curve25519 -- we could also solve this issue by eliminating - CREATE_FAST/CREATED_FAST entirely, which would also help our - security margin a little.) + While undergoing these following substeps, Bob SHOULD continue to + proceed with Step 4, below, in parallel, as an optimization for + speeding up circuit construction. - The CREATE/CREATE_FAST distinction is not the only way for a - bridge's guard to tell bridges from orginary clients, however. - Most importantly, a busy bridge will open far more circuits than a - client would. More subtly, the timing on response from the client - will be higher and more highly variable that it would be with an - ordinary client. I don't think we can make bridges behave wholly - indistinguishably from clients: that's why we should go with guard - nodes for bridges. + 3.a. Create Cells + + Bob sends the appropriate type of create cell to Guillaume. + For ORs new enough to support the NTor handshake (nearly all + of them at this point), Bob sends a CREATE2 cell. Otherwise, + for ORs which only support the older TAP handshake, Bob sends + either a CREATE or CREATE_FAST cell, using the same + decision-making logic as OPs. + + See §4.1 for more information the distinguishability of + bridges based upon whether they use CREATE versus + CREATE_FAST. Also note that the CREATE2 cell has since + become ubiquitous after this proposal was originally drafted. + Thus, because we prefer ORs which use loose-source routing to + behave (as much as possible) like OPs, we now prefer to use + CREATE2. + + 3.b. Created Cells + + Later, when Bob receives a corresponding CREATED/CREATED_FAST + or CREATED2 cell from Guillaume, Bob extracts key material + for the shared forward and reverse keys, KG_f and KG_b, + respectively. + + 3.c. Extend Cells + + When N > 1, for each additional hop, H_i, in [H_2, ..., H_N], + Bob chooses the appropriate type of extend cell for H_i, and + sends this extend cell to H_i-1, who transforms it into a + create cell in order to perform the extension. To choose + which type of extend cell to send, Bob uses the same + algorithm as an OP to determine whether to use EXTEND or + EXTEND2. Similar to the CREATE* cells above, for most modern + ORs, this will very likely mean an EXTEND2 cell. + + 3.d. Extended Cells + + When a corresponding EXTENDED/EXTENDED2 cell is received for + an additional hop, H_i, Bob extracts the shared forward and + reverse keys, Ki_f and Ki_b, respectively. + + 4. Responding to the OP + + Now that the additional hops in Bob's loose-source routed circuit + are chosen, and construction of the loose-source routed circuit + has begun, Bob answers Alice's original CREATE/CREATE_FAST or + CREATE2 cell (from Step 1) by sending the corresponding created + cell type. + + Alice has now built a circuit through Bob, and the two share the + negotiated forward and reverse keys, KB_n and KB_p, respectively. + + Note that Bob SHOULD do this step in tandem with the loose-source + routed circuit construction procedure outlined in Step 3, above. + + 5. OP Circuit Extension + + Alice then wants to extend the circuit to node Charlie. She makes + a hybrid-encrypted onionskin, encrypted to Charlie's public key, + containing her chosen g^x value. She puts this in an extend cell: + "Extend (Charlie's address) (Charlie's OR Port) (Onionskin) + (Charlie's ID)". She encrypts this with KB_n and sends it as a + RELAY_EARLY cell to Bob. + + Bob's behaviour is now dependent on whether the loose-source + routed circuit construction steps (as outlined in Step 3, above) + have already completed. + + 5.a. The Loose-Source Routed Circuit Construction is Incomplete + + If Bob has not yet finished the loose-source routed circuit + construction, then Bob MUST store the first outgoing + (i.e. exitward) RELAY_EARLY cell received from Alice until + the loose-source routed circuit construction has been + completed. + + If any incoming (i.e. toward the OP) RELAY* cell is received + while the loose-source routed circuit is not fully + constructed, Bob MUST drop the cell. + + If Bob has already stored Alice's first RELAY_EARLY cell, and + Alice sends any additional RELAY* cell, then Bob SHOULD mark + the entire circuit for close with END_CIRC_REASON_TORPROTOCOL. + + 5.b. The Loose-Source Routed Circuit Construction is Completed + + Later, when the loose-source routed circuit is fully + constructed, Bob MUST send any stored cells from Alice + outward by following the procedure described in Step 6.a. + + 6. Relay Cells + + When receiving a RELAY* cell in either direction, Bob MAY keep + statistics on the number of relay cells encountered, as well as + the number of relay cells relayed. + + 6.a. Outgoing Relay Cells + + Bob decrypts the RELAY* cell with KB_n. If the cell becomes + recognized, Bob should now follow the relay command checks + described in Step 6.c. + + Bob MUST encrypt the relay cell's underlying payload to each + additional hop in the loose-source routed circuit, in + reverse: for each additional hop, H_i, in [H_N, ..., H_1], + Bob encrypts the relay cell payload to Ki_f, the shared + forward key for the hop H_i. + + Bob MUST update the forward digest, DG_f, of the relay cell, + regardless of whether or not the cell is recognized. See + 6.c. for additional information on recognized cells. + + Bob now sends the cell outwards through the additional hops. + At each hop, H_i, the hop removes a layer of the onionskin by + decrypting the cell with Ki_f, and then hop H_i forwards the + cell to the next addition additional hop H_i+1. When the + final additional hop, H_N, received the cell, the OP's cell + command and payload should be processed by H_N in the normal + manner for an OR. + + 6.b. Incoming Relay Cells + + Bob MUST decrypt the relay cell's underlying payload from + each additional hop in the loose-source routed circuit (in + forward order, this time): For each additional hop, H_i, in + [H_1, ..., H_N], Bob decrypts the relay cell payload with + Ki_b, the shared backward key for the hop H_i. + + If the cell has becomes recognized after all decryptions, Bob + should now follow the relay command checks described in Step + 6.c. + + Bob MUST update the backward digest, DG_b, of the relay cell, + regardless of whether or not the cell is recognized. See + 6.c. for additional information on recognized cells. + + Bob encrypts the cell towards the OP with KB_p, and sends the + cell inwards. + + 6.c. Recognized Cells + + If a relay cell, either incoming or outgoing, becomes + recognized (i.e. Bob sees that the cell was intended for him) + after decryption, and there is no stream attached to the + circuit, then Bob SHOULD mark the circuit for close if the + relay command contained within the cell is any of the + following types: + + - RELAY_BEGIN + - RELAY_CONNECTED + - RELAY_END + - RELAY_RESOLVE + - RELAY_RESOLVED + - RELAY_BEGIN_DIR + + Apart from the above checks, Bob SHOULD essentially treat + every cell as "unrecognized" by following the en-/de-cryption + procedures in Steps 6.a. and 6.b. regardless of whether the + cell is actually recognized or not. That is, since this is a + loose-source routed circuit, Bob SHOULD relay cells not + intended for him *and* cells intended for him through the + leaky pipe, no matter what the cell's underlying payload and + command are. + +3.1.2. Example Loose-Source Circuit Construction + + For example, given the following circuit path chosen by Alice: + + Alice -> Bob -> Charlie -> Deidra + + when Alice wishes to extend to node Charlie, and Bob the bridge is + using only one additional loose-source routed hop, Guillaume, as his + bridge guard, the following steps are taken: + + - Alice packages the extend into a RELAY_EARLY cell and encrypts + the RELAY_EARLY cell with KB_f to Bob. + + - Bob receives the RELAY_EARLY cell from Alice, and he follows + the procedure (outlined in §3.1.1. Step 6.a.) by: + + * Decrypting the cell with KB_f, + * Encrypting the cell to the forward key, KG_f, which Bob + shares with his guard node, Guillaume, + * Updating the cell forward digest, DG_f, and + * Sending the cell as a RELAY_EARLY cell to Guillaume. + + - When Guillaume receives the cell from Bob, he processes it by: + + * Decrypting the cell with KG_f. Guillaume now sees that it + is a RELAY_EARLY cell containing an extend cell "intended" + for him, containing: "Extend (Charlie's address) (Charlie's + OR Port) (Onionskin) (Charlie's ID)". + * Performing the circuit extension to the specified node, + Charlie, by acting accordingly: creating a connection to + Charlie if he doesn't have one, ensuring that the ID is as + expected, and then sending the onionskin in a create cell + on that connection. Note that Guillaume is behaving + exactly as a regular node would upon receiving an Extend + cell. + * Now the handshake finishes. Charlie receives the onionskin + and sends Guillaume "CREATED g^y,KH". + * Making an extended cell for Bob which contains + "E(KG_b, EXTENDED g^y KH)", and + * Sending the extended cell to Bob. Note that Charlie and + Guillaume are both still behaving in a manner identical to + regular ORs. + + - Bob receives the extended cell from Guillaume, and he follows + the procedure (outlined in §3.1.1. Step 6.b.) by: + + * Decrypting the cell with KG_b, + * Encrypting the cell to Alice with KB_b, + * Updating the cell backward digest, DG_b, and + * Sending the cell to Alice. + + - Alice receives the cell, and she decrypts it with KB_b, just + as she would have if Bob had extended to Charlie directly. + She then processes the extended cell contained within to + extract shared keys with Charlie. Note that Alice's behaviour + is identical to regular OPs. + +3.2. Additional Notes on the Construction -4.2. Client-enforced bridge guards + Note that this design does not require that our stream cipher + operations be commutative, even though they are. + + Note also that this design requires no change in behavior from any + node other than Bob, and as we can see in the above example in §3.1.2 + for Alice's circuit extension, Alice, Guillaume, and Charlie behave + identical to a normal OP and normal ORs. + + Finally, observe that even though the circuit N hops longer than it + would be otherwise, no relay's count of permissible RELAY_EARLY cells + falls lower than it otherwise would. This is because the extra hop + that Bob adds is done with RELAY_EARLY cells, then he continues to + relay Alice's cells as RELAY_EARLY, until the appropriate maximum + number of RELAY_EARLY cells is reached. Afterwards, further + RELAY_EARLY cells from Alice are repackaged by Bob as normal RELAY + cells. + +4. Alternative designs + +4.1. Client-enforced bridge guards What if Tor didn't have loose source routing? We could have bridges tell clients what guards to use by advertising those guard @@ -191,7 +398,7 @@ Status: Open Fortunately, we don't need to go down this path. So let's not! -4.3. Separate bridge-guards and client-guards +4.2. Separate bridge-guards and client-guards In the design above, I specify that bridges should use the same guard nodes for extending client circuits as they use for their own @@ -209,11 +416,84 @@ Status: Open through the bridge to a node it controls, and finding out where the extend request arrives from). -5. Other considerations +5. Additional bridge enumeration methods and protections + + In addition to the design above, there are more ways to try to + prevent enumeration. + + Right now, there are multiple ways for the node after a bridge to + distinguish a circuit extended through the bridge from one + originating at the bridge. (This lets the node after the bridge + tell that a bridge is talking to it.) + +5.1. Make it harder to tell clients from bridges + + When using the older TAP circuit handshake protocol, one of the + giveaways is that the first hop in a circuit is created with + CREATE_FAST cells, but all subsequent hops are created with CREATE + cells. + + However, because nearly everything in the network now uses the newer + NTor circuit handshake protocol, clients send CREATE2 cells to all + hops, regardless of position. Therefore, in the above design, it's + no longer quite so simple to distinguish an OP connecting through + bridge from an actual OP, since all of the circuits that extend + through a bridge now reach its guards through CREATE2 cells (whether + the bridge originated them or not), and only as a fallback (e.g. if + an additional node in the loose-source routed path does not support + NTor) will the bridge ever use CREATE/CREATE_FAST. (Additionally, + when using the fallback mathod, the behaviour for choosing either + CREATE or CREATE_FAST is identical to normal OP behaviour.) + + The CREATE/CREATE_FAST distinction is not the only way for a + bridge's guard to tell bridges from orginary clients, however. + Most importantly, a busy bridge will open far more circuits than a + client would. More subtly, the timing on response from the client + will be higher and more highly variable that it would be with an + ordinary client. I don't think we can make bridges behave wholly + indistinguishably from clients: that's why we should go with guard + nodes for bridges. + + [XXX For further research: we should study the methods by which a + bridge guard can determine that they are acting as a guard for a + bridge, rather than for a normal OP, and which methods are likely to + be more accurate or efficient than others. -IL] + +5.2. Bridge Reachability Testing + + Currently, a bridge's reachability is tested both by the bridge + itself (called "self-testing") and by the BridgeAuthority. + +5.2.1. Bridge Reachability Self-Testing + + Before a bridge uploads its descriptors to the BridgeAuthority, it + creates a special type of testing circuit which ends at itself: + + Bob -> Guillaume -> Charlie -> Bob + + Thus, going to all this trouble to later use loose-source routing in + order to relay Alice's traffic through Guillaume (rather than + connecting directly to Charlie, as Alice intended) is diminished by + the fact that Charlie can still passively enumerate bridges by + waiting to be asked to connect to a node which is not contained + within the consensus. + + We could get around this option by disabling self-testing for bridges + entirely, by automatically setting "AssumeReachable 1" for all bridge + relays… although I am not sure if this is wise. + +5.2.2. Bridge Reachability Testing by the BridgeAuthority + + After receiving Bob's descriptors, the BridgeAuthority attempts to + connect to Bob's ORPort by making a direct TLS connection to the + bridge's advertised ORPort. + + Should we change this behaviour? One the one hand, at least this + does not enable any random OR in the entire network to enumerate + bridges. On the other hand, any adversary who can observe packets + from the BridgeAuthority is capable of enumeration. + +6. Other considerations What fraction of our traffic is bridge traffic? Will this alter our circuit selection weights? - - Are the current guard selection/evaluation/replacement mechanisms - adequate for bridge guards, or do bridges need to get more - sophisticated? -- 2.1.4

1 0

Re: [tor-dev] How to introduce new RELAY_COMMAND_* types and newcell fields?
by tordev123＠Safe-mail.net 10 Sep '15

10 Sep '15

>> 3. Additional RELAY_COMMAND_* types for clients to request out-of-band >> HMAC request cells for Proposal 253. Do you need to request that data? How about always sending it from middle nodes? (Less leakage about the client.) >> 4. Additional RELAY_COMMAND_* opcodes for clients to request padding >> from relays (for an upcoming padding negotiation proposal). >> >> However, for items #3 and #4, if I introduce a new RELAY_COMMAND type >> and send it to a relay that doesn't support it, then that relay will >> emit a warning log message from connection_edge_process_relay_cell() in >> relay.c. How should I detect support? Based on advertised relay version >> in the consensus? What about non-standard relay implementations that >> don't use Tor's versioning? > > I don't think you can use the consensus for this: for HS connections you > wont have any relay info. Wouldn't matter for 4, right? > How about introducing very basic version info into > created2/extended/rendezvous cells? You'd have to be really careful to > not leak too much version info, of course. It seems much better to avoid that, if there is a workaround.

2 1

How to introduce new RELAY_COMMAND_* types and new cell fields?
by Mike Perry 10 Sep '15

10 Sep '15

I'm writing a handful of proposals that need to introduce either new cell sub-payloads or new command types. Specifically, I want to add: 1. Sub-fields to CELL_PADDING to allow clients to tell relays about the amount of link padding they want for Proposal 251. Mobile clients will probably want less or no padding, and need some way to say so. Adding this information to CELL_PADDING itself seemed to be an easy choice. 2. Sub-fields to RELAY_COMMAND_TRUNCATED and CELL_DESTROY to include HMAC data for Proposal 253. 3. Additional RELAY_COMMAND_* types for clients to request out-of-band HMAC request cells for Proposal 253. 4. Additional RELAY_COMMAND_* opcodes for clients to request padding from relays (for an upcoming padding negotiation proposal). I *think* that fields for items #1 and #2 can be simply added to the existing cell definitions, since we specify that cells should already be 0-filled, and I can rely on 0 fields to mean that the additional fields are absent. It should be OK if relays simply ignore/omit these fields until those proposals are implemented. Is this OK? However, for items #3 and #4, if I introduce a new RELAY_COMMAND type and send it to a relay that doesn't support it, then that relay will emit a warning log message from connection_edge_process_relay_cell() in relay.c. How should I detect support? Based on advertised relay version in the consensus? What about non-standard relay implementations that don't use Tor's versioning? -- Mike Perry

2 1

Fwd: Reproducibility of Pluggable Transports python.msi
by Joseph Bisch 09 Sep '15

09 Sep '15

I'm resending this, because tor-dev didn't accept my message originally. Sorry David and Jeremy for the double message. ____________________________________________________ Hello David and Jeremy, I am the "reproducible build specialist" that Jeremy was referring to, though I don't know if I would call myself that. As Jeremy said, Armory isn't yet reproducible on Windows, but hopefully some of this information will be helpful. First of all, none of the following has been wrapped in Gitian and checked for reproducibility yet, because I was primarily concerned with just getting cross-compiling of Python working first. But it is at least possible to cross-compile Python 3.4.3. I have a PR[0] open for Armory that covers the whole exe creation process using pyqtdeploy, but a lot of that is probably not relevant to Tor if something other than pyqtdeploy is used. So I will go over the parts specifically having to do with building Python from source. Also, the PR isn't fully working yet. I found that Python 2 is much more difficult to get cross-compiling than Python 3. So I decided to go with Python 3. I used the patches[1] that the Arch Linux Mingw Python 3 package uses. Before compiling Python, you will want to compile OpenSSL. I have instructions for building Python 3 from source. Since steps two and three are probably only relevant in Armory's case (we are building Python statically and then "freezing" it with pyqtdeploy, so all modules need to be embedded), you just have to download the Python 3.4.3 source and the Arch patches and run the build_py3.sh script[3]. That is the latest script, so don't use the one in the PR. The script is still a little fragile (everything seems to work with the build I did on my desktop, but importing json with the build I did on my laptop gives a PyInit_Thread error and importing hashlib gives a page fault). Also, I'm not sure if every flag/variable is necessary for the build to work. Also, my latest comment[4] on the PR explains some things I had to do to the OpenSSL and Python source trees to get the build working. I should probably make patches out of them and they may be appropriate for integration into OpenSSL and Python upstream. Though I don't know how my changes affect other system configurations, such as when someone is compiling on Windows. Hope that helps. I'd also appreciate any feedback anyone might have. Also, let me know if there is a better channel to talk about this, because getting Python to cross-compile isn't strictly tor-dev related. [0] - https://github.com/etotheipi/BitcoinArmory/pull/303 [1] - https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-python3 [2] - https://github.com/josephbisch/BitcoinArmory/blob/autotools-windows/mingwbu… [3] - https://gist.github.com/josephbisch/cf2d32a798f063f0bf71 [4] - https://github.com/etotheipi/BitcoinArmory/pull/303#issuecomment-138762224 Cheers, Joseph

1 0

fteproxy depends on obfsproxy...
by Kevin P Dyer 09 Sep '15

09 Sep '15

...and it shouldn't. Fortunately, the dependency is isolated to a single file. See [1]. My understanding is that pyptlib [2] is no longer maintained. wiley/asn/etc. - What's the proper way to remove this dependency, but make it easy for fteproxy to be a PT? -Kevin [1] https://github.com/kpdyer/fteproxy/blob/master/fteproxy/cli.py#L248 [2] https://pypi.python.org/pypi/pyptlib

4 7

[RFC] On new guard algorithms and data structures
by George Kadianakis 09 Sep '15

09 Sep '15

Hello there, recently we've been busy specifying various important improvements to entry guard security. For instance see proposals 250, 241 and ticket #16861. Unfortunately, the current guard codebase is dusty and full of problems (see #12466, #12450). We believe that refactoring and cleaning up the entry guard code is essential before we proceed to more advanced security improvements. We've been working on new algorithms and data structures for guard nodes as part of ticket #12595. In this mail I include some pseudocode for this new algorithm with the hope that it will act as a draft for implementing these changes. You can find the pseucode here: https://gitweb.torproject.org/user/asn/tor.git/tree/src/or/guardlist.c?h=bu… A short description of the algorithm is included on top, and then various methods and functions are prototyped underneath to make the logic more concrete. Apart from the comments and XXXs on the code, here are some more thoughts on this work: - This new design focuses on protecting against path bias attacks, by slightly damaging our reachability. Specifically, the old design is better at recovering in filtered networks, because it will keep on adding new nodes till one succeeds. In this new design, we will not try more than 80 relays per time. So if none of them passes the filtered network, bad luck no Tor. While this failure mode should not happen much, it's bad news for users behind FascistFirewalls which are actually quite frequent. A quick fix here would be to always add an 80/443 guard on our list, however as it stands only 30% of the guards are 80/443 guards, so this has bad anonymity consequences. - To improve our algorithm and make it more robust we need to understand further what kind of path bias attacks are relevant here. The adversary here is a network adversary (like a gateway) that can block our connections to certain guards. What nasty attacks can this adversary do? If we can't find bad attacks here, then maybe we should stop worrying about those path bias attacks so much. For example a threat here with the old guard logic, is that if we used this evil gateway just for 10 minutes (in an airport), the adversary could launch a path bias attack and force us to connect to her guard node. Then even after we left that airport, we would still stick to the evil guard node which is bad. Also, an adversary that manages to own our guard using path bias attacks, then has further possibilites for biasing the rest of the circuit. What can this adversary do? - Notice that the pseudocode contains no logic about bridges. I'm not sure how bridges should be handled here. - I tried to keep the dirguard logic very simple, hoping that we can eventually forget about dirguards entirely when #12538 is done. The main dirguard feature is that we assume that populate_live_entry_guards() and add_an_entry_guard() will return dirguards when the circuit is a directory circuit. Maybe we should consider introducing the "primary dirguard" concept as well. And maybe also add some logic where Tor will move on to the next dirguard if it failed to receive a document from the current dirguard. - I used the ATTEMPTED_THRESHOLD concept of prop241, but did not use the NET_THRESHOLD and CONNECTED_THRESHOLD ideas. I removed NET_THRESHOLD because I increased the value of ATTEMPTED_THRESHOLD to the point that it can also be used as a network down indicator. Also, I was not sure what CONNECTED_THRESHOLD was useful for, and there were certain engineering issues with it (Like, if that threshold is hit, we need a logic that will *only retry the successfully connected guards*, and not all guards). - There is no log message warning the user of path bias attacks or bad network or anything. That's because there is no way to figure out what's the problem, and issuing an alarming log message here would confuse and panic the user. If we want to inform the user anyhow, maybe if the user is *actively* trying to visit a destination, and we've been cycling through our guard list for ages, maybe we should then issue a log message telling the user that something is wrong with the network. - In general, I tried to keep the number of heuristics and kludges to the minimum to keep the logic simple. Unfortunately, it seems that without a "network down" indicator (#16120) there is no way to avoid edge cases and false positives here. We should try to fix all problems here that can occur frequently or have security consequences, but there will always be scenarios where Tor will end up thinking there is no network while it's actually on a filternet. For this reason, we should give plenty of testing to this feature before we ship it to real users! - Finally, all the constants & parameters in the pseudocode are subject to change. I tried to motivate some of them, but others are just arbitrary. Feedback is very welcome and please let me know of any issues with security or reachability that you find! Or of how the pseudocode should be altered to make it more useful for implementors. Cheers!

9 19

Erebus Code Review
by Damian Johnson 09 Sep '15

09 Sep '15

Hi Cristobal, just took a peek at Erebus and looks good! There's some low hanging fruit for improvement (unused functions and such) but generally looks good. Cheers! -Damian ====================================================================== erebus/server/handlers/log.py ====================================================================== 9 import erebus.util.log Perfectly fine for now, but obviously most of this is copy/paste from Nyx. Both Nyx and Erebus are smack in the middle of development so this is great! Do whatever you find most expedient. But obviously in the long run we'll want to revisit this. ---------------------------------------------------------------------- 16 def conf_handler(key, value): 17 if key == 'features.log.maxLinesPerEntry': 18 return max(1, value) 19 elif key == 'features.log.prepopulateReadLimit': 20 return max(0, value) 21 elif key == 'features.log.maxRefreshRate': 22 return max(10, value) 23 elif key == 'cache.log_panel.size': 24 return max(1000, value) I know this is copy-and-paste from Nyx but personally I wouldn't bother. Many of the config keys are unused, and at present users don't have a way of customizing them. Nyx's support for a nyxrc is a very rarely used feature, so I would leave this out for now. ---------------------------------------------------------------------- 40 TOR_EVENT_TYPES = { 41 'd': 'DEBUG', 42 'i': 'INFO', 43 'n': 'NOTICE', 44 'w': 'WARN', 45 'e': 'ERR', I did this for Nyx because because... well, curses. For Erebus we can do better. ;) Rather than 'key => event' mapping maybe Erebus should show all the event types, with checkboxes for selection? ---------------------------------------------------------------------- 155 def _register_event(self, event): 156 output = { 'message': '', 'type': ''} 157 if event.type not in self._logged_events: 158 return 159 #self._event_log.add(event) 160 #self._log_file.write(event.display_message) 161 162 # notifies the display that it has new content 163 if self._filter.match(event.display_message): 164 self._has_new_event = True 165 output['message'] = event.display_message 166 output['type'] = event.type.lower() 167 ws = ws_controller() 168 if ws is not None: 169 ws.send_data(WebSocketType.LOG, output) You can drop the line with self._has_new_event. It's an unused attribute, and doesn't appear outside of this line. The 'output' is only ever used on this last line, so might as well just call... ws.send_data(WebSocketType.LOG, {'message': event.display_message, 'type': event.type.lower()}) ---------------------------------------------------------------------- 171 def log_handler(): 172 """ 173 Singleton for getting our log hander. 174 175 :returns: :class:`~erebus.server.handlers.log.LogHandler` 176 """ 177 178 return LOG_HANDLER 179 180 def init_log_handler(logged_events): 181 """ 182 """ 183 global LOG_HANDLER 184 LOG_HANDLER = LogHandler(logged_events) 185 return LOG_HANDLER Seems from a quick look that these might be unused. The controller.py and starter.py call init_log_handler, but not spotting anything that ever uses LOG_HANDLER. ---------------------------------------------------------------------- 195 response = None 196 if controller is not None: 197 response = controller.get_info('events/names', None) 198 if response is None: 199 return [] # GETINFO query failed This is equivalent to... response = controller.get_info('events/names', []) if controller else [] ====================================================================== erebus/util/controller.py ====================================================================== 29 def erebus_controller(): 30 """ 31 Provides the erebus controller instance. 32 """ 33 34 return EREBUS_CONTROLLER 35 36 def init_controller(control_port, control_socket, logged_events): 37 """ 38 Spawns the controller 39 """ 40 41 global EREBUS_CONTROLLER 42 EREBUS_CONTROLLER = Controller(control_port, control_socket, logged_events) Again, you call init_controller() but looks as though EREBUS_CONTROLLER is never used. ---------------------------------------------------------------------- 61 self._logged_events = logged_events 62 63 self._bw_handler = None 64 self._status_handler = None 65 self._log_handler = None All unused. ---------------------------------------------------------------------- 86 def _stop_loop_check(self): 87 """ 88 Stop the looping task when is not necessary 89 """ 90 91 self._loop_call.stop() Only called once, in _conn_starter(). Might as well just call this there (it's just a minor alias). ---------------------------------------------------------------------- 156 def heartbeat_check(is_unresponsive): 157 """ 158 Logs if its been ten seconds since the last BW event. 159 160 Arguments: 161 is_unresponsive - flag for if we've indicated to be responsive or not 162 """ 163 164 controller = tor_controller() 165 last_heartbeat = controller.get_latest_heartbeat() 166 167 if controller.is_alive(): 168 if not is_unresponsive and (time.time() - last_heartbeat) >= 10: 169 is_unresponsive = True 170 log.notice('Relay unresponsive (last heartbeat: %s)' % time.ctime(last_heartbeat)) 171 elif is_unresponsive and (time.time() - last_heartbeat) < 10: 172 # really shouldn't happen (meant Tor froze for a bit) 173 is_unresponsive = False 174 log.notice('Relay resumed') 175 176 return is_unresponsive Unused. ====================================================================== erebus/util/websockets.py ====================================================================== 41 def add_websocket(self, ws_type, ws): 42 """ 43 Add an object to the list of active websockets of certain type. 44 45 Arguments: 46 ws_type - type of websocket to be fetched 47 ws - websocket object to be added 48 """ 49 50 if self._websockets[ws_type] is not None: 51 if ws not in self._websockets[ws_type]: 52 self._websockets[ws_type].append(ws) 53 else: 54 log.notice('Undefined type (%s) when adding websocket' % ws_type) Your 'if self._websockets[ws_type] is not None' check will never evaluate to False. You default all WebSocketType values to an empty list, and if it doesn't belong to that enum then it'll raise a KeyError. You want... if ws_type in self._websockets: Also, if self._websockets had sets rather than lists then you could drop the second check. Btw, lets use Sphinx pydocs. The documentation style you're using here is adopted from Nyx before I started using Sphinx, and I'm replacing that documentation style as I rewrite it.

2 1

Troubleshooting Tor 0.2.7.1+ ephemeral hidden services
by Micah Lee 09 Sep '15

09 Sep '15

I'm trying to add support for ephemeral hidden services to OnionShare. They'll solve a wide variety of problems related to OnionShare using a system Tor instead of Tor browser, and running in environments like Tails or Whonix. I'd rather never write the hidden service info to disk anyway. I noticed that Tor Browser 5.5a2 comes with Tor 0.2.7.2-alpha, so it's easy to test. But so far I'm not having any success, at least with using stem. Before I open a bug report, either against stem or against tor, am I doing this right? I've made a simple test project: https://github.com/micahflee/ephemeral-hs-test It successfully appears to create the hidden service, but it never succeeds in connecting to itself. I also can't connect to it using Tor Browser. -- Micah Lee

2 2

Proposal: Out of Band Circuit HMACs
by Mike Perry 08 Sep '15

08 Sep '15

This proposal exists in my torspec remote here: https://gitweb.torproject.org/user/mikeperry/torspec.git/tree/proposals/xxx… Including it in-line for ease of reply: =============================== Filename: xxx-oob-hmac.txt Title: Out of Band Circuit HMACs Authors: Mike Perry Created: 01 September 2015 Status: Draft 0. Motivation It is currently possible for Guard nodes (and MITM adversaries that steal their identity keys) to perform a "tagging" attack to influence circuit construction and resulting relay usage[1]. Because Tor uses AES as a stream cipher, malicious or intercepted Guard nodes can simply XOR a unique identifier into the circuit cipherstream during circuit setup and usage. If this identifier is not removed by a colluding exit (either by performing another XOR, or making use of known plaintext regions of a cell to directly extract a complete side-channel value), then the circuit will fail. In this way, malicious or intercepted Guard nodes can ensure that all client traffic is directed only to colluding exit nodes, who can observe the destinations and deanonymize users. Most code paths in the Tor relay source code will emit loud warnings for the most obvious instances circuit failure caused by this attack. However, it is very difficult to ensure that all such error conditions are properly covered such that warnings will be emitted. This proposal aims to provide a mechanism to ensure that tagging and related malleability attacks are cryptographically detectable when they happen. 1. Overview Since Tor Relays are already storing a running hash of all data transmitted on their circuits (via the or_circuit_t::n_digest and or_circuit_t::p_digest properties), it is possible to compute an out-of-band HMAC on circuit data, and verify that it is as expected. This proposal first defines an OOB_HMAC primitive that can be included standalone in a new relay cell command type, and additionally in other cell types. Use of the standalone relay cell command serves to ensure that circuits that are successfully built and used were not manipulated at a previous point. By altering the RELAY_COMMAND_TRUNCATED and CELL_DESTROY cells to also include the OOB_HMAC information, it is similarly possible to detect alteration of circuit contents that cause failures before the point of usage. 2. The OOB_HMAC primitive The OOB_HMAC primitive uses the existing rolling hashes present in or_circuit_t to provide a Tor OP (aka client) with the hash history of the traffic that a given relay has seen it so far. Note that to avoid storing an additional 64 bytes of SHA256 digest for every circuit at every relay, we use SHA1 for the hash logs, since the circuits are already storing SHA1 hashes. It's not immediately clear how to upgrade the existing SHA1 digests to SHA256 with the current circuit protocol, either, since matching hash algorithms are essential to the 'recognized' relay cell forwarding behavior. The version field exists primarily for this reason, should the rolling circuit hashes ever upgrade to SHA256. The OOB_HMAC primitive is specified in Trunnel as follows: struct oob_hmac_body { /* Version of this section. Must be 1 */ u8 version; /* SHA1 hash of all client-originating data on this circuit (obtained from or_circuit_t::n_digest). */ u8 client_hash_log[20]; /* Number of cells processed in this hash, mod 2^32. Used to spot-check hash position */ u32 client_cell_count; /* SHA1 hash of all server-originating data on this circuit (obtained from or_circuit_t::p_digest). */ u8 server_hash_log[20]; /* Number of cells processed in this hash, mod 2^32. Used to spot-check hash position. XXX: Technically the server-side is not needed. */ u32 server_cell_count; /* HMAC-SHA-256 of the entire cell contents up to this point, using or_circuit_t::p_crypto as the hmac key. XXX: Should we use a KDF here instead of p_crypto directly? */ u8 cell_hmac_256[32]; }; 3. Usage of OOB_HMAC The OOB_HMAC body will be included in three places: 1. In a new relay cell command RELAY_COMMAND_HMAC_SEND, which is sent in response to a client-originating RELAY_COMMAND_HMAC_GET on stream 0. 2. In CELL_DESTROY, immediately after the error code 3. In RELAY_COMMAND_TRUNCATED, immediately after the CELL_DESTROY contents 3.1. RELAY_COMMAND_HMAC_GET/SEND relay commands Clients should use leaky-pipe topology to send RELAY_COMMAND_HMAC_GET to the second-to-last node (typically the middle node) in the circuit at three points during circuit construction and usage: 1. Immediately after the last RELAY_EARLY cell is sent 2. Upon any stream detachment, timeout, or failure. 3. Upon any OP-initiated circuit teardown (including timed-out partially built circuits). We use RELAY_EARLY as the point at which to send these cells to avoid leaking the path length to the middle hop. 3.2. Alteration of CELL_DESTROY and RELAY_COMMAND_TRUNCATED In order to provide an HMAC even when a circuit is torn down before use due to failure, the behavior for generating and handling CELL_DESTROY and RELAY_COMMAND_TRUNCATED should be modified as follows: Whenever an OR sends a CELL_DESTROY for a circuit towards the OP, if that circuit was already properly established, the OR should include the contents of oob_hmac_body immediately after the reason field. The HMAC must cover the error code from CELL_DESTROY. Upon receipt of a CELL_DESTROY, and in any other case where an OR would generate a RELAY_COMMAND_TRUNCATED due to error, a conformant relay would include the CELL_DESTROY oob_hmac_body, as well as its own locally created oob_hmac_body. The locally created oob_hmac_body must cover the entire payload contents of RELAY_COMMAND_TRUNCATED, including the error code and the CELL_DESTROY oob_hmac_body. Here is a new Trunnel specification for RELAY_COMMAND_TRUNCATED: struct relay_command_truncated { /* Error code */ u8 error_code; /* Number of oob_hmacs. Must be 0, 1, or 2 */ u8 num_hmac; /* If there are 2 hmacs, the first one is from the CELL_DESTROY, and the second one is from the truncating relay. If num_hmac is 0, then this came from a relay without support for oob_hmac. */ struct oob_hmac_body[num_hmac]; }; The usage of a strong HMAC to cover the entire CELL_DESTROY contents also allows an OP to properly authenticate the reason a remote node needed to close a circuit, without relying on the previous hop to be honest about it. 4. Ensuring proper ordering with respect to hashes 4.1. RELAY_COMMAND_HMAC_GET/SEND The in-order delivery guarantee of circuits will mean that the incoming hashes will match upon receipt of the RELAY_COMMAND_HMAC_SEND cell, but any outgoing traffic the OP sent since RELAY_COMMAND_HMAC_GET will not have been seen by the responding OR. Therefore, immediately upon sending a RELAY_COMMAND_HMAC_GET, the OP must record and store its current outgoing hash state for that circuit, until the RELAY_COMMAND_HMAC_SEND arrives, and use that stored hash value for comparison against the oob_hmac_body's client_hash_log field. The server_hash_log should be checked against the corresponding crypt_path_t entry in origin_circuit_t for the relay that the command was sent to. 4.2. RELAY_COMMAND_TRUNCATED Since RELAY_COMMAND_TRUNCATED may be sent in response to any error condition generated by a cell in either direction, the OP must check that its local cell counts match those present in the oob_hmac_body for that hop. If the counts do not match, the OP may generate a RELAY_COMMAND_HMAC_GET to the hop that sent RELAY_COMMAND_TRUNCATED, prior to tearing down the circuit. 4.3. CELL_DESTROY If the cell counts of the destroy cell's oob_hmac_body do not match what the client sent for that hop, unfortunately that hash must be discarded. Otherwise, it may be checked against values held from before processing the RELAY_COMMAND_TRUNCATED envelope. 5. Security concerns and mitigations 5.1. Silent circuit failure attacks The primary way to game this oob-hmac is to omit or block cells containing HMACs from reaching the OP, or otherwise tear down circuits before responses arrive with proof of tampering. If a large fraction of circuits somehow fail without any RELAY_COMMAND_TRUNCATED oob_hmac_body payloads present, and without any responses to RELAY_COMMAND_HMAC_GET requests, the user should be alerted of this fact as well. This rate of silent circuit failure should be kept as an additional, separate per-Guard Path Bias statistic, and the user should be warned if this failure rate exceeds some (low) threshold for circuits containing relays that should have supported this proposal. 5.2. Malicious/colluding middle nodes If the adversary is prevented from causing silent circuit failure without the client being able to notice and react, their next available vector is to ensure that circuits are only built to middle nodes that are malicious and colluding with them (or that do not support this proposal), so that they may lie about the proper hash values that they see (or omit them). Right now, the current path bias code also does not count circuit failures to the middle hop as circuit attempts. This was done to reduce the effect of ambient circuit failure on the path bias accounting (since an average ambient circuit failure of X per-hop causes the total circuit failure middle+exit circuits to be 2X). Unfortunately, not counting middle hop failure allows the adversary to only allow circuits to colluding middle hops to complete, so that they may lie about their hash logs. All failed circuits to non-colluding middle nodes could be torn down before RELAY_COMMAND_TRUNCATED is sent. For this reason, the per-Guard Path Bias counts should be augmented to additionally track middle-node-only failure as a separate statistic as well, and the user should be warned if middle-node failure drops below a similar threshold as the current end-to-end failure. 5.3. Side channel issues, mitigations, and limitations Unfortunately, leaking information about circuit usage to the middle node prevents us from sending RELAY_COMMAND_HMAC_GET cells at more optimal points in circuit usage (such as immediately upon open, immediately after stream usage, etc). As such, we are limited to waiting until RELAY_EARLY cells stop being sent. It is debatable if we should send hashes periodically (perhaps with windowing information updates?) instead. 6. Alternatives A handful of alternatives to this proposal have already been discussed, but have been dismissed for various reasons. Per-hop cell HMACs were ruled out because they will leak the total path length, as well as the current hop's position in the circuit. Wide-block ciphers have been discussed, which would provide the property that attempts to alter a cell at a previous hop would render it completely corrupted upon its final destination, thus preventing untagging and recovery, even by a colluding malicious peer. Unfortunately, performance analysis of modern provably secure versions of wide-block ciphers has shown them to be at least 10X slower than AES-NI[2]. 1. https://lists.torproject.org/pipermail/tor-dev/2012-March/003347.html 2. http://archives.seul.org/tor/dev/Mar-2015/msg00137.html -- Mike Perry

1 0