- tor-dev - lists.torproject.org

Proposal 321: Better performance and usability for the MyFamily option (v2)
by Nick Mathewson 27 May '20

27 May '20

``` Filename: 321-happy-families.md Title: Better performance and usability for the MyFamily option (v2) Author: Nick Mathewson Created: 27 May 2020 Status: Open ``` ## Problem statement. The current family mechanism allows well-behaved relays to identify that they all belong to the same 'family', and should not be used in the same circuits. Right now, families work by having every family member list every other family member in its server descriptor. This winds up using O(n^2) space in … [View More]microdescriptors and server descriptors. (For RAM, we can de-duplicate families which sometimes helps.) Adding or removing a server from the family requires all the other servers to change their torrc settings. The is growth in size is not just a theoretical problem. Family declarations currently make up a little over 55% of the microdescriptors in the directory--around 24% after compression. The largest family has around 270 members. With Walking Onions, 270 members times a 160-bit hashed identifier leads to over 5 kilobytes per SNIP, which is much greater than we'd want to use. This is an updated version of proposal 242. It differs by clarifying requirements and providing a more detailed migration plan. ## Design overview. In this design, every family has a master ed25519 "family key". A node is in the family iff its server descriptor includes a certificate of its ed25519 identity key with the family key. The certificate format the one in the tor-certs.txt spec; we would allocate a new certificate type for this usage. These certificates would need to include the signing key in the appropriate extension. Note that because server descriptors are signed with the node's ed25519 signing key, this creates a bidirectional relationship between the two keys, so that nodes can't be put in families without their consent. ## Changes to router descriptors We add a new entry to server descriptors: "family-cert" NL "-----BEGIN FAMILY CERT-----" NL cert "-----END FAMILY CERT-----". This entry contains a base64-encoded certificate as described above. It may appear any number of times; authorities MAY reject descriptors that include it more than three times. ## Changes to microdescriptors We add a new entry to microdescriptors: `family-keys`. This line contains one or more space-separated strings describing families to which the node belongs. These strings MUST be sorted in lexicographic order. Clients MUST NOT depend on any particular property of these strings. ## Changes to voting algorithm We allocate a new consensus method number for voting on these keys. When generating microdescriptors using a suitable consensus method, the authorities include a "family-keys" line if the underlying server descriptor contains any valid family-cert lines. For each valid family-cert in the server descriptor, they add a base-64-encoded string of that family-cert's signing key. > See also "deriving family lines from family-keys?" below for an > interesting but more difficult extension mechanism that I would > not recommend. ## Relay configuration There are several ways that we could configure relays to let them include family certificates in their descriptors. The easiest would be putting the private family key on each relay, so that the relays could generate their own certificates. This is easy to configure, but slightly risky: if the private key is compromised on any relay, anybody can claim membership in the family. That isn't so very bad, however -- all the relays would need to do in this event would be to move to a new private family key. A more orthodox method would be to keep the private key somewhere offline, and using it to generate a certificate for each relay in the family as needed. These certificates should be made with long-enough lifetimes, and relays should warn when they are going to expire soon. ## Changes to relay behavior Each relay should track which other relays they have seen using the same family-key as itself. When generating a router descriptor, each relay should list all of these relays on the legacy 'family' line. This keeps the "family" lines up-to-date with "family-keys" lines for compliant relays. Relays should continue listing relays in their family lines if they have seen a relay with that identity using the same family-key at any time in the last 7 days. The presence of this line should be configured by a network parameter, `derive-family-line`. Relays whose family lines do not stay at least mostly in sync with their family keys should be marked invalid by the authorities. ## Client behavior Clients should treat node A and node B as belonging to the same family if ANY of these is true: * The client has descriptors for A and B, and A's descriptor lists B in its family line, and B's descriptor lists A in its family line. * Client A has descriptors for A and B, and they both contain the same entry in their family-keys or family-cert. ## Migration For some time, existing relays and clients will not support family certificates. Because of this, we try to make sure above the well-behaved relays will list the same entries in both places. Once enough clients have migrated to using family certificates, authorities SHOULD disable `derive-family-line`. ## Security Listing families remains as voluntary in this design as in today's Tor, though bad-relay hunters can continue to look for families that have not adopted a family key. A hostile relay family could list a "family" line that did not match its "family-certs" values. However, the only reason to do so would be in order to launch a client partitioning attack, which is probably less valuable than the kinds of attacks that they could run by simply not listing families at all. ## Appendix: deriving family lines from family-keys? As an alternative, we might declare that _authorities_ should keep family lines in sync with family-certs. Here is a design sketch of how we might do that, but I don't think it's actually a good idea, since it would require major changes to the data flow of the voting system. In this design, authorties would include a "family-keys" line in each router section in their votes corresponding to a relay with any family-cert. When generating final microdescriptors using this method, the authorities would use these lines to add entries to the microdescriptors' family lines: 1. For every relay appearing in a routerstatus's family-keys, the relays calculate a consensus family-keys value by listing including all those keys that are listed by a majority of those voters listing the same router with the same descriptor. (This is the algorithm we use for voting on other values derived from the descriptor.) 2. The authorities then compute a set of "expanded families": one for each family key. Each "expanded family" is a set containing every router in the consensus associated with that key in its consensus family-keys value. 3. The authorities discard all "expanded families" of size 1 or smaller. 4. Every router listed for the "expanded family" has every other router added to the "family" line in its microdescriptor. (The "family" line is then re-canonicalized according to the rules of proposal 298 to remove its ) 5. Note that the final microdescriptor consensus will include the digest of the derived microdescriptor in step 4, rather than the digest of the microdescriptor listed in the original votes. (This calculation is deterministic.) The problem with this approach is that authorities would have to s to fetch microdescriptors they do not have in order to replace their family lines. Currently, voting never requires an authority to fetch a microdescriptor from another authority. If we implement vote compression and diffs as in the Walking Onions proposal, however, we might suppose that votes could include microdescriptors directly. Still, this is likely more complexity than we want for a transition mechanism. ## Appendix: Deriving family-keys from families?? We might also imagine that authorities could infer which families exist from the graph of family relationships, and then include synthetic "family-keys" entries for routers that belong to the same family. This has two challenges: first, to compute these synthetic family keys, the authorities would need to have the same graph of family relationships to begin with, which once again would require them to include the complete list of families in their votes. Secondly, finding all the families is equivalent to finding all maximal cliques in a graph. This problem is NP-hard in its general case. Although polynomial solutions exist for nice well-behaved graphs, we'd still need to worry about hostile relays including strange family relationships in order to drive the algorithm into its exponential cases. ## Appendix: New assigned values We need a new assigned value for the certificate type used for family signing keys. We need a new consensus method for placing family-keys lines in microdescriptors. ## Appendix: New network parameters * `derive-family-line`: If 1, relays should derive family lines from observed family-keys. If 0, they do not. Min: 0, Max: 1. Default: 1. [View Less]

1 0

So Windows is Adding a Package Manager.....
by Keifer Bly 24 May '20

24 May '20

2 1

tor-dev Digest, Vol 112, Issue 21
by Support Team 20 May '20

20 May '20

Thank you for the insights. On Wed, May 20 2020, at 12:00 PM, <tor-dev(a)lists.torproject.org> Send tor-dev mailing list submissions to tor-dev(a)lists.torproject.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev or, via email, send a message with subject or body 'help' to tor-dev-request(a)lists.torproject.org You can reach the person managing the list at tor-dev-owner(a)lists.torproject.org When replying, … [View More]please edit your Subject line so it is more specific than "Re: Contents of tor-dev digest..." Today's Topics: 1. Re: Proposal 319: RELAY_FRAGMENT cells (Nick Mathewson) 2. Re: Proposal 320: Removing TAP usage from v2 onion services (Nick Mathewson) 3. Re: Proposal 320: Removing TAP usage from v2 onion services (Nick Mathewson) 4. Re: Proposal 320: Removing TAP usage from v2 onion services (Sebastian Hahn) ---------------------------------------------------------------------- Message: 1 Date: Tue, 19 May 2020 13:29:54 -0400 From: Nick Mathewson To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Proposal 319: RELAY_FRAGMENT cells Message-ID: Content-Type: text/plain; charset="UTF-8" On Thu, May 14, 2020 at 3:15 PM David Goulet wrote: > > On 11 May (16:47:24), Nick Mathewson wrote: [...] > > # Onion service concerns. > > > > We allocate a new extension for use in the ESTABLISH_INTRO by onion services, > > to indicate that they can receive a wide INTRODUCE2 cell. This extension > > contains: > > > > struct wide_intro2_ok { > > u16 max_len; > > } > > > > We allocate a new extension for use in the 'ESTABLISH_RENDEZVOUS' > > cell, to indicate acceptance of wide 'RENDEZVOUS2' cells. This > > extension contains: > > > > struct wide_rend2_ok { > > u16 max_len; > > } > > > > (Note that 'ESTABLISH_RENDEZVOUS' cells do not currently have a an > > extension mechanism. They should be extended to use the same > > extension format as 'ESTABLISH_INTRO' cells, with extensions placed > > after the rendezvous cookie.) > > Why would a client need to announce wide cells in the ESTABLISH phase as > opposed to using protover "Relay=N" ? This is not for announcing support of wide cells -- this is for reporting a setting for how wide fragmented cells should be. > The maximum length of a fragmented cell is capped to 2^16 (u16) so we don't > really need the establish process to inform us of the maximum expected length > but rather use the max_len in the first fragment? This all comes back to an earlier part of the proposal: Not all lengths up to 65535 are valid lengths for a fragmented cell. Any length under 499 bytes SHOULD cause the circuit to close, since that could fit into a non-fragmented RELAY cell. Parties SHOULD enforce maximum lengths for cell types that they understand. In other words, I'm imagining that there is a maximum length for each cell type that is much shorter than 65535, even though we're using two bytes for the length field. The extension in the establish_intro cell is to tell the intro point the longest introduce1 cell that it should accept; this extension in the establish_rend cell is to tell the rendezvous point the longest rendezvous1 cell that it should accept. Another way we could do this would be with a set of network parameters to describe the maximum length of each fragmented cell. Do you think that would be simpler? (I can't quite remember why I specified it this way in the first place.) > Furthermore, ESTABLISH_INTRO has extensions (only 1 as of today) so they could > also be fragments themselves and thus I'm not sure I see the point of having > two different ways of "expecting" fragments for the ESTABLISH_* cells and the > INTRO/RENDEZVOUS cells? The difference thing here is that everybody can tell which protocols that a relay supports, but there is no automatic way to tell which protocols an onion service or client supports. Since INTRODUCE2/RENDEZVOUS2 cells are handled by these clients, they need to get opted into by the relays. (I'm not sure I understood the question completely.) > > # Compatibility > > > > This proposal will require the allocation of a new 'Relay' protocol version, > > to indicate understanding of the RELAY_FRAGMENTED command. > > Here is a thought about a DoS vector. Here goes: > > As an upper limit of 65KB total fragment size, it represents ~126 cells in > total so I could basically send *125* cells and then stop which will put in > memory a bit more than 64KB and it will stay there until the last fragment is > received. > > And then I do that on 1000 different circuits bringing the total count in > memory to 64GB. All stuck there, all "waiting" for the last fragment. > > Our OOM would kick in killing circuits but it just seems to me a very easy way > to continously kick the OOM of a _service_ which is pretty bad side channel. A few responses here: First, we shouldn't allow 65535-byte fragmented cells. The actual maximum length should be something more like 1024 or 4096 bytes. Second, we should make sure that when we are reassembling cells, we use the same buf_t buffers that we use for other stuff. Our buffers are timestamped, so we can tell which buffer has had data stalling for the longest, and we should use that to make sure we're killing off the right circuits preferentially. Third, fragments should only be allowed at an onion service for INTRODUCE2, and those should only come one at a time from each introduction point, so the number that it's reassembling at the time will be limited by the number of intro circuits it has open. It'll be the the intro points that have to be keeping a bunch of cells in assembly at once, and be ready to kill off circuits that dawdle too long. Does this make more sense? If so I'll try to clarify it in the proposal. [View Less]

1 0

Proposal 319: RELAY_FRAGMENT cells
by Nick Mathewson 19 May '20

19 May '20

``` Filename: 319-wide-everything.md Title: RELAY_FRAGMENT cells Author: Nick Mathewson Created: 11 May 2020 Status: Open ``` (This proposal is part of the Walking Onions spec project.) # Introduction Proposal 249 described a system for `CREATE` cells to become wider, in order to accommodate hybrid crypto. And in order to send those cell bodies across circuits, it described a way to split `CREATE` cells into multiple `EXTEND` cells. But there are other cell types that can need to be wider … [View More]too. For example, `INTRODUCE` and `RENDEZVOUS` cells also contain key material used for a handshake: if handshakes need to grow larger, then so do these cells. This proposal describes an encoding for arbitrary "wide" relay cells, that can be used to send a wide variant of anything. To be clear, although this proposal describes a way that all relay cells can become "wide", I do not propose that wide cells should actually be _allowed_ for all relay cell types. # Proposal We add a new relay cell type: `RELAY_FRAGMENT`. This cell type contains part of another relay cell. A `RELAY_FRAGEMENT` cell can either introduce a new fragmented cell, or can continue one that is already in progress. The format of a RELAY_FRAGMENT body is one of the following: // First body in a series struct fragment_begin { // What relay_command is in use for the underlying cell? u8 relay_command; // What will the total length of the cell be once it is reassembled? u16 total_len; // Bytes for the cell body u8 body[]; } // all other cells. struct fragment_continued { // More bytes for the cell body. u8 body[]; } To send a fragmented cell, first a party sends a RELAY_FRAGMENT cell containing a "fragment_begin" payload. This payload describes the total length of the cell, the relay command Fragmented cells other than the last one in sequence MUST be sent full of as much data as possible. Parties SHOULD close a circuit if they receive a non-full fragmented cell that is not the last fragment in a sequence. Fragmented cells MUST NOT be interleaved with other relay cells on a circuit, other than cells used for flow control. (Currently, this is only SENDME cells.) If any party receives any cell on a circuit, other than a flow control cell or a RELAY_FRAGEMENT cell, before the fragmented cell is complete, than it SHOULD close the circuit. Parties MUST NOT send extra data in fragmented cells beyond the amount given in the first 'total_len' field. Not every relay command may be sent in a fragmented cell. In this proposal, we allow the following cell types to be fragmented: EXTEND2, EXTENDED2, INTRODUCE1, INTRODUCE2, RENDEZVOUS. Any party receiving a command that they believe should not be fragmented should close the circuit. Not all lengths up to 65535 are valid lengths for a fragmented cell. Any length under 499 bytes SHOULD cause the circuit to close, since that could fit into a non-fragmented RELAY cell. Parties SHOULD enforce maximum lengths for cell types that they understand. All `RELAY_FRAGMENT` cells for the fragmented cell must have the same Stream ID. (For those cells allowed above, the Stream ID is always zero.) Implementations SHOULD close a circuit if they receive fragments with mismatched Stream ID. # Onion service concerns. We allocate a new extension for use in the ESTABLISH_INTRO by onion services, to indicate that they can receive a wide INTRODUCE2 cell. This extension contains: struct wide_intro2_ok { u16 max_len; } We allocate a new extension for use in the `ESTABLISH_RENDEZVOUS` cell, to indicate acceptance of wide `RENDEZVOUS2` cells. This extension contains: struct wide_rend2_ok { u16 max_len; } (Note that `ESTABLISH_RENDEZVOUS` cells do not currently have a an extension mechanism. They should be extended to use the same extension format as `ESTABLISH_INTRO` cells, with extensions placed after the rendezvous cookie.) # Handling RELAY_EARLY The first fragment of each EXTEND cell should be tagged with `RELAY_EARLY`. The remaining fragments should not. Relays should accept `EXTEND` cells if and only if their _first_ fragment is tagged with `RELAY_EARLY`. > Rationale: We could allow any fragment to be tagged, but that would give > hostile guards an opportunity to move RELAY_EARLY tags around and build a > covert channel. But if we later move to a relay encryption method that > lets us authenticate RELAY_EARLY, we could then require only that _any_ > fragment has RELAY_EARLY set. # Compatibility This proposal will require the allocation of a new 'Relay' protocol version, to indicate understanding of the RELAY_FRAGMENTED command. [View Less]

4 3

Re: [tor-dev] Chutney code refactoring
by c 17 May '20

17 May '20

On Fri, 15 May 2020 11:20:54 +1000 teor <teor(a)riseup.net> wrote: > You could try deleting functions, and then running tor's > "make test-network-all". If that test still passes, then the code is probably > unused (in practice). > > Chutney's CI does a few more tests for different tor versions, but I'm pretty > sure they all use the same code. This is what I will end up doing. I planned on it, but humans tend to know the purpose of the code better, and computers just … [View More]

1 0

Re: [tor-dev] Chutney code refactoring
by c 17 May '20

17 May '20

On Thu, 14 May 2020 19:58:45 -0400 Nick Mathewson <nickm(a)freehaven.net> wrote: > isInExpectedDirInfoDocs looks like it might once have done something useful. My impression is to remove it unless we anticipate needing it in the near future. > configure and restart and wait_for_bootstrap are all in use; they are > invoked by name, from the command line, at the end of runConfigFile, > where it says `return getattr(network, verb)()`. Ah, I see now, thank you. I think it … [View More]

1 0

Deprecating Tor Protocol Versions
by teor 15 May '20

15 May '20

Hi all, Nick and I were talking about how we remove legacy features in tor, and their corresponding subprotocol versions. Here is a list of the current subprotocol versions: https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n2049 Here's a recent protocol version proposal, which deals with recommending and requiring new features: https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-remov… But we don't have a similar proposal for removing support for older protocol … [View More]versions from the tor codebase. For an example of what that proposal could look like, see our proposal for deprecating consensus methods: https://gitweb.torproject.org/torspec.git/tree/proposals/290-deprecate-cons… Here's the original conversation Nick and I had: https://github.com/torproject/tor/pull/1874#discussion_r423713579 But after reading our consensus methods deprecation proposal, I've changed my mind. I think that we should check for "protocol N, and any later version" by default. That's what we do for consensus methods, and it seems to work well. We can drop the earliest consensus methods, and recent tor versions just keep working. If we need an incompatible change, we can make it another protocol version, and recommend then require support for it. So here's an edited version of my notes on that ticket: There are a few instances of ">=" and "=" confusion in protocol versions. We should try to fix them all. It only matters when we remove protocol versions. We haven't really specified, tested, or exercised this functionality in practice. And so our reviewers lack experience. (And when we did discover a need for it with NSS and LinkAuth, it was more complicated than we expected.) I'd like to see a proposal that tells us how to check future protocol versions as they are added. Along with a migration plan for disabling protocol versions. So let's also open a ticket to check for "any future version". We should replace all "=" checks with ">=". Let's make sure we check all the places where we use protocol versions, even if they don't have a summary flag. Overall, I think it would be helpful if future protocol versions were orthogonal. Or if they depend on earlier features, that dependency should be clearly documented. (For example, Relay=3 IPv6 extends depends on Relay=2 EXTEND2 cells. So if we were checking EXTEND2 cell support, it would be Relay=2 or Relay=3.) T -- teor ---------------------------------------------------------------------- [View Less]

2 2

Re: [tor-dev] Chutney code refactoring
by c 15 May '20

15 May '20

On Fri, 17 Apr 2020 18:01:42 -0400 Nick Mathewson <nickm(a)freehaven.net> wrote: > If you want to work on this it would be helpful to maybe start by > listing (here or elsewhere) some places where you *don't* feel like > you could (or would want to) write documentation: those would be a > good target for devs who _have_ worked on Chutney before. I came across Node.specialize() which does not seem to be called elsewhere, and I cannot guess at its purpose. So, if anyone knows … [View More]

3 3

woswos - GSoC 2020
by Barkin Simsek 12 May '20

12 May '20

Hi everyone, I'm Barkin (woswos in IRC), and I study Computer Engineering at New York University Abu Dhabi. This summer, I will be working on the "Cloudflare CAPTCHA Monitoring" project as a part of the GSoC program with my mentors Georg and Roger. I’m very excited about working on this project! The Cloudflare CAPTCHA Monitoring project aims to track how often Cloudflare fronted webpages return CAPTCHAs to Tor clients. The project aims to achieve this by fetching webpages via both Tor and … [View More]

1 0

Proposal 318: Limit protover values to 0-63
by Nick Mathewson 12 May '20

12 May '20

``` Filename: 318-limit-protovers.md Title: Limit protover values to 0-63. Author: Nick Mathewson Created: 11 May 2020 Status: Open ``` # Limit protover values to 0-63. I propose that we no longer accept protover values higher than 63, so that they can all fit nicely into 64-bit fields. (This proposal is part of the Walking Onions spec project.) ## Motivation Doing this will simplify our implementations and our protocols. Right now, an efficient protover implementation needs to use ranges … [View More]

2 1