- tor-dev - lists.torproject.org

Re: [tor-dev] Special-use-TLD support
by str4d 29 Sep '15

29 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Jeff Burdges wrote: > Very interesting. Yes, this sounds reasonable in the short run. In > the longer run, there are several people with an interest in > externalizing Tor's DNS handling, which changes things. I'll > check out OnioNS and discuss this with people at the meeting. > Thanks for making this proposal! I am very interested in seeing it become a more general-purpose method for externalizing DNS handling. I2P has supported pluggable NamingServices for a long time, and it is trivial to add new methods. Now that Tor is considering them, I would like to see whatever protocol is decided on standardized in some way, so that GNS, Namecoin etc. can implement one protocol and be interoperable with both Tor and I2P. I include some comments below to this effect. I would expect that if this went ahead, this spec would be split into a) a general spec for the RPC protocol, and b) how Tor itself uses the protocol. > > > Filename: xxx-special-use-tld-support.txt Title: Special-Use TLD > Support Author: Jeffrey Burdges Created: ?? Sept 2015 Status: Draft > Implemented-In: ? > > Abstract > > Suppose Special-Use TLDs in Tor via external Domain Name System > (DNS) suppliers, such as the GNU Name System and Namecoin. > > Background > > Special-use TLD supplier software integrates with the host > operating system's DNS layer so that other software resolves the > special-use TLD identically to standard DNS TLDs. On Linux for > example, a Special -Use TLD package could create a plugin for the > Name Service Switch (NSS) subsystem of the GNU C Library. > > Tor cannot safely use the local system's own DNS for name > resolution, as doing so risks deanonmizing a user through their > DNS queries. Instead Tor does DNS resolution at a circuit's exit > relay. It follows that Tor users cannot currently use special-use > TLDs packages in a safe manor. > > In addition, there are projects to add public key material to DNS, > like TLSA records and DNSSEC, that necessarily go beyond NSS. > > Design > > We denote by N an abstract name service supplier package. There > are two steps required to integrate N safely with Tor : > > Of course, N must be modified so as to (a) employ Tor for its own > traffic and (b) to use Tor in a safe way. We deem this step > outside the scope of the present document since it concerns > modifications to N that depend upon N's design. We caution > however that peer-to-peer technologies are famous for sharing > unwanted information and producing excessively distinctive traffic > profiles, making (b) problematic. Another proposal seeks to > provide rudimentary tools to assist with (a). This "must" would need to be generalized to "employ anonymizing services for its own traffic" if left in the general spec. In practice this will probably just be Tor if exit nodes are the means for communicating with clearnet N nodes, but the spec shouldn't be restrictive in this regard (e.g. N nodes run from Tor HSs or inside I2P) . > > We shall instead focus on modifying Tor to route some-but-not-all > DNS queries to N. For this, we propose a NameService configuration > option that tells Tor where to obtain the DNS record lying under > some specific TLD. > > Anytime Tor resolves a DNS name ending in an Special-Use TLD > appearing in an NameService configuration line then Tor makes an > RPC request for the name record using given UNIX domain socket or > address and port. > > We should allow CNAME records to refer to .onion domains, and to > regular DNS names, but care must be taken in handling CNAME records > that refer to Special-Use TLDs handled by NameService lines. Tor > should reject CNAME records that refer to .exit domains. > > Configuration > > We propose two Tor configuration options : > > NameService [.]<dnspath> <socketspec> [noncannonical] [timeout=num] > [-- service specific options] > > We require that <socketspec> be either the path to a UNIX domain > socket or an address of the form IP:port. We also require that > each <dnspath> be a string conforming to RFC 952 and RFC 1123 sec. > 2.1. In other words, a dnsspec consists of a series of labels > separated by periods . with each label of up to 63 characters > consisting of the letters a-z in a case insensitive manor, the > digits 0-9, and the hyphen -, but hyphens may not appear at the > beginning or end of labels. > > NameService rules route matching queries to appropriate name > service supplier software. If a trailing substring of a query > matches <dnspath> then a query is sent to the <socketspec> using > the RPC protocol described below. NameService rules are applied > only after all MapAddress rules. > > There is no way to know in advance if N handles caching itself, > much less if it handles caching in a way suitable for Tor. > Ideally, we should demands that N return an appropriate expiration > time, which Tor can respect without harming safety or > performance. If this proves problematic, then configuration > options could be added to adjust Tor's caching behavior. > > Seconds is the unit for the timeout option, which defaults to 60 > and applies only to the name service supplier lookup. Tor DNS > queries, or attempts to contact .onion addresses, that result from > CNAME records should be given the full timeout allotted to > standard Tor DNS queries, .onion lookups, etc. > > Any text following -- is passed verbatim to the name service > supplier as service specific options, according to the RPC > protocol described below. > > Control Protocol > > An equivalent of NameService should be added to the Tor control > protocol, so that multiple users using the same Tor daemon can > have different name resolution rules. > > RPC protocol > > We require an RPC format that communicates two values, first any > service specific options give on the NameService line, and second > the query name itself of course. We might however discuss if there > are any standardized flags, distinct from these options, and > whether they should be communicated separately. > > In principle, Tor could make due with simply receiving a strong in > "strong" -> "string" > return. We recommend however that Tor expect a return format as > or more powerful than full DNS queries. In particular, we should > endever to return TLSA records at the same time as the underlying > DNS record, so that Tor Browser can utilize that key material. > The GNS Record format used by the GNU Name System addresses this > and other issues, so it should be taken as a candidate. See : > https://github.com/GNUnet/gnunet/blob/gnunet/src/include/gnunet_gns > > > record_lib.h For host names that the external service maps to I2P servers, I2P would ideally like the record to contain the full Destination, because then only one lookup is required. This is not a domain name-like value, like foo.onion or bar.b32.i2p, so the ability for the returned record to support these kinds of return values should be considered. If a domain name-like value is required, then I2P would need to do two lookups - one for spam.gnu -> bar.b32.i2p, and then a second (internal) for bar.b32.i2p -> DESTINATION. This partly also depends on what kind of records the external service supports, but IMHO the RPC protocol should not be limiting in this regard, given that it will likely only be used by networks like Tor and I2P that cater to more than IPv4 and IPv6. > > Sepcial-use tLD suppliers should internally process CNAME records "Sepcial" -> "Special" > that fall into their own domains, but they should return CNAME > records to Tor that refer to .onion or .exit domains, or to normal > DNS names. Initially, Tor should issue an error if it receives a > CNAME record that matches an NameService line. If however that > NameService line contains the noncannonical option, then CNAME > records should instead bypass it, and use Tor's DNS system. It's not clear to me a) where this is useful for Tor, and b) where this is useful in general. Can you provide an example? > > At present, alternative DNS packages should not pass CNAME records > between themselves, despite speaking the same RPC protocol, as > this creates unknown risks. As such forwarding can be done most > safely by Tor itself, the Tor Project reserves the right to > forward CNAME records between NameService lines in the future. > Applications should therefore not depend upon the above error > being returned. One thing that the I2P NamingService impl doesn't do is recursive lookups, because we haven't had a need for them yet (we instead have an optional fallback system where if a lookup fails with one configured service, it can try another one). But it wouldn't be hard to implement, and I agree with having the RPC user handle CNAME records instead of the external service. str4d > > Variations > > Tor would conceivably benefit from externalizing its own DNS > handling as a separate process. This might however require that > Tor have the ability to start name service suppliers. A fuller > consideration of this might alter our design of the NameService > configuration option. > > Acknowledgments > > Based on extensive discussions with Christian Grothoff and George > Kadianakis. > > > > > > _______________________________________________ tor-dev mailing > list tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJWCfHfAAoJEBO17ljAn7Pg5xsP/A+6JT4/ZZ0ZIBJ3YTPefAd7 D42WCHDKc8F5u2SC3NzqlryvfmXN6INlEDHx7JzGg2j52AdjhXbzpR58U6rZS1LE El/mzh3ZnIV8PpzfojY+SU7eQ/4cAgr8hJW3/x8rIRBlt2ZRBpWyKYPKBDtZtOzp ptbNs7rjio28ymaPEpdyT7by9SzE6GiC4k4LGJnDxMUhk/f2EzjwpQJ7Um4pkKmV ziV1FpUcuWbnXlz4rVs08DO/aULQs6d7He494XoA06HgFnTqXRirg3DdsxffbDDR 7UkL5DzM1IXb0yv2VXyB3DfUI4L5w5kZoDtaGsYtUoIZdtHJlrcV96eodAjp1PTJ PpubSRZc/pC0tL5ol9FyGkDg/Gh+HWSHyVYkY+RJo7/4pSJc6lt/ruUQNvpRqYty BpW8YtVHURRoThjkJ5SAWjXC96ET0yLpAXo3O9aFeNQt36vG33GmqdNHzA6x2vb8 yHjB7YlxtDsoj+o+O9cJ/9vPLKeeMjOhd6xFQaujFIOCxFT374R3WMUyniIEmrPv dI86kalXyDM8HZ80Q4QVqDZkEZKSWdigp149cTMhU279CpmsIVPjAXWucXChtT6E +u2MSFsGSN6mRZEGosJty18HBgS218IvYUI1IQbYfGM3mD6gL0+SEq8hNT/Bj500 r8yIwBccfe1IXEXdxyn7 =X7ur -----END PGP SIGNATURE-----

1 0

Interruption in patch workshop and (online) tor dev meeting this week
by Nick Mathewson 28 Sep '15

28 Sep '15

Hi, all! I hope things are going well for you. This week, many tor developers are all at a meeting in Berlin. So, there will probably be far fewer people than usual at the Tuesday patch workshop time or at the Wednesday tor status meeting. We should be back as usual next week! peace, -- Nick

1 0

Reminder for weekly OONI dev meeting Monday 2015-09-28 17:00 UTC (19:00 CET)
by Vasilis 28 Sep '15

28 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, This is a reminder that today there will be the weekly OONI meeting. It will happen as usual on the #ooni channel on irc.oftc.net at 17:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). Everybody is welcome to join us and bring their questions and feedback. See you later, ~ Vasilis -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJWCVPXAAoJEF+/cLHRJgFiyfEP/3MXP6nHjk9Q9ixRS+FOyKgt qqyeDWdrgrO5wrtoz3cuaeKqwmvACPAJdlXJUNhVOtwM7msziNZug9g7wWi/W7Tv 0LJZ0+YTRyQ9RPGV+ldAM+n0ZELywP6l3trh9Yoa9Q/wgOwhrulVpsuxtgh2DkfJ KZXGI5dR1io+N90/9f5lk0Ed45P1w4JWMw4qtOGJ+5YA0W3Du0R25shgkfglZJpt Wx9hZilingxqrbtqczc8JFet8QFmkZifrO/+SkX8iTCYGTxhRihsLjlE4yxZd5hS k0PNeFlfB13x3tgESwY4prIe1oTch5qVWAgGDcMeSgnK/2Om8xiwdWMufTuHHgE/ 2GkbQ0pRNNn8a1B+MnVOelVktC7Cv8znAP9Aq6V7bwLSRKaBIYBjKyAd+5NmpuGl UxGPeUvSPSyIDlnPV68tfHtNrj65HuZ2mcs7a8dWmD6ODboETxTXkf5aDAMG2sly wQ6GuNCZKCJa0bHhQE9ADk7QnR8LUDRbzi4hJ9TSQrFmi1hlJhHZGDxO3ihw2nGe qz0PBkagbb6RIMHbXy6GObezIf9l14N+CUiCsg93CKKZaPpjgyqR2kkCsSX/Vi9r eN9CjxPP6rbQ/8H0T5MQy2V6Oq8YAoEGOGTUpOnwfnONk4Za39nKwwuk5JeGSTyr NL6UnDm/bhvSd24Sp4Vv =xZmR -----END PGP SIGNATURE-----

1 0

Simplifying load balancing by removing Guard+Exit?
by Mike Perry 28 Sep '15

28 Sep '15

Hey all, I am playing with the load balancing equations in 3.8.3 in dir-spec.txt (https://gitweb.torproject.org/torspec.git/tree//dir-spec.txt#n2454) in order to account for padding, hidden/onion services, and directory traffic, and I've noticed that the fact that nodes can be both Exits and Guards not only vastly complicates the current equations, but will make any attempt at accounting for this additional overhead nearly impossible. If you've previously tried to understand the current load balancing equations, or wondered why they have 3 major sub-cases and roughly 9 sub-cases, this is entirely to account for Guard+Exit nodes, and to handle relative scarcity of pure Guard and pure Exit nodes in various conditions. Removing the Guard+Exit ('D') nodes makes all of those cases condense to one single, simple set of solutions: ⎧ E + G + M E + G + M 2⋅E - M - G 2⋅G - M - E ⎫ ⎨Wee: ─────────, Wgg: ─────────, Wme: ───────────, Wmg: ────────────⎬ ⎩ 3⋅E 3⋅G 3⋅E 3⋅G ⎭ No missing constraints, no sub-cases, no boundary conditions. Easy to reason about. Easy to intuitively convince yourself that the solutions are correct. What I want to do is add two variables to the consensus that allow us to specify expected (or measured) overhead at Guard nodes (G_o) and at Middle nodes (M_o). These two variables would represent the sum of padding, directory overhead, and hidden service traffic experienced by Guard and Middle nodes, respectively. If we ignore Guard+Exit nodes, this makes the load balancing equations become: (1-G_o)*Wgg*G == (1-M_o)*(M + Wme*E + Wmg*G) # Guard bw == middle bw (1-G_o)*Wgg*G == Wee*E # Guard bw == exit bw Wmg*G + Wgg*G == G # Guards can be middles or Guards Wme*E + Wee*E == E # Exits can be middles or Exits These solutions are a bit longer so I won't paste them here, but they are still reasonable if we ditch the Guard+Exit idea. They can be simplified even further by using a similar approach as the GuardFraction idea from Proposal 236 (Section 1.3): We can simply adjust each Guard's bandwidth by G_o either before solving the equations, or by adjusting Wgg and Wmg after the solution, then we only need to include the (1-M_o) factor in the equations themselves. We can't really do this simplification if we keep Guard+Exit, because of the interplay between Exit and Guard weights in the system of equations and cases. Additionally, if we keep Guard+Exit, the solutions literally consume an entire page of text to represent symbolically, and the sub-cases become even more complicated to evaluate and check for proper bounds and correctness. In fact, George has already run into such edge cases while trying to implement Proposal 236 (See https://trac.torproject.org/projects/tor/ticket/16255) Moreover, I've been thinking about the surveillance incentives for Guard+Exit nodes, and I actually think it is a net risk to the network for nodes to be both Guards and Exits. It's easy to find an excuse to watch an Exit node. If anything bad ever happens from an Exit, you may be able to convince someone to let you keep an eye on it** "just in case". A Guard-only node shouldn't cause anybody any problems, though, and should be much harder to justify monitoring. So this means that Guard+Exit nodes allow an adversary to justify monitoring large portions of both the entry and exit traffic of the network without any strange/technical/esoteric arguments about the need to perform complicated statistical attacks with questionable accuracy and high collateral damage, and instead force them to justify their monitoring another way. I also hear from large node operators that it is still the Exit flag that causes their nodes to really become overloaded, so if that's actually true, then it's likely that Exit nodes make poor Guards anyway, especially if Exit bandwidth is scarce (which it often is). So, what do people think? Does anyone violently disagree with removing the Guard flag from Guard+Exit nodes? ** I used to run an Exit node, and my lawyer at the time was told exactly this at one point during a police visit about a nonsensical bomb threat to some 4chan equivalent. He was specifically told "We will be monitoring this node from now on" by one of the visiting officers. I was unable to ever discover what that actually meant, and the EFF unfortunately did not feel it was worth pursuing to help me find out. It may have been a scare tactic, but it could just as easily have not been, and for all I know they may have been actually able to convince a judge to let them do this. -- Mike Perry

3 2

Two new Onionoo versions 2.4 and 2.5 add new "effective_family" and "measured" fields
by Karsten Loesing 27 Sep '15

27 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi devs, I was reminded by Harmony that I said a while back [0] that I would announce new Onionoo versions on this list and not only in private message to Harmony for including them in the next TWN issue. Oops. So, version 2.4 added a new "effective_family" field to details documents, listing all the relays with which the relay in question is in an effective, mutual family relationship. The main goal here is to make it easier to detect misconfigured relay families. This can be relay operators or friendly people watching over the Tor network and reminding relay operators to fix their configurations. [1] The new version 2.5 that I deployed this week adds the optional "measured" field to details documents. The main idea behind this new field is that relay operators and Tor network debuggers can now figure out easily whether a relay is affected by not being measured by a sufficient number of bandwidth authorities and as a result has lower usage than it could handle. This field is not yet displayed by the Onionoo clients Atlas or Globe, but it's accessible via Onionoo's API [2]. More details are on ticket #16020 [3]. All the best, Karsten [0] onionoo-announce@ posting from April 27, 2015 [1] https://lists.torproject.org/pipermail/tor-news/2015-August/000109.html [2] https://onionoo.torproject.org/protocol.html#details [3] https://trac.torproject.org/projects/tor/ticket/16020 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV1HYeAAoJEJD5dJfVqbCrFJAIAIQYIC0CiSx6GJJikQ6CTxgw +AljFxABfnLtHu58EtBLx1jRJo0u9fStCMA/YFWLiq3FmEhCb5YvoL4zwa92PwlN bIoEricgq3sXjOm6TxJ75/6OBeH8AVIsah6qxHFkVfCPwjmt0bsDyaE7VqlCv2tN tAunIZcDyhsStGOOErr5uc9FHgPquvb66Iy7YFwnJxioMRE3yp4GBTDv78WyUEuR TZOWBh/cbJYrwSu5b+vU3C+t9eAI9ErQ8BDYlwy0C6zgJO+paSUkxWkCvuI3nv2l e/Ol6M40c9syZEb5jexn3K3NTOpJCKKhjXoXtP7UHXz8ZSdhZw2gxi31ZXYrSkQ= =/rwg -----END PGP SIGNATURE-----

3 3

Tor on iOS 9
by Conrad Kramer 27 Sep '15

27 Sep '15

Hi all! I'm new to the list, my name is Conrad. As some of you may or may not have heard, Apple's latest version of iOS enables functionality that allows for a system-wide Tor client. A few people in the community, myself included, started working on a project to get this client into the App Store. I wrote up a small document (attached) that explains the project in detail, so that you can learn about it if you're interested! I'm very excited to get Tor into the hands of many more users. Conrad

1 0

0.2.8 triage at dev meeting!
by Isabela 25 Sep '15

25 Sep '15

Hello everyone! We are planing on doing a triage session for 0.2.8 release in Berlin! Below is some guidance of what you can do to prepare for that, also what you can do if you are planning on being there but want to help build the release. Who should participate? Core Tor developers, obfuscation developers, Tor Browser developers, Hidden Services developers, and of course anyone else To help with the triage we are asking everyone to add the following information to tickets you think should be part of 0.2.8 * keyword field: 0.2.8.x-triage * Size field: an estimation of how long you think it will take to get it done (not counting review time, just execution): * small < 1 day * medium < 1 week * large > 1 week * Priority field: * blocker * critical * major * normal * minor * trivial * Sponsor field: if it's related to a sponsor deliverable For 0.2.7 triage we did a few rounds until we ended with a good chunk of work for the release. The release is coming out this month and here is some numbers related to it if you would like to use as a reference: https://storm.torproject.org/shared/TLgTuwGNswrIbfhj9NUctPKj6iG9V7KGHpwy7yz… I will send more updates on how this first triage session will be done in Berlin and how we will get remote folks to participate. Also, this does not mean what we do in Berlin is final, things can be refined after too. Cheers, Isabela -- PM at TorProject.org gpg fingerprint = 8F2A F9B6 D4A1 4D03 FDF1 B298 3224 4994 1506 4C7B @isa

4 4

Desired exit node diversity
by Virgil Griffith 25 Sep '15

25 Sep '15

Let's try a simple special case. In an idealized Tor network, what would the distribution of exit nodes look like? * Would each exit node have the same bandwidth? Or would there instead be only one exit node per AS? * Would the number of exit nodes constitute exactly 1/3 of all Tor nodes? Would the total exit node bandwidth constitute 1/3 of all Tor bandwidth? * Would their distribution of AS numbers match the distribution of AS numbers of Tor clients? The distribution of Internet users? Exit nodes seem a nice place to start concretizing what's meant when we say we want relay diversity. Comments immensely appreciated because as-is I don't know the answers to these questions. -V

8 11

[Measurement Team] Next IRC meeting happens on Thursday, Sep 24, 14:00 UTC in #tor-dev
by Karsten Loesing 23 Sep '15

23 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Thursday, September 24, 2015, 14:00 UTC in #tor-dev https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150924T14 This will be the last IRC meeting before the Berlin dev meeting. Maybe we'll hold an in-person team meeting next week at the same time. We'll resume the IRC meeting format the week after. Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWArn+AAoJEJD5dJfVqbCrm/YIAMTJvTAPpmbg0B85By/GfzLL mtTRB108vCLl+mkAOOkefo3o0u7RXcGlWeuSIUsqfoS4AcNpKVTF8dMrOyxCm80c nIOcwC8QX9gHGNXJXKIXaDCSeZ5MRcSLa2gLpllc/M844LbrfILIVQiF9AKafEiM DhJ7Hw41E9dNma1Qfm8LQXYwd9Sp1DLjRIFuGs3hOurZH9b2H+9Z9S+8M4D0I/hM FDyDo11MbRHEWcombVAX6IdkCUBjcZnjOUIu3tvca3pvf67or3gsRKI1GCLIRwan /Ff5TxDPkltTQwmU2E4xLpuI09baKcRpdiInkrsg1E6JfH1tT2VESJxNYb+1STM= =QMdU -----END PGP SIGNATURE-----

1 0

Looking for resources
by TOR USER 22 Sep '15

22 Sep '15

Hello, I was going through the Tor proposals which have now been closed. I have been looking for discussion regarding those proposals (here - https://gitweb.torproject.org/torspec.git) Could you link me to some discussion threads because I couldn't find much on them? Thank you

2 1