- tor-dev - lists.torproject.org

Fwd: Orbot 16.2.0 RC-1 (tor-0.4.2.7)
by Nathan Freitas 04 May '20

04 May '20

Now with integrated MOTE bridge request support, just like on Onion Browser iOS, courtesy of Benjamin Erhart (aka tla). -------- Forwarded Message -------- Subject: Orbot 16.2.0 RC-1 (tor-0.4.2.7) Date: Mon, 4 May 2020 12:03:13 -0400 From: Nathan of Guardian <nathan(a)guardianproject.info> Organization: Guardian Project To: Guardian Dev <guardian-dev(a)lists.mayfirst.org> Orbot 16.2.0 RC-1 (tor-0.4.2.7) Tag: https://gitweb.torproject.org/orbot.git/tag/?h=16.2.0-RC-1-tor-0.4.2.7 APKs: - https://github.com/guardianproject/orbot/releases/tag/16.2.0-RC-1-tor-0.4.2… - Universal: https://guardianproject.info/releases/Orbot-16.2.0-RC-1-tor-0.4.2.7-1-geed7… (.asc) - ARM64: https://guardianproject.info/releases/Orbot-16.2.0-RC-1-tor-0.4.2.7-1-geed7… (.asc) - F-Droid soon... Thanks to the fantastic contributions by @bitmold @tladesignz and @Hashik-Donthineni Highlights Resolved many issues related to startup of Tor and Obfs4/Meek bridges on Android Q/10 VPN fixes resolved through unifying 2 background services (TorVPNService, TorService) into one (OrbotService) Update tor to 0.4.2.7 and obfs4proxy to 0.0.12 latest Brand new "beta" integrated bridge fetch feature through Tor's MOTE service (just like in Tor Browser) Brand new custom bridge configuration, setup UI Zulu language random switch bug FINALLY fixed.... now the language you choose remains! (sorry, Zulu!) and much more... /** 16.2.0-RC-1-tor-0.4.2.7 / 4 May 2020 / 0a3a4f7 **/ 0a3a4f7 (HEAD -> master) update version code to 16202001 a631077 (HEAD -> master, origin/master, origin/HEAD) Merge branch 'tladesignz-master' 875f18a (tladesignz-master) Merge branch 'master' of https://github.com/tladesignz/orbot into tladesignz-master 85e027c update version code cabf328 don't start tun2socks/VPN if it is already running 718bb42 Issue #324: Implemented CustomBridgesActivity analogous to Onion Browser iOS. 12a3f4a (tag: 16.2.0-BETA-4-tor-0.4.2.7) remove TorVpnService from manifest to address #327 crash 22d7ef8 (tag: 16.2.0-BETA-3-tor-0.4.2.7) update to 16201003 22bd248 update binary install scripts, to fix pdnsd install - now pdnsd is packaged as libpdsnd.so to work on newer platforms e93c8bc (tag: 16.2.0-BETA-2-tor-0.4.2.7) update version code to 16201002 d0fec8e update tor and obfs4proxy binary libs to address #326 hopefully 3ac5a70 Issue #309: Activate newly required bridges, fixed accidental reset through Tor status notification. e1425d3 Issue #309: Also disable CAPTCHA solution EditText, as long as there's no CAPTCHA to solve and while letting it check. ff7cbcf Issue #309: Error alert button should say "OK" instead of "Cancel", since there's nothing to cancel, really, just to acknowledge the error. 54f5f31 Issue #309: Hide refresh option menu item instead of disable, because Android doesn't visualize that adequately. Additionally: Fix issue where you couldn't refresh after a network error. 3bfaa8a Issue #309: Fixed potential crash, when activity went away. c41d453 (public/master) fix unneeded includes 4f4300a (tag: 16.2.0-BETA-1A-tor-0.4.2.7, tag: 16.2.0-BETA-1-tor-0.4.2.7, ghdev/master) update version to 16201001 7d7b2e9 remove app updater code to address concerns in #323 ec47151 major VPN refactor with Android Q updates for #263 #261 #151 #316 and #303 - integrate TorVPNService directly into OrbotService, so just one service now - removed VPNEnableActivity, and just have VPN activated by service now - changes address start on boot with VPN enable - also improvements to managing PDNSD daemon d64b4d0 (tag: 16.1.5-BETA-2-tor-0.4.2.7) update version code to 16150001 d5bc374 update to SDK 29 919c211 improve pdnsd pid checking e78c27b remove unneeded commands in tor config and startup 5cd6cc8 update tor to 0.4.2.7 785fa8f Merge branch 'master' of github.com:guardianproject/orbot 93a0786 Merge pull request #322 from bitmold/AboutDialogCrashFix b24750e Fixes #321 IlleglaStateException on about dialog 2e6c9b1 (tag: 16.1.5-BETA-1-tor-0.4.2.5-rc) small tweaks to request bridge strings 3c80df0 Merge branch 'bitmold-compile_erorr+refactor' 08de553 (bitmold-compile_erorr+refactor) Merge branch 'compile_erorr+refactor' of https://github.com/bitmold/orbot into bitmold-compile_erorr+refactor df847ff Merge branch 'tladesignz-master' b2646a3 update AndroidPT to latest 1.0.8 with fixes for obfs4 and meek on Android 10 feedb3b fix string variable values 155980f Change from using String resources for a bundle key in Onboarding ec0ad26 Fixes compile error on master branch right now f2c8473 Merge branch 'master' of https://github.com/tladesignz/orbot into tladesignz-master 732c153 Merge branch 'master' of https://github.com/guardianproject/orbot 09ded59 (githubgp/master) Merge pull request #311 from bitmold/email_bridges_fix b1049ca Merge branch 'master' of https://github.com/guardianproject/orbot b9804c1 Merge pull request #305 from Hashik-Donthineni/AppIntroOrientationFix 410ae03 tweaks to app data storage and control port interaction - in some cases, Orbot connection to local control port can hang. Make changes to address that. 19d5054 Color indeterminate ProgressBar. 1749167 Fixed behaviour on device turn. f94931e Don't allow refresh button to trigger while request ongoing. 7a9b9d1 Adapted to naming convention of project. 964c311 Fixed MoatActivity startup when Tor is not running. Reset old bridge configuration, when not completed. 5c99859 Finish BridgeWizardActivity on successful MOAT. a34ee22 Improved UI: show progress spinner, start request on keyboard enter. a46e463 Updated Gradle to latest version. c026e20 Check OrbotService status, start Tor, if down, switch to Meek, when started, fetch CAPTCHA, as soon as done with that. Proxy Volley correctly. Improved option menu handling. Not working, yet: Tor doesn't come up. Why? e7cfe46 Added first stab at a MoatActivity. Doesn't change bridges automatically, yet. Also, Volley needs to be proxied. 469d4cf Fixed most warnings and errors in OrbotMainActivity. 3eb76e6 Updated Gradle. 4283221 Updated BUILD info. 446dfa6 Fixes bugs in #289 pertaining to parsing bridges 8627989 Merge pull request #310 from bitmold/zulu_context_menu 29541f3 Fixes #300 Where Locale is set to Zulu when settings are opened bfbb85b Changed variable names for better understanding a09b4eb Fixed Unused Imports f4580cf Added comments for better understanding 6c09f67 Extracted String Resources ee3e054 Fixed blank screen when orintation changed in AppIntro 8a3727a Merge pull request #301 from PLNech/patch-1 0dcba0d fix FAQ URL in README 73b0b0d handle NPE since node is not yet set Assets 6 Orbot-16.2.0-RC-1-tor-0.4.2.7-1-geed732a3-fullperm-arm64-v8a-release.apk 11.2 MB Orbot-16.2.0-RC-1-tor-0.4.2.7-1-geed732a3-fullperm-arm64-v8a-release.apk.asc 833 Bytes Orbot-16.2.0-RC-1-tor-0.4.2.7-1-geed732a3-fullperm-universal-release.apk 32.2 MB Orbot-16.2.0-RC-1-tor-0.4.2.7-1-geed732a3-fullperm-universal-release.apk.asc 833 Bytes Source code (zip) Source code (tar.gz)

1 0

Proposal Idea: IPv6-Only Exit Relays
by teor 01 May '20

01 May '20

Hi all, The network team is making it easier to set up a dual-stack Tor relay. We're currently working on: * IPv6 reachability self-tests * IPv6 address auto-detection * IPv6 relay statistics We're also thinking about the next IPv6 project. One possibility is adding IPv6-only exits to the Tor network. We'd need to solve two technical problems: 1. stop requiring IPv4 in the consensus and circuit code 2. avoiding delays when using an IPv6-only exit for an IPv4-only site And one deployment problem: 3. Get more dual-stack middle relays (50% - 75%) Here's a quick sketch of a solution to the IPv4-only site issue: Clients open two exit streams for every site. One of those streams must use an exit with IPv4 support. The client uses whichever stream succeeds first. Just like the "happy eyeballs" protocol, there is a delay between launching the two streams: https://www.rfc-editor.org/rfc/rfc8305.txt Here are some benefits of a universal change: * code and protocol consistency * makes traffic analysis harder * also helps with other exit failures, for a better user experience Here are some drawbacks: * extra load We can change the delay to manage the tradeoff between load, user experience, and traffic analysis resistance. Making the delay longer reduces the load, but makes the user experience worse. Traffic analysis resistance is a bit more complex. Consistency across different clients is important, but we also don't want to create obvious traffic patterns. So there are also some benefits to randomisation. If I get time, I'd like to turn these ideas into a proposal. T -- teor ----------------------------------------------------------------------

1 0

Tor Relay wont connect to private IP address
by Eli Vakrat 29 Apr '20

29 Apr '20

Hey guys! So thanks to teor's insightful response yesterday I decided to try to run a second tor relay (my middle node) on my private network. Unfortunately, I can't do it with Chutney because my python client is running on a windows machine. But I do have 3 machines at my disposal: 1. A windows machine (the python client) 2. A mac (the guard node) 3. Another mac (the middle node) However, after connecting all three machines to my private LAN (meaning they now all have local IP addresses), the EXTEND from the guard node to the middle node fails. When my guard node tries to connect to my middle node after receiving from the client a RELAY_EXTEND cell, the guard node logs the following error: Apr 28 17:00:31.000 [info] circuit_extend: Client asked me to extend to a private address Apr 28 17:00:31.000 [info] circuit_receive_relay_cell: connection_edge_process_relay_cell (away from origin) failed. Apr 28 17:00:31.000 [info] command_process_relay_cell: circuit_receive_relay_cell (forward) failed. Closing. So regarding this, I have two questions: 1. Is there a way for me to change something in my torrc file to override this error and allow my relay to extend to private IP addresses? My torrc is currently configured as such (Notice I put some place holders for the drectories and for the ip address tha aren't actually whats written there): ContactInfo e <draftkingschaching(a)gmail.com>mail(a)example.com ControlPort 9051 DataDirectory </path/to/data/dir> ExitPolicy reject *:* ExitRelay 0 GeoIPFile </path/to/geo/ip/file> GeoIPv6File </path/to/geo/ipv6/file> Log notice file <path/to/log/dirs/>/notice.log Log debug file <path/to/log/dirs/>/debug.log Log warn file <path/to/log/dirs/>/warn.log Nickname vtoria ORPort 443 NoAdvertise ORPort Relay.Public.IP.Example:443 <http://79.183.54.194:443/> NoListen SafeLogging 0 ExtendAllowPrivateAddresses 1 2. Would there maybe be a better way to run this private tor network (without chutney)? Thanks in advance for any answers! Eli

2 1

Relay Truncated Cell Example
by Eli Vakrat 28 Apr '20

28 Apr '20

Hello again everyone! For anyone who doesn't know\remember, my name is Eli, I'm a high school student from Israel and I'm currently trying to implement a TOR Client in Python. Currently, my project is configured so that my python client (OP) has its guard node set as my local machine (which is running a downloaded version of TOR). I do this for debugging purposes so that if I send a malformed cell as the implemented client, I can read the debug log that the OR generates and see what I did wrong. Last time I sent a question to the dev-list I was stuck trying to get my CREATE cells to work, and Nick Mathewson immediately answered my question and helped me a lot! Thanks Nick! I've made pretty good progress since then, and as of writing this, my implemented OP can successfully send a CREATE cell and even an EXTEND cell to my guard node. However, I am running into a problem while trying to extend my circuit to the third node (my exit node in this case). After sending a RELAY_EXTEND cell to my middle node (which is encrypted twice with my middle node and then guard node forward key using AES128 in CTR mode), the Response I get back is a relay cell with the correct Circuit Id and command (the command I get is the RELAY command which is represented as 3), BUT the payload of the response cell is very weird. I am unable to 'recognize' it (as specified in section 6.1 of the tor-spec). and furthermore, it does not seem to be any type of cell, it just seems like a bunch of nonsense. Seeing this I initially thought that the response cell I kept getting was a RELAY_EXTENDED cell that I couldn't 'recognize' due to an error while decrypting the cell payload. But then I looked at the debug log of my guard node (remember that my guard node is on my local machine) and it said that it had received a DESTROY Cell back from the middle node and was passing on a RELAY_TRUNCATED cell to me: *Apr 26 16:11:03.014 [debug] command_process_destroy_cell: Received for circID 2297363203.* <------this is the circuit ID between my gaurd node and the middle node *Apr 26 16:11:03.014 [debug] command_process_destroy_cell: Delivering 'truncated' back.Apr 26 16:11:03.014 [debug] relay_send_command_from_edge_: delivering 9 cell backward.* To my understanding what this log means is that some part of the EXTEND cell I sent to the middle node was wrong or malformed and because of this when the middle node tried to extend the circuit, an error occurred, and the circuit needed to be torn down. This is very weird because when I send an EXTEND cell that is meant for my guard node (meaning I want to extend the circuit from one hop to two hops) everything works fine, and I can even successfully derive the shared key material for the middle node. So I have several questions regarding this: 1. First of all, I didn't quite understand the exact format of a RELAY_TRUNCATED cell. Does it contain a relay cell command + recognized+field +digest and so on? or is it just a single octet that immediately follows the cell command field? If some could show me an example of the cell, it would be much appreciated... 2. What are some common errors that would make an OR drop a RELAY EXTEND cell? I thought maybe it was a problem with my TAP handshake data, but after extensive checking that doesn't seem to be the case. 3. If someone could describe the exact steps of extending a circuit to a third node, it would greatly help me to make sure that I didn't miss a step or do something wrong. Thanks in advance for any answers, examples, or comments! I am having a lot of fun doing this project so far and I hope to hear back from anyone who has an answer :) Regards, Eli

2 1

Walking onions status update: week 8
by Nick Mathewson 27 Apr '20

27 Apr '20

Walking Onions: week 8 update On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send out these updates once a week. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html Week 2: Specifying details of SNIP and ENDIVE formats, in CDDL. https://lists.torproject.org/pipermail/tor-dev/2020-March/014181.html Week 3: Expanding ENDIVEs into SNIPs. https://lists.torproject.org/pipermail/tor-dev/2020-March/014194.html Week 4: Voting (part 1) and extending circuits. https://lists.torproject.org/pipermail/tor-dev/2020-March/014207.html Week 5: Voting (part 2) https://lists.torproject.org/pipermail/tor-dev/2020-April/014218.html Week 6: Editing, voting rules, and client behavior (part 1) https://lists.torproject.org/pipermail/tor-dev/2020-April/014222.html Week 7: Exit policies: How do they work? https://lists.torproject.org/pipermail/tor-dev/2020-April/014232.html == Onion services: surprisingly easy! (but not?) This week I started with a look at onion services and how to make them work with Walking Onions [ONIONSPEC]. Most of my initial thought had gone into the problem of finding and using HSDirs, and so specifying this part went smoothly. I already havd a link specifier for asking for a span of several entries in an ENDIVE, and I already had a notion of truncating index values based on a network parameter. The harder part came when I wanted to talk about how to specify introduction points and rendezvous points by their SNIPs. The hard part here was managing the migration path. To avoid fingerprinting, when we make changes to the onion service protocol of this kind, we need to give a lot of onion services and clients the opportunity to start supporting the new protocol all at once, and then to stop supporting the old one all at once. This requires more network parameters, somewhat carefully specified. I'm hoping we can turn this into a more general pattern. And the most irritating part is when I ran into the issue of supporting HSv2 handshakes. Were it not for those, we wouldn't have to keep TAP keys around any more -- and TAP keys would bloat SNIPs to a pretty huge degree. The best answer here seems to be migrating our legacy HSv2 services so they can use ntor onion keys for intro and rendezvous points. That would also need to be done in a phased way. [TAP-OUT-AGAIN]. The alternative way to keep HSv2 working with walking onions would be to allow TAP keys to be distributed as part of the SNIP system. That seems like a fair amount of work, though, and unlike using ntor, it wouldn't have any positive benefit. ((Yes, HSv2 is deprecated. But I don't want to give people who still use it a reason to avoid the walking onions transition.)) == Relay honesty and ENDIVE swapping One of the issues with our current directory system is that multiple consensus network documents are valid at the same time. In practice, clients will accept fairly old documents, to avoid running into trouble if the authorities have failed to vote. But this client behavior gives hostile directory guards an opportunity to select which of several recent documents to serve. Walking Onions would make this problem worse, since every relay, when it serves a SNIP, is serving part of a directory that it knows a client is about to use. If we did nothing to mitigate the problem, then hostile relays could mount an "ENDIVE swapping" attack where they look at a client's choice of routing index, and then see whether any live ENDIVE contains a SNIP that the attacker likes at that position. In [HONESTY] I described a few responses to this problem. The first is that we should limit the rate at which the index layouts change. The second is that all parties that look up SNIPs through a relay should verify that the "published" times from each relay are monotonically increasing. The third response seems like the most useful to me: we would recognize that under ordinary circumstances, the permissible lifetime for an ENDIVE would be much shorter than it would be under emergency (consensus failed) circumstances. That is, if everything is going well, each relay should get a new ENDIVE every hour or so. Only if there is no new ENDIVE should clients accept SNIPs that are extremely old. With this in mind, I'm proposing a rule that if a client sees _any_ valid SNIP published at time `T` or later, it should not accept any SNIP published before time `T-Delta`. (Delta is a network parameter.) With this rule in place, clients will insist on fresh SNIPs when the network is working well, but will tolerate old ones if the authorities are failing to reach consensus for a while. == More questions of migration One hard step in any proposal is dealing with migration questions. I've stared to try to lay out a timeline in a new section [MIGRATION]. This part isn't looking great to me right now. It seems like it won't be safe to turn on Walking Onions for clients until it's supported by most relays. Moreover, clients that use Walking Onions won't be able to use relays (or at least, non-exit relays) without support for it. This implies a longer migration path than I'd like, but I don't see a way around it. == And what about families? Our existing family representation is quadratic in family size, since every relay needs to list every other relay in the family. This gets worse with Walking Onions, since SNIPs can't benefit from compressing the repeated family entries. At first I had hoped to use some kind of a max-clique algorithm to infer the families at the authorities, and then encode clique identifiers in the SNIPs. This isn't such a great idea, though: I wasn't able to persuade myself that an adversary couldn't force the authorities' burden to become exponential here. Instead I think we should resurrect proposal 242, in an improved version [BETTER-FAMILIES]. Instead of having to list every node in a family, each relay in a family would include a signed certificate from a "family key" attesting to their membership in the family. The proposal addresses some tricky compatibility issues. == Coming up next This week is going to be rather busy on my end: I'll be a bit startled if I get very much done with Walking Onions at all. My hope is to edit the proposal, fill in gaps, and address pending comments. [BETTER-FAMILIES] https://github.com/nmathewson/walking-onions-wip/blob/master/other-proposal… [HONESTY] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/08-track… [MIGRATION] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/09-migra… [ONIONSPEC] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/07-onion… [TAP-OUT-AGAIN] https://github.com/nmathewson/walking-onions-wip/blob/master/other-proposal…

1 0

Building a privacy-preserving "contact tracing" app
by Adrien Luxey 24 Apr '20

24 Apr '20

Hi all, The French state is making a glosing about the "privacy-preserving", "anonymous" contact tracing app they are developing with Inria (national informatics research agency). You can check about the protocol proposal, ROBERT, here: https://github.com/ROBERT-proximity-tracing/documents (in English!) As you would expect, the proposal is not privacy-preserving unless you believe the State would never ever misbehave, e.g. link IP address to identity with the help of ISPs etc. There is some relevant criticism here: https://github.com/ROBERT-proximity-tracing/documents/issues/6 I'd like to propose a really private "contact tracing" counter-proposal, which would use Tor's onion services for sender-receiver anonymity. Not that I am a proponent of the idea, but we need to come up with alternatives in the debate. My question is: /would the Tor community agree on having StopCovid's traffic go through its net?/ In my proposal, I would only broadcast a message from a declared COVID-positive person to all its recent contacts; the proximity sensing would remain Bluetooth. Still, deployed at the scale of a country, it's a lot of traffic. Thank you for your time, I will keep you posted, Adrien

4 3

Proposal 314: Allow Markdown for proposal format.
by Nick Mathewson 23 Apr '20

23 Apr '20

``` Filename: 314-allow-markdown-proposals.md Title: Allow Markdown for proposal format. Author: Nick Mathewson Created: 23 April 2020 Status: Open ``` # Introduction This document proposes a change in our proposal format: to allow Markdown. ## Motivation Many people, particularly researchers, have found it difficult to write text in the format that we prefer. Moreover, we have often wanted to add more formatting in proposals, and found it nontrivial to do so. Markdown is an emerging "standard" (albeit not actually a standardized one), and we're using it in several other places. It seems like a natural fit for our purposes here. # Details We should pick a particular Markdown dialect. "CommonMark" seems like a good choice, since it's the basis of what github and gitlab use. We should also pick a particular tool to use for validating Markdown proposals. We should continue to allow text proposals. We should continue to require headers for our proposals, and do so using the format at the head of this document: wrapping the headers inside triple backticks.

1 0

Re: [tor-dev] Chutney code refactoring
by c 22 Apr '20

22 Apr '20

On Fri, 17 Apr 2020 18:01:42 -0400 Nick Mathewson <nickm(a)freehaven.net> wrote: > If you want to work on this it would be helpful to maybe start by > listing (here or elsewhere) some places where you *don't* feel like > you could (or would want to) write documentation: those would be a > good target for devs who _have_ worked on Chutney before. Sounds good, I will work on this. > Though I think this is likely to be an ongoing change that we see > over time, since Was this sentence supposed to be longer? > * More tests for the various parts of chutney, and refactoring to > make them more testable. Internal unit testing seems like a good idea, yes. > * Making chutney comply with one or more of the python style > checking tools that are out there. I can look into this; personally I have no experience with Python style-checking or linting tools, but now's a good time as ever for me to learn what is available and how it would work for Chutney. > This is an good vision for how Chutney should go; I really support > this idea. Glad you are on board :) Caitlin

3 2

Fallback Directory Handover
by teor 22 Apr '20

22 Apr '20

Hi all, Here's a summary of the current state of fallback directory mirrors. Overall Design This repository contains a list of potential fallback directory mirrors (a fallback "offer list"), and a script that checks each mirror for speed and reliability: https://gitweb.torproject.org/fallback-scripts.git/ There is a "Fallback Scripts" component for tickets: https://trac.torproject.org/projects/tor/query?status=!closed&component=Cor… The fallback system is designed to gracefully degrade as fallback directory mirrors fail. Failures shift load to directory authorities, and cause brief delays during client bootstrap. We expect the system to operate well, even if all the fallbacks have failed. But we try to keep the fallback failure rate below 20-30%. When the failure rate gets too high, we rebuild the fallback list. Regular Tasks This ticket is the parent ticket for the next fallback rebuild: https://trac.torproject.org/projects/tor/ticket/30971 This ticket contains the "offer list" changes that relay operators have requested. I usually commit them all at once, but you should feel free to do them incrementally: https://trac.torproject.org/projects/tor/ticket/30972 Sometimes, we don't have enough relays on the offer list, and we have to ask relay operators to opt-in to the list. Ideally, we want at least 100 fallbacks, we usually have between 120-160. Future Work It's hard to verify changes to the offer list. Changes are usually sent by email or through trac tickets. There's no reliable trust path from the relay key to the email or ticket. The opt-in process is also a manual process. It can be time-consuming. To resolve these issues, I had planned to add a signed fallback offer line to relay descriptors: https://trac.torproject.org/projects/tor/ticket/24839 Instead of checking the list in the fallback-scripts repository, the script can check relay descriptors instead. (Or check both, during the transition period.) Unresolved Issues Fallbacks eventually see the entire set of clients. Clients that are active all the time may only ever contact one fallback. (Clients re-use the same fallback for authority keys, and then switch to the consensus as soon as possible.) But clients whose consensuses have expired will choose new fallbacks at random. Ideally, clients should select fallback (and maybe authority) guards. That is, they should retry previously-selected fallbacks. There are some tradeoffs here: a bad fallback guard can continue to manipulate its client's view of the network. We can avoid this issue by selecting multiple fallback guards. Clients will need persistent state to remember their guards, so transient systems like TAILS won't benefit from this change. T -- teor ----------------------------------------------------------------------

2 2

Walking Onions: week 7 update
by Nick Mathewson 17 Apr '20

17 Apr '20

Walking Onions: week 7 update On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send out these updates once a week. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html Week 2: Specifying details of SNIP and ENDIVE formats, in CDDL. https://lists.torproject.org/pipermail/tor-dev/2020-March/014181.html Week 3: Expanding ENDIVEs into SNIPs. https://lists.torproject.org/pipermail/tor-dev/2020-March/014194.html Week 4: Voting (part 1) and extending circuits. https://lists.torproject.org/pipermail/tor-dev/2020-March/014207.html Week 5: Voting (part 2) https://lists.torproject.org/pipermail/tor-dev/2020-April/014218.html Week 6: Editing, voting rules, and client behavior (part 1) https://lists.torproject.org/pipermail/tor-dev/2020-April/014222.html == Clustering exit ports This week I tried to figure out how exits work with walking onions. A naive approach might have 65535 different routing indices in the ENDIVE (one for each TCP port), but that would give horrible performance: it would make the expanded ENDIVEs absolutely huge. Instead, we can cluster ports into "classes" based on which are always supported or not supported together. (For example, on the day I scanned the directory, every relay supporting any port in the range 49002-50000 supported all of them.) There are about 200 such classes on the current Tor network. But 200 exit indices is still probably too much: it seems like we'll need to cluster exit ports together in a more lossy way. For example, not every exit that supports port 80 supports port 443 or vice versa. But around 98% of them do -- so it we wouldn't be losing too much by using one exit index for "443 and 80", and not listing the 2% of exits that only support one of those ports. I wrote a short program [EXIT-ANALYSIS] to experiment with clustering strategies here, and settled on a greedy algorithm to get down to a smaller number of port classes by iteratively combining whichever pair of classes will cause us to lose the fewest number of (relay:port) possibilities. We can get down to 16 classes and lose only 0.07% of all the relay:port possibilities; we can get down to 8 classes and lose only 0.2%. That's not the whole story, though. Some of what we're losing would be things we'd miss. When the algorithm gives us 16 classes, it loses 75% of port 10000, 70% of port 23, and so on. Clearly not all ports are equivalent, and we should tune this approach to take that into account. We should also consider the bandwidth impact of these changes, and retool the algorithm so that it minimizes the bandwidth-per-port combination we have. We'd also likely want to hand-edit these port lists so that they make more sense semantically on their own. That way, we could take Teor's suggestion from last month and try to give exit relays the options to opt in to or out of human-intelligible partitions. Further, we could save some complexity by requiring dual-stack exits to support the same ports on IPv4 and IPv6. That way we wouldn't need to worry about divergent sets of port classes. == So what do we do with these clusters? If a client wants to connect to port 25, and uses a port-25-only index, that leaks the client's intention to the middle node. That's not so good. Instead, in the tech report [WHITEPAPER], one or both of Chelsea Komlo and Ian Goldberg came up with a mechanism called Delegated Verifiable Selection (DVS). With DVS, when the client sends a BEGIN cell, the relay can respond that it does not support the requested port, along with a SNIP from a relay that does. To prevent the relay from choosing an arbitrary SNIP, the BEGIN cell would have to include an auxiliary index value to be used when DVS is in place, and the END cell would need room for a SNIP. (Alternatively, clients could just make a 3-hop circuit and ask it for SNIPs to be used as exits on some other circuit.) == Voting on policy sets These policy clusters (and a few other things!) are too complex for the voting operations I had before. Fortunately, it's safe to treat them as opaque. I've added a new voting operation to allow authorities to vote on a bstr that is then decoded and treated as a cbor object (if it is valid and well-formed). It should be useful in other places too. == Migrating port classes Migrating from one set of port classes to another is nontrivial. Clients who have the old root document will expect the indices to correspond to one partition of ports, whereas clients with the new root document will expect another partition. To solve this, I've designed the system so that two sets of exit indices can exist concurrently while clients are moving to the newer partition of ports. Alternatively, we could give SNIPs a mechanism to contain a field saying "Your root document must be at least this new." I don't know which approach would be simpler; both seem like too much complexity for a feature that would only be used occasionally. == Next steps My next plan here is to write up onion services; I believe I've got the design there mostly figured out, and just need to write it down. After that, I'm planning to turn back and fill in all the missing subsections that I've written so far, and write a short introduction. There are a lot of missing sections, so that won't be super fast. I'm also going to need to show the whole thing to a beta reader or two to answer the question, "Would this make sense to somebody who doesn't already know what it says?" == Fun facts The script to generate exit port classes [EXIT-ANALYSIS] was written in Rust. If I'm counting right, it's only my second Rust program that actually does something useful! If you are one of those Rust enthusiasts who would like to critique it, I wouldn't mind at all -- just send me comments offline or make notes on the repository. (Only do this if you like critiquing newbie Rust that was never actually meant for production use. Don't do this if you just want to point out all the things that make it unsuitable for production use.) == Not-so-fun facts This week I've been fairly overwhelmed with the lead up to and down from the recent layoffs at Tor, in response to a downturn in funding during the COVID-19 pandemic [BLOGPOST]. Tor will continue, and this work will continue with it--but it is still a hard blow to have to lose so many excellent coworkers. (Yes, I'm still at Tor, for the record.) Do you know somebody who is interested in hiring one or more amazingly talented programmers, engineers, communications specialists, or project managers -- all with experience at productive remote work and all with dedication to improving people's lives? Please pass their information along to tor-alums(a)lists.torproject.org , and it will (after moderation) get forwarded to those affected. [BLOGPOST] https://blog.torproject.org/covid19-impact-tor [EXIT-ANALYSIS] https://github.com/nmathewson/walking-onions-wip/tree/master/exit-analysis [EXIT-PARTITION] https://lists.torproject.org/pipermail/tor-dev/2020-March/014182.html [WHITEPAPER] https://crysp.uwaterloo.ca/software/walkingonions/

1 0