- tor-dev - lists.torproject.org

Erebus Code Review
by Damian Johnson 09 Sep '15

09 Sep '15

Hi Cristobal, just took a peek at Erebus and looks good! There's some low hanging fruit for improvement (unused functions and such) but generally looks good. Cheers! -Damian ====================================================================== erebus/server/handlers/log.py ====================================================================== 9 import erebus.util.log Perfectly fine for now, but obviously most of this is copy/paste from Nyx. Both Nyx and Erebus are smack in the middle of development so this is great! Do whatever you find most expedient. But obviously in the long run we'll want to revisit this. ---------------------------------------------------------------------- 16 def conf_handler(key, value): 17 if key == 'features.log.maxLinesPerEntry': 18 return max(1, value) 19 elif key == 'features.log.prepopulateReadLimit': 20 return max(0, value) 21 elif key == 'features.log.maxRefreshRate': 22 return max(10, value) 23 elif key == 'cache.log_panel.size': 24 return max(1000, value) I know this is copy-and-paste from Nyx but personally I wouldn't bother. Many of the config keys are unused, and at present users don't have a way of customizing them. Nyx's support for a nyxrc is a very rarely used feature, so I would leave this out for now. ---------------------------------------------------------------------- 40 TOR_EVENT_TYPES = { 41 'd': 'DEBUG', 42 'i': 'INFO', 43 'n': 'NOTICE', 44 'w': 'WARN', 45 'e': 'ERR', I did this for Nyx because because... well, curses. For Erebus we can do better. ;) Rather than 'key => event' mapping maybe Erebus should show all the event types, with checkboxes for selection? ---------------------------------------------------------------------- 155 def _register_event(self, event): 156 output = { 'message': '', 'type': ''} 157 if event.type not in self._logged_events: 158 return 159 #self._event_log.add(event) 160 #self._log_file.write(event.display_message) 161 162 # notifies the display that it has new content 163 if self._filter.match(event.display_message): 164 self._has_new_event = True 165 output['message'] = event.display_message 166 output['type'] = event.type.lower() 167 ws = ws_controller() 168 if ws is not None: 169 ws.send_data(WebSocketType.LOG, output) You can drop the line with self._has_new_event. It's an unused attribute, and doesn't appear outside of this line. The 'output' is only ever used on this last line, so might as well just call... ws.send_data(WebSocketType.LOG, {'message': event.display_message, 'type': event.type.lower()}) ---------------------------------------------------------------------- 171 def log_handler(): 172 """ 173 Singleton for getting our log hander. 174 175 :returns: :class:`~erebus.server.handlers.log.LogHandler` 176 """ 177 178 return LOG_HANDLER 179 180 def init_log_handler(logged_events): 181 """ 182 """ 183 global LOG_HANDLER 184 LOG_HANDLER = LogHandler(logged_events) 185 return LOG_HANDLER Seems from a quick look that these might be unused. The controller.py and starter.py call init_log_handler, but not spotting anything that ever uses LOG_HANDLER. ---------------------------------------------------------------------- 195 response = None 196 if controller is not None: 197 response = controller.get_info('events/names', None) 198 if response is None: 199 return [] # GETINFO query failed This is equivalent to... response = controller.get_info('events/names', []) if controller else [] ====================================================================== erebus/util/controller.py ====================================================================== 29 def erebus_controller(): 30 """ 31 Provides the erebus controller instance. 32 """ 33 34 return EREBUS_CONTROLLER 35 36 def init_controller(control_port, control_socket, logged_events): 37 """ 38 Spawns the controller 39 """ 40 41 global EREBUS_CONTROLLER 42 EREBUS_CONTROLLER = Controller(control_port, control_socket, logged_events) Again, you call init_controller() but looks as though EREBUS_CONTROLLER is never used. ---------------------------------------------------------------------- 61 self._logged_events = logged_events 62 63 self._bw_handler = None 64 self._status_handler = None 65 self._log_handler = None All unused. ---------------------------------------------------------------------- 86 def _stop_loop_check(self): 87 """ 88 Stop the looping task when is not necessary 89 """ 90 91 self._loop_call.stop() Only called once, in _conn_starter(). Might as well just call this there (it's just a minor alias). ---------------------------------------------------------------------- 156 def heartbeat_check(is_unresponsive): 157 """ 158 Logs if its been ten seconds since the last BW event. 159 160 Arguments: 161 is_unresponsive - flag for if we've indicated to be responsive or not 162 """ 163 164 controller = tor_controller() 165 last_heartbeat = controller.get_latest_heartbeat() 166 167 if controller.is_alive(): 168 if not is_unresponsive and (time.time() - last_heartbeat) >= 10: 169 is_unresponsive = True 170 log.notice('Relay unresponsive (last heartbeat: %s)' % time.ctime(last_heartbeat)) 171 elif is_unresponsive and (time.time() - last_heartbeat) < 10: 172 # really shouldn't happen (meant Tor froze for a bit) 173 is_unresponsive = False 174 log.notice('Relay resumed') 175 176 return is_unresponsive Unused. ====================================================================== erebus/util/websockets.py ====================================================================== 41 def add_websocket(self, ws_type, ws): 42 """ 43 Add an object to the list of active websockets of certain type. 44 45 Arguments: 46 ws_type - type of websocket to be fetched 47 ws - websocket object to be added 48 """ 49 50 if self._websockets[ws_type] is not None: 51 if ws not in self._websockets[ws_type]: 52 self._websockets[ws_type].append(ws) 53 else: 54 log.notice('Undefined type (%s) when adding websocket' % ws_type) Your 'if self._websockets[ws_type] is not None' check will never evaluate to False. You default all WebSocketType values to an empty list, and if it doesn't belong to that enum then it'll raise a KeyError. You want... if ws_type in self._websockets: Also, if self._websockets had sets rather than lists then you could drop the second check. Btw, lets use Sphinx pydocs. The documentation style you're using here is adopted from Nyx before I started using Sphinx, and I'm replacing that documentation style as I rewrite it.

2 1

Troubleshooting Tor 0.2.7.1+ ephemeral hidden services
by Micah Lee 09 Sep '15

09 Sep '15

I'm trying to add support for ephemeral hidden services to OnionShare. They'll solve a wide variety of problems related to OnionShare using a system Tor instead of Tor browser, and running in environments like Tails or Whonix. I'd rather never write the hidden service info to disk anyway. I noticed that Tor Browser 5.5a2 comes with Tor 0.2.7.2-alpha, so it's easy to test. But so far I'm not having any success, at least with using stem. Before I open a bug report, either against stem or against tor, am I doing this right? I've made a simple test project: https://github.com/micahflee/ephemeral-hs-test It successfully appears to create the hidden service, but it never succeeds in connecting to itself. I also can't connect to it using Tor Browser. -- Micah Lee

2 2

Proposal: Out of Band Circuit HMACs
by Mike Perry 08 Sep '15

08 Sep '15

This proposal exists in my torspec remote here: https://gitweb.torproject.org/user/mikeperry/torspec.git/tree/proposals/xxx… Including it in-line for ease of reply: =============================== Filename: xxx-oob-hmac.txt Title: Out of Band Circuit HMACs Authors: Mike Perry Created: 01 September 2015 Status: Draft 0. Motivation It is currently possible for Guard nodes (and MITM adversaries that steal their identity keys) to perform a "tagging" attack to influence circuit construction and resulting relay usage[1]. Because Tor uses AES as a stream cipher, malicious or intercepted Guard nodes can simply XOR a unique identifier into the circuit cipherstream during circuit setup and usage. If this identifier is not removed by a colluding exit (either by performing another XOR, or making use of known plaintext regions of a cell to directly extract a complete side-channel value), then the circuit will fail. In this way, malicious or intercepted Guard nodes can ensure that all client traffic is directed only to colluding exit nodes, who can observe the destinations and deanonymize users. Most code paths in the Tor relay source code will emit loud warnings for the most obvious instances circuit failure caused by this attack. However, it is very difficult to ensure that all such error conditions are properly covered such that warnings will be emitted. This proposal aims to provide a mechanism to ensure that tagging and related malleability attacks are cryptographically detectable when they happen. 1. Overview Since Tor Relays are already storing a running hash of all data transmitted on their circuits (via the or_circuit_t::n_digest and or_circuit_t::p_digest properties), it is possible to compute an out-of-band HMAC on circuit data, and verify that it is as expected. This proposal first defines an OOB_HMAC primitive that can be included standalone in a new relay cell command type, and additionally in other cell types. Use of the standalone relay cell command serves to ensure that circuits that are successfully built and used were not manipulated at a previous point. By altering the RELAY_COMMAND_TRUNCATED and CELL_DESTROY cells to also include the OOB_HMAC information, it is similarly possible to detect alteration of circuit contents that cause failures before the point of usage. 2. The OOB_HMAC primitive The OOB_HMAC primitive uses the existing rolling hashes present in or_circuit_t to provide a Tor OP (aka client) with the hash history of the traffic that a given relay has seen it so far. Note that to avoid storing an additional 64 bytes of SHA256 digest for every circuit at every relay, we use SHA1 for the hash logs, since the circuits are already storing SHA1 hashes. It's not immediately clear how to upgrade the existing SHA1 digests to SHA256 with the current circuit protocol, either, since matching hash algorithms are essential to the 'recognized' relay cell forwarding behavior. The version field exists primarily for this reason, should the rolling circuit hashes ever upgrade to SHA256. The OOB_HMAC primitive is specified in Trunnel as follows: struct oob_hmac_body { /* Version of this section. Must be 1 */ u8 version; /* SHA1 hash of all client-originating data on this circuit (obtained from or_circuit_t::n_digest). */ u8 client_hash_log[20]; /* Number of cells processed in this hash, mod 2^32. Used to spot-check hash position */ u32 client_cell_count; /* SHA1 hash of all server-originating data on this circuit (obtained from or_circuit_t::p_digest). */ u8 server_hash_log[20]; /* Number of cells processed in this hash, mod 2^32. Used to spot-check hash position. XXX: Technically the server-side is not needed. */ u32 server_cell_count; /* HMAC-SHA-256 of the entire cell contents up to this point, using or_circuit_t::p_crypto as the hmac key. XXX: Should we use a KDF here instead of p_crypto directly? */ u8 cell_hmac_256[32]; }; 3. Usage of OOB_HMAC The OOB_HMAC body will be included in three places: 1. In a new relay cell command RELAY_COMMAND_HMAC_SEND, which is sent in response to a client-originating RELAY_COMMAND_HMAC_GET on stream 0. 2. In CELL_DESTROY, immediately after the error code 3. In RELAY_COMMAND_TRUNCATED, immediately after the CELL_DESTROY contents 3.1. RELAY_COMMAND_HMAC_GET/SEND relay commands Clients should use leaky-pipe topology to send RELAY_COMMAND_HMAC_GET to the second-to-last node (typically the middle node) in the circuit at three points during circuit construction and usage: 1. Immediately after the last RELAY_EARLY cell is sent 2. Upon any stream detachment, timeout, or failure. 3. Upon any OP-initiated circuit teardown (including timed-out partially built circuits). We use RELAY_EARLY as the point at which to send these cells to avoid leaking the path length to the middle hop. 3.2. Alteration of CELL_DESTROY and RELAY_COMMAND_TRUNCATED In order to provide an HMAC even when a circuit is torn down before use due to failure, the behavior for generating and handling CELL_DESTROY and RELAY_COMMAND_TRUNCATED should be modified as follows: Whenever an OR sends a CELL_DESTROY for a circuit towards the OP, if that circuit was already properly established, the OR should include the contents of oob_hmac_body immediately after the reason field. The HMAC must cover the error code from CELL_DESTROY. Upon receipt of a CELL_DESTROY, and in any other case where an OR would generate a RELAY_COMMAND_TRUNCATED due to error, a conformant relay would include the CELL_DESTROY oob_hmac_body, as well as its own locally created oob_hmac_body. The locally created oob_hmac_body must cover the entire payload contents of RELAY_COMMAND_TRUNCATED, including the error code and the CELL_DESTROY oob_hmac_body. Here is a new Trunnel specification for RELAY_COMMAND_TRUNCATED: struct relay_command_truncated { /* Error code */ u8 error_code; /* Number of oob_hmacs. Must be 0, 1, or 2 */ u8 num_hmac; /* If there are 2 hmacs, the first one is from the CELL_DESTROY, and the second one is from the truncating relay. If num_hmac is 0, then this came from a relay without support for oob_hmac. */ struct oob_hmac_body[num_hmac]; }; The usage of a strong HMAC to cover the entire CELL_DESTROY contents also allows an OP to properly authenticate the reason a remote node needed to close a circuit, without relying on the previous hop to be honest about it. 4. Ensuring proper ordering with respect to hashes 4.1. RELAY_COMMAND_HMAC_GET/SEND The in-order delivery guarantee of circuits will mean that the incoming hashes will match upon receipt of the RELAY_COMMAND_HMAC_SEND cell, but any outgoing traffic the OP sent since RELAY_COMMAND_HMAC_GET will not have been seen by the responding OR. Therefore, immediately upon sending a RELAY_COMMAND_HMAC_GET, the OP must record and store its current outgoing hash state for that circuit, until the RELAY_COMMAND_HMAC_SEND arrives, and use that stored hash value for comparison against the oob_hmac_body's client_hash_log field. The server_hash_log should be checked against the corresponding crypt_path_t entry in origin_circuit_t for the relay that the command was sent to. 4.2. RELAY_COMMAND_TRUNCATED Since RELAY_COMMAND_TRUNCATED may be sent in response to any error condition generated by a cell in either direction, the OP must check that its local cell counts match those present in the oob_hmac_body for that hop. If the counts do not match, the OP may generate a RELAY_COMMAND_HMAC_GET to the hop that sent RELAY_COMMAND_TRUNCATED, prior to tearing down the circuit. 4.3. CELL_DESTROY If the cell counts of the destroy cell's oob_hmac_body do not match what the client sent for that hop, unfortunately that hash must be discarded. Otherwise, it may be checked against values held from before processing the RELAY_COMMAND_TRUNCATED envelope. 5. Security concerns and mitigations 5.1. Silent circuit failure attacks The primary way to game this oob-hmac is to omit or block cells containing HMACs from reaching the OP, or otherwise tear down circuits before responses arrive with proof of tampering. If a large fraction of circuits somehow fail without any RELAY_COMMAND_TRUNCATED oob_hmac_body payloads present, and without any responses to RELAY_COMMAND_HMAC_GET requests, the user should be alerted of this fact as well. This rate of silent circuit failure should be kept as an additional, separate per-Guard Path Bias statistic, and the user should be warned if this failure rate exceeds some (low) threshold for circuits containing relays that should have supported this proposal. 5.2. Malicious/colluding middle nodes If the adversary is prevented from causing silent circuit failure without the client being able to notice and react, their next available vector is to ensure that circuits are only built to middle nodes that are malicious and colluding with them (or that do not support this proposal), so that they may lie about the proper hash values that they see (or omit them). Right now, the current path bias code also does not count circuit failures to the middle hop as circuit attempts. This was done to reduce the effect of ambient circuit failure on the path bias accounting (since an average ambient circuit failure of X per-hop causes the total circuit failure middle+exit circuits to be 2X). Unfortunately, not counting middle hop failure allows the adversary to only allow circuits to colluding middle hops to complete, so that they may lie about their hash logs. All failed circuits to non-colluding middle nodes could be torn down before RELAY_COMMAND_TRUNCATED is sent. For this reason, the per-Guard Path Bias counts should be augmented to additionally track middle-node-only failure as a separate statistic as well, and the user should be warned if middle-node failure drops below a similar threshold as the current end-to-end failure. 5.3. Side channel issues, mitigations, and limitations Unfortunately, leaking information about circuit usage to the middle node prevents us from sending RELAY_COMMAND_HMAC_GET cells at more optimal points in circuit usage (such as immediately upon open, immediately after stream usage, etc). As such, we are limited to waiting until RELAY_EARLY cells stop being sent. It is debatable if we should send hashes periodically (perhaps with windowing information updates?) instead. 6. Alternatives A handful of alternatives to this proposal have already been discussed, but have been dismissed for various reasons. Per-hop cell HMACs were ruled out because they will leak the total path length, as well as the current hop's position in the circuit. Wide-block ciphers have been discussed, which would provide the property that attempts to alter a cell at a previous hop would render it completely corrupted upon its final destination, thus preventing untagging and recovery, even by a colluding malicious peer. Unfortunately, performance analysis of modern provably secure versions of wide-block ciphers has shown them to be at least 10X slower than AES-NI[2]. 1. https://lists.torproject.org/pipermail/tor-dev/2012-March/003347.html 2. http://archives.seul.org/tor/dev/Mar-2015/msg00137.html -- Mike Perry

1 0

[Measurement Team] Next IRC meeting happens on Wednesday, Sep 9, 14:00 UTC in #tor-project
by Karsten Loesing 08 Sep '15

08 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Wednesday, September 9, 2015, 14:00 UTC in #tor-project https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150909T14 Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV7zD0AAoJEJD5dJfVqbCrNlAIAIHDOy3J3Rw7Tg8zbIECZMdU bLwUE31jnsIE54fPBv2+lofWq0FlciMNR/UGmuCDCvgKy1tzLAReWvZzZc4kB3hg sUMOYqB757/4IBFuh/6YWqrC7jOEuVEhky7/J/1obHj1pSzWY/Tt+whhC4Sfl4h5 WRHAUQUYPVIDaa8FjYiHCnHcTCLDiuu+FoODCD6LjtBX1dQYABkbUVm8Vo1vH/u+ UimxN+EhrJc/u5WA3Eb4YDURhikzUkdMM680gyZXrCip7klvX4fGlojBZHur6VkS p3LSSWfZ7o+mPBqyord7CPs+1o66sQLZdG6tzeOm3nQugqE2DgF9fNlnBkBt1D4= =sCDs -----END PGP SIGNATURE-----

1 0

Is there a way to do the man in the middle on hidden service?
by Liste 08 Sep '15

08 Sep '15

Sometime i can find the wrong SSL certificate on my hidden service.

1 0

Re: [tor-dev] Let's identify which measurement-related tools n eedwork when relays switch from RSA identities to ed25519 identities
by tordev123＠Safe-mail.net 08 Sep '15

08 Sep '15

From: Damian Johnson <atagar(a)torproject.org> To: "tor-dev(a)lists.torproject.org" <tor-dev(a)lists.torproject.org> Subject: Re: [tor-dev] Let's identify which measurement-related tools needwork when relays switch from RSA identities to ed25519 identities Date: Tue, 8 Sep 2015 09:16:15 -0700 > On a side note it would be nice to have a spec patch before changing > things this time. +1 The dir spec is still broken and doesn't cover the new descriptor format with stuff like: extra-info-digest 0589E8C4C8A522814EE8D7DC1A3703920B57208C Qychl+FIG6YEbXgGJaG13mMHW9z9Tci5D9DWCXOwmNg dir-spec: "extra-info-digest" digest NL [At most once] "Digest" is a hex-encoded digest (using upper-case characters) of the router's extra-info document, as signed in the router's extra-info (that is, not including the signature). (If this field is absent, the router is not uploading a corresponding extra-info document.) [Versions before 0.2.0.1-alpha don't recognize this]

1 0

Let's identify which measurement-related tools need work when relays switch from RSA identities to ed25519 identities
by Karsten Loesing 08 Sep '15

08 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi everyone, Sebastian suggested to me in one of our last measurement team 1-1-1 task exchange rounds [0] to think about the following question: Which measurement-related tools need work when relays switch from RSA identities to ed25519 identities? You'll find below what I came up with. Please send any feedback and corrections to this list. I'll incorporate everything and send an updated document to this list next week. Thanks! All the best, Karsten The switch from RSA to ed25519 will happen in multiple steps (need to confirm these!): 0. Before ed25519 identities were used, each relay had a unique RSA identity. 1. Relays can create an optional ed25519 identity, change it, or stop using it at will. 2. Relays can create an ed25519 identity, but are not allowed to change it or stop using it. 3. Relays can stop using their earlier RSA identity. 4. Relays only use their ed25519 identity. (Add a notion of timeframe here. It seems unlikely that 3 or even 4 will happen before 2017 or even 2018, given how long it takes to deploy a new major Tor version in the network. Sebastian thinks that the timeline may not contain step 1.) ## Directory authority tools: Torflow, Guardfraction There are several tools that provide data to directory authorities for voting on relays. These tools include bandwidth authorities and the yet-to-be-finished Guardfraction tool. They will have to handle new ed25519 identities and provide their data in a format that directory authorities will be able to use for voting. ## Network health scanners: Exitmap, Sybilhunter It's unclear how much work remains to update network scanners like Exitmap and Syilbhunter. ## Descriptor parsing libraries: metrics-lib, Stem, Zoossh Descriptor parsing libraries need to support parsing new fields and later need to support that fields containing RSA fingerprints have become optional. Stem has already implemented ed25519 identities, and there's Java code that metrics-lib could re-use in CollecTor's bridge descriptor sanitizer. Zoossh is probably not updated yet. ## Descriptor archiver: CollecTor The part of CollecTor that archives relay descriptors relies heavily on RSA identities and SHA-1 digests. This includes its implementation of the Tor directory protocol and its structure for storing fetched descriptors in tarballs. Another part of CollecTor that needs to be updated is the bridge descriptor sanitizer. That is already done to the extent of sanitizing ed25519-related descriptor lines. The part that is not updated yet is the tarball structure that still uses SHA-1 digests. ## Network health checkers: DocTor, DepicTor Network health checkers like DocTor and DepicTor use relay fingerprints, but they focus more on network consensus meta data provided by the directory authorities. Updating to ed25519 identities shouldn't be hard. ## Statistics portal: Metrics Most of the data-aggregating modules behind the Metrics website use RSA identities to ensure uniqueness of reported statistics. Switching to another fingerprint format or length should be doable, but will cause some effort. ## Relay address database: ExoneraTor The relay address database ExoneraTor only uses the RSA identity to ensure uniqueness of database entries and for display purposes. Supporting new identities should not be that difficult. ## Onionoo server and clients: Onionoo, Atlas, Globe, Compass, OnionTip, Roster The Onionoo server heavily depends on RSA fingerprints of relays and hashed RSA fingerprints of bridges, both internally and in its API. Supporting ed25519 identities in addition to RSA fingerprints would not be hard, but stopping to support RSA fingerprints will require rewriting major parts of Onionoo. Onionoo clients use the RSA fingerprint provided by the Onionoo server to uniquely identify relays and bridges. They will have to adapt to changes in Onionoo that would first support ed25519 identities and later stop supporting RSA fingerprints. This shouldn't be difficult. ## OONI It's unclear how much effort is needed to update OONI to support new ed25519 identities. ## Exit address scanners: TorDNSEL, TorBEL, Check TorDNSEL and its planned successor TorBEL include RSA identities in their output format. It might be sufficient to simply add ed25519 identities there and leave out RSA identities once they are not used anymore. Check will have to be updated to understand these new formats. [0] 1-1-1 task exchange: you get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do for you within 1 week (review a document, write some analysis code, fix a small bug, etc.; better come prepared to get the most out of this; give 1, take 1) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV7uf1AAoJEJD5dJfVqbCrBJUH/A78UIeNy5UIvctO6EjMp6KU XXeB7jxDoA9RmdhNV7v5MzhSCjblQLfG5tyLmeFV/opX+xG0fMvqFdUB9jupXccV HQXLjeSuNF4Tenayn3W49EUF/O75BHcfwe0fHzmV8KEIhPf66KQn6gNBFFnWr4S0 OfvoSWbIb9Wu7hCya4C7J69FVAhwXPH8JpgpSpKLFYSZWQo1aKbgVqBDKNe/lmrV UO04Ws/O04osG9WWX/HcZgSCfSaN68nBMllnu0yCTJY+fEqFruxYhAaZZV6tXIfG E66vZc2K3S6Rk/1nwHEkNeZKCfjyRG1LthzH28HdCEMzl1mdQhUPkJawNaQXoDM= =rXAd -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] Proposal: Single onion services
by tordev123＠Safe-mail.net 05 Sep '15

05 Sep '15

-------- Original Message -------- From: John Brooks <john.brooks(a)dereferenced.net> To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Proposal: Single onion services Date: Fri, 4 Sep 2015 15:31:15 -0600 > tordev123(a)Safe-mail.net wrote: > > > Doesn't your proposal imply that you are turning all relays into > > exit-nodes lite? The last relay in the path will know what service you are > > connecting to (at least if that service is hosted with a unique relay), > > right? > > A single onion service operates its own server(s). These servers accept OR > connections like a relay does, but they aren’t required to be in the > consensus or to relay traffic. They are the servers listed in the > descriptor. Right, that's how I understood things. > A client connects by extending a circuit to the single onion server. This is > not the same as an exit connection: tor relays will extend circuits to > relays they don't know about, as long as the destination speaks the tor > protocol. It’s possible for any tor relay to be used as the last one before > the single onion server. > > If the single onion server isn’t also a tor relay, it’s possible for the > previous relay to guess the service you’re connecting to. That is a large part of my concern. > This isn’t a risk > to client anonymity, because tor clients will always choose the first three > hops in a circuit before extending to one they didn’t choose. I'm not concerned about clients. > The final circuit looks like: > > Client -> Guard -> Middle -> Middle -> Single Onion > > The client’s traffic is encrypted through to the single onion server as > well. IMO, the second Middle relay can be considered serving as an exit with regards to Single Onion services - that's what I meant with 'exit node lite'. There was the case of an Austrian exit node operator getting prosecuted. It will sometimes be possible to attribute traffic relating to specific transactions to the second Middle node in the path (e.g. when the single onion server keeps detailed logs). So the circumstances of that case could apply to a non-exit relay operator as well. Your proposal is shifting non-exit relays towards performing a role that can be considered exit-like, even if that role is much more limited than normal exits (and there is an additional Tor protocol layer involved).

2 1

Proposal: Single onion services
by John Brooks 04 Sep '15

04 Sep '15

Here’s a delayed shipment from the hidden services hackfest: Single onion services are a modified form of onion services, which trade service-side location privacy for improved performance, reliability, and scalability. Single onion services have a .onion address identical to any other onion service. The descriptor contains information sufficient to do a relay extend of a circuit to the onion service and to open a stream for the onion address. The introduction point and rendezvous protocols are bypassed for these services. The full proposal by Paul, Roger, and myself is below, and available from: https://gitweb.torproject.org/user/special/torspec.git/plain/proposals/idea… Thanks especially to Paul, who is behind this whole concept, and all of the other participants of the Arlington Accords. - special ----- Filename: xxx-single-onion.txt Title: Single Onion Services Author: John Brooks, Paul Syverson, Roger Dingledine Created: 2015-07-13 Status: Draft 1. Overview Single onion services are a modified form of onion services, which trade service-side location privacy for improved performance, reliability, and scalability. Single onion services have a .onion address identical to any other onion service. The descriptor contains information sufficient to do a relay extend of a circuit to the onion service and to open a stream for the onion address. The introduction point and rendezvous protocols are bypassed for these services. We also specify behavior for a tor instance to publish a single onion service, which requires a reachable OR port, without necessarily acting as a public relay in the network. 2. Motivation Single onion services have a few benefits over double onion services: * Connection latency is much lower by skipping rendezvous * Stream latency is reduced on a 4-hop circuit * Removing rendezvous circuits improves service scalability * A single onion service can use multiple relays for load balancing Single onion services are not location hidden on the service side, but clients retain all of the benefits and privacy of onion services. More details, relation to double onion services, and the rationale for the 'single' and 'double' nomenclature are further described in section 7.4. We believe these improvements, along with the other benefits of onion services, will be a significant incentive for website and other internet service operators to provide these portals to preserve the privacy of their users. 3. Onion descriptors The onion descriptor format is extended to add: "service-extend-locations" NL encrypted-string [At most once] A list of relay extend info, which is used instead of introduction points and rendezvous for single onion services. This field is encoded and optionally encrypted in the same way as the "introduction-points" field. The encoded contents of this field contains no more than 10 entries, each containing the following data: "service-extend-location" SP link-specifiers NL [At start, exactly once] link-specifiers is a base64 encoded link specifier block, in the format described by proposal 224 [BUILDING-BLOCKS] and the EXTEND2 cell. "onion-key" SP key-type NL onion-key [Exactly once] Describes the onion key that must be used when extending to the single onion service relay. The key-type field is one of: "tap" onion-key is a PEM-encoded RSA relay onion key "ntor" onion-key is a base64-encoded NTOR relay onion key [XXX: Should there be some kind of cookie to prove that we have the desc? See also section 7.1. -special] A descriptor may contain either or both of "introduction-points" and "service-extend-locations"; see section 5.2. [XXX: What kind of backwards compatibility issues exist here? Will existing relays accept one of those descriptors? -special] 4. Reaching a single onion service as a client Single onion services use normal onion hostnames, so the client will first request the service's descriptor. If the descriptor contains a "service-extend-locations" field, the client should ignore the introduction points and rendezvous process in favor of the process defined here. The descriptor's "service-extend-locations" information is sufficient for a client to extend a circuit to the onion service, regardless of whether it is also listed as a relay in the network consensus. This extend info must not be used for any other purpose. If multiple extend locations are specified, the client should randomly select one. The client uses a 3-hop circuit to extend to the service location from the descriptor. Once this circuit is built, the client sends a BEGIN cell to the relay, with the onion address as hostname and the desired TCP port. If the circuit or stream fails, the client should retry using another extend location from the descriptor. If all extend locations fail, and the descriptor contains an "introduction-points" field, the client may fall back to a full rendezvous operation. 5. Publishing a single onion service To act as a single onion service, a tor instance (or cooperating group of tor instances) must: * Have a publicly accessible OR port * Publish onion descriptors in the same manner as any onion service * Include a "service-extend-locations" section in the onion descriptor * Accept RELAY_BEGIN cells for the service as defined in section 5.3 5.1. Configuration options The tor server operating a single onion service must accept connections as a tor relay, but is not required to be published in the consensus or to allow extending circuits. To enable this, we propose the following configuration option: RelayAllowExtend 0|1 If set, allow clients to extend circuits from this relay. Otherwise, refuse all extend cells. PublishServerDescriptor must also be disabled if this option is disabled. If ExitRelay is also disabled, this relay will not pass through any traffic. 5.2. Publishing descriptors A single onion service must publish descriptors in the same manner as any onion service, as defined by rend-spec and section 3 of this proposal. Optionally, a set of introduction points may be included in the descriptor to provide backwards compatibility with clients that don't support single onion services, or to provide a fallback when the extend locations fail. 5.3. RELAY_BEGIN When a RELAY_BEGIN cell is received with a configured single onion hostname as the destination, the stream should be connected to the configured backend server in the same manner as a service-side rendezvous stream. All relays must reject any RELAY_BEGIN cell with an address ending in ".onion" that does not match a locally configured single onion service. 6. Other considerations 6.1. Load balancing High capacity services can distribute load by including multiple entries in the "service-extend-locations" section of the descriptor, or by publishing several descriptors to different onion service directories, or by a combination of these methods. 6.2. Benefits of also running a Tor relay If a single onion service also acts as a published tor relay, it will keep connections to many other tor relays. This can significantly reduce the latency of connections to the single onion service, and also helps the tor network. 6.3. Proposal 224 ("Next-Generation Hidden Services") This proposal is compatible with proposal 224, with small changes to the service descriptor format. In particular: The "service-extend-location" sections are included in the encrypted portion of the descriptor, adjacent to any "introduction-point" sections. The "service-extend-locations" field is no longer present. An onion service is also single onion service if any "service-extend-location" field is present. 6.4. Proposal 246 ("Merging Hidden Service Directories and Intro Points") This proposal is compatible with proposal 246. The onion service will publish its descriptor to the introduction points in the same manner as any other onion service. The client may choose to build a circuit to the specified relays, or to continue with the rendezvous protocol. The client should not extend from the introduction point to the single onion service's relay, to avoid overloading the introduction point. The client may truncate the circuit and extend through a new relay. 7. Discussion 7.1. Authorization Client authorization for a single onion service is possible through encryption of the service-extend-locations section in the descriptor, or "stealth" publication under a new onion address, as with traditional onion services. One problem with this is that if you suspect a relay is also serving a single onion service, you can connect to it and send RELAY_BEGIN without any further authorization. To prevent this, we would need to include a cookie from the descriptor in the RELAY_BEGIN information. 7.2. Preventing relays from being unintentionally published Many single onion servers will not want to relay other traffic, and will set 'PublishServerDescriptor 0' to prevent it. Even when they do, they will still generate a relay descriptor, which could be downloaded and published to a directory authority without the relay's consent. To prevent this, we should insert a field in the relay descriptor when PublishServerDescriptor is disabled that instructs relays to never include it as part of a consensus. [XXX: Also see task #16564] 7.3. Ephemeral single onion services (ADD_ONION) The ADD_ONION control port command could be extended to support ephemerally configured single onion services. We encourage this, but specifying its behavior is out of the scope of this proposal. 7.4. Onion service taxonomy and nomenclature Onion services in general provide several benefits. First, by requiring a connection via Tor they provide the client the protections of Tor and make it much more difficult to inadvertently bypass those protections than when connecting to a non .onion site. Second, because .onion addresses are self-authenticating, onion services have look-up, routing, and authentication protections not provided by sites with standard domain addresses. These benefits apply to all onion services. Onion services as originally introduced also provide network location hiding of the service itself: because the client only ever connects through the end of a Tor circuit created by the onion service, the IP address of the onion service also remains protected. Applications and services already exist that use existing onion service protocols for the above described general benefits without the need for network location hiding. This Proposal is accordingly motivated by a desire to provide the general benefits, without the complexity and overhead of also protecting the location of the service. Further, as with what had originally been called 'location hidden services', there may be useful and valid applications of this design that are not reflected in our current intent. Just as 'location hidden service' is a misleading name for many current onion service applications, we prefer a name that is descriptive of the system but flexible with respect to applications of it. We also prefer a nomenclature that consistently works for the different types of onion services. It is also important to have short, simple names lest usage efficiencies evolve easier names for us. For example, 'hidden service' has replaced the original 'location hidden service' in Tor Proposals and other writings. For these reasons, we have chosen 'onion services' to refer to both those as set out in this Proposal and those with the client-side and server-side protections of the original---also for referring indiscriminately to any and all onion services. We use 'double-onion service' to refer to services that join two Tor circuits, one from the server and one from the client. We use 'single-onion' when referring to services that use only a client-side Tor circuit. In speech we sometimes use the even briefer, 'two-nion' and 'one-ion' respectively.

2 1

tor-browser git repository
by tordev123＠Safe-mail.net 04 Sep '15

04 Sep '15

Is there some documentation, how the tor-browser git repository is set up? In particular, how are new Firefox releases imported? How can I get a diff of Tor changes to the underlying Firefox release?

3 2