- tor-dev - lists.torproject.org

Proposal 351: Making SOCKS5 authentication extensions extensible
by Nick Mathewson 16 Sep '24

16 Sep '24

(You can see this proposal rendered at https://spec.torproject.org/proposals/351-socks-auth-extensions.html ) ``` Filename: 351-socks-auth-extensions.md Title: Making SOCKS5 authentication extensions extensible Author: Nick Mathewson Created: 9 September 2024 Status: Open ``` ## Introduction Currently, Tor implementations use the SOCKS5 username and password fields to pass parameters for stream isolation. (See the `IsolateSocksAuth` flag in the C tor manual, and the "Stream isolation" … [View More]section ([forthcoming](https://gitlab.torproject.org/tpo/core/torspec/-/merge_request…) in our [socks extensions](../socks-extensions.md) spec.) > Tor implementations also support SOCKS4 and SOCKS4a, > but they are not affected by this proposal. > > The C Tor implementation also supports other proxy types besides SOCKS. > They are not affected by this proposal > because they either have other means to extend their protocols > (as with HTTP headers in HTTP CONNECT) > or no means to pass extension information > (as for DNS proxies, iptables transparent proxies, etc). Until now, the rules for interpreting these fields have been simple: all values are permitted, and streams with unequal values may not share a circuit. But in order to integrate SOCKS connections into Arti's RPC protocol, we additionally want the ability to send RPC "Object IDs"[^ObjectId] in these fields. To do this, we will need some way to tell when we have received an object ID, when we have received an isolation parameter, and to avoid confusing them with one another. > Note that some confusion will necessarily remain possible: > Since current Tor clients are allowed to send any value > as SOCKS username and password, > any value we specify here will be one which a client in principle > _might_ have sent under the old protocol. Additionally, since we are adding complexity to the interpretation of these fields, it's possible we'll want to change this complexity in the future. To do this, we'll want a versioning scheme to premit changes. ## Proposal > If accepted, the following can be incorporated into > our [socks extensions](../socks-extensions.md) spec.) We support a series of extensions in SOCKS5 Username/Passwords. Currently, these extensions can encode a stream isolation parameter (used to indicate that streams may share a circuit) and an RPC object ID (used to associate the stream with an entity in an RPC session). These extensions are in use whenever the SOCKS5 Username begins with the 8-byte "magic" sequence `[3c 74 6f 72 53 30 58 3e]`. (This is the ASCII encoding of `<torS0X>`). If the SOCKS5 Username/Password fields are present but the Username does not begin with this byte sequence, it indicates _legacy isolation_. New client implementations SHOULD NOT use legacy isolation. A SocksPort may be configured to reject legacy isolation. When these extensions are in use, the next byte of the username after the "magic" sequence indicate a version number. Any implementation receiving an unrecognized or missing version MUST reject the socks request. When the version number is `[30]` (the ascii encoding of `0`), we interpret the rest of the Username field and the Password field as follows: The remainder of the Username field encodes an RPC Object ID. (If the remainder of the Username field is empty, there is no RPC object.) The Password field is stream isolation parameter. If it is empty, the stream isolation parameter is an empty string. ### Stream isolation > This replaces the corresponding part of > the "Stream isolation" section > ([forthcoming](https://gitlab.torproject.org/tpo/core/torspec/-/merge_request…) > in our [socks extensions](../socks-extensions.md) spec. Two streams are considered to have the same SOCKS authentication values if and only if one of the following is true: - They are both SOCKS4 or SOCKS4a, with the same user "ID" string. - They are both SOCKS5, with no authentication. - They are both SOCKS5 with USERNAME/PASSWORD authentication, using legacy isolation parameters, and they have identical usernames and identical passwords. - They are both SOCKS5 using the extensions above, with the same stream isolation parameter. ### A further extension for integration with Arti SOCKS > We should add the following to a specification, > though it is not clear whether it goes in the Arti RPC spec > or in the socks extensions spec. In some cases, the RPC Object ID may denote an object that already includes information about its intended stream isolation. In such cases, the stream isolation MUST be blank. Implementations MUST reject non-blank stream isolation in such cases. In some cases, the RPC object ID may denote an object that already includes information about its intended destination address and port. In such cases, the destination address MUST be `0.0.0.0` or `::` (encoded either as an IPv4 address, an IPv6 address, or a hostname) and the destination port MUST be 0. Implementations MUST reject other addresses in such cases. ----- > (Here the specifications end. > The rest of this proposal is discussion.) ## Design considerations Our use of SOCKS5 Username/Passwords here (as opposed to some other, new authentication type) is based on the observation that many existing SOCKS5 implementations support Username/Password, but comparatively few support arbitrary plug-in authentication. The magic "`<torS0X>`" prefix is chosen to be 8 characters long so that existing client implementations that generate random strings will not often generate it by mistake. The version number is chosen to be an ASCII `0` rather than a raw 0 byte, for compatibility with existing SOCKS5 client implementations that do not support non-ASCII username/password values. ## C Tor migration When this proposal is accepted, we *should* configure C tor to implement it as follows: - To reject any SOCKS5 Username starting with `<torS0X>` unless it is exactly `<torS0X>0`. This behavior is sufficient to give correct isolation behavior, to reject any connection including an RPC object ID, and to reject any as-yet-unspecified isolation mechanisms. [^ObjectId]: An ObjectId is used in the Arti RPC protocol to associate a SOCKS request with some existing Client object, or with a preexisting DataStream. [View Less]

6 10

changes in user-agent spoofing in the Tor Browser 14.0 series
by Morgan 09 Sep '24

09 Sep '24

tldr; We're planning on disabling HTTP-header user agent spoofing on the basis that it (in our analysis) does little if any good while causing breakage on the internet when the JS and HTTP user agents do not match. However, if there is something critical we're missing here, we'd certainly love to hear. In the meantime it seems like an easy win for some improved usability. best, -morgan Excerpt from 14.0a4's blog post follows: > User Agent Spoofing Changes > > Historically, … [View More]Tor Browser has spoofed the browser user agent found in HTTP headers, while not spoofing the user agent returned by the Navigator.userAgent property in JavaScript. The logic behind the HTTP header spoofing was to prevent passive tracking of users' operating system by websites (when using the 'Safest' security level) and by malicious exit nodes (or their upstream routers) passively listening in on unencrypted HTTP traffic. We left the JavaScript query intact for the purposes of website compatibility and usability. We also left it enabled because there are already many ways of detecting a user's real operating system when JavaScript is enabled (e.g. via font enumeration). > > With Tor Browser 14.0a4, we have introduced the boolean preference privacy.resistFingerprinting.spoofOsInUserAgentHeader. When this pref is set to true (which is currently the default), Tor Browser will follow the previously described legacy behaviour. However, if you set this preference (accessible in about:config) to false, Tor Browser will never spoof the user agent and will report your operating system family (i.e. Windows, macOS, Linux, or Android) when requested. We are considering changing Tor Browser to make this the new default behaviour. > > So, why are we considering making this change? Basically, asymmetrically spoofing the user agent causes website breakage seemingly due to bot-detection scripts. And (in our analysis) it also provides only a negligible amount of benefit to the user in terms of additional linkability (i.e. cross-site tracking, fingerprinting) protections, and only then when JavaScript is disabled. Tor Browser's default HTTPS-Only mode (and much of the web having moved to HTTPS) has also significantly reduced the utility of passively sniffing HTTP traffic for user agents as well. > > We would be very curious to hear from users and domain experts as to whether user agent spoofing is providing any other privacy benefits. In the meantime, disabling spoofing is available to users on an opt-in basis. For more information and to join the conversation, please see the Gitlab ticket tor-browser#42467. [View Less]

1 0

Proposal for a New Web App to Configure Tor Nodes
by Alessandro Greco 30 Aug '24

30 Aug '24

Hello everyone, I have been thinking about creating a web app that generates a script to configure a Tor node based on the settings defined by the user. Let me explain a bit further. This web app could work either entirely on the client side using JavaScript or on the server side. I believe a client-side-only approach is preferable because it avoids handling user data in any way and reduces the risk of man-in-the-middle attacks, although it doesn't completely eliminate it. The main use case … [View More]

5 6

terminology explaination: identifier.
by eff_03675549＠posteo.se 27 Aug '24

27 Aug '24

Hi, On the following URL : http://hctxrvjzfpvmzh2jllqhgvvkoepxb4kfzdjm6h7egcwlumggtktiftid.onion/stats… the quote: *All per-graph statistics files are available for download via an URL of the form:* *https://metrics.torproject.org/identifier.csv* Remains ambiguous to me... *object of this email: *What is an /identifier/ ? Do the author imply the Nickname / Fingerprint ?... Do they imply the |KP_relayid_ed| that requires specific relay permission when the mere fingerprint is safer ?.… [View More]

2 1

Cancel
by tj 21 Aug '24

21 Aug '24

Sent from my iPhone

2 1

txtorcon 24.8.0
by meejah 20 Aug '24

20 Aug '24

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I'm gruntled to announce txtorcon 24.8.0 with the following changes: * Fix (test) issues with Twisted 24.7.0 (https://github.com/meejah/txtorcon/pull/400) * Remove usage of "six" (https://github.com/meejah/txtorcon/issues/395) from https://github.com/a-detiste txtorcon is an implementation of the "control-spec" for Tor using the "Twisted" networking library. This is the library to use if you want to write event-based software (including … [View More]

1 0

Tor community papers at Usenix Security
by Roger Dingledine 26 Jul '24

26 Jul '24

Here are seven papers that I think our community will find interesting from the upcoming Usenix Security conference, August 14-16. Many of them are written by our very own university relay operators, too. :) --Roger "Snowflake, a censorship circumvention system using temporary WebRTC proxies" Cecylia Bocovich, Tor Project; Arlo Breault, Wikimedia Foundation; David Fifield and Serene, unaffiliated; Xiaokang Wang, Tor Project https://www.usenix.org/conference/usenixsecurity24/presentation/… [View More]

1 0

Fwd: Introducing & Discussing "Reflec-Tor"s as concept | Exit-Relay as Entry-Relay | Tor & Echo | Adding Entry-Relays as Reflec-Tor to Exit-Nodes
by Sam 16 Jul '24

16 Jul '24

System wrote via Tor Project <noreply(a)forum.torproject.org>:Feedback: Please submit the proposal also to tor-dev: tor-dev Info PageIntroducing & Discussing "Reflec-Tor"s as concept | Exit-Relay as Entry-Relay | Tor & Echo | Adding Entry-Relays as Reflec-Tor to Exit-Nodes https://www.reddit.com/r/TOR/comments/1e4te8m/introducing_discussing_reflec… ==============================================================Introducing & Discussing "Reflec-Tor"s as concept | Exit-Relay as … [View More]Entry-Relay | Tor & Echo | Adding Entry-Relays as Reflec-Tor to Exit-Nodes *Tor-Messaging: Introducing & Discussing "Reflec-Tor"s as concept | Exit-Relay as Entry-Relay |* Hello, I think this belongs to this core, general, relay topic-forum, as it is also a development & community issue, request and efford, I post it here into the reddit forum for your core discussion: The idea is to add next to Bridges, Relays and Exit-Nodes also "Entry-Nodes" as "Reflec-Tor"(s) to the point of Exit-Nodes. Hence: Exit-Nodes are developed futher to be also an Entry-Node. Some may remember when gnutella got hybrid with edonkey and then also with torrent, Mike Stokes from Shareaza did that. The idea today, 20 years later, is to add some Echo-capabilities to Tor in regard of the servers for Exit and Entry. *Vision: Every (updated) Exit-Node is an Echo-Server - For a better Tor-Messaging.* What does that mean? An Echo-Server is a server for chat-messaging to send an incomming message packet again out to all connected clients at the moment. Ping-in and Ping-Out to all. That simple, that's why it is called the Echo. Like a shout in front of a forrest, all connected users can hear and get the (encrypted) shout or message or packet back from the tree wall. There are 3 software applications for Echo-Servers: - https://github.com/textbrowser/spot-on (Desktop, sercer in the Listener Tab) - https://github.com/textbrowser/smokestack (java for Android) - https://github.com/textbrowser/spot-on-lite (headless deamon for Linux) Now, the Listener function with ping in and ping out should be implemented within a Tor-Exit-Node. When it comes to Tor-Messaging, there are some pathes possible: A) Tor-Chat-Client-Alice - Tor - Internet - Echo-Server - Internet - Tor - Tor-Chat-Client-Bob (Tor-proxied Chat-Server, which only accepts encrypted packets) B) Tor-Chat-Client-Alice - Echo - Tor - Tor - Echo - Tor-Chat-Client-Bob (Echo Tor-tunneled) C) Tor-Chat-Client-Alice - Echo-Server - Tor-Chat-Client-Bob (direct connection to Echo-Server with only encypted packets) The request here is to build and develop option A. That is just right now also possible, by an Exit-Node of Tor running the Echo-Server-Software on the same machine in parallel. Just the port is different. This is an idea for some might be new, but thinking Tor Messaging a bit, it comes quickly to this ideas and better soluttion. The way to connect D) Tor-Chat-Client-Alice - Hidden Onion Server v3 - Tor - Tor - Hidden Onion Server v3 - Tor-Chat-Client-Bob is the current given way for clientes like RicoChet Refresh, Quiet or Cwtch. Similar to Briar, even developers of such clients above tell the loss of messages and low reliability of the hidden to hidden path. Some of you might know, that there were use cases with missing messages in a range of 35-45 %. Don't quote me on that, but as core developers and community members you might be in contact with those who experience this. Furthermore the Messaging clients are not advanced in functionality, nor advanced in strong encryption. It would be third a long development way to got that route. It is cost effective and needs cappable developers. Some project have stamped on and made a workable client, but does that unite all our power in the sense of Tor-Messaging? Messaging needs a Vision and Statement from the Tor-Core-Developer team with a discussion in the community in that regard with honor to the individual projects and also with support for their chosen path (Model D). At the same time we have to state that it is as it is, a HTTP-Server in the middle like in Model A is faster than Model D. In the graph-path the Echo-Server in the middle handles only encrypted traffic, so it is just like another Relay. We can call it "Reflec-Tor". The only sense it to multiply incomming encrypted packets from one node to all connected nodes. With that Idea, the Messenger Spot-On could be used as a Tor-Messenger. This Messenger has stong encryption and is full of feature for messaging and also cryptography. it is like adding Firefox to Tor-Browsing, when Spot-On is added to Tor-Messaging. Something to read at the community forum: https://www.reddit.com/r/Spot_On_Encryption/ Also there is a Mobile Client for Android, which also connects to Reflec-Tors, find "Smoke" Messenger at F-Droid.org. Please, get this right, it is not about a technical view on slow and failing chat-packets to hidden servers, and it is not about those start-up clients using this inside technology, some do a good project. It is about the idea, that an Reflec-Tor mirroring and pinging back packets on the Exit-Node this hop within the path of tor is not outside, it is always fully encryted for the messaging packets and should be seen as one Tor-Path, especially if the Listener-Ping-Back function (the Echo capabilities) is build in the Exit-Node-Tor Software. Spot-On is already a Tor-Messenger - as it uses also HTTPs and it sends only encrypted packets. *A test is easy to make:* (1) Start the Tor-Browser, which has always the Port 9051 to Tor, Tor is running. Next to Surfing with Tor-Browser Firefox the Chat with Tor-Messenger Spot-On can start. (2) Start Spot-On on a webserver and create in the Listeners Tab a Listener on Port 4710. (3) Start two Spot-On Instances Alice and Bob on two Laptops, in the Neighbors Tab you connect the Webserver via Tor: Add the IP and Port 4710 to the neighbor and choose Proxy: 127.0.0.1 Port 4710. You get the the Path: Tor-Chat-Client-Alice - Tor - Internet - Echo-Server - Internet - Tor - Tor-Chat-Client-Bob The idea is now to integrate this a bit more: Tor-Chat-Client-Spot-On-Alice - Tor - [Tor-Exit-Node also Reflec-Tor (Echo-Server)] - Tor - Tor-Chat-Client-Spot-On-Bob You see, the way stays all within the Tor-Family. For sure, in case Alice does not want to use Tor, she also can message to Bob, who is behind Tor. Tor-Chat-Client-Spot-On-Alice --> [Tor-Exit-Node as Entry-Node also Reflec-Tor (Echo-Server)] - Tor - Tor-Chat-Client-Spot-On-Bob The IP of an Entry-Node is shown in the Tor-Browser and can get a port added. Then two user can simply cata over that node. We need in times of surveillance, data retention, chat control and for sake of the needs of whistleblowser and people who want to chat privat and anonymously more decentral and open source chat server. *The mission is: Every Tor-Exit Node is an Entry-Node for Chat.* It should be not a lot of code to be added to the ports of an Exit-Node and displaying the Port of the Exit-Node in the Tor-Browser path icon. This makes sense in several effects to be discussed and developed further: - Taking the next Development Step for Tor-Messaging: BTW, A Forum about Tor-Messaging could be made as a category here in the forum please. - directly support Tor-based Messaging for the Spot-On Messaging client To be developed and discussed is, if this infra-structure could help to - support bootstrapping of Tor - support Censorship circumvention of Tor Reflec-Tors as SnowFlakes over the Messenger with EPKS - Accept, that some routing to an HTTPS internet server at the Tor-Node is faster than to an hidden onion server at the Tor Node. Well, to add some "ping-in-and-out" for an packet is what every netcat and socat under linux can do. It is a small development step to make each Tor-Exit node a free chat server for messaging, which is a big step for mankind to have a network of free messaging chat servers. Lets also see, how users and community will test and develop this messaging. So it is not only a discussion for developers, it is also a step forward the needs of the communities *for a free internet:* - *for chat and their discussions.* A few code lines to exit nodes make them a Reflec-Tor and messaging over Tor can start really decentral and open source and free. *What do you think?* does this privacy-concept bring more privacy and reliability in packet delivery to messaging with Tor? Regards [View Less]

1 0

Some Gosling Updates!
by Richard Pospesel 06 Jul '24

06 Jul '24

Hi Everyone! It's been a minute since the last time I posted to this list about Gosling (December, 2021 what the actual heck: https://lists.torproject.org/pipermail/tor-dev/2021-December/014684.html ). Well, since then I have been quite busy leading the applications team and shipping browsers by day, and developing Ricochet-Refresh and Gosling by night! To summarise, gosling is a Rust crate (and cgosling is a C-FFI library) which generalises and encapsulates all the hard parts of … [View More]building peer-to-peer applications using tor, where a peer's long-term identity is an onion-service id. My vision and hope is that this allows for the creation new peer-to-peer applications with all of the various properties and guarantees of tor baked in, and that developers won't need to go spend some years becoming tor experts to do it correctly and safely. The next version of Ricochet-Refresh will be using Gosling as its foundation, and so of course heavily informed Gosling's features and development direction. The specific properties/features offered by Gosling: - long-term peer identity: an onion-service id - anonymous: it's onion-services and tor - secure: end-to-end encrypted from tor-client to tor-client because tor - nat-punching: more tor - censorship-circumvention: tor + pluggable-transports and bridges Ok and the novel/harder stuff: - authentication: cryptographically prove connecting clients control the private key of their claimed onion-service identity and their provided client-auth keys - customisable/application-specific handshakes: on top of the baseline cryptographic authentication, the 'add me as a peer' handshake also allows for an additional challenge+response requirement; imagine for instance a one-time code verification, password verification, socialist millionaire scheme, proof-of-work or stake, etc - metadata-resistant: the only metadata available to untrusted clients is your primary identity's onion-service's online/offline status; this onion-service is not required to be online to connect with trusted clients (unlike with other Ricochet-derived IM clients) If any of this sounds exciting or useful to you, I'd love to invite you over to the project Github to take a look: - https://github.com/blueprint-freespeech/gosling I'm hoping for Gosling to hit version 1.0.0 and lock-down/converge on a plausible stable API before the end of this year. The current API surface is very informed by my experience with what would be useful for Ricochet-Refresh, but I'm sure there are blind-spots here or improvements that could be made for other potential use-cases. The goal from day one has been to make this code usable outside the Rust ecosystem. To that end, I've tried to make the C-FFI API design consistent, familiar and safe as possible, since it eventually will be used from Ricochet-Refresh's C++ codebase. --- The vast majority of the protocol work and initial implementation was completed around the beginning of 2023. So what have I been up to for the past couple years? Basically, all of the 'boring' (quite exciting actually) software engineering work to get Gosling both usable and trustworthy. To be specific, I've been working on the following areas: - code-cleanup: refactored Gosling into separate crates which may be individually useful: - honk-rpc: the bson-based, json-rpc inspired RPC protocol - tor-interface: probably should have a better name, but this encapsulates the tor-daemon configuration and lifecycle, control-port communications, and connectivity - gosling: the Rust protocol implementation - cgosling: gosling wrapped in a safe+friendly C-FFI - improved CMake project layout; cgosling should be relatively usable/integratable into existing CMake projects - build+test automation: build and run tests on github CI/CD Linux, macOS, and Windows runners, publish code-coverage, generated Rust documentation, doxygen C/C++ documentation - fuzzing: fuzz the Rust crates including the C-FFI - identified and fixed a lot of bugs in the entire stack this year :D - language bindings: build a cbindgen -> c-header -> json idl pipeline for the cgosling crate; json idl used to automatically generate: - C/C++ headers, static+shared libraries - python bindings to shared library - Java/JNI shared library and jar - packaging/deployment - published Rust crates to crates.io - deb-source package generation for debian-based Linux - homebrew flask formula generation for macOS - MSYS2 pkgbuild generation for Windows - some minor new features: - connect API to connect to non-gosling onion-services and clearnet targets using tor - use an already-running tor rather than running/configuring own tor process - configure bundled tor with all the familiar stuff: proxy, firewall, pluggable transports, and bridges - initial protoyping for in-proc arti-client backend (currently on version 0.18 so a bit behind) The remaining planned work in this area before hopefully shifting my focus back on Ricochet-Refresh are a documentation pass on all my Rust crates, cleaning up/documenting the example projects, and completing a security audit. The Java and Python bindings could also use some love and automation. -- Anyway, I think that's enough technical non-sense for your inbox. If any of this sounds interesting, come check things out the project website ( https://gosling.technology/ ) which links to all the relevant stuff. best, -richard [View Less]

1 0

Proposal 350: A phased plan to remove TAP onion keys
by Nick Mathewson 31 May '24

31 May '24

``` Filename: 350-remove-tap.md Title: A phased plan to remove TAP onion keys Author: Nick Mathewson Created: 31 May 2024 Status: Open ``` ## Introduction Back in [proposal 216], we introduced the `ntor` circuit extension handshake. It replaced the older `TAP` handshake, which was badly designed, and dependent on insecure key lengths (RSA1024, DH1024). With the [final shutdown of v2 onion services][hsv2-deprecation], there are no longer any supported users of TAP anywhere in the Tor … [View More]protocols. Anecdotally, a relay operator reports that fewer than one handshake in 300,000 is currently TAP. (Such handshakes are presumably coming from long-obsolete clients.) Nonetheless, we continue to bear burdens from TAP support. For example: - TAP keys compose a significant (but useless!) portion of directory traffic. - The TAP handshake requires cryptographic primitives used nowhere else in the Tor protocols. Now that we are implementing [relays in Arti], the time is ripe to remove TAP. (The only alternative is to add a useless TAP implementation in Arti, which would be a waste of time.) This document outlines a plan to completely remove the TAP handshake, and its associated keys, from the Tor protocols. This is, in some respects, a modernized version of [proposal 245]. ## Migration plan Here is the plan in summary; we'll discuss each phase below. - Phase 1: Remove TAP dependencies - Item 1: Stop accepting TAP circuit requests. - Item 2: Make TAP keys optional in directory documents. - Item 3: Publish dummy TAP keys. - Phase 2: After everybody has updated - Item 1: Allow TAP-free routerdescs at the authorities - Item 2: Generate TAP-free microdescriptors - Phase 3: Stop publishing dummy TAP keys. Phase 1 can begin immediately. Phase 2 can begin once all supported clients and relays have upgraded to run versions with the changes made in Phase 1. Phase 3 can begin once all authorities have made the changes described in phase 2. ### Phase 1, Item 1: Stop accepting TAP circuit requests. (All items in phase 1 can happen in parallel.) Immediately, Tor relays should stop accepting TAP requests. This includes all CREATE cells (not CREATE2), and any CREATE2 cell whose type is TAP (0x0000). When receiving such a request, relays should respond with DESTROY. Relays MAY just drop the request entirely, however, if they find that they are getting too many requests. Such relays must stop reporting `Relay=1` among their supported protocol versions. (This protocol version is not currently required or recommended.) > If this proposal is accepted, > we should clarify the protocol version specification > to say that `Relay=1` specifically denotes TAP. ### Phase 1, Item 2: Make TAP keys optional in directory documents. (All items in phase 1 can happen in parallel.) In C tor and Arti, we should make the `onion-key` entry and the `onion-key-crosscert` entry optional. (If either is present, the other still must be present.) When we do this, we should also modify the authority code to reject any descriptors that do not have these fields. (This will be needed so that existing Tor instances do not break.) In the microdescriptor documents format, we should make the _object_ of the `onion-key` element optional. (We do not want to make the element itself optional, since it is used to delimit microdescriptors.) We use new protocol version flags (Desc=X, Routerdesc=Y) to note the ability to parse these documents. ### Phase 1, Item 3: Publish dummy TAP keys (All items in phase 1 can happen in parallel.) Even after step 2 is done, many clients and relays on the network will still require TAP keys to be present in directory documents. Therefore, we can't remove those keys right away. Relays, therefore, must put _some_ kind of RSA key into their `onion-key` field. I'll present three designs on what relays should do. We should pick one. > #### Option 1 (conservative) > > Maybe, we should say that a relay > should generate a TAP key, generate an onion-key-crosscert, > and then discard the private component of the key. > #### Option 2 (low-effort) > > In C tor, we can have relays simply proceed as they do now, > maintaining TAP private keys and generating crosscerts. > > This has little real risk beyond what is described in Option 1. > #### Option 3 (nutty) > > We _could_ generate a global, shared RSA1024 private key, > to be used only for generating onion-key-crosscerts > and placing into the onion-key field of a descriptor. > > We would say that relays publishing this key MUST NOT > actually handle any TAP requests. > > The advantage of this approach over Option 1 > would be that we'd see gains in our directory traffic > immediately, since all identical onion keys would be > highly compressible. > > The downside here is that any client TAP requests > sent to such a relay would be decryptable by anybody, > which would expose long-obsolete clients to MITM attacks > by hostile guards. We would control the presence of these dummy TAP keys using a consensus parameter: `publish-dummy-tap-key` — If set to 1, relays should include a dummy TAP key in their routerdescs. If set to 0, relays should omit the TAP key and corresponding crosscert. (Min: 0, Max, 1, Default: 0.) We would want to ensure that all authorities voted for this parameter as "1" before enabling support for it at the relay level. ### Phase 2, Item 1: Allow TAP-free routerdescs at the authorities Once all clients and relays have updated to a version where the `onion-key` router descriptor element is optional (see phase 1, item 2), we can remove the authority code that requires all descriptors to have TAP keys. ### Phase 2, Item 2: Generate TAP-free microdescriptors Once all clients and descriptors have updated to a version where the `onion-key` body is optional in microdescriptors (see phase 1, item 2), we can add a new consensus method in which authorities omit the body when generating microdescriptors. ### Phase 3: Stop publishing dummy TAP keys. Once all authorities have stopped requiring the `onion-key` element in router descriptors (see phase 2, item 1), we can disable the `publish-dummy-tap-key` consensus parameter, so that relays will no longer include TAP keys in their router descriptors. [proposal 216]: ./216-ntor-handshake.txt [proposal 245]: ./245-tap-out.txt [hsv2-deprecation]: https://support.torproject.org/onionservices/v2-deprecation/ [relays in Arti]: https://gitlab.torproject.org/tpo/team/-/wikis/Sponsor%20141 [View Less]

2 1

Key Blinding Secrets
by Bellebaum, Thomas 04 May '24

04 May '24

Hello everyone, I am a researcher currently looking into different schemes for what you call Keyblinding in the rendevouz spec. https://spec.torproject.org/rend-spec/keyblinding-scheme.html I noticed that your description there mentiones a secret `s` to be hashed into the blinding factor, and have a few questions about it: 1. Is this secret currently being used / intended to be used? If so, how? 2. What kinds of security (formally or informally) would you expect from using a secret in the … [View More]

2 1

Proposal for Tor (Relay) Watchdog (Telegram) Bot
by Alessandro Greco 29 Apr '24

29 Apr '24

Dear Tor Project Developers, I hope this email finds you well. I am writing to share with you a project [1] I have been working on called Tor Watchdog Bot [2], and I believe it may be of interest to you. Tor Watchdog Bot is a Telegram bot designed to monitor the status of Tor relays and notify users when relays go offline. The intent behind creating this bot was to develop a user-friendly tool that allows anyone to easily keep track of their Tor nodes. I am fully aware that the bot is not … [View More]

2 1

Proposal #349: Command state validation (for dropmark attacks)
by Mike Perry 07 Mar '24

07 Mar '24

We've merged a draft of Proposal #349 to torspec: https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/349-co… Also available on the tor spec site at: https://spec.torproject.org/proposals/349-command-state-validation.html This proposal is meant to deal with the third class of highly severe protocol side channels in tor: Dropped Cells. See Prop #344 for background: https://spec.torproject.org/proposals/344-protocol-info-leaks.html Note that there still are some details that … [View More]

1 0

Fact-checking a claim about relay/bridge fingerprint authentication
by David Fifield 25 Feb '24

25 Feb '24

Cecylia, Arlo, Serene, Shelikhoo, and I are writing a research paper about Snowflake. Here is a draft: https://www.bamsoftware.com/papers/snowflake/snowflake.20231003.e6e1c30d.pdf We're writing to check a factual claim in the section about having multiple backend bridges. Basically, we wanted it to be possible for there to be multiple Snowflake bridge sites run by different groups of people, and we did not want to share the same relay identity keys across all bridge sites, because of the … [View More]increased risk of the keys being exposed. Therefore every bridge site has its own relay identity, which requires the client to know the relay fingerprints in advance and that it be the client (and not, e.g., the broker) that decides which bridge to use. 1. Is our general description (quoted below) of the design constraints as they bear on Tor correct? 2. Is §4.2 "CERTS cells" the right part of tor-spec to cite to make our point? https://gitlab.torproject.org/tpo/core/torspec/-/blob/b345ca044131b2eb18e6a… https://github.com/turfed/snowflake-paper/blob/e6e1c30dde6716dc5e54a32f2134… A Tor bridge is identified by a long-term identity public key. If, on connecting to a bridge, the client finds that the bridge's identity is not the expected one, the client will terminate the connection \cite[\S 4.2]{tor-spec}. The Tor client can configure at most one identity per bridge; there is no way to indicate (with a certificate, for example) that multiple identities should be considered equivalent. This constraint leaves two options: either all Snowflake bridges must share the same cryptographic identity, or else it must be the client that makes the choice of what bridge to use. While the former option is possible to do (by synchronizing identity keys across servers), every added bridge would increase the risk of compromising the all-important identity keys. Our vision was that different bridge sites would run in different locations with their own management teams, and that any compromise of a bridge site should affect that site only. In my own experiments, providing an incorrect relay fingerprint leads to errors in connection_or_client_learned_peer_id: https://gitlab.torproject.org/tpo/core/tor/-/blob/tor-0.4.7.13/src/core/or/… [warn] Tried connecting to router at 192.0.2.3:80 ID=<none> RSA_ID=2B280B23E1107BB62ABFC40DDCC8824814F80A71, but RSA + ed25519 identity keys were not as expected: wanted 1111111111111111111111111111111111111111 + no ed25519 key but got 2B280B23E1107BB62ABFC40DDCC8824814F80A72 + 1zOHpg+FxqQfi/6jDLtCpHHqBTH8gjYmCKXkus1D5Ko. [warn] Problem bootstrapping. Stuck at 14% (handshake): Handshaking with a relay. (Unexpected identity in router certificate; IDENTITY; count 1; recommendation warn; host 1111111111111111111111111111111111111111 at 192.0.2.3:80) [View Less]

2 3

Re: [tor-dev] UDP Application Support in Tor
by Vilgot Bergquist 13 Feb '24

13 Feb '24

Hi, I looked at the suggested solutions and I think there is another approach, which is much easier. I C it's pretty easy to encapsulate UDP segments inside TCP segments. Hence there is no need to re-organize the connection logic of tor relays. Instead it should be possible to make Guards, when receiving an UDP packet, to just add a TCP header and then it goes through the normal process. The exit nodes than removed the TCP header and pass the UDP segment on. Regards Vilgot tor-dev-… [View More]request(a)lists.torproject.org <tor-dev-request(a)lists.torproject.org> schrieb am Donnerstag, 25. Januar 2024 um 18:49: > > > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. New Proposal - UDP Application Support in Tor > (Micah Elizabeth Scott) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 25 Jan 2024 09:49:02 -0800 > From: Micah Elizabeth Scott beth(a)torproject.org > > To: tor-dev(a)lists.torproject.org > Subject: [tor-dev] New Proposal - UDP Application Support in Tor > Message-ID: 79aa2de8-1ff9-4579-b87f-ee9792e9cbb0(a)torproject.org > > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > Hello tor-dev folks! > > Late last year I started taking a close look at what it would take to > support applications on Tor which rely on UDP networking. This was > originally to be based on Nick's proposal, 339-udp-over-tor. > > The scope of this work so far has been specifically focused on end-user > application compatibility, and excludes fundamental changes to Tor's > network structure or protocols for now. > > This combination of approach and scope left me with more questions than > answers, so I started looking deeper into the available solutions along > with their expected benefits and risks. This proposal is the result of > that investigation. > > Please find the text attached, or in the torspec repo as proposal #348: > > https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/348-ud… > > Unlike a typical proposal, this does not recommend any specific change > to the Tor implementation. Several possible changes are presented, but > ultimately the recommended approach is to use application-specific UDP > relays to achieve compatibility. > > Integrated approaches are also presented, where Tor does involve itself > in the transit of individual datagrams. These approaches offer > advantages only when they are also part of a long-term plan to offer > transport features beyond those offered by TCP. Until such a plan is > in-scope, specific UDP extensions cannot be offered with confidence. > > I would appreciate any feedback on this proposal, whether it's about > this particular shorter-term context or about longer-term plans to > achieve some kind of optional unreliable transport. > > Thanks for your time! > > --beth > [View Less]

3 2

Announcing Onionspray 1.6.0 with a SECURITY fix for Onion Services rewriting proxies
by rhatto 09 Feb '24

09 Feb '24

Hello everyone, I'd like to announce Onionspray, a tool for setting up Onion Services for existing public websites, working as a HTTPS rewriting proxy: https://tpo.pages.torproject.net/onion-services/onionspray/ It's a fork of Alec Muffett's EOTK (https://github.com/alecmuffett/eotk) with many enhancements but retaining compatibility, and relying on C Tor until an alternative in Arti is available. The first Onionspray version is 1.6.0, following the pre-existing version sequence from EOTK. … [View More]Security fixes: * This release fixes a CRITICAL security vulnerability related to upstream HTTPS certificate verification, which is detailed at https://tpo.pages.torproject.net/onion-services/onionspray/security/advisor… A related fix is also available for EOTK: https://github.com/alecmuffett/eotk/pull/116 We urge Onionspray users that were testing the software while it was being on it's early stages to upgrade ASAP to 1.6.0 and update their configurations, and we recommend that EOTK to the same with the corresponding patch. This issue might also affect other similar rewriting proxy setups, and we urge operators to review and fix their Onion Service configurations. Main improvements over EOTK: * MetricsPort support (for gathering metrics data from the tor instances). * Denial of Service (DoS) protections. * Circuit ID exporting to NGINX logs and optionally to the upstream proxy (through the X-Onion-CircuitID HTTP header). * Onionbalance v3 support ("softmaps" are working again). * Revamped documentation. * Installation procedures added for recent Debian and Ubuntu releases. * Tor and OpenResty upgraded to the latest versions. * Option to keep Onionspray running in the foreground (`--no-daemonize`). * Local healthcheck action (`--health-local`), useful for containerized execution. The full ChangeLog is available at https://tpo.pages.torproject.net/onion-services/onionspray/changelog/ For those wishing to switch from EOTK to Onionspray, there's a migration guide at https://tpo.pages.torproject.net/onion-services/onionspray/migrating/ We also welcome people to report issues, send merge requests etc: https://tpo.pages.torproject.net/onion-services/onionspray/contact/ And we have a bunch of issues waiting for contributions: https://gitlab.torproject.org/tpo/onion-services/onionspray/-/issues Finally, I'd like to thank Alec Muffett for his important work with EOTK and for promoting Onion Services all these years :) Thanks! -- Silvio Rhatto pronouns he/him [View Less]

1 0

docker bridge suddenly fails startup with "Directory /var/lib/tor cannot be read: Permission denied"
by gentle.luck4713＠doublequote.org 01 Feb '24

01 Feb '24

Hi tor-dev@, I have been running a bridge following the instructions at <https://community.torproject.org/relay/setup/bridge/docker/> on a Debian 12.4 system but when I just tried to make sure everything is up to date I got the following error message from docker-compose logs: obfs4-bridge_1 | Using NICKNAME=DockerObfs4Bridge, OR_PORT=3845, PT_PORT=443, and EMAIL=redacted(a)example.com. obfs4-bridge_1 | Additional properties from 'OBFS4V_' environment variables processing enabled … [View More]obfs4-bridge_1 | Overriding 'AddressDisableIPv6' with value '1' obfs4-bridge_1 | Starting tor. obfs4-bridge_1 | Jan 31 22:49:20.501 [notice] Tor 0.4.8.10 running on Linux with Libevent 2.1.12-stable, OpenSSL 3.0.11, Zlib 1.2.13, Liblzma 5.4.1, Libzstd 1.5.4 and Glibc 2.36 as libc. obfs4-bridge_1 | Jan 31 22:49:20.501 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://support.torproject.org/faq/staying-anonymous/ obfs4-bridge_1 | Jan 31 22:49:20.501 [notice] Read configuration file "/etc/tor/torrc". obfs4-bridge_1 | Jan 31 22:49:20.502 [notice] Based on detected system memory, MaxMemInQueues is set to 6264 MB. You can override this by setting MaxMemInQueues by hand. obfs4-bridge_1 | Jan 31 22:49:20.503 [notice] Opening OR listener on 0.0.0.0:3845 obfs4-bridge_1 | Jan 31 22:49:20.503 [notice] Opened OR listener connection (ready) on 0.0.0.0:3845 obfs4-bridge_1 | Jan 31 22:49:20.503 [notice] Opening OR listener on [::]:3845 obfs4-bridge_1 | Jan 31 22:49:20.503 [notice] Opened OR listener connection (ready) on [::]:3845 obfs4-bridge_1 | Jan 31 22:49:20.503 [notice] Opening Extended OR listener on 127.0.0.1:0 obfs4-bridge_1 | Jan 31 22:49:20.503 [notice] Extended OR listener listening on port 44595. obfs4-bridge_1 | Jan 31 22:49:20.503 [notice] Opened Extended OR listener connection (ready) on 127.0.0.1:44595 obfs4-bridge_1 | Jan 31 22:49:20.504 [warn] Directory /var/lib/tor cannot be read: Permission denied obfs4-bridge_1 | Jan 31 22:49:20.504 [notice] Closing partially-constructed OR listener connection (ready) on 0.0.0.0:3845 obfs4-bridge_1 | Jan 31 22:49:20.504 [notice] Closing partially-constructed OR listener connection (ready) on [::]:3845 obfs4-bridge_1 | Jan 31 22:49:20.504 [notice] Closing partially-constructed Extended OR listener connection (ready) on 127.0.0.1:44595 obfs4-bridge_1 | Jan 31 22:49:20.504 [warn] Failed to parse/validate config: Couldn't create private data directory "/var/lib/tor" obfs4-bridge_1 | Jan 31 22:49:20.504 [err] Reading config failed--see warnings above. I confirmed that my docker-compose.yaml is unmodified and my .env is innocuous. I know that I can blow away the docker volumes but that'd remove the private keys etc. and that'd doesn't seem desirable. Any advice? Thanks, Hein [View Less]

1 0

New Proposal - UDP Application Support in Tor
by Micah Elizabeth Scott 25 Jan '24

25 Jan '24

Hello tor-dev folks! Late last year I started taking a close look at what it would take to support applications on Tor which rely on UDP networking. This was originally to be based on Nick's proposal, 339-udp-over-tor. The scope of this work so far has been specifically focused on end-user application compatibility, and excludes fundamental changes to Tor's network structure or protocols for now. This combination of approach and scope left me with more questions than answers, so I … [View More]

1 0

Timers in Arti?
by Michael Rogers 15 Jan '24

15 Jan '24

Hi, If I understand right, the C implementation of Tor has various state machines that are driven by local libevent timers as well as events from the network. For example, when building a circuit, if there hasn't been any progress for a certain amount of time then a timer fires to handle the timeout. When running Tor on Android, we need to prevent the OS from suspending so that these timers fire when they're supposed to. This uses a lot of battery, because normally the OS would spend … [View More]

4 8

txtorcon 23.11.0
by meejah 20 Nov '23

20 Nov '23

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I'm energized to announce txtorcon 23.11.0 with the following changes: * Fix test-failures on Python 3.12 * Particular GETINFO hanging (https://github.com/meejah/txtorcon/issues/389) (ultra-long lines over 16KiB caused problems in the protocol) * Use built-in `mock` only (https://github.com/jelly) * Remove `incremental` (https://github.com/gdrosos) txtorcon is an implementation of the "control-spec" for Tor using the "Twisted" networking … [View More]

1 0

Proposal 347: Domain separation for certificate signing keys
by Nick Mathewson 19 Oct '23

19 Oct '23

To see this rendered, go to https://spec.torproject.org/proposals/347-domain-separation.html ``` Filename: 347-domain-separation.md Title: Domain separation for certificate signing keys Author: Nick Mathewson Created: 19 Oct 2023 Status: Open ``` ## Our goal We'd like to be able to use the "family key" from proposal 321 as a general purpose signing key, to authenticate other things than the membership of a family. For example, we might want to have a challenge/response mechanism where the … [View More]challenger says, "If you want to log in as the owner of the account corresponding to this family, sign the following challenge with your key. Or we might want to have a message authentication scheme where an operator can sign a message in a way that proves key ownership. We _might_ also like to use relay identity keys or onion service identitiy keys for the same purpose. ## The problem When we're using a signing key for two purposes, it's important to perform some kind of domain separation so that documents signed for one purpose can't be mistaken for documents signed for the other. For example, in the challenge/response example, it would be bad if the challenger could provide a challenge string that would cause the signer to inadvertently authenticate an incorrect family. These keys are currently used in some places with no personalization. Their signature format is as described in `cert-spec.txt`, which says: > The signature is created by signing all the fields in > the certificate up until "SIGNATURE" (that is, signing > sizeof(ed25519_cert) - 64 bytes). ## One solution This one is pretty easy: we would extend `cert-spec` as follows. > ### Using signing keys for other purposes. > > Other protocols may introduce other uses for the signing keys > in these certificates other than those specified here. When > they do, they MUST ensure that the documents being signed > cannot be confused with the certificate bodies of this document. > > In some existing cases in the Tor protocols, we achieve this by > specifying an ASCII prefix string that must be prepended to the > other protocol's signed object before it is signed. > > For future protocols, we recommend that this be done by > specifying that the signing key is to be used to sign a > cSHAKE digest (or other secure customizable digest) > of the other protocol's signed object, using > a customization string unique to the other protocol. We would also make this amendment: > ### Future versions of this specification > > In order to maintain the domain separation that currently exists > between the signatures on these certificates and other documents > signed with the same keys, it suffices (for now!) that these > certificates begin with the version byte `[01]`, whereas > the other documents are in printable ASCII, which never > includes `[01]`. > > Future versions of this specification SHOULD move to using > an ed25519-prehash construction, using a customizable hash > with built-in domain separation. [View Less]

2 1

Proposal 346: Clarifying and extending the use of protocol versioning
by Nick Mathewson 19 Oct '23

19 Oct '23

Note: we've updated our specs website! You can read a rendered version of this proposal at https://spec.torproject.org/proposals/346-protovers-again.html . ``` Filename: 346-protovers-again.md Title: Clarifying and extending the use of protocol versioning Author: Nick Mathewson Created: 19 Oct 2023 Status: Open ``` # Introduction In proposal 264, we introduced "subprotocol versions" as a way to independently version different pieces of the Tor protocols, and communicate which parts of the … [View More]Tor protocols are supported, recommended, and required. Here we clarify the semantics of individual subprotocol versions, and describe more ways to use and negotiate them. # Semantics: Protocol versions are feature flags One issue we left unclarified previously is the relationship between two different versions of the same subprotocol. That is, if we know the semantics of (say) `Handshake=7`, can we infer anything about a relay that supports `Handshake=8`? In particular, can we infer that it supports all of the same features implied by `Handshake=7`? If we want to know "does this relay support some feature supported by `Handshake=7`", must we check whether it supports `Handshake=7`, or should we check `Handshake=x for any x≥7`? In this proposal, we settle the question as follows: subprotocol versions are flags. They do not have any necessary semantic relationship between them. We reject the `≥` interpretation for several reasons: * It's tricky to implement. * It prevents us from ever removing a feature. * It requires us to implement features in the same order across all Tor versions. ## ...but sometimes a flag _is_ a version! There _are_ some places in our protocol (notably: directory authority consensus methods, and channel protocol versions) where there _is_ a semantic relationship between version numbers. Specifically: "higher numbers are already better". When parties need to pick a one of _these_ versions, they always pick the highest version number supported by enough of them. When this kind of _real_ version intersects with the "subprotocol versions" system, we use the same numbers: * `Link` subprotocols correspond one-to-one with the version numbers sent in a VERSIONS cell. * `Microdesc` and `Cons` subprotocols correspond to a _subset_ of the version numbers of consensus methods. ## How to document subprotocol versions When describing a subprotocol, we should be clear what relationship, if any, exists between its versions and any versions negotiated elsewhere in the specifications. Unless otherwise documented, all versions can be in use _at the same time_: if only one can exist at once (on a single circuit, a single document, etc), this must be documented. > Implication: This means that we must say (for example) that you > can't use Link=4 and Link=5 on the same channel. # Negotiating protocol versions in circuit handshakes. Here we describe a way for a client to opt into features as part of its circuit handshake, in order to avoid proliferating negotiating extensions. ## Binary-encoding protocol versions. We assign a one-byte encoding for each protocol version number, ordered in the same way as in tor-spec. | Protocol | Id | | --- | --- | | Link | 0 | | LinkAuth | 1 | | Relay | 2 | | DirCache | 3 | | HSDir | 4 | | HSIntro | 5 | | HSRend | 6 | | Desc | 7 | | MicroDesc| 8 | | Cons | 9 | | Padding | 10 | | FlowCtrl | 11 | | Conflux | 12 | | Datagram | 13 | > Note: This is the same encoding used in the [walking onions > proposal][prop323]. It takes its order from the ordering of > protocol versions in > [tor-spec][subprotocol-versioning] > and matches up with the values defined in for `protocol_type_t` in C > tor's `protover.h`. ## Requesting an opt-in circuit feature When a client wants to request a given set of features, it sends an `ntor_v3` extension containing: ``` struct subproto_request { struct req[..]; // up to end of extension } struct req { u8 protocol_id; u8 protovol_version; } ``` > Note 1: The above format does not include any parameters for each > req. Thus, if we're negotiating an extension that requires a client- > supplied parameter, it may not be appropriate to use this > request format. > > Note 2: This proposal does not include any relay extension > acknowledging support. In the case of individual subprotocols, we could > later say "If this subprotocol is in use, the relay MUST also send > extension foo". > > Note 3: The existence of this extension does not preclude the later > addition of other extensions to negotiate featuress differently, or > to do anything else. Each `req` entry corresponds to a single subprotocol version. A client MUST NOT send any `req` entry unless: * That subprotocol version is advertised by the relay, * OR that subprotocol version is listed as required for relays in the current consensus, using `required-relay-protocols`. > Note: We say above that a client may request a _required_ subprotocol > even if the relay does not advertise it. This is what allows > clients to send a `req` extension to introduction points and > rendezvous points, even when we do not recognize the relay from the > consensus. > > Note 2: If the intro/rend point does not support a _required_ protocol, > it should not be on the network, and the client/service should not have > selected it. If a relay receives a `subproto_request` extension for any subprotocol version that it does not support, it MUST reject the circuit with a DESTROY cell. > Alternatives: we _could_ give the relay the option to > decline to support an extension, and we _could_ require the > relay to acknowledge which extensions it is providing. > We aren't doing that, in the name of simplicity. Only certain subprotocol versions need to be negotiated in this way; they will be explicitly listed as such in our specifications, with language like "This extension is negotiated as part of the circuit extension handshake". Other subprotocol versions MUST NOT be listed in this extension; if they are, the relay SHOULD reject the circuit. > Alternative: We _could_ allow the client to list other subprotocols > that the relay supports which are nonetheless irrelevant to > the circuit protocol, like `Microdesc`, or ones that don't currently need > to be negotiated, like `HsRend`. > > This is not something we plan to do. Currently specified subprotocol versions which can be negotiated using this extension are: * FlowCtrl=2 (congestion control) * Packed-and-fragmented support (proposal 340) The availability of the `subproto_request` extension itself will be indicated by a new Relay=X flag. When used, it will supplant several older `ntor_v3` extensions, including: * (TODO: list these here, if we find any. I think FlowCtrl has an extension?) That is, if using `subproto_request`, there is no need to send the (TODO) extensions. ## Making features that can be disabled. Sometimes, we will want the ability to make features that can be enabled or disabled from the consensus. But if we were to make a single flag that can turn the feature on and off, we'd run into trouble: after the feature was turned off, every relay would stop providing it right away, but there would be a delay before clients realized that the relays had stopped advertising the feature. During this interval, clients would try to enable the feature, and the relays would reject their circuits. To solve this problem, we need to make features like these controlled by a _pair_ of consensus parameters: one to disable advertising the feature, and one to disable the feature itself. To disable a feature, first the authorities would tell relays to stop advertising it, and only later tell the relays to stop supporting it. (If we want to _enable_ a previously disabled feature, we can turn on advertisement and support at the same time.) These parameters would be specified something like this (for a hypthetical `Relay=33` feature). * `support-relay-33`: if set to 1, relays that can provide `Relay=33` should do so. * `advertise-relay-33`: if set to 1, relays that are providing `Relay=33` should include it in their advertised protocol versions. Note: as a safety measure, relays MUST NOT advertise any feature that they do not support. This is reflected in the descriptions of the parameters above. When we add a new feature of this kind, we should have the `advertise-*` flag parameter be `1` by default, and probably we should have `support-*` be `1` by default oo. # Subprotocol versions in onion services Here we describe how to expand the onion service protocols in order to better accomodate subprotocol versions. ## Advertising an onion service's subprotocols In its encrypted descriptor (the innermost layer), the onion service adds a new entry: * `"protocols"` - A list of supported subprotocol versions, in the same format as those listed in a microdescriptor or descriptor. Note that this is NOT a complete list of all the subprotocol versions actually supported by the onion service. Instead, onion services only advertise a subprotocol version if they support it, _and_ it is documented in the specs as being supported by onion services. > Alternative: I had considered having a mask that would be put in the > consensus document, telling the onion services which subprotocols > to advertise. I don't think that's a great idea, however. Right now, the following protocols should be advertised: - FlowCtrl - Conflux (?? Doesn't this take parameters? TODO) - Pow (??? Doesn't this take parameters? If we do it, we need to allocate a subprotocol for it. TODO) ## Negotiating subprotocols with an onion service. In the `hs_ntor` handshake sent by the client, we add an encrypted `subproto_request` extension of the same format, with the same semantics, as used in the `ntor-v3` handshake. This supplants the following: - (Congestion Control; Pow? TODO) ## Advertising other relays' subprotocols? > Alternative: I had previously considered a design where the introduction points > in the onion service descriptor would be listed along with their > subprotocols, and the `hs_ntor` handshake would contain the > subprotocols of the rendezvous point. > > I'm rejecting this design for now because it gives the onion service > and the client too much freedom to lie about relays. In the future, > the walking onions design would solve this, since the contact > information for intro and rend points would be authenticated. # Appendix New numbers to reserve: * An extension ID for the `ntor_v3` handshake `subproto_request` extension. * An extension ID for the `hs_ntor` handshake `subproto_request` extension. * A Relay= subprotocol indicating support for the ntor-v3 and hs_ntor extensions. * The numeric encoding of each existing subprotocol, in the table above. [prop323]: https://spec.torproject.org/proposals/323-walking-onions-full.html [subprotocol-versioning]: https://spec.torproject.org/tor-spec/subprotocol-versioning.html [View Less]

1 0

PT bridge reporting significant numbers of <OR> IP addresses after upgrade to 0.4.8.6; Conflux related?
by David Fifield 10 Oct '23

10 Oct '23

The Snowflake bridges do not expose their plain ORPort (they have `ORPort 127.0.0.1:auto` in torrc), and consequently they have always reported ≈0 <OR> IP addresses in the bridge-ip-transports line of bridge-extra-info descriptors, with virtually all connecting IP addresses being instead attributed to the snowflake transport. (The <OR> count is low but nonzero, which I have rationalized as tor making some small number of internal connections, or something.) I upgraded one of the … [View More]two Snowflake bridges from 0.4.7.13 to 0.4.8.6 on 2023-09-24. Since then, the number of <OR> IP addresses has been roughly equal to the number of snowflake IP addresses. The ORPort is still not exposed; these are not external vanilla bridge users. Did something change between these versions that might cause PT connections to be double-counted, once for the transport and once for <OR>? Here are excerpted bridge-extra-info descriptors from before and after the version upgrade. Note the bridge-ip-transports lines. ``` @type bridge-extra-info 1.3 extra-info crusty5 91DA221A149007D0FD9E5515F5786C3DD07E4BB0 master-key-ed25519 1y9CAtinlbrhDuYBSOBNiCU9Ck1lcY7LErxnzhtxVks published 2023-09-19 10:46:08 transport snowflake bridge-ip-versions v4=13528,v6=1384 bridge-ip-transports <OR>=8,snowflake=14912 ``` ``` @type bridge-extra-info 1.3 extra-info crusty5 91DA221A149007D0FD9E5515F5786C3DD07E4BB0 master-key-ed25519 1y9CAtinlbrhDuYBSOBNiCU9Ck1lcY7LErxnzhtxVks published 2023-09-29 17:33:20 transport snowflake bridge-ip-versions v4=2880,v6=336 bridge-ip-transports <OR>=1632,snowflake=1592 ``` Below is a time series for a whole month. The thing to notice is not the overall decrease in users on 2023-09-20; that's due to https://forum.torproject.org/t/problems-with-snowflake-since-2023-09-20-bro…. Rather, the thing to notice is that before 2023-09-24, <OR> ≈ 0; and after, <OR> ≈ snowflake. date <OR> snowflake 2023-08-30 4 45240 ← running 0.4.7.13 here 2023-08-31 21 193412 2023-09-01 30 195647 2023-09-02 40 200405 2023-09-03 40 201730 2023-09-04 42 195097 2023-09-05 35 190842 2023-09-06 30 189694 2023-09-07 26 191524 2023-09-08 35 195833 2023-09-09 32 196244 2023-09-10 27 146972 2023-09-11 6 55491 2023-09-12 18 182519 2023-09-13 17 173536 2023-09-14 21 161396 2023-09-15 21 166379 2023-09-16 16 174350 2023-09-17 17 165063 2023-09-18 23 164734 2023-09-19 32 164075 2023-09-20 23 121623 2023-09-21 2 25268 2023-09-22 0 4314 2023-09-23 0 746 2023-09-24 239 606 ← upgrade to 0.4.8.6 on this day 2023-09-25 1919 1900 2023-09-26 3533 3470 2023-09-27 11919 11687 2023-09-28 20242 19890 2023-09-29 26583 26100 2023-09-30 25929 25400 [View Less]

2 2

Proposal 345: Migrating the tor specifications to mdbook
by Nick Mathewson 03 Oct '23

03 Oct '23

``` Filename: 345-specs-in-mdbook.md Title: Migrating the tor specifications to mdbook Author: Nick Mathewson Created: 2023-10-03 Status: Open ``` # Introduction I'm going to propose that we migrate our specifications to a set of markdown files, specifically using the [`mdbook`] tool. This proposal does _not_ propose a bulk rewrite of our specs; it is meant to be a low-cost step forward that will produce better output, and make it easier to continue working on our specs going forward. That … [View More]said, I think that this change will enable rewrites in the future. I'll explain more below. # What is mdbook? Mdbook is a tool developed by members of the Rust community to create books with [Markdown]. Each chapter is a single markdown file; the files are organized into a book using a [`SUMMARY.md`] file. Have a look at the [`mdbook` documentation][`mdbook`]; this is what the output looks like. Have a look at this [source tree]: that's the input that produces the output above. Markdown is extensible: it can use numerous [plugins] to enhance the semantics of the the markdown input, add diagrams, output in more formats, and so on. # What would using mdbook get us _immediately_? There are a bunch of changes that we could get immediately via even the simplest migration to mdbook. These immediate benefits aren't colossal, but they are things we've wanted for quite a while. * We'll have a document that's easier to navigate (via the sidebars). * We'll finally have good HTML output. * We'll have all our specifications organized into a single "document", able to link to one another and cross reference one another. * We'll have perma-links to sections. * We'll have a built-in text search function. (Go to the [`mdbook`] documentation and hit "`s`" to try it out.) ## How will mdbook help us _later on_ as we reorganize? Many of the benefits of mdbook will come later down the line as we improve our documentation. * Reorganizing will become much easier. * Our links will no longer be based on section number, so we won't have to worry about renumbering when we add new sections. * We'll be able to create redirects from old section filenames to new ones if we need to rename a file completely. * It will be far easier to break up our files into smaller files when we find that we need to reorganize material. * We will be able make our documents even _easier_ to navigate. * As we improve our documentation, we'll be able to use links to cross-reference our sections. * We'll be able to include real diagrams and tables. * We're already using the [`mermaid`] tool in Arti in generate [protocol diagrams] and [architecture diagrams]; we can use this in our specs too, instead of hand-drawn ASCII art. * We'll be able to integrate proposals more easily. * New proposals can become new chapters in our specification simply by copying them into a new 'md' file or files; we won't have to decide between integrating them into existing files or creating a new spec. * Implemented but unmerged proposals can become additional chapters in an appendix to the spec. We can refer to them with permalinks that will still work when they move to another place in the specs. # How should we do this? ## Strategy My priorities here are: * no loss of information, * decent-looking output, * a quick automated conversion process that won't lose a bunch of time. * a process that we can run experimentally until we are satisfied with the results With that in mind, I'm writing a simple set of [`torspec-converter`] scripts to convert our old torspec.git repository into its new format. We can tweak the scripts until we like the that they produce. After running a recent `torspec-converter` on a fairly recent torspec.git, here is how the branch looks: https://gitlab.torproject.org/nickm/torspec/-/tree/spec_conversion?ref_type… And here's the example output when running mdbook on that branch: https://people.torproject.org/~nickm/volatile/mdbook-specs/index.html > Note: these is not a permanent URL; we won't keep the example output > forever. When we actually merge the changes, they will move into > whatever final location we provide. The conversion script isn't perfect. It only recognizes three kinds of content: headings, text, and "other". Content marked "other" is marked with \`\`\` to reneder it verbatim. The choice of which sections to split up and which to keep as a single page is up to us; I made some initial decisions in the file above, but we can change it around as we please. See the [configuration section] at the end of the `grinder.py` script for details on how it's set up. ## Additional work that will be needed Assuming that we make this change, we'll want to build an automated CI process to build it as a website, and update the website whenever there is a commit to the specifications. (This automated CI process might be as simple as `git clone && mdbook build && rsync -avz book/ $TARGET`.) We'll want to go through our other documentation and update links, especially the permalinks in spec.torproject.org. It might be a good idea to use spec.torproject.org as the new location of this book, assuming weasel (who maintains spec.tpo) also thinks it's reasonable. If we do that, we need to decide on what we want the landing page to look like, and we need _very much_ to get our permalink story correct. Right now I'm generating a .htaccess file as part of the conversion. ## Stuff we shouldn't do. I think we should continue to use the existing torspec.git repository for the new material, and just move the old text specs into a new archival location in torspec. (We *could* make a new repository entirely, but I don't think that's the best idea. In either case, we shouldn't change the text specifications after the initial conversion.) We'll want to figure out our practices for keeping links working as we reorganize these documents. Mdbook has decent redirect support, but it's up to us to actually create the redicrets as necessary. # The transition, in detail * Before the transition: - Work on the script until it produces output we like. - Finalize this proposal and determine where we are hosting everything. - Develop the CI process as needed to keep the site up to date. - Get approval and comment from necessary stakeholders. - Write documentation as needed to support the new way of doing things. - Decide on the new layout we want for torspec.git. * Staging the transition: - Make a branch to try out the transition; explicitly allow force-pushing that branch. (Possibly nickm/torspec.git in a branch called mdbook-demo, or torspec.git in a branch called mdbook-demo assuming it is not protected.) - Make a temporary URL to target with the transition (possibly spec-demo.tpo) - Once we want to do the transition, shift the scripts to tpo/torspec.git:main and spec.tpo, possibly? * The transition: - Move existing specs to a new subdirectory in torspec.git. - Run the script to produce an mdbook instance in torspec.git with the right layout. - Install the CI process to keep the site up to date. * Post-transition - Update links elsewhere. - Continue to improve the specs. # Integrating proposals We could make all of our proposals into a separate book, like rust does at https://rust-lang.github.io/rfcs/ . We could also leave them as they are for now. (I don't currently think we should make all proposals part of the spec automatically.) # Timing I think the right time to do this, if we decide to move ahead, is before November. That way we have this issue as something people can work on during the docs hackathon. # Alternatives I've tried experimenting with Docusaurus here, which is even more full-featured and generates pretty react sites [like this](https://docusaurus.io/). (We're likely to use it for managing the Arti documentation and website.) For the purposes we have here, it seems slightly overkill, but I do think a migration is feasible down the road if we decide we _do_ want to move to docusaurus. The important thing is the ability to keep our URLs working, and I'm confident we could do that The main differences for our purposes here seem to be: * The markdown implementation in Docusaurus is extremely picky about stuff that looks like HTML but isn't; it rejects it, rather than passing it on as text. Thus, using it would require a more painstaking conversion process before we could include text like `"<state:on>"` or `"A <-> B"` as our specs do in a few places. * Instead of organizing our documents in a `SUMMARY.md` with an MD outline format, we'd have to organize them in a `sidebar.js` with a javascript syntax. * Docusaurus seems to be far more flexible and have a lot more features, but also seems trickier to configure. <-- References --> [`mdbook`]: https://rust-lang.github.io/mdBook/ [source tree]: https://github.com/rust-lang/mdBook/tree/master/guide/src/ [Markdown]: https://en.wikipedia.org/wiki/Markdown [`SUMMARY.md`]: https://rust-lang.github.io/mdBook/format/summary.html [plugins]: https://github.com/rust-lang/mdBook/wiki/Third-party-plugins [`mermaid`]: https://mermaid.js.org/ [architecture diagrams]: https://gitlab.torproject.org/tpo/core/arti/-/blob/main/doc/dev/Architectur… [protocol diagrams]: https://gitlab.torproject.org/tpo/core/arti/-/blob/main/doc/dev/hs-overview… [`torspec-converter`]: https://gitlab.torproject.org/nickm/torspec-converter [configuration section]: https://gitlab.torproject.org/nickm/torspec-converter/-/blob/main/grinder.p… [View Less]

1 0

[Update] Network Status APIs
by Mattia Righetti 03 Oct '23

03 Oct '23

Greetings Tor devs, I wanted to give a quick update on the progress of the Network Status APIs[0]. First of all, a couple of weeks ago we have deployed an initial version of the service internally so that we can benchmark and test it and see how it holds. Plus, we now have developed three more APIs: `/details`, `/clients`, `/weights`. In total, we now have covered almost every endpoint that is currently provided by onionoo. Just to give you a little bit more insights, `/summary` and `/… [View More]

1 0