- tor-dev - lists.torproject.org

GSoC 2022 - Tor Weather: Improving the Tor Network
by Sarthik Gupta 19 Sep '22

19 Sep '22

Hello everyone, I’m Sarthik Gupta (irc: sarthikg), a recent grad from Punjab Engineering College, Chandigarh & currently working as a Software Engineer at Soroco. This summer, I’ll be working on GSoC Project: “Tor Weather: Improving the Tor Network” with my mentors GeKo & Silvia. I’m extremely excited to be a part of Tor and am interested in working on this project! Briefing about the project: As of now, if any relay disappears from the tor-network, no one will know. This causes the network to lose out on bandwidth from nodes which have been down for hours because no-one knew they were down. Tor-Weather aims at solving this by creating a notification service which relay operators can subscribe to in order to get various types of updates for their relays. The tor-weather service will offer a plethora of notifications options for the relays. These include, the node being down, running on EOL/Outdated version, losing a flag, ranking in top 20/50/100, etc. These notifications can be subscribed & customised by the relay operators to fit their needs using a web-frontend. A detailed proposal for this project is available at: <https://gitlab.torproject.org/sarthikg/tor-weather/-/wikis/Proposal> https://gitlab.torproject.org/sarthikg/tor-weather/-/wikis/Proposal. I plan to keep this wiki updated with the progress & design decisions taken throughout the development of the project. Suggestions are always welcomed! Please reach out to us in irc (#tor-dev) for any ideas, questions, or suggestions you might have. Thanks, Sarthik Gupta

1 5

Shortcuts to data-heavy parts of the bootstrapping process
by Holmes Wilson 07 Sep '22

07 Sep '22

I was just at an event with a slow (but reliable) network and Tor was not able to connect; the "Loading relay descriptors" step just took too long. At some point I got an error message that indicated that it was giving up but that I had enough information to connect to onion addresses. I can't reproduce the problem now on a normal network, and I just went through the Tor code looking for the error message I saw, but I couldn't find it. But I figured I'd ask here to see if anyone was familiar with shortcuts Tor can take in its connection process that safely save time and bandwidth on slow connections if the only thing I intend to use Tor for is connecting to onion addresses. Are there any steps in bootstrapping that can be skipped if I only care about making and receiving onion address connections? Holmes

2 2

Introducing a Conjure PT for Tor
by Cecylia Bocovich 31 Aug '22

31 Aug '22

Conjure[0] is an anti-censorship tool in the refraction networking (a.k.a. decoy routing) lineage of circumvention systems. The key innovation of Conjure is to turn the unused IP address space of deploying ISPs into a large pool of "phantom" proxies that users appear to connect to. Due to the size of unused IPv6 address space and the potential for collateral damage against real websites hosted by the deploying ISPs, Conjure provides a possible solution to the problem of censors enumerating and blocking deployed bridges or proxies. I've been working with Jack Wampler and Eric Wustrow at the University of Colorado to implement a Conjure PT for Tor that uses the existing deployed Conjure stations[1]. # How it works Conjure clients first perform a registration step with the Conjure station to negotiate a phantom IP address to connect through. The idea of a registration phase that precedes the proxied connection is common among anti-censorship tools, including Snowflake. Like Snowflake, there are a few options for making the registration process censorship-resistant. Conjure currently offers unidirectional registration via Tapdance[2] decoy routing, and bidirectional registration via domain fronting. Once the client and station have negotiated a phantom IP address, the client makes a connection to the phantom address. The Conjure station sees this connection and instead proxies traffic using the haproxy[3] protocol to a client-specified destination. Conjure currently supports 3 different transport options for the connection between the client and the phantom proxy: a minimal SSH-based transport, obfs4, and webrtc. The Tor conjure pluggable transport[4] uses the gotapdance library[5] to register and connect to the production Conjure station through the negotiated phantom IP address. We have a deployed Tor bridge (named Haunt) that accepts haproxy connections from an allowlist of known Conjure stations. You can see details and usage metrics on Relay Search: https://metrics.torproject.org/rs.html#details/A84C946BF4E14E63A3C92E140532… # Next steps This PT is currently in development and only recommended for testing. We still have some work to do before the Tor Conjure PT can be rolled out to a large user base. The PT in its current form is very minimal in its features. We're reaching out to the development community now for initial feedback and testing. We are planning a slow ramp of client traffic to avoid placing stress on the stations and improve the reliability and censorship resistance of our setup. Immediate planned work includes: - securing the haproxy connection between the Conjure station and the bridge - making the registration connection censorship resistant - ongoing testing and development of the obfs4 transport integration - usability improvements and resilience in the event of overloaded Conjure stations - reproducible builds for Tor Browser For more details, check out the issue tracker for this project[6]. We have a milestone for shipping conjure support in alpha version of Tor Browser[7], which is the next major step in the rollout process. # Try it out yourself Instructions for cloning and building this PT are in the repository[4]. In the client/ directory there are two provided torrc files. For testing the production Conjure deployment, you may use the one named `torrc`. The other file, `torrc-testing` is for use with the libvirt-based testing and development environment[8]. Successful bootstrapping can take a few tries due to high load at the station. More detailed log messages will be written out to a log file conjure.log by default. This can be changed by modifying the -log argument in the ClientTransportPlugin line of the torrc file. If the station is overloaded, you will see a message like the following in conjure.log: ``` [11:32:12] [1-d9b572] failed to dial phantom [scrubbed]: dial tcp [scrubbed]: i/o timeout ``` Please try it out, open issues, ask questions, provide feedback! # Thanks! Thanks again to Jack and Eric for their efforts on this project, and to whole team of Conjure researchers, developers, and admins for their work on deploying Conjure and making it available for use. [0] https://censorbib.nymity.ch/pdf/Frolov2019b.pdf [1] https://censorbib.nymity.ch/pdf/Frolov2017a.pdf [2] https://censorbib.nymity.ch/pdf/Wustrow2014a.pdf [3] https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt [4] https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/conj… [5] https://github.com/refraction-networking/gotapdance [6] https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/conj… [7] https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/conj… [8] https://gitlab.torproject.org/cohosh/phantombox

1 0

Counting HS descriptor uploads
by Michael Rogers 17 Aug '22

17 Aug '22

Hi, The Briar team is working on a new app that uses Tor hidden services, and we're trying to work out when the server publishing a hidden service should consider the service to be reachable by clients. We've tried counting HS descriptor uploads via control port events, but we found that when republishing a hidden service that has been published before, the number of descriptor uploads seems to vary. When republishing a hidden service, is it guaranteed that at least one copy of the descriptor will be uploaded? Or are there circumstances where Tor might decide that enough copies were previously uploaded, and still contain up-to-date information about introduction points etc, so no new copies need to be uploaded? Thanks, Michael

3 3

TOR socket for P2P in Python
by Martin Neuenhofen 16 Aug '22

16 Aug '22

Dear Tor Developers, in my application, a client connects to a server via: *client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)client_socket.connect(ip,port)* I want to replace these two lines to create a client_socket whose IP address cannot be seen by the server. The application is e-polling. I looked into - torpy (too many timeouts), - socks (requires too much external configuration), - and I have tor installed (unknown how to use from within python). I just need a replacement for the two-liner code above. *The ideal solution:* is device & OS agnostic, portable, uses no peripherals or configuration (i.e., just one pip package to install), and works reliably. Thus, the below torpy solution would have been ideal if it had worked reliably: *from torpy import TorClient # pip install torpy* *class Torsocket:* * def __init__(self,ip,port): self.mgr1 = TorClient() self.tor = type(self.mgr1).__enter__(self.mgr1) self.mgr2 = self.tor.create_circuit(3) self.circuit = type(self.mgr2).__enter__(self.mgr2) self.mgr3 = self.circuit.create_stream((ip,port)) self.socket = type(self.mgr3).__enter__(self.mgr3) def send(self,data): self.socket.send(data) def recv(self,size): return self.socket.recv(size) def __del__(self): for bla in [self.mgr3,self.mgr2,self.mgr1]: type(bla).__exit__(bla, None, None, None)* It replaces the two-liner from the beginning with the below one-liner *client_socket = Torsocket(*server_data.server_address)* My questions/requests: - Is torpy connected to the real tor network (or is it a little toy twin)? If it is the real tor network then I will close this issue and open another one. - Are practical solutions available to my problem already?, and which of them works best for user-friendliness? - Can a pip package for a torsocket be made, as described above? i.e., something like torpy, but connecting to the real network? - Can I offer anything so that the ideal solution would be crafted and packaged?* *) If it helps, I could tell a lot about my project and how I believe it is significant and will be for the greater good of people. It is a fully implemented 1000LOC LSAG+WOT decentral P2P e-polling program with GUI that shall return trust and sovereignty into the people's hands. The only missing bit is a reliable tor connection.

2 3

bridge:// URI and QR codes
by meskio 10 Aug '22

10 Aug '22

Hello, We have a messy situation to share bridges between devices. BridgeDB and Tor Browser have QR codes but they use different format for them. There has being some attempts to standarize a bridge URI[0][1], but it looks like we never finished that work. Pier has produced an RFC of a bridge URI proposal: https://gitlab.torproject.org/tpo/applications/team/-/wikis/Design-Document… If you have an application that uses (or plan to use) bridge URIs or QR codes I'm interested on your take on that proposal. Do your application already have any kind of support for bridge URIs or QR codes? If so how do they look like? Will it break things if we change them? What do you think of the proposal? How can we improve it? Thank you. [0] https://gitlab.torproject.org/legacy/trac/-/issues/15035 [1] https://gitlab.torproject.org/tpo/applications/fenix/-/issues/40189 -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

5 8

Proposal 341: A better algorithm for out-of-sockets eviction
by Nick Mathewson 27 Jul '22

27 Jul '22

``` Filename: 341-better-oos.md Title: A better algorithm for out-of-sockets eviction Author: Nick Mathewson Created: 25 July 2022 Status: Open ``` # Introduction Our existing algorithm for handling an out-of-sockets condition needs improvement. It only handles sockets used for OR connections, and prioritizes those with more circuits. Because of these weaknesses, the algorithm is trivial to circumvent, and it's disabled by default with `DisableOOSCheck`. Here we propose a new algorithm for choosing which connections to close when we're out of sockets. In summary, the new algorithm works by deciding which kinds of connections we have "too many" of, and then by closing excess connections of each kind. The algorithm for selecting connections of each kind is different. # Intuitions behind the algorithm below We want to keep a healthy mix of connections running; favoring one kind of connection over another gives the attacker a fine way to starve the disfavored connections by making a bunch of the favored kind. The correct mix of connections depends on the type of service we are providing. Everywhere _except_ authorities, for example, inbound directory connections are perfectly fine to close, since nothing in our protocol actually generates them. In general, we would prefer to close DirPort connections, then Exit connections, then OR connections. The priority with which to close connections is different depending on the connection type. "Age of connection" or "number of circuits" may be a fine metric for how truly used an OR connection is, but for a DirPort connection, high age is suspicious. # The algorithm Define a "candidate" connection as one that has a socket, and is either an exit stream, an _inbound_ directory stream, or an OR connection. (Note that OR connections can be from clients, relays, or bridges. Note that ordinary relays should not get directory streams that use sockets, since clients always use `BEGIN_DIR` to create tunneled directory streams.) In all of the following, treat subtraction as saturating at zero. In other words, when you see "A - B" below, read it as "MAX(A-B, 0)". ## Phase 1: Deciding how many connections to close When we are find that we are low on sockets, we pick a number of sockets that we want to close according to our existing algorithm. (That is, we try to close 1/4 of our maximum sockets if we have reached our upper limit, or 1/10 of our maximum sockets if we have encountered a failure from socket(2).) Call this `N_CLOSE`. Then we decide which sockets to target based on this algorithm. 1. Consider the total number of sockets used for exit streams (`N_EXIT`), the total number used for _inbound_ directory streams (`N_DIR`), and the total number used for OR connections (`N_OR`). (In these calculations, we exclude connections that are already marked to be closed.) Call the total `N_CONN = N_DIR + N_OR + N_EXIT`. Define `N_RETAIN = N_CONN - N_CLOSE`. 2. Compute how many connections of each type are "in excess". First, calculate our target proportions: * If we are an authority, let `T_DIR` = 1. Otherwise set `T_DIR = 0.1`. * If we are an exit or we are running an onion service, let `T_EXIT = 2`. Otherwise let `T_EXIT = 0.1`. * Let `T_OR` = 1. > TODO: Should those numbers be consensus parameters? These numbers define the relative proportions of connections that we would be willing to retain retain in our final mix. Compute a number of _excess_ connections of each type by calculating. ``` T_TOTAL = T_OR + T_DIR + T_EXIT. EXCESS_DIR = N_DIR - N_RETAIN * (T_DIR / T_TOTAL) EXCESS_EXIT = N_EXIT - N_RETAIN * (T_EXIT / T_TOTAL) EXCESS_OR = N_OR - N_RETAIN * (T_OR / T_TOTAL) ``` 3. Finally, divide N_CLOSE among the different types of excess connections, assigning first to excess directory connections, then excess exit connections, and finally to excess OR connections. ``` CLOSE_DIR = MIN(EXCESS_DIR, N_CLOSE) N_CLOSE := N_CLOSE - CLOSE_DIR CLOSE_EXIT = MIN(EXCESS_EXIT, N_CLOSE) N_CLOSE := N_CLOSE - CLOSE_EXIT CLOSE_OR = MIN(EXCESS_OR, N_CLOSE) ``` We will try to close `CLOSE_DIR` directory connections, `CLOSE_EXIT` exit connections, and `CLOSE_OR` OR connections. ## Phase 2: Closing directory connections We want to close a certain number of directory connections. To select our targets, we sort first by the number of directory connections from a similar address (see "similar address" below) and then by their age, preferring to close the oldest ones first. > This approach defeats "many requests from the same address" and "Open > a connection and hold it open, and do so from many addresses". It > doesn't do such a great job with defeating "open and close frequently > and do so on many addresses." > Note that fallback directories do not typically use sockets for > handling directory connections: theirs are usually created with > BEGIN_DIR. ## Phase 3: Closing exit connections. We want to close a certain number of exit connections. To do this, we pick an exit connection at random, then close its circuit _along with all the other exit connections on the same circuit_. Then we repeat until we have closed at least our target number of exit connections. > This approach probabilistically favors closing circuits with a large > number of sockets open, regardless of how long those sockets have been > open. This defeats the easiest way of opening a large number of exit > streams ("open them all on once circuit") without making the > counter-approach ("open each exit stream on a its own circuit") much > more attractive. ## Phase 3: Closing OR connections. We want to close a certain number of OR connections, to clients, bridges, or relays. To do this, we first close OR connections with zero circuits. Then we close all OR connections but the most recent 2 from each "similar address". Then we close OR connections at random from among those _not_ to a recognized relay in the latest directory. Finally, we close OR connections at random. > We used to unconditionally prefer to close connections with fewer > circuits. That's trivial for an adversary to circumvent, though: they > can just open a bunch of circuits on their bogus OR connections, and > force us to preferentially close circuits from real clients, bridges, > and relays. > Note that some connections that seem like client connections ("not > from relays in the latest directory") are actually those created by > bridges. ## What is "A similar address"? We define two connections as having a similar address if they are in the same IPv4 /30, or if they are in the same IPv6 /90. # Acknowledgments This proposal was inspired by a set of [OOS improvements](https://gitlab.torproject.org/tpo/core/tor/-/issues/32794 ) from `starlight`.

2 1

Shrink Tor binary size
by Sergey Ponomarev 17 Jul '22

17 Jul '22

I’m working on a firmware for routers based on OpenWrt and it needs Tor out of the box for NAT punching i.e. SSH and Web admin access. It will expose a Single Onion Service i.e. not "hidden" with just 3 hop for a better performance. In fact it just needs to connect to some random relay to have a tunnel and encrypt/decrypt traffic. The full hidden service functionality with IP/RP dances is not needed: I'm not trying to hide the location of the router. In fact most users even don't need anonymity with the 3 hop to access their router but Tor requires 3. So this is probably a kind of abuse of the Tor network but this is the only free network of proxies. At the same time a user will have an encryption while accessing the router's admin panel. I may enable HTTPS with a self signed cert but users must accept it manually which is hard for inexperienced users. The Tor is already ported to OpenWrt but its binary is more than 2mb. For 16mb routers this is not critical but a lot and I need to keep space. So is it possible to compile Tor without some features? As far as I understood from autoconf I can disable Relay functionality. But maybe I can also disable some admin api parts and SOCKS proxy. Also OpenWrt out of the box has WolfSSL so is it possible to compile tor with it instead of OpenSSL? Another problem is files sizes inside of /var/lib/tor/: 20,442 cached-certs 2,303,443 cached-microdesc-consensus 8,725,598 cached-microdescs.new 3,531 state I think that all these files may be minimized. E.g. cached-certs contains keys in PEM format which is just Base64 form of DER. So switching to plain DER certs can significantly reduce size. As a side question: is it possible to make a small relay proxy that can work part time when I sleep? Imagine that each router already has a Tor. This is potentially thousands of relay nodes and all of them have a motivation to support the network which they are using themselves. But users don’t want to lose bandwidth. I see that I can set some traffic or bandwidth limits. But maybe I can write some script that will enable or disable the relay by schedule in the evening. As far I know relays must be always online so this will make the relay unstable and it won’t be used by Tor. Is it technically possible? Regards, Sergey Ponomarev, stokito.com

2 1

Want to build circuit via unpublished relay, can't add microdesc using control port
by ValdikSS 11 Jul '22

11 Jul '22

Hello everyone. I'm experimenting with Tor network and wanted to build a circuit via unpublished relay (PublishServerDescriptor 0). To do so, I set up a relay, got its authority descriptor, imported it with +POSTDESCRIPTOR in the client using control port and tried to build a circuit using its fingerprint with EXTENDCIRCUIT. Everything works fine with UseMicrodescriptors 0, however with MicroDescriptors activated, the client does not detect imported (full) descriptor, returning: 552 No descriptor for "fingerprint here" I found no way to import microdescriptor using control port and no other means to use unpublished nodes. How can I add microdesc which is not in consensus into router? Or how can I build circuit with a relay not in consensus otherwise?

3 2

Tor can't read HiddenServicePort unix socket through group permissions when starting as root and using setgid?
by keyandthegate 06 Jul '22

06 Jul '22

Permissions are set so tor should be able to access through the `postfix-test-queue` user: > $ sudo ls -l /var/spool/postfix-test/public/smtpd > srw-rw-rw- 1 postfix-test postfix-test 0 █████ /var/spool/postfix-test/public/smtpd > $ sudo ls -l /var/spool/postfix-test > # ... > drwx--x--- 2 postfix-test postfix-test-queue 4096 █████ public > $ sudo -u _tor-test id > uid=130(_tor-test) gid=141(_tor-test) groups=141(_tor-test),1006(postfix-test-queue) > $ cat /etc/tor/instances/test/torrc | grep HiddenServicePort > HiddenServicePort 25 unix:/var/spool/postfix-test/public/smtpd > $ cat /run/tor-instances/test.defaults | grep User > User _tor-test Running `tor@test` via the default systemctl config shows: > $ ps -ax -o uid,gid,supgid,command | grep /usr/bin/tor > 130 141 141 /usr/bin/tor --defaults-torrc /run/tor-instances/test.defaults -f /etc/tor/instances/test/torrc Which is missing the `postfix-test-queue` `1006` user which, for example shows up if I do: > $ sudo -u _tor-test sleep 1000 & ps -ax -o uid,gid,supgid,command | grep sleep > [1] 132314 > 0 141 141,1006 sudo -u _tor-test sleep 1000 Connecting using `sudo -u` works (the message indicates successful connection): > $ sudo -u _tor-test curl --unix-socket /var/spool/postfix-test/public/smtpd http://localhost > curl: (1) Received HTTP/0.9 when not allowed But connecting via tor does not: > $ torsocks --ipv6 curl http://█████.onion:25 > █████ ERROR torsocks[134873]: Host unreachable (in socks5_recv_connect_reply() at socks5.c:539) > curl: (7) Couldn't connect to server But does if I allow access to the socket to everyone: > $ sudo chmod "o+x" /var/spool/postfix-test/public/ > $ torsocks --ipv6 curl http://█████.onion:25 > curl: (1) Received HTTP/0.9 when not allowed Tor's relevant source code: [Tor: lib/process/setuid.c Source File](https://tpo.pages.torproject.net/core/doc/tor/setuid_8c_source.html)

1 1