[tor-relays] How to reduce tor CPU load on a single bridge?

David Fifield david at bamsoftware.com
Sat Jan 1 02:11:08 UTC 2022


On Thu, Dec 30, 2021 at 10:42:51PM -0700, David Fifield wrote:
> One complication we'll have to work out is that ptadapter doesn't have a
> setting for ExtORPort forwarding. ptadapter absorbs any ExtORPort
> information and forwards an unadorned connection onward. The idea I had
> to to work around this limitation is to have ptadapter, rather than
> execute snowflake-server directly, execute a shell script that sets
> TOR_PT_EXTENDED_SERVER_PORT to a hardcoded address (i.e., to haproxy)
> before running snowflake-server. Though, I am not sure what to do about
> the extended_orport_auth_cookie file, which will be different for
> different tor instances.

There are a number of potential ways to deal with the complication of
ExtORPort authentication, from alternative ExtORPort authentication
types, to ExtORPort-aware load balancing. With a view towards deploying
something in the near future, I wrote this program that enables an
external pluggable transport to talk to tor's ExtORPort and authenticate
as if it had an unchanging authentication cookie.

https://gitlab.torproject.org/dcf/extor-static-cookie

The difficulty with load-balancing multiple tor instances, with respect
to ExtORPort, is that to authenticate with the ExtORPort you need to
read a cookie from a file on disk, which tor overwrites randomly every
time it starts. If you do not know which instance of tor will receive
your forwarded traffic, you do not know which ExtORPort cookie to use.

The extor-static-cookie program presents an ExtORPort interface, however
it reads its authentication cookie that is independent of any instance
of tor, which you can write once and then leave alone. The external
server pluggable transport can read from the shared authentication
cookie file as well. Every instance of tor runs a copy of
extor-static-cookie, all using the same authentication cookie file. The
extor-static-cookie instances receive ExtORPort authentication from the
external server pluggable transport, along with the USERADDR and
TRANSPORT metadata, then re-authenticate and echo that information to
their respective tor's ExtORPort.

So we change from this:
	                                    ___
	                                .->|tor|
	   ________________    _______  |   ---
	->|snowflake-server|->|haproxy|-+->|tor|
	   ----------------    -------  |   ---
	                                '->|tor|
	                                    ---
to this:
	                                    ___________________    ___
	                                .->|extor-static-cookie|->|tor|
	   ________________    _______  |   -------------------    ---
	->|snowflake-server|->|haproxy|-+->|extor-static-cookie|->|tor|
	   ----------------    -------  |   -------------------    ---
	                                '->|extor-static-cookie|->|tor|
	                                    -------------------    ---

I have a similar setup running now on a test bridge, with one instance
of obfs4proxy load-balancing to two instances of tor.


## Setup notes

Install extor-static-cookie:
	# apt install golang
	# git clone https://gitlab.torproject.org/dcf/extor-static-cookie
	# (cd extor-static-cookie && go build)
	# install -o root -g root extor-static-cookie/extor-static-cookie /usr/local/bin/

Generate a shared authentication cookie file:
	# mkdir -m 755 /var/lib/extor-static-cookie
	# extor-static-cookie/gen-auth-cookie > /var/lib/extor-static-cookie/static_extended_orport_auth_cookie

Install a first instance of tor and configure it as a bridge:
	# apt install tor
	# tor-instance-create o1
/etc/tor/instances/o1/torrc:
	BridgeRelay 1
	PublishServerDescriptor 0
	AssumeReachable 1
	SocksPort 0
	ORPort 127.0.0.1:auto
	ExtORPort auto
	ServerTransportPlugin extor_static_cookie exec /usr/local/bin/extor-static-cookie /var/lib/extor-static-cookie/static_extended_orport_auth_cookie
	ServerTransportListenAddr extor_static_cookie 127.0.0.1:10001
Notice we set `ExtORPort auto` (this is tor's own ExtORPort), and also
pass `127.0.0.1:10001` to extor-static-cookie, which is the ExtORPort
that the external server pluggable transport will talk to. Start the
first instance, which will generate keys:
	systemctl start tor at o1

Install a second instance of tor and configure it as a bridge (with a
different ServerTransportListenAddr port):
	# tor-instance-create o2
/etc/tor/instances/o2/torrc:
	BridgeRelay 1
	PublishServerDescriptor 0
	AssumeReachable 1
	SocksPort 0
	ORPort 127.0.0.1:auto
	ExtORPort auto
	ServerTransportPlugin extor_static_cookie exec /usr/local/bin/extor-static-cookie /var/lib/extor-static-cookie/static_extended_orport_auth_cookie
	ServerTransportListenAddr extor_static_cookie 127.0.0.1:10002
But before starting the second instance the first time, copy keys from
the first instance:
	# cp -r /var/lib/tor-instances/o1/keys /var/lib/tor-instances/o2/
	# chown -R _tor-o2:_tor-o2 /var/lib/tor-instances/o2/keys/
	# systemctl start tor at o2

The two instances should have the same fingerprint:
	# cat /var/lib/tor-instances/*/fingerprint
	Unnamed 4808CD98E4C1D4F282DA741A860A44D755701F2F
	Unnamed 4808CD98E4C1D4F282DA741A860A44D755701F2F

Install haproxy and configure it to forward to the two instances of
extor-static-cookie (which will then forward to the ExtORPort of their
respective tor instances):
	# apt install haproxy
/etc/haproxy/haproxy.cfg:
	frontend tor
		mode tcp
		bind 127.0.0.1:10000
		default_backend tor-o
	backend tor-o
		mode tcp
		server o1 127.0.0.1:10001
		server o2 127.0.0.1:10002
Restart haproxy with the new configuration:
	# systemctl restart haproxy

Instead of ptadapter, I found it more convenient to start the external
server pluggable transport with a shell script that sets up the
necessary variables:
extor.sh:
	#!/bin/sh

	# Usage: extor.sh 127.0.0.1:10000 /var/lib/extor-static-cookie/static_extended_orport_auth_cookie /usr/bin/obfs4proxy

	set -e

	EXTOR_ADDR="${1:?missing ExtORPort address}"
	EXTOR_COOKIE_FILE="${2:?missing ExtORPort auth cookie file}"
	shift 2

	BINDADDR='[::]:443'
	TRANSPORT=obfs4

	TOR_PT_MANAGED_TRANSPORT_VER=1 \
	TOR_PT_SERVER_TRANSPORTS="$TRANSPORT" \
	TOR_PT_SERVER_BINDADDR="$TRANSPORT"-"$BINDADDR" \
	TOR_PT_EXTENDED_SERVER_PORT="$EXTOR_ADDR" \
	TOR_PT_AUTH_COOKIE_FILE="$EXTOR_COOKIE_FILE" \
	TOR_PT_STATE_LOCATION=pt_state \
	TOR_PT_EXIT_ON_STDIN_CLOSE=1 \
	exec "$@"

Then I run the shell script, giving the address of the haproxy frontend,
the path to the shared authentication cookie file, and a command to run:
	# ./extor.sh 127.0.0.1:10000 /var/lib/extor-static-cookie/static_extended_orport_auth_cookie /usr/bin/obfs4proxy

On the client, make a torrc file with the information from
pt_state/obfs4_bridgeline.txt:
	UseBridges 1
	SocksPort auto
	Bridge obfs4 172.105.3.197:443 4808CD98E4C1D4F282DA741A860A44D755701F2F cert=1SCzqyYyPh/SiXTJa9nLFxMyjWQITVCKeICME+SwxgNcTTSUQ7+vM/ghofU7oaalIRBILg iat-mode=0
	ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy
	DataDir datadir
Then run tor with the torrc:
	$ tor -f torrc


More information about the tor-relays mailing list