Hello everyone!
As many of you know we have been working on a simple bandwidth scanner
(sbws) over the last years to deal with the old and unmaintained Torflow
code on our bandwidth authorities. We learned a lot during that process,
especially after we started to replace Torflow with the new sbws code on
bandwidth authorities (today sbws is powering a majority of them, yay!).
We realized that we needed to improve the robustness and testability of
the code significantly, in particular as we want to build exciting
features like loadbalancing capability on top of it and decided it would
be easier to do so in a new tool: welcome onbasca. Onbasca takes the
best out of sbws and makes the code easier to test and more robust for
future improvements by deploying a database management system where we
previously dealt with JSON files only.[1]
The code repository lives in our network-health group[2] and is open for
contributions.
Get in touch in case you are interested, everyone is welcome!
Georg
[1] For a more detailed overview over the bandwidth scanner history,
design issues we encountered in sbws, and decisions we made, see:
https://tpo.pages.torproject.net/network-health/onbasca/history.html.
[2] https://gitlab.torproject.org/tpo/network-health/onbasca
Hi,
below is a partial proposal draft for human readable relay operator IDs that are authenticated
by directory authorities. If there is any interest in implementing
something like this I'll complete the draft and submit
it via gitlab.
kind regards,
nusenu
# HAROI: Human Readable Authenticated Relay Operator Identifier
A HAROI is a DNS FQDN that can be chosen by a relay operator and
linked to one or more relays in a verifiable way.
The link is authenticated in the sense that it requires some control
over the relay and the FQDN (on the DNS or webroot level).
A relay can only be linked to a single HAROI.
Relays publish their HAROI via their descriptor.
Directory authorities verify the link and publish the verification result in their vote.
## Motivation
Many tools in the tor ecosystem display at least some relay (operator)
information (nickname, ContactInfo, ...), some examples are:
- Tor Browser
- Relay Search [1]
- Onioncircuits by tails [2]
- ExoneraTor [3]
- allium [4]
unfortunatelly there is no authenticated operator ID available, these tools could display,
so they might display misleading information, something that has been exploited in the wild for
impersonation attacks.
There is a value in giving relay operators the possibility to
declare an identifier that is globally unique, consistent and can not be easily spoofed by adversaries
so they do not become easy victims of impersonation attacks.
The tor ecosystem would benefit from an operator ID that can not be spoofed.
This proposal is not replacing the relay ContactInfo.
## Design
* A HAROI is optional and not mandatory.
* A HAROI is verified by directory authorities in one of two ways, depending on the proof type.
In the distant future where relays have no RSA1024 keys, Ed25519 proof types are added.
* The used proof type can be selected by the relay operator.
Successfully verified proofs are cached by directory authorities for 30 days.
After 30 days proofs are verified again.
Relay operators that want to make use of HAROI can participate without
requiring a domain since many free services offer custom free subdomains
(example: GitLab or GitHub pages).
## Proof Types
Relay operators, that chose to set a HAROI,
can select their preferred a proof type.
### uri-rsa
This proof type can be verified by fetching
the list of relay RSA SHA1 fingerprints from
the FQDN via HTTPS using the well-known URI
as defined in proposal 326
https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/326-to…
Example: if the FQDN is example.com, the url to fetch the list of fingerprints is:
https://example.com/.well-known/tor-relay/rsa-fingerprint.txt
The TLS certificate used on the webserver must be from a trusted CA and logged in a public certificate transparency log.
For N relays using the same FQDN only a single HTTPS GET request is needed.
### dns-rsa
This proof type can be verified by performing
DNS lookups. To make use of this proof type the DNS zone MUST be DNSSEC signed.
The DNS query is constructed by combining the relay's fingerprint and the FQDN:
relay-fingerprint.<FQDN>
example:
relay-fingerprint.example.com value: “we-run-this-tor-relay”
relay-fingerprint is the 40 character RSA SHA1 fingerprint of the tor relay.
Each relay has its own DNS record, a single TXT record MUST be returned per relay only.
content of the TXT record MUST match:
"we-run-this-tor-relay”
Each relay requires a single DNS lookup (less scalable than the uri-rsa option).
## Relay Configuration
Relay operators can specify their identifier via a torrc option.
OperatorIdentifier <FQDN> <proof-type>
example:
OperatorIdentifier example.com dns-rsa
## Length Limit
Although DNS FQDNs can be rather long, we limit HAROIs to 40 characters.
(As of December 2021 the longest observed HAROI is 28 characters long.)
Operators will not be able to specify a HAROI longer than 40 characters in their torrc.
Directory authorities refuse to accept them as well.
## Relays Descriptor Changes
This proposal defines a new optional descriptor field that is
automatically verified and voted on by tor directory authorities.
operatorid <FQDN> <proof-type>
## Directory Authorities
Directory authorities refuse to accept domains on the public suffix list [5].
XXX TODO
## Security Considerations
Relay operators can trick directory authorities into performing DNS lookups
and HTTPS GETs on arbitrary domains.
Possible solutions:
Directory authorities could required PTR+A DNS records on the same domain as the given FQDN for the relays IP address.
[1] https://metrics.torproject.org/rs.html
[2] https://gitlab.tails.boum.org/tails/onioncircuits
[3] https://metrics.torproject.org/exonerator.html
[4] https://yui.cat/
[5] https://publicsuffix.org/list/public_suffix_list.dat
--
https://nusenu.github.io
Hi,
tldr:
- more outdated relays
(that is a claim I'm making and you could
easily proof me wrong by recreating the 0.3.3.x alpha
repos and ship 0.3.3.7 in them and see how things evolve
after a week or so)
- more work for the tpo website maintainer
- less happy relay operators [3][4]
- more work for repo maintainers? (since a new repo needs to be created)
When the tor 0.3.4 alpha repos (deb.torproject.org) first appeared on 2018-05-23
I was about to submit a PR for the website to include it in the sources.list
generator [1] on tpo but didn't do it because I wanted to wait for a previous PR to be merged first.
The outstanding PR got merged eventually (2018-06-28) but I still did not submit a PR to
update the repo generator for 0.3.4.x nonetheless and here is why.
Recently I was wondering why are there so many relays running tor version 0.3.3.5-rc? (see OrNetStats or Relay Search)
(> 3.2% CW fraction)
Then I realized that this was the last version the tor-experimental-0.3.3.x-*
repos were shipping before they got abandoned due to the new 0.3.4.x-* repos
(I can no longer verify it since they got removed by now).
Peter made it clear in the past that the current way to
have per-major-version debian alpha repos (i.e. tor-experimental-0.3.4.x-jessie)
will not change [2]:
> If you can't be bothered to change your sources.list once or twice a
> year, then you probably should be running stable.
but maybe someone else would be willing to invoke a
"ln" commands everytime a new new alpha repo is born.
tor-alpha-jessie -> tor-experimental-0.3.4.x-jessie
once 0.3.5.x repos are created the link would point to
tor-alpha-jessie -> tor-experimental-0.3.5.x-jessie
It is my opinion that this will help reduce the amount of relays running
outdated versions of tor.
It will certainly avoid having to update the tpo website, which isn't a big task
and could probably be automated but it isn't done currently.
"..but that would cause relay operators to jump from i.e. 0.3.3.x to 0.3.4.x alphas
(and break setups)!"
Yes, and I think that is better than relays stuck on an older version because
the former repo no longer exists and operators still can choose the old repos
which will not jump to newer major versions.
[1] https://www.torproject.org/docs/debian.html.en#ubuntu
[2] https://trac.torproject.org/projects/tor/ticket/14997#comment:3
[3] https://lists.torproject.org/pipermail/tor-relays/2018-June/015549.html
[4] https://trac.torproject.org/projects/tor/ticket/26474
--
https://twitter.com/nusenu_https://mastodon.social/@nusenu
Sections 3 and 6 of the Quux paper have some relevant discussion [1]
> Unfortunately, it appears that the Quux QUIC paper studied QUIC at the
> wrong position - between relays, and the QuTor implementation is
> unclear. This means that it may still be an open question as to if
> QUIC's congestion control algorithms will behave optimally in a
> Tor-like network under heavy load.
As Reardon and Goldberg noted in concluding remarks, approaches other
than hop-by-hop will incur an extra cost for retransmissions, since
these must be rerouted through a larger part of the network [RG09].
As Tschorsch and Scheuermann discuss [TS12], due to the longer RTT of
TCP connections, end-to-end approaches will also take longer to “ramp
up” through slow start and up to a steady state.
Both of these factors (not to mention increased security risk of
information leakage [DM09]) suggest that hop-by-hop designs are likely
to yield beer results. In fact, the hop-by-hop approach may be viewed as
an instance of the Split TCP Performance-Enhancing Proxy design, whereby
arbitrary TCP connections are split in two to negate the issues noted
above.
> Unfortunately, the Quux implementation appears to use QUIC at a
> suboptimal position -- they replace Tor's TLS connections with QUIC,
> and use QUIC streams in place of Tor's circuit ID -- but only between
> relays. This means that it does not benefit from QUIC's end-to-end
> congestion control for the entire path of the circuit. Such a design
> will not solve the queuing and associated OOM problems at Tor relays,
> since relays would be unable to drop cells to signal backpressure to
> endpoints. Drops will instead block every circuit on a connection
> between relays, and even then, due to end-to-end reliability, relays
> will still need to queue without bound, subject to Tor's current (and
> inadequate) flow control.
A fully QUIC relay path (with slight modication to fix a limit on
internal buffer sizes) would allow end-to-end backpressure to be used
from the client application TCP stream up to the exit TCP stream.
Leaving aside Tor’s inbound rate limit mechanism but retaining the
global outbound limit, this design would allow max-min fairness to be
achieved in the network, as outlined by Tschorsch and Scheuermann
[TS11].
...
Once implemented however, backpressure would allow Tor to adopt a
signicantly improved internal design. In such a design, a Tor relay
could read a single cell from one QUIC stream’s read buffer, onion crypt
it, and immediately place it onto the write buffer of the next stream in
the circuit. This process would be able to operate at the granularity of
a single cell because the read and write operations for QUIC are very
cheap user-space function calls and not syscalls as for host TCP.
The schedule of this action would be governed by the existing EWMA
scheduler for circuits that have both a readable stream and a writeable
stream (and as allowed by a global outgoing token bucket), allowing
optimal quality of service for circuits.
It’s expected that backpressure implemented in this way will yield
signicant performance and fairness gains on top of the performance
improvement found in this thesis.
One issue for Quux was that it used the Chromium demo QUIC server code as the
basis for its implementation, which was fine for performance research but not
such a good choice for Tor's networking stack.
Several Rust implementations have been released with server-side (not just
client-side) usage, so I expect that to be much less of an issue today.
io_uring is also a significant development since Quux was developed, as
it can reduce the performance hit for host-TCP syscalls, or for using
recvmsg instead of recvmmsg with QUIC if the implementation makes
it difficult to use recvmmsg on the listener side.
[1] https://www.benthamsgaze.org/wp-content/uploads/2016/09/393617_Alastair_Cla…
The following paper has in-depth discussion, but I don't have a copy to
hand unfortunately:
Ali Clark. Tor network performance — transport and flow control. Technical report, University College London, April 2016
Hi there. I had an idea recently for an onion service to improve the UX
of sites that require a login. The site would have two onions: one for
those who want to use onion auth and another for those who don't or are
still setting it up. A user would first sign in with a username+password
on the unauthenticated onion and click a button to generate a
certificate associated with their account. Then they would add the
public key to their browser and visit the authenticated onion. The
application server would then match the pubkey used to authenticate with
an account in the database, and log them in automatically.
I've looked in the mailing list archives and `man 1 tor` but didn't find
anything that would facilitate this. The closest, it seems, is
HiddenServiceExportCircuitID, but that is for *circuit* IDs, not
*client* IDs. Is this possible to implement, either as an operator or as
a Tor developer?
As an operator, an alternative would be to generate one (authenticated)
onion service per user and route them all to the same place with
different Host headers, but that seems rather inefficient, and I don't
know how well the tor daemon scales up to hundreds of onion services anyway.
P.S. I didn't find an easy way to do full text search on the mailing
list archives, so I wrote a little script to download them all. I've
attached it in case it ends up useful. It requires python3.8+ and you'll
need to `pip install aiohttp anyio BeautifulSoup4` first. After that you
can run `./pipermail_fetch.py
https://lists.torproject.org/pipermail/tor-dev/` and then something like
`rg --context 3 --search-zip '^[^>].*search term here'` will do the trick.
Hi guys
A complete non-dev/programmer here so please go easy on me.
I'm trying to compile torsocks with 'mock', while 'tor'
compiles okey for 'torsock' I fail with:
...
configure.ac:20: comes from Automake 1.16.1. You should
recreate
configure.ac:20: aclocal.m4 with aclocal and run automake
again.
WARNING: 'automake-1.16' is probably too old.
You should only need it if you modified
'Makefile.am' or
'configure.ac' or m4 files included by
'configure.ac'.
The 'automake' program is part of the GNU Automake
package:
<http://www.gnu.org/software/automake>
It also requires GNU Autoconf, GNU m4 and Perl in
order to run:
<http://www.gnu.org/software/autoconf>
<http://www.gnu.org/software/m4/>
<http://www.perl.org/>
make[1]: *** [Makefile:600: Makefile.in] Error 63
make: *** [Makefile:466: all-recursive] Error 1
...
My setup is Centos 9 and obviously I do it with an rpm/spec.
Would you guys be kind to guide me to troubleshoot this?
many thanks, L.
Hi,
I'm wondering if I can access the shared random value[1] while
developing a
protocol/application on top of Tor onion services. The application is
still in
early development, but it would be great if I could depend on the shared
random
value.
If this is not the correct mailing list for this question, I would be
glad if
you could point me to one.
--Sebastian
[1]: PUB-SHAREDRANDOM in
https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt
Hi,
when adding MetricsPort support to ansible-relayor
I realized that many operators that run more than one tor instance per server
will run into an issue because tor's relay prometheus metrics has no identifying label like
fingerprint=
or similar to tell tor instances appart. The instance=
default label can have the same value for all tor instances on a given server
so that can not be used.
To avoid using nickname (might not be set) the easiest option is probably to use
the relay's SHA1 fingerprint or alternatively the IP:ORPort combination
which is unique per server but not necessarily globally unique (RFC1918 IPs).
Another neat option for operators is to use node_exporter's textfile collector to
collect tor's MetricsPort content to avoid having to run an additional webserver
for TLS and authentication (because unlike tor's exporter node_exporter comes with TLS and authentication builtin).
In that case the suggested solution would be even more needed because in that case relabling
via prometheus' scrape config is no longer possible.
What do you think about this suggestion?
kind regards,
nusenu
--
https://nusenu.github.io