Hello Everyone,
I recently inadvertently opened a much larger can of worms than I'd
intended when fixing a bug reported downstream where cURL would,
when configured with certain DNS backends, fail to resolve .onion
addresses.
https://bugs.gentoo.org/887287
After doing some digging I discovered that the c-ares library was
updated in 2018 to intentionally fail to resolve .onion addresses
in line with RFC 7686, and another reported 'Bug' in cURL for
leaking .onion DNS requests:
https://github.com/…
[View More]c-ares/c-ares/issues/196https://github.com/curl/curl/issues/543
I took the obviously sane and uncontroversial approach of making
sure that cURL would always behave the same way regardless of the
DNS backend in use, and that it would output a useful error message
when it failed to resolve a .onion address.
Unfortunately, this has made a lot of people very angry and been
~~widely regarded as a bad move~~panned by a small subset of
downstream cURL users:
https://github.com/curl/curl/discussions/11125https://infosec.exchange/@harrysintonen/110977751446379372https://gitlab.torproject.org/tpo/core/torspec/-/issues/202
I accept that, in particular, transproxy users are being inconvenienced,
but I also don't want to go back to 'cURL leaks .onion DNS requests
_sometimes_'. As a career sysadmin and downstream bug triager: this
is the stuff that keeps me up late at night. Quite literally, far too
often.
I have found, however that the downstreams that I expected to be
inconvenienced
most (Whonix and Tails) simply use socks:
https://github.com/Kicksecure/sdwdate/commit/5724d83b258a469b7a9a7bbc651539…https://github.com/Kicksecure/tb-updater/commit/d040c12085a527f4d39cb1751f2…https://github.com/Kicksecure/usability-misc/blob/8f722bbbc7b7f2f3a35619a5a…https://gitlab.tails.boum.org/tails/tails/-/issues/19488https://gitlab.tails.boum.org/tails/tails/-/merge_requests/1123
I've asked in the Tor Specifications issue (inspired by Silvio's
suggestions), and in the cURL issue, but I seem to be getting nowhere
and the impacted users are clamouring for a quick band-aid solution,
which I feel will work out worse for everyone in the long run:
>How can client applications (safely):
>
>1.discover that they're in a Tor-enabled environment
>2.resolve onion services only via Tor in that circumstance
>3.not leak .onion resolution attempts at all
>
>Right now, not making these requests in the first place is the
>safest (and correct) thing to do, however inconvenient it may be.
>Rather than immediately trying to come up with a band-aid approach
>to this problem, a sane mechanism needs to be implemented to:
>
>1.prevent each application from coming up with their own solution
>2.prevent inconsistency in .onion resolution (i.e. no "oh it only
>leaks if DO_ONION_RESOLUTION is set")
>3.provide a standardised mechanism for applications that want to be Tor
>aware to discover that they're in a Tor-enabled environment.
I'm not particularly attached to that last point, but it's worth discussing.
On a related note:
-is the use of a transparent proxy recommended?
-is there a sane alternative that involves as minimal configuration
as possible for these users?
I'm not sure what the best way forward is here, but I'm hoping that
actual Tor developers might have a useful opinion on the matter, or
at least be able to point me in the right direction.
Thanks for your time,
Cheers,
Matt
[View Less]
(You can see this proposal rendered at
https://spec.torproject.org/proposals/351-socks-auth-extensions.html )
```
Filename: 351-socks-auth-extensions.md
Title: Making SOCKS5 authentication extensions extensible
Author: Nick Mathewson
Created: 9 September 2024
Status: Open
```
## Introduction
Currently, Tor implementations use
the SOCKS5 username and password fields
to pass parameters for stream isolation.
(See the `IsolateSocksAuth` flag in the C tor manual,
and the "Stream isolation" …
[View More]section
([forthcoming](https://gitlab.torproject.org/tpo/core/torspec/-/merge_request…)
in our [socks extensions](../socks-extensions.md) spec.)
> Tor implementations also support SOCKS4 and SOCKS4a,
> but they are not affected by this proposal.
>
> The C Tor implementation also supports other proxy types besides SOCKS.
> They are not affected by this proposal
> because they either have other means to extend their protocols
> (as with HTTP headers in HTTP CONNECT)
> or no means to pass extension information
> (as for DNS proxies, iptables transparent proxies, etc).
Until now, the rules for interpreting these fields have been simple:
all values are permitted,
and streams with unequal values may not share a circuit.
But in order to integrate SOCKS connections into Arti's RPC protocol,
we additionally want the ability to send RPC "Object IDs"[^ObjectId]
in these fields.
To do this, we will need some way to tell
when we have received an object ID,
when we have received an isolation parameter,
and to avoid confusing them with one another.
> Note that some confusion will necessarily remain possible:
> Since current Tor clients are allowed to send any value
> as SOCKS username and password,
> any value we specify here will be one which a client in principle
> _might_ have sent under the old protocol.
Additionally,
since we are adding complexity to the interpretation of these fields,
it's possible we'll want to change this complexity in the future.
To do this, we'll want a versioning scheme to premit changes.
## Proposal
> If accepted, the following can be incorporated into
> our [socks extensions](../socks-extensions.md) spec.)
We support a series of extensions in SOCKS5 Username/Passwords.
Currently,
these extensions can encode a stream isolation parameter
(used to indicate that streams may share a circuit)
and an RPC object ID
(used to associate the stream with an entity in an RPC session).
These extensions are in use whenever the SOCKS5 Username
begins with the 8-byte "magic" sequence `[3c 74 6f 72 53 30 58 3e]`.
(This is the ASCII encoding of `<torS0X>`).
If the SOCKS5 Username/Password fields are present
but the Username does not begin with this byte sequence,
it indicates _legacy isolation_.
New client implementations SHOULD NOT use legacy isolation.
A SocksPort may be configured to reject legacy isolation.
When these extensions are in use,
the next byte of the username after the "magic" sequence
indicate a version number.
Any implementation receiving an unrecognized or missing version
MUST reject the socks request.
When the version number is `[30]` (the ascii encoding of `0`),
we interpret the rest of the Username field and the Password field
as follows:
The remainder of the Username field encodes an RPC Object ID.
(If the remainder of the Username field is empty, there is no RPC object.)
The Password field is stream isolation parameter.
If it is empty, the stream isolation parameter is an empty string.
### Stream isolation
> This replaces the corresponding part of
> the "Stream isolation" section
> ([forthcoming](https://gitlab.torproject.org/tpo/core/torspec/-/merge_request…)
> in our [socks extensions](../socks-extensions.md) spec.
Two streams are considered to have the same SOCKS authentication values
if and only if one of the following is true:
- They are both SOCKS4 or SOCKS4a, with the same user "ID" string.
- They are both SOCKS5, with no authentication.
- They are both SOCKS5 with USERNAME/PASSWORD authentication,
using legacy isolation parameters,
and they have identical usernames
and identical passwords.
- They are both SOCKS5 using the extensions above,
with the same stream isolation parameter.
### A further extension for integration with Arti SOCKS
> We should add the following to a specification,
> though it is not clear whether it goes in the Arti RPC spec
> or in the socks extensions spec.
In some cases,
the RPC Object ID may denote an object
that already includes information about its intended stream isolation.
In such cases, the stream isolation MUST be blank.
Implementations MUST reject non-blank stream isolation in such cases.
In some cases, the RPC object ID may denote an object
that already includes information
about its intended destination address and port.
In such cases, the destination address MUST be `0.0.0.0` or `::`
(encoded either as an IPv4 address, an IPv6 address, or a hostname)
and the destination port MUST be 0.
Implementations MUST reject other addresses in such cases.
-----
> (Here the specifications end.
> The rest of this proposal is discussion.)
## Design considerations
Our use of SOCKS5 Username/Passwords here
(as opposed to some other, new authentication type)
is based on the observation
that many existing SOCKS5 implementations support Username/Password,
but comparatively few support arbitrary plug-in authentication.
The magic "`<torS0X>`" prefix is chosen to be 8 characters long
so that existing client implementations that generate random strings
will not often generate it by mistake.
The version number is chosen to be an ASCII `0` rather than a raw 0 byte,
for compatibility with existing SOCKS5 client implementations
that do not support non-ASCII username/password values.
## C Tor migration
When this proposal is accepted,
we *should* configure C tor to implement it as follows:
- To reject any SOCKS5 Username starting with `<torS0X>`
unless it is exactly `<torS0X>0`.
This behavior is sufficient to give correct isolation behavior,
to reject any connection including an RPC object ID,
and to reject any as-yet-unspecified isolation mechanisms.
[^ObjectId]: An ObjectId is used in the Arti RPC protocol
to associate a SOCKS request with some existing Client object,
or with a preexisting DataStream.
[View Less]
tldr;
We're planning on disabling HTTP-header user agent spoofing on the basis
that it (in our analysis) does little if any good while causing breakage
on the internet when the JS and HTTP user agents do not match. However,
if there is something critical we're missing here, we'd certainly love
to hear. In the meantime it seems like an easy win for some improved
usability.
best,
-morgan
Excerpt from 14.0a4's blog post follows:
> User Agent Spoofing Changes
>
> Historically, …
[View More]Tor Browser has spoofed the browser user agent found in HTTP headers, while not spoofing the user agent returned by the Navigator.userAgent property in JavaScript. The logic behind the HTTP header spoofing was to prevent passive tracking of users' operating system by websites (when using the 'Safest' security level) and by malicious exit nodes (or their upstream routers) passively listening in on unencrypted HTTP traffic. We left the JavaScript query intact for the purposes of website compatibility and usability. We also left it enabled because there are already many ways of detecting a user's real operating system when JavaScript is enabled (e.g. via font enumeration).
>
> With Tor Browser 14.0a4, we have introduced the boolean preference privacy.resistFingerprinting.spoofOsInUserAgentHeader. When this pref is set to true (which is currently the default), Tor Browser will follow the previously described legacy behaviour. However, if you set this preference (accessible in about:config) to false, Tor Browser will never spoof the user agent and will report your operating system family (i.e. Windows, macOS, Linux, or Android) when requested. We are considering changing Tor Browser to make this the new default behaviour.
>
> So, why are we considering making this change? Basically, asymmetrically spoofing the user agent causes website breakage seemingly due to bot-detection scripts. And (in our analysis) it also provides only a negligible amount of benefit to the user in terms of additional linkability (i.e. cross-site tracking, fingerprinting) protections, and only then when JavaScript is disabled. Tor Browser's default HTTPS-Only mode (and much of the web having moved to HTTPS) has also significantly reduced the utility of passively sniffing HTTP traffic for user agents as well.
>
> We would be very curious to hear from users and domain experts as to whether user agent spoofing is providing any other privacy benefits. In the meantime, disabling spoofing is available to users on an opt-in basis. For more information and to join the conversation, please see the Gitlab ticket tor-browser#42467.
[View Less]