Per-peer stream isolation for Bitcoin clients
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Tor-Dev, I'm trying to gauge the consensus (or lack thereof) in the Tor development community on whether it's desirable for Bitcoin clients (e.g. Bitcoin Core) to use stream isolation such that each peer is accessed over a different circuit. Some random thoughts on the matter: 1. Bitcoin Core accesses 8 peers by default, so per-peer stream isolation would use 8 circuits instead of 1. 2. Per-peer stream isolation prevents a single exit relay from feeding the user a chain that's not the longest chain, so it's desirable from a Bitcoin security point of view. 3. Per-peer stream isolation would mean more potential for one of the circuits being deanonymizable, via traffic analysis etc. It's not clear to me whether this amount of increased circuits is harmful, or how it compares to other common usage of Tor such as Tor Browser (which uses first-party stream isolation, so a user with a lot of tabs open may very well have 8 or more circuits in use at once). 4. Per-peer stream isolation puts more load on the Tor network. It's not clear to me whether this increased load (8 circuits instead of 1) is so much that it's harmful. 5. Bitcoin Core does do per-peer stream isolation by default. The relevant PR is https://github.com/bitcoin/bitcoin/pull/5911 6. Whonix's tor-service-defaults-torrc chooses to disable automatic per-peer stream isolation for Bitcoin's SOCKS port, and states "Makes too many connections to different servers. Should not hurt if they get through the same circuit." No citation was given for either claim. 7. The behavior in Bitcoin Core's PR was ACKed by Isis Lovecruft, and an unspecified Tor developer whom Greg Maxwell talked to. So, it sounds like there is apparently some disagreement between the two Tor devs who ACKed this behavior, and the Whonix devs who decided not to enable it. Curious what the general feeling in the community is. (I understand that Isis no longer is active in Tor, so I'm not CC'ing them. I am CC'ing Patrick from Whonix in case he wants to weigh in.) Cheers, - -- - -Jeremy Rand Lead Application Engineer at Namecoin Mobile email: jeremyrandmobile@airmail.cc Mobile OpenPGP: 2158 0643 C13B B40F B0FD 5854 B007 A32D AB44 3D9C Send non-security-critical things to my Mobile with OpenPGP. Please don't send me unencrypted messages. My business email jeremy@veclabs.net is having technical issues at the moment. -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEVB3fdzAraEcoBtkSs/LRZXhtZXAFAl0VKAIACgkQs/LRZXht ZXD3rRAAydyiDv2NVU8rPopr+FkTVclNAA3gM3BKjhZH1EGqya54Xr0OdsTR3VUQ gBMz5HiexbFnE7VTysnB6ePP/PpLRlAlTj1zLXFCT90H+s1kj84WpjxNZJaguLIJ zGOQKhqgId9f1pjv7or8DYDrdzT0bfOzKREFM+Yx5TTgPfvxXC9Ug6U1sx5sp7AA jCAZnrjqWSK+IrNnwrayL2vSPByt+djsK2YPLnq13frKMdonMw+IITVOvT9w5LZt I/MQa2UujG5DU0owI0t5XmJ7ZQxehcHdAuqU/BfZ5OQVjCH3QgVNwRjlGPaaag+f y9wMVG0X4cbuvg5x8sar/n8UCR1sDJmlSuN7C6AwE2/e0RU0g67wfTgNcqFlIHta 9t8ORsFUxN6rDtRkbq0FE7xgSYbT9cDtjDFJMhG8oOc0Ibc9EhIJkGETz1amHNUo GFEBAEircjPJ+KZBsHsQGrLUwpiM6zPbPfkVWwA7dTn4peESSed0Yjwawcn2wpiE X2TMzhhXpzTjtqpBcroSDKRMum5z/VksEZ3d2MyfIj/caYVXXxPZYS1DqOGtKxYc uiDCRpL4b/ouJ1FbT1sxHpwTp5e+62XFnfXsoz3Wa9jkrA75kgnnb2kfcjNW/Fq/ 6HgLhasriz4jg2iya9rScuUnmU1Ewa8st0bNDGK83qYHofCdvCg= =1u2L -----END PGP SIGNATURE-----
Hi Jeremy,
On 28 Jun 2019, at 06:35, Jeremy Rand <jeremyrand@airmail.cc> wrote:
2. Per-peer stream isolation prevents a single exit relay from feeding the user a chain that's not the longest chain, so it's desirable from a Bitcoin security point of view.
Tor itself uses 3 directory guards to make sure that one guard can't feed it bad data. If you use one exit, and it happens to be malicious, all your peers are accessed via that bad exit. But if you use 8 exits, then it's very unlikely that all of them will be malicious in exactly the same way. But it makes it 8x more likely that any one of your exits will be malicious. It's also worth using multiple circuits for redundancy, so a single exit can't deny service all 8 of your peers. (As noted on the pull request.) So it's a tradeoff, based on the protocol. For example: * If you were mainly worried about exploits, then you'd want one exit. * If you are worried about short chains or denial of service, then you'd want many exits. If you don't know which one to choose, follow Tor Browser, which uses many exits. (The right way to deal with exploits is to fix those bugs, and use coding practices that make them less likely.)
3. Per-peer stream isolation would mean more potential for one of the circuits being deanonymizable, via traffic analysis etc. It's not clear to me whether this amount of increased circuits is harmful, or how it compares to other common usage of Tor such as Tor Browser (which uses first-party stream isolation, so a user with a lot of tabs open may very well have 8 or more circuits in use at once).
It's not quite that simple: If you send traffic from all 8 peers over one circuit, then a pattern might be visible on you client to guard, and exit to internet connections. That pattern makes correlation easier. Using 8 circuits splits the traffic pattern across 8 different exit locations on the internet, so correlation is harder. I'd follow Tor Browser on this one, too.
4. Per-peer stream isolation puts more load on the Tor network. It's not clear to me whether this increased load (8 circuits instead of 1) is so much that it's harmful.
Every Tor client tries to keep 6+ pre-emptive circuits open, so that new requests have lower latency. So you're really looking at doubling your circuit load, not multiplying it by 8. Once the circuits are built, there's not much extra load splitting the streams across 8 circuits, because it's already split into streams and cells within the protocol. And you're right, Tor Browser can use lots more than 8 circuits, so I wouldn't worry about it. Do you know how much load Bitcoin places on the Tor network? If it's a lot, one good answer is to encourage users to run relays, or to donate to organisations that run relays. (Or donate to Tor, so we can make the network more efficient.) T
On Fri, Jun 28, 2019 at 07:53:54AM +1000, teor wrote:
And you're right, Tor Browser can use lots more than 8 circuits, so I wouldn't worry about it.
Do you know how much load Bitcoin places on the Tor network?
If it's a lot, one good answer is to encourage users to run relays, or to donate to organisations that run relays. (Or donate to Tor, so we can make the network more efficient.)
Right -- my first question would be "8 circuits per what?" That is, how often does it use eight new circuits? If it makes 8 circuits and then holds them open and uses each of them for a long period of time, that sounds like a solid win -- you get isolation between streams with little downside. If we're talking 8 circuits per new gadget, and the new gadgets are pretty frequent, then the tradeoff becomes more complicated. --Roger
Roger Dingledine wrote:
On Fri, Jun 28, 2019 at 07:53:54AM +1000, teor wrote:
And you're right, Tor Browser can use lots more than 8 circuits, so I wouldn't worry about it.
Do you know how much load Bitcoin places on the Tor network?
If it's a lot, one good answer is to encourage users to run relays, or to donate to organisations that run relays. (Or donate to Tor, so we can make the network more efficient.)
Right -- my first question would be "8 circuits per what?" That is, how often does it use eight new circuits?
If it makes 8 circuits and then holds them open and uses each of them for a long period of time, that sounds like a solid win -- you get isolation between streams with little downside.
If we're talking 8 circuits per new gadget, and the new gadgets are pretty frequent, then the tradeoff becomes more complicated.
--Roger
It would hold on for the circuits until the peers on the other side of each circuit disconnect or disappear. Usually Bitcoin full nodes are steady and kept up (no recent disconnects of any sort, unless the peer is unreliable). But this is not the proper way to use Bitcoin behind Tor. So stream isolation for clearnet type circuits shouldn't even be a concern. Whonix's tor-service-defaults-torrc chooses to disable automatic per-peer stream isolation for Bitcoin's SOCKS port and I think it does the right thing, because this is not how Bitcoin should be used behind Tor. Jeremy, when Bitcoin (Core) is used with Tor, the proper and recommended way is to set in bitcoin.conf `onion=127.0.0.1:9050` (substitute with the SocksPort of your system's Tor instance). This will teach Bitcoin that we are behind Tor, so it should prefer .onion peers instead, this way you won't be dependent on clearnet type circuits, and one (or 8) exit nodes seeing all your peers or tampering with them. Bitcoin has 3 peer families: -IPv4 -IPv6 -Tor (onion) There is even a bitcoin option to make it ONLY connect to .onion peers: `onlynet=tor`. This will eliminate Exit nodes out of the equation entirely. There are lots of .onion peers. I run 3 (but there are many more) "hybrid bridges", like nodes open to all 3 peer families so that there .onion peers and clearnet peers are very well connected and synced and the effect of "isolation/island" is not created. It's of course desirable to prefer .onion peers while behind Tor. Otherwise the peers will see one Exit node IP address as 'too many peers' and give it bad score, as Bitcoin keeps a score for the reliability overall of each peer, so you can understand it's quite problematic for many "different peers" to connect to a peer with same IP address. Of course when more peers connect to your .onion, you still technically see one remote IP address (127.0.0.1) but at least this is coded as Tor and behaves differently than the score system for clearnet IP addresses. Also .onion traffic is end-to-end encrypted and self-authenticated so you eliminate the MITM attack type (given Bitcoin peer to peer traffic is not encrypted). You are not forced to also listen on a .onion if you use `onlynet=tor`, you can set `listenonion=0` -- you can play with it how you want: connect to one or more families (any combination) and listen to one or more families (any combination) or don't listen at all. There's also an option where you can set the hostname like `externalip=<hostname>.onion`, etc. (re. teor): The Bitcoin traffic in Tor network / onion land is not negligible. Look at just one of my bridge nodes: Jun 27 17:05:51.000 [notice] Heartbeat: Tor's uptime is 15 days 6:00 hours, with 34 circuits open. I've sent 19.65 GB and received 5.70 GB. There is only Bitcoin traffic here because there's nothing else. As you can see we sent more than 3x what we received, meaning we helped more nodes to bootstrap the blockchain (new started nodes or nodes that are not kept on / connected 24x24). So yeah, it would be nice to encourage the community towards running or sponsoring relays to assist in maintaining a good capacity and diversity of the Tor network, it is very important and widely used as you can see. Also there is a ticket open by me: https://github.com/bitcoin/bitcoin/issues/9214 to support v3 onion address types. Currently Bitcoin Core only supports v2 legacy onion addresses, which are heavier on the network because use TAP handshake and etc. v3 onion new address types are superior of course, so getting this fixed will decrease the load on the Tor network and increase the efficiency in Bitcoin onionland. Better work on this.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Thanks everyone for the excellent feedback, that was very helpful in understanding the issues at play. s7r:
But this is not the proper way to use Bitcoin behind Tor. So stream isolation for clearnet type circuits shouldn't even be a concern. Whonix's tor-service-defaults-torrc chooses to disable automatic per-peer stream isolation for Bitcoin's SOCKS port and I think it does the right thing, because this is not how Bitcoin should be used behind Tor.
Yes, I'm aware that Bitcoin Core supports stream isolation without relying on a torrc setting. Even if Whonix is doing the right thing here, the comments in Whonix's file suggest that they're doing it for the wrong reason. It should also be noted that not all Bitcoin clients do what Bitcoin Core does (and in fact part of the motivation for my inquiry was to determine if I should be submitting patches to those clients to make them mimic what Bitcoin Core does). Using a torrc setting would probably provide some useful defense-in-depth in case a Bitcoin client isn't doing stream isolation on its own. Cheers, - -- - -Jeremy Rand Lead Application Engineer at Namecoin Mobile email: jeremyrandmobile@airmail.cc Mobile OpenPGP: 2158 0643 C13B B40F B0FD 5854 B007 A32D AB44 3D9C Send non-security-critical things to my Mobile with OpenPGP. Please don't send me unencrypted messages. My business email jeremy@veclabs.net is having technical issues at the moment. -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEVB3fdzAraEcoBtkSs/LRZXhtZXAFAl0bYTcACgkQs/LRZXht ZXCSeg//Q9HztuCzK8iVNeu193xr711MquKabRUH/d08NRHTFW7Ltuwns31jGPel JPcX0e01u/2yT7Rzzc2Ryx5JcGHv6kOjOmY2OD3QHnVUHVhZBEyvQtlvYw2SKKli 8nEUmZotLOxv2X3/tc7RgNCHn0982F1BcJRuZHd8KpswV0tcG801HzkZz+W5NJ/4 WQO6VNnYOlv4ARddjVuxWhkqkHACmnQJ/z7nKqO7O2b54gWy7FEPhfWjHsbHPlo8 MaXwQdunOYtkYJ/afQA4bLb7Rmg50xjRzoO1oWK68yKqqX2hOe1dV+WfGak63eR/ eY3xjM6gLqR8jxdtW491BgatAh0hNr7q/V55VEXoMq1g/QL2JW43LhCfJI0SuT+h K3LVnX9iuq+MIMEHuJ1eVM7rWVO5g2thpFfvnCR82faCUKZa6VRoS/4r1XGqU0rW 1pg1b67Cu4iVVmKdSmlOHDNhk1yZOY5HU3bpM4BeZi/1dqvZG+yOMtVgYGVPl1ma iXsfMcbTWIXCEAobAanbgG6NEgiUQUCm39yp+mqQErd4VkRc79kCHII4VznjuoZ7 jLpRT28OA6Xgzvby+AfFEXX/Mot2BwX33aoQX4nKk+aWmg1CG6xtbfjLKgRhNsMx Goca1Cbc8zqsBZzCjOxgDWFSDrj9Yxg5+cn5AG4kAN2M5aVEw6A= =pE+P -----END PGP SIGNATURE-----
participants (4)
-
Jeremy Rand -
Roger Dingledine -
s7r -
teor