- tor-dev - lists.torproject.org

Fwd: Re: Onion Service - Intropoint DoS Defenses
by juanjo 08 Jul '19

08 Jul '19

-------- Forwarded Message -------- Subject: Re: [tor-dev] Onion Service - Intropoint DoS Defenses Date: Thu, 4 Jul 2019 20:38:48 +0200 From: juanjo <juanjo(a)avanix.es> To: David Goulet <dgoulet(a)torproject.org> These experiments and final note confirm what I thought about this rate limiting feature from the start: it is missing important parts. Ok, you can protect the network a little and the HS, but the general availability is not affected so it actually does not help for that. I wanna make a proposal including many things at the same time, but I don't have much time to follow the guidelines to make a official proposal. Maybe in some weeks? Again, I repeat: things that should be done now: -Authenticated rend signature. This would help a lot I think. -Mid-term: PoW for the client when reaching the 305prop limit instead of denying access? IDK, all always configurable. -Deprecate clients or allow the Hidden Service to configure the IP to allow access for old version clients (not supporting new antiDoS features) or not. If we allow old version without protections, all security measures are useless. And just a new idea: what about make the rotation of IP dynamic based on this prop305 values? + time based rotation: One of the goal for rotation was defending against correlation attacks: if we set a lower limit we have a potential DoS (right now), if we set it high we have a potential correlation attack, bigger surface. What about we join time based rotation (ex. 24 hours) + or limit reached based on the prop305 values. On 3/7/19 20:37, David Goulet wrote: > On 30 May (09:49:26), David Goulet wrote: >> Greetings! > [snip] > > Hi everyone, > > I'm writing here to update on where we are about the introduction rate > limiting at the intro point feature. > > The branch of #15516 (https://trac.torproject.org/15516) is ready to be merged > upstream which implements a simple rate/burst combo for controlling the amount > of INTRODUCE2 cells that are relayed to the service. > > As previously detailed in this thread, the default values are a rate of 25 > introduction per second and a burst of 200 per second. These values can be > controlled by consensus parameters meaning they can be changed network wide. > > We've first asked big service operators, I'm not going to detail the values > they provided us in private, but those defaults are quite large enough to > sustain heavy traffic from what we can tell from what they gave us. > > The second thing we did is do experimental testing to see how CPU usage and > availability is affected. We've tested this with 3 _fast_ introduction points > and then 3 rate limited introduction points. > > The good news is that once the attack stops, the rain of introduction requests > to the service stops very quickly. > > With the default rate/burst values, on a Intel(R) Xeon(R) CPU E5-2650 v4 @ > 2.20GHz (8 cores), the tor service CPU doesn't go above ~60% (on one single > core). And almost drops to 0 as soon as the attack ends. > > The bad news is that availability is _not_ improved. One of the big reasons > for that is because the rate limit defenses, once engaged at the intro point, > will send back a NACK to the client. A vanilla tor client will stop using that > introduction point away for 120 seconds if it gets 3 NACKs from it. This leads > to tor quickly giving up on trying to connect and thus telling the client that > connection is impossible to the .onion. > > We've hacked a tor client to play along and stop ignoring the NACKs to see how > much time it would take to reach it. On average, a client would roughly need > around 70 seconds with more than 40 NACKs on average. > > However, it varied a _lot_ during our experiments with many outliers from 8 > seconds with 1 NACK up to 160 seconds with 88 NACKs. (For this, the > SocksTimeout had to be bumped quite a bit). > > There is an avenue of improvement here to make the intro point sends a > specific NACK reason (like "Under heavy load" or ...) which would make the > client consider it like "I should retry soon-ish" and thus making the client > possibly able to connect after many seconds (or until the SocksTimeout). > > Another bad news there! We can't do that anytime soon because of this bug that > basically crash clients if an unknown status code is sent back (that is a new > NACK value):https://trac.torproject.org/30454. So yeah... quite unfortunate > there but also a superb reason for everyone out there to upgrade :). > > One good news is that it seems that having fast intro points instead of slow > IPs doesn't change much on the overall load on the service so this for now, > our experiment, shows it doesn't matter. > > Overall, this rate limit feature does two things: > > 1. Reduce the overall network load. > > Soaking the introduction requests at the intro point helps avoid the > service creating pointless rendezvous circuits which makes it "less" of an > amplification attack. > > 2. Keep the service usable. > > The tor daemon doesn't go in massive CPU load and thus can be actually used > properly during the attack. > > The problem with (2) is the availability part where for a legit client to > reach the service, it is close to impossible for a vanilla tor without lots of > luck. However, if let say the tor daemon would be configured with 2 .onion > where one is public and the other one is private with client authorization, > then the second .onion would be totally usable due to the tor daemon not being > CPU overloaded. > > As a third thing we did about this. In order to make this feature a bit more > "malleable", we are working onhttps://trac.torproject.org/30924 which is > proposal 305. > > In short, torrc options are added so an operator can change the rate/burst > that the intro points will use. We can do that using the ESTABLISH_INTRO cell > that will have an extension to define the DoS defensavailability > e parameters (proposal > 305). > > That way, a service operator can disable this feature, or turn the knobs on > the rate/burst in order to basically adjust the defenses. > > At this point in time, we don't have a good grasp on what happens in terms of > CPU if the rate or the burst is bumped up or even how availability is > affected. During our experimentation, we did observed a "sort of" linear > progression between CPU usage and rate. But we barely touched the surface > since it was changed from 25 to 50 to 75 and that is it. > > We would require much more experimentation which is something we want to avoid > as much as possible on the real network. > > Finally, many more changes are cooking up. One in particular is > https://trac.torproject.org/projects/tor/ticket/26294 that will make tor to > only rotate its intro points when the number of introduction requests is > between 150k to 300k (random value) which currently is between 16k and 32k. > See the ticket for the benefits here which mostly helps with (1). > > There has been much talk about a client PoW (see the proposal 305 thread on > this list) which in theory would help out with service availability. > > We will also soon merge upstream this tickethttps://trac.torproject.org/24962 > which goes one step further at denying single-hop connections to the > HSDir/Intro in order to try as much as possible to shutdown the Tor2web > connections (or any attacker that speeds things up on their side by single > hoping). > > We are making progress here... This is really a non trivial problem and > solution for service availability are not that simple. Our priority is to > protect the network as much as possible and then move to possible solutions > for availability. > > I'll stop for now. Huge thanks to everyone who provided service logs, ideas, > code review and future testers :). > > Cheers! > David > > > _______________________________________________ > tor-dev mailing list > tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

2 1

Onion Service - Intropoint DoS Defenses
by David Goulet 05 Jul '19

05 Jul '19

Greetings! As some of you know, a bunch of onion services were or are still under heavy DDoS on the network. More specifically, they are bombarded with introduction requests (INTRODUCE2 cells) which forces them to rendezvous for each of them by creating a ton of circuits. This basically leads to a resource exhaustion attack on the service side with its CPU massively used for path selection, opening new circuits and continously handling INTRODUCE2 cells. Unfortunately, our circuit-level flow control does not apply to the service introduction circuit which means that the intro point is allowed, by the Tor protocol, to send an arbitrary large amount of cells down the circuit. This means for the service that even after the DoS has stopped, it would still receive massive amounts of cells because some are either inflight on the circuit or queued at the intro point ready to be sent (towards the service). That being all said, our short-term goal here is to add INTRODUCE2 rate-limiting (similar to the Guard DoS subsystem deployed early last year) *at* the intro point but much simpler. The goal is to soak up the introduction load directly at the intro points which would help reduce the load on the network overall and thus preserve its health. Please have a look at https://trac.torproject.org/15516 for some discussions and ongoing code work. We are at the point where we have a branch that rate limits INTRODUCE2 cells at the intro point but we need to figure out proper values for the rate per second and the burst allowed. One naive approach is to see how much cells an attack can send towards a service. George and I have conducted experiment where with 10 *modified* tor clients bombarding a service at a much faster rate than 1 per-second (what vanilla tor does if asked to connect a lot), we see in 1 minute ~15000 INTRODUCE2 cells at the service. This varies in the thousands depending on different factors but overall that is a good average of our experiment. This means that 15000/60 = 250 cells per second. Considering that this is an absurd amount of INTRODUCE2 cells (maybe?), we can put a rate per second of let say a fifth meaning 50 and a burst of 200. Over the normal 3 intro points a service has, it means 150 introduction per-second are allowed with a burst of 600 in total. Or in other words, 150 clients can reach the service every second up to a burst of 600 at once. This probably will ring alarms bell for very popular services that probably gets 1000+ users a second so please check next section. I'm not that excited about hardcoded network wide values so this is why the next section is more exciting but much more work for us! One step further: we have not really decided yet if this is something we want nor have time to tackle but an idea here would be for a service to inform the intro point, using the ESTABLISH_INTRO cell payload, on the parameters it wants for its DoS defenses. So let say a very popular .onion with OnionBalance and 10 intro points, could tell to its intro points that it wants much higher values for the DoS defenses (or even put it off). However, it doesn't change the initial building block of being able to rate limit at the introduction point. As a second step, we can add this new type of ESTABLISH_INTRO cell. It is always dicy to introduce a new cell since for instance this would leak information to the intro point that the service is ">= version". Thus, this needs to be done with carefully. Time for your thoughts and help! :) Thanks everyone! David -- g6fXJTHx8+aXYB1ZLPzFft/VxRukF1Td5nborX04XM0=

6 15

Proposal 303: When and how to remove support for protocol versions
by Nick Mathewson 05 Jul '19

05 Jul '19

Filename: 303-protover-removal-policy.txt Title: When and how to remove support for protocol versions Author: Nick Mathewson Created: 21 May 2019 Status: Draft 1. Background With proposal 264, added support for "subprotocol versions" -- a means to declare which features are required for participation in the Tor network. We also created a mechanism (refined later in proposal 297) for telling Tor clients and relays that they cannot participate effectively in the Tor network, and they need to shut down. In this document, we describe a policy according to which these decisions should be made in practice. 2. Recommending features (for clients and relays) A subprotocol version SHOULD become recommended soon after all release series that did not provide it become unsupported (within a month or so). For example, the current oldest LTS release series is 0.2.9; when it becomes unsupported in 2020, the oldest supported release series will be 0.3.5. Suppose that 0.2.9 supports a subprotocol Cupcake=1, and that all stable 0.3.5.x versions support Cupcake=1-3. Around one month after the end of 0.2.9 support, Cupcake=3 should become a _recommended_ protocol for clients and relays. Additionally, a feature can become _recommended_ because of security reasons. If we believe that it is a terrible idea to run an old protocol, we can make it _recommended_ for relays or clients or both. We should not do this lightly, since it will be annoying. 3. Requiring features (for relays) We regularly update the directory authorities to require relays to run certain versions of Tor or later. We generally do this after a short outreach campaign to get as many relays as possible to upgrade. We MAY make a feature required for relays one month after every version without it is obsolete and unsupported, though it is better to wait three months if possible. We SHOULD make a feature required for relays within 12 months after every version without it is obsolete and unsupported. 4. Requiring features (for clients) Clients take the longest time to update, and are often the least able to fetch upgrades. Because of this, we should be very careful about making subprotocol versions required on clients, and should only do so for fairly compelling reasons. We SHOULD NOT make a feature required for clients until it has been _recommended_ for clients for at first 9 months. We SHOULD make a feature required for clients if it has been _recommended_ for clients for at least 18 months.

4 3

Per-peer stream isolation for Bitcoin clients
by Jeremy Rand 02 Jul '19

02 Jul '19

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Tor-Dev, I'm trying to gauge the consensus (or lack thereof) in the Tor development community on whether it's desirable for Bitcoin clients (e.g. Bitcoin Core) to use stream isolation such that each peer is accessed over a different circuit. Some random thoughts on the matter: 1. Bitcoin Core accesses 8 peers by default, so per-peer stream isolation would use 8 circuits instead of 1. 2. Per-peer stream isolation prevents a single exit relay from feeding the user a chain that's not the longest chain, so it's desirable from a Bitcoin security point of view. 3. Per-peer stream isolation would mean more potential for one of the circuits being deanonymizable, via traffic analysis etc. It's not clear to me whether this amount of increased circuits is harmful, or how it compares to other common usage of Tor such as Tor Browser (which uses first-party stream isolation, so a user with a lot of tabs open may very well have 8 or more circuits in use at once). 4. Per-peer stream isolation puts more load on the Tor network. It's not clear to me whether this increased load (8 circuits instead of 1) is so much that it's harmful. 5. Bitcoin Core does do per-peer stream isolation by default. The relevant PR is https://github.com/bitcoin/bitcoin/pull/5911 6. Whonix's tor-service-defaults-torrc chooses to disable automatic per-peer stream isolation for Bitcoin's SOCKS port, and states "Makes too many connections to different servers. Should not hurt if they get through the same circuit." No citation was given for either claim. 7. The behavior in Bitcoin Core's PR was ACKed by Isis Lovecruft, and an unspecified Tor developer whom Greg Maxwell talked to. So, it sounds like there is apparently some disagreement between the two Tor devs who ACKed this behavior, and the Whonix devs who decided not to enable it. Curious what the general feeling in the community is. (I understand that Isis no longer is active in Tor, so I'm not CC'ing them. I am CC'ing Patrick from Whonix in case he wants to weigh in.) Cheers, - -- - -Jeremy Rand Lead Application Engineer at Namecoin Mobile email: jeremyrandmobile(a)airmail.cc Mobile OpenPGP: 2158 0643 C13B B40F B0FD 5854 B007 A32D AB44 3D9C Send non-security-critical things to my Mobile with OpenPGP. Please don't send me unencrypted messages. My business email jeremy(a)veclabs.net is having technical issues at the moment. -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEVB3fdzAraEcoBtkSs/LRZXhtZXAFAl0VKAIACgkQs/LRZXht ZXD3rRAAydyiDv2NVU8rPopr+FkTVclNAA3gM3BKjhZH1EGqya54Xr0OdsTR3VUQ gBMz5HiexbFnE7VTysnB6ePP/PpLRlAlTj1zLXFCT90H+s1kj84WpjxNZJaguLIJ zGOQKhqgId9f1pjv7or8DYDrdzT0bfOzKREFM+Yx5TTgPfvxXC9Ug6U1sx5sp7AA jCAZnrjqWSK+IrNnwrayL2vSPByt+djsK2YPLnq13frKMdonMw+IITVOvT9w5LZt I/MQa2UujG5DU0owI0t5XmJ7ZQxehcHdAuqU/BfZ5OQVjCH3QgVNwRjlGPaaag+f y9wMVG0X4cbuvg5x8sar/n8UCR1sDJmlSuN7C6AwE2/e0RU0g67wfTgNcqFlIHta 9t8ORsFUxN6rDtRkbq0FE7xgSYbT9cDtjDFJMhG8oOc0Ibc9EhIJkGETz1amHNUo GFEBAEircjPJ+KZBsHsQGrLUwpiM6zPbPfkVWwA7dTn4peESSed0Yjwawcn2wpiE X2TMzhhXpzTjtqpBcroSDKRMum5z/VksEZ3d2MyfIj/caYVXXxPZYS1DqOGtKxYc uiDCRpL4b/ouJ1FbT1sxHpwTp5e+62XFnfXsoz3Wa9jkrA75kgnnb2kfcjNW/Fq/ 6HgLhasriz4jg2iya9rScuUnmU1Ewa8st0bNDGK83qYHofCdvCg= =1u2L -----END PGP SIGNATURE-----

4 4

Changed Network Team Meeting Times
by teor 01 Jul '19

01 Jul '19

Hi all, We've moved the first network team meeting of each month to Wednesday 2300 UTC. (It used to be Tuesday.) We have also moved the weekly "patch party" to Wednesday 2300 UTC in the other weeks of the month. The regular team meeting time is still the same, Monday 1700 UTC. Here is our meeting schedule for July: * Wednesday 3 July 2300 UTC - Changed Day! * Monday 8 July at 1700 UTC * (In-person meeting 12-14 July) * Monday 22 July 1700 UTC * Monday 29 July 1700 UTC T -- teor ----------------------------------------------------------------------

1 0

tor writen in golang
by liuyihen 30 Jun '19

30 Jun '19

Hi tor-dev@mailing list, I want to know is there somebody has been re-writing tor in golang? or is there a repo have been opened?

2 1

Release: obfs4proxy-0.0.11
by Yawning Angel 20 Jun '19

20 Jun '19

Hello, I just tagged obfs4proxy-0.0.11. The primary changes are an alteration to how the obfs4 transport handles connections after the handshake failed, based on a report and assistance from Sergey Frolov. Tarball/Signature: https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.1… https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.1… Changes in version 0.0.11 - 2019-06-21: - Update my e-mail address. - Change the obfs4 behavior for handling handshake failure to be more uniform. Thanks to Sergey Frolov for assistance. - Bump the version of the utls fork. Regards, -- Yawning Angel

1 0

Proposal 305: ESTABLISH_INTRO Cell DoS Defense Extension
by David Goulet 20 Jun '19

20 Jun '19

9 19

Shortcomings of the pluggable transports specification?
by Philipp Winter 20 Jun '19

20 Jun '19

We are working on improving Tor's pluggable transports specification: <https://spec.torproject.org/pt-spec> The goal is to make the spec useful to more people and fix issues that have accumulated over the years. For more context, have a look at the following ticket, which we use to coordinate this effort: <https://bugs.torproject.org/29285> Before changing the spec, we need to understand its shortcomings and what issues implementers have run into. For those of you who have experience with the spec -- either Tor's version 1.0 or version 2.1 maintained by pluggabletransports.info -- please let us know: * What version of the PT specification and what library implementation (if any) are you using? * What has your experience been with the PT specification? * How would you improve the specification? Thanks, Philipp

3 2

User Data Exchange with Network Automata (re: Prop 305, ex energy)
by grarpamp 17 Jun '19

17 Jun '19

> And this is really off topic for this list. General Tor energy bits moved here... https://lists.torproject.org/pipermail/tor-talk/2019-June/045256.html re 305 etc It wouldn't be unusual for an app to pop up some form of captcha, challenge, or data exchange where needed. Some of that model exists in form of onion service authentication config... some helper data that grants access, makes things happen, whether passive or active interrupt. The controller would be the interface. Yet tor currently has no mechanism on platforms to handle such protocols and popups... thus like onion auth, the user has know they need it for something and configure it in advance. Try investigating an ssh-agent like tor tool... preloaded with users Proof-of-*, 2FA, consumables, etc to dole out, even automatically, where needed and permitted.

1 0