[tor-talk] IDEA: Compress traffic at exit

[Roger Dingledine]

On Fri, Jan 20, 2012 at 03:29:13PM -0500, Andrew Lewman wrote:
> >    The network has a high bandwidth usage due to it massive user-base.
> >    This idea should reduce the outgoing traffic of the exit and take
> >    much load from the mid-relay and the entry point.
> 
> Lack of bandwidth isn't so much the problem. Lack of tcp sockets and cpu
> power for onionskins at the relays is the problem. The higher bandwidth
> the relay, the worse these two problems become. 

I think these are all important problems to tackle. It is certainly the
case that the Tor network doesn't have enough capacity these days to
handle all the users who want to use it. Tor relays that have a huge
amount of capacity start to hit up against CPU and file descriptor
limits. Most Tor relays are bandwidth-limited though.

https://metrics.torproject.org/network.html#bandwidth

I'm told that networking people think of networks as hopelessly congested
when their load passes 10% or 20% of capacity. We have a long way to go
before we can reach that. :)

> Exit relays are already overloaded, adding compression overhead will
> not make this better. 

For huge relays, I tend to agree. For most of the rest, it would help.

Things get complexer though because even if the exit relay has plenty
of spare capacity, that doesn't mean the middle or entry relay in the
circuit does. And the exit relay isn't supposed to know the topology of
the circuit.

> Lots of traffic on the Internet is already compressed between the
> client and server. What tor does in the middle is irrelevant. See the
> SPDY proposal,
> https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/ideas/xxx-using-spdy.txt

Right -- I think it's clearly better for the application level to compress
its traffic, rather than having Tor do it blindly.

> Generally, the tor relays do not want to touch nor modify user traffic
> in any way. Once we can detect "compress or not", it's a quick slide
> down a slippery slope to "censor or not". See
> https://trac.torproject.org/projects/tor/wiki/doc/TorFAQ#YoushouldchangeTortopreventusersfrompostingcertaincontent.

I think blindly doing compression wouldn't run afoul of the "you're
looking at content" issues. But I totally agree that examining the
content to decide what (or whether) to compress is madness.

--Roger