[tor-bugs] #21693 [Core Tor/Tor]: prop224: HS descriptors do wasteful double-base64 encoding

[Tor Bug Tracker & Wiki]

#21693: prop224: HS descriptors do wasteful double-base64 encoding
-----------------------------+------------------------------------
 Reporter:  asn              |          Owner:  asn
     Type:  task             |         Status:  assigned
 Priority:  Medium           |      Milestone:  Tor: 0.3.1.x-final
Component:  Core Tor/Tor     |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:  tor-hs, prop224  |  Actual Points:
Parent ID:                   |         Points:  4
 Reviewer:                   |        Sponsor:  SponsorR-can
-----------------------------+------------------------------------

Comment (by asn):

 Replying to [comment:6 nickm]:
 > Something that would be almost as good: whenever we are going to
 encrypt, first compress, and then encrypt.  That would mitigate most of
 the trouble.

 Nice suggestion. Let's see how that could work out.

 As discussed before here is how the descriptor layers work:

 `middle_layer = b64(encrypt(client_auth_data + b64(encrypt(inner_layer)) +
 nul_padding))`
 `outer_layer = header + middle_layer`

 There are two encrypts here. We should probably '''not''' compress the
 plaintext of the middle layer, as that includes the NUL padding, which
 would basically get annihilated by the compression and lose all of its
 metadata hiding properties (we are using the NUL padding to hide the
 number of intro points and client auth details).

 However, compressing the `inner_layer` before `b64(encrypt(inner_layer))`
 could make sense. The inner layer is basically a small header with a bunch
 of intro points. It's usually about 3k bytes (for 3-4 intro points), and
 it compresses down to about 2k bytes using zlib (based on some testing I
 just did).

 So by compressing the inner layer before encrypting we can save about 1k
 bytes (which is also about the cost of double-base64 encoding).

 Do you think we should do this?
 Is the zlib API a PITA to use, or do you think it's gonna be a
 straightforward patch?

 ----

 That said, the above change will '''not''' change the final size of the HS
 descriptor because of the padding that gets applied on the layer above.

 If we wanted to also reduce the size of the final descriptor we could
 relax the padding logic from padding to 10k bytes (total desc size: 14k
 bytes), to padding to 8k bytes or so (total desc size: 11.5k bytes). I
 think that might be OK to do, if we feel that having 14k bytes descriptors
 is a stupid thing.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21693#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online