[tor-bugs] #12857 [meek]: Use streaming downloads

[Tor Bug Tracker & Wiki]

#12857: Use streaming downloads
-----------------------------+-----------------
     Reporter:  dcf          |      Owner:  dcf
         Type:  enhancement  |     Status:  new
     Priority:  major        |  Milestone:
    Component:  meek         |    Version:
   Resolution:               |   Keywords:
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+-----------------

Comment (by dcf):

 I've been working on this feature in a [https://gitweb.torproject.org
 /pluggable-transports/meek.git/log/?h=stream stream] branch (currently at
 [https://gitweb.torproject.org/pluggable-
 transports/meek.git/log/?h=stream&id=d3c344ea50d7e3405bb1772a2d6069c450454cec
 d3c344ea50]).

 At this point, the branch only does two things:
  1. Allows meek-client to read response bodies of any length (previously
 it was limited to 64 KB).
  2. Changes the protocol between meek-client and the helper to be able to
 express bodies of any length (previously the body was a base64-encoded
 part of the JSON blob, so it could not be too large and could not be
 streamed piece by piece).
 By themselves, these changes don't change anything (though they are safe
 to merge and compatible with the current network). They will also need a
 change to meek-server that allows it to send response bodies of any length
 (currently they are [https://gitweb.torproject.org/pluggable-
 transports/meek.git/tree/meek-server/meek-
 server.go?id=30db0cba962d4620c8787bc455d8f5e8eb2b8d8c#n178 limited to 64
 KB]). But this will need a protocol change, as currently deployed meek-
 clients [https://gitweb.torproject.org/pluggable-transports/meek.git/tree
 /meek-client/meek-
 client.go?id=30db0cba962d4620c8787bc455d8f5e8eb2b8d8c#n168 only look at
 the first 64 KB of a response]. What I'm thinking of is sending a new
 version header field (like "X-Meek-Version: 1" or "X-V: 1"); when meek-
 server sees that, it knows it can should send bodies without a length
 limit. If meek-server does not receive the header, then it will limit
 itself to 64 KB as before.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/12857#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online