[tor-bugs] #12428 [meek]: Make it possible to have multiple requests and responses in flight

[Tor Bug Tracker & Wiki]

#12428: Make it possible to have multiple requests and responses in flight
-------------------------+---------------------
 Reporter:  dcf          |          Owner:  dcf
     Type:  enhancement  |         Status:  new
 Priority:  normal       |      Milestone:
Component:  meek         |        Version:
 Keywords:               |  Actual Points:
Parent ID:               |         Points:
-------------------------+---------------------
 meek segments a data stream into multiple HTTP request–response pairs. In
 order to keep the segments in order, meek-client strictly serializes
 requests: it won't issue a second request until after it receives the
 response to its first request, even if there is buffered data waiting to
 be sent.

 The limit of one latent request–response is restricting possible
 throughput. For instance, if a user is located 200 ms from App Engine, and
 receives up to 64 KB per request, then their downstream throughput can be
 no greater than 64 KB/200 ms = 320 KB/s, even if everything after App
 Engine were instantaneous. Longer delays lead to even lower throughput.

 The problem is how to deal with out-of-order arrivals, and retransmissions
 when an HTTP transaction fails. My plan is to add sequence numbers and
 acknowledgements to upstream and downstream HTTP headers, similar to what
 we did in OSS (https://www.bamsoftware.com/papers/oss.pdf section 4). The
 seq number is the index of the first byte of a payload in the overall
 stream. The ack number is the index of the next byte we're expecting from
 the other side. We can implement this idea in a backward-compatible way,
 by having the server guess in seq and ack fields when they are missing;
 old clients that serialize will continue to work.

 There's a complication related to the protocol's polling nature. During a
 big download, we want multiple downstream responses to be in flight. In
 order to get that, we need to speculatively send a bunch of requests and
 see if they get responses that have data. My thinking is to do something
 like [https://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm TCP
 congestion avoidance], where we increment the number of speculative probes
 we send by 1 every time we get a response back with data. Maybe only when
 we get a full-sized response. Reset the number to 1 when there is a loss
 event.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/12428>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online