[tor-dev] OnionShare bug that's possibly caused by an upstream v3 onion bug

[Micah Lee]

I've been working on a major OnionShare release that, among other 
things, will use v3 onion services by default. But it appears that 
either something in stem or in Tor deals with v3 onions differently than 
v2 onions, and causes a critical bug in OnionShare. It took a lot of 
work to track down exactly how to reproduce this bug, and I haven't 
opened an upstream issue for either stem or tor because I feel like I 
don't understand it enough yet.

Here is the OnionShare issue [1]. Does anyone know if this is an issue 
with stem or tor, and how to go about mitigating it? Here's some 
background:

When you share files with OnionShare it starts a local web server and 
then makes it an ephemeral onion service. Someone else loads this onion 
service in Tor Browser and downloads your files. If the setting "Stop 
sharing after first download" is checked (this is the default behavior), 
then as soon as the download completes, OnionShare stops the onion 
service and web server.

If I share a 1mb file and a Tor Browser client makes an HTTP request to 
download it, OnionShare will respond with a 1mb HTTP response. As soon 
as it's done sending the 1mb, OnionShare stops the server and alerts the 
user that transfer is complete -- however, it's not actually complete 
yet. If you look at the client in Tor Browser, it's still downloading. 
It depends on the Tor circuit, but it might be only ~32% done, and you 
have to wait a few seconds for it to finish. This is because when the 
final byte of the 1mb file leaves the OnionShare web server, it takes a 
few seconds for that final byte to make it through the onion circuit and 
into Tor Browser download.

This works fine with v2 onion services.

But with v3 onion services, as soon as the OnionShare web server 
finishes sending the full HTTP response, the torified HTTP client stops 
downloading. I made a small python script, onion-bug.py, that reproduces 
the issue that you can test [2].

This script connects to the default Tor control port for Tor Browser, so 
open Tor Browser in the background. It then start an HTTP server and 
creates an onion service, and if you make a GET request to the server it 
responds with 2mb of "A" characters, and then immediately stops the web 
server and onion service. You have to pass in either "v2" or "v3", 
depending on which type of onion service to make. Like OnionShare, this 
script uses stem, and specifically the 
Controller.create_ephemeral_hidden_service method [3].

I just ran "./onion-bug.py v2" in one terminal and waited for it to 
start:

```
$ ./onion-bug.py v2
http service: http://127.0.0.1:8080/

starting onion service with: key_type='NEW', key_content='RSA1024'
http://d7vomh45i7ryhhfw.onion/
```

Then in a second terminal, I made the GET request:

```
$ torify curl http://d7vomh45i7ryhhfw.onion/ > out2
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  
Current
                                  Dload  Upload   Total   Spent    Left  
Speed
100 1024k    0 1024k    0     0   125k      0 --:--:--  0:00:08 --:--:-- 
  245k
```

The script in the first terminal finished and quit (while curl on the 
second term finished the download for a few more seconds):

```
127.0.0.1 - - [24/Nov/2018 21:12:52] "GET / HTTP/1.1" 200 -
shutting down http server
```

The file, out1, is exactly 1mb.

Now I tried doing the same, but with a v3 onion service:

```
$ ./onion-bug.py v3
http service: http://127.0.0.1:8080/

starting onion service with: key_type='NEW', key_content='ED25519-V3'
http://htrzngqgb7ogyifsvah6qz7t6spe3rr7bikdws7d3rigkpwy5kuyrlqd.onion/

127.0.0.1 - - [24/Nov/2018 21:14:22] "GET / HTTP/1.1" 200 -
shutting down http server
```

And in the other terminal:

```
$ torify curl 
http://htrzngqgb7ogyifsvah6qz7t6spe3rr7bikdws7d3rigkpwy5kuyrlqd.onion/ > 
out2
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  
Current
                                  Dload  Upload   Total   Spent    Left  
Speed
100 29830    0 29830    0     0   3399      0 --:--:--  0:00:08 --:--:-- 
  3399
```

It only downloaded 29830 bytes before the download connection stopped, 
so out2 is only 30kb. I just tried it again, and this time only got 12kb 
downloaded. And I tried it again, and got 41kb downloaded. It appears to 
depend on the speed of the Tor circuit.

The download only stops when the v3 onion service is shut down -- if you 
uncomment out the time.sleep(5) before calling 
remove_ephemeral_hidden_service at the end of the script, you'll be able 
to download the full 1mb using the v3 onion server.

I'm pretty stumped on how to fix this and I'd appreciate any help.

Until it's fixed, we'll either have to not include support for v3 onion 
services in OnionShare in the next major release (version 2.0), or 
include support but with the "Stop sharing after first download" only 
available for v2 onion services -- so probably we'd default to using v2 
onion services, and users could optionally use v3 if they don't need the 
autostop feature. Of course, I'd much prefer to find a way to solve this 
and release with v3 onion services by default.


[1] https://github.com/micahflee/onionshare/issues/812
[2] https://gist.github.com/micahflee/bade960e96d35007bc5c182a0ca61b56
[3] 
https://stem.torproject.org/api/control.html#stem.control.Controller.create_ephemeral_hidden_service