[tor-bugs] #9379 [Tor]: Tor descriptor fetching is unreliable

Sun Aug 4 19:15:35 UTC 2013

#9379: Tor descriptor fetching is unreliable
--------------------+-------------------------------------------------------
 Reporter:  atagar  |          Owner:     
     Type:  defect  |         Status:  new
 Priority:  normal  |      Milestone:     
Component:  Tor     |        Version:     
 Keywords:          |         Parent:     
   Points:          |   Actualpoints:     
--------------------+-------------------------------------------------------

Comment(by atagar):

 Huh. The thick plottens. I recalled while writing this module that
 fetching plaintext descriptors was pretty reliable so I decided to test
 that...

 {{{
 atagar at odin:~/Desktop/tor/tor-utils$ cat puller.py
 import stem.descriptor.remote
 import stem.util.log

 stem.util.log.log_to_stdout(stem.util.log.TRACE)

 while True:
   query = stem.descriptor.remote.Query(
     '/tor/server/all',
     block = True,
     timeout = 60,
   )

   if not query.error:
     print "Results downloaded: %i" % len(list(query))
   else:
     print "Got an error: %s" % query.error
 }}}

 When I run the above...

 {{{
 atagar at odin:~/Desktop/tor/tor-utils$ python puller.py
 08/04/2013 12:10:03 [TRACE] Descriptors retrieved from
 'http://212.112.245.170:80/tor/server/all' in 37.85s
 Results downloaded: 5768
 08/04/2013 12:10:57 [TRACE] Descriptors retrieved from
 'http://86.59.21.38:80/tor/server/all' in 36.47s
 Results downloaded: 5767
 08/04/2013 12:11:51 [TRACE] Descriptors retrieved from
 'http://212.112.245.170:80/tor/server/all' in 37.28s
 Results downloaded: 5765
 08/04/2013 12:12:44 [TRACE] Descriptors retrieved from
 'http://208.83.223.34:443/tor/server/all' in 35.98s
 Results downloaded: 5766
 }}}

 ... but when I change the resource to 'all.z'...

 {{{
 atagar at odin:~/Desktop/tor/tor-utils$ python puller.py
 08/04/2013 12:02:38 [DEBUG] Unable to download descriptors from
 'http://208.83.223.34:443/tor/server/all.z' (2 retries remaining): Error
 -5 while decompressing data: incomplete or truncated stream
 08/04/2013 12:02:59 [TRACE] Descriptors retrieved from
 'http://154.35.32.5:80/tor/server/all.z' in 21.75s
 Results downloaded: 5763
 08/04/2013 12:03:37 [DEBUG] Unable to download descriptors from
 'http://194.109.206.212:80/tor/server/all.z' (2 retries remaining): Error
 -5 while decompressing data: incomplete or truncated stream
 08/04/2013 12:03:55 [DEBUG] Unable to download descriptors from
 'http://194.109.206.212:80/tor/server/all.z' (1 retries remaining): Error
 -5 while decompressing data: incomplete or truncated stream
 08/04/2013 12:03:55 [DEBUG] Unable to download descriptors from
 'http://171.25.193.9:443/tor/server/all.z': <urlopen error [Errno 111]
 Connection refused>
 Got an error: <urlopen error [Errno 111] Connection refused>
 08/04/2013 12:03:56 [DEBUG] Unable to download descriptors from
 'http://171.25.193.9:443/tor/server/all.z' (2 retries remaining): <urlopen
 error [Errno 111] Connection refused>
 08/04/2013 12:03:56 [DEBUG] Unable to download descriptors from
 'http://171.25.193.9:443/tor/server/all.z' (1 retries remaining): <urlopen
 error [Errno 111] Connection refused>
 08/04/2013 12:04:14 [TRACE] Descriptors retrieved from
 'http://86.59.21.38:80/tor/server/all.z' in 18.09s
 Results downloaded: 5762
 08/04/2013 12:04:49 [TRACE] Descriptors retrieved from
 'http://86.59.21.38:80/tor/server/all.z' in 18.10s
 Results downloaded: 5763
 08/04/2013 12:06:16 [DEBUG] Unable to download descriptors from
 'http://76.73.17.194:9030/tor/server/all.z' (2 retries remaining): Error
 -5 while decompressing data: incomplete or truncated stream
 08/04/2013 12:07:09 [DEBUG] Unable to download descriptors from
 'http://194.109.206.212:80/tor/server/all.z' (1 retries remaining): Error
 -5 while decompressing data: incomplete or truncated stream
 08/04/2013 12:07:37 [DEBUG] Unable to download descriptors from
 'http://128.31.0.39:9131/tor/server/all.z': Error -5 while decompressing
 data: incomplete or truncated stream
 Got an error: Error -5 while decompressing data: incomplete or truncated
 stream
 08/04/2013 12:07:38 [DEBUG] Unable to download descriptors from
 'http://171.25.193.9:443/tor/server/all.z' (2 retries remaining): <urlopen
 error [Errno 111] Connection refused>
 08/04/2013 12:07:58 [DEBUG] Unable to download descriptors from
 'http://194.109.206.212:80/tor/server/all.z' (1 retries remaining): Error
 -5 while decompressing data: incomplete or truncated stream
 08/04/2013 12:08:46 [DEBUG] Unable to download descriptors from
 'http://76.73.17.194:9030/tor/server/all.z': Error -5 while decompressing
 data: incomplete or truncated stream
 Got an error: Error -5 while decompressing data: incomplete or truncated
 stream
 }}}

 The issue definitely seems to be related to the compression.

 > In theory, you could use your script to download from a directory
 mirror, right?

 Yup.

 > Can you find a directory mirror that logs on debug level that you then
 request server or extra-info descriptors from?

 Presently the only relay I have any level of access to is amunet, and we
 definitely don't want to log at debug level there.

 Cheers! -Damian

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9379#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online