[tor-bugs] #21377 [Core Tor/Tor]: DirAuths should expose bwauth bandwidth files

[Tor Bug Tracker & Wiki]

#21377: DirAuths should expose bwauth bandwidth files
-------------------------------------------------+-------------------------
 Reporter:  tom                                  |          Owner:  juga
     Type:  enhancement                          |         Status:
                                                 |  needs_review
 Priority:  High                                 |      Milestone:  Tor:
                                                 |  0.4.0.x-final
Component:  Core Tor/Tor                         |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  tor-dirauth, metrics, tor-bwauth,    |  Actual Points:
  035-removed-20180711, 040-roadmap-proposed     |
Parent ID:  #25925                               |         Points:
 Reviewer:  ahf                                  |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by juga):

 Replying to [comment:47 teor]:
 > I think nickm's comments might be the cause of your issue:

 yes, that makes sense

 > is the test actually running the compression code?

 the test is using the compression code implemented in
 `directory_handle_command_get`, but because the file is wrote with
 `connection_write_to_buf_mock` (`connection_buf_add_compress` is not
 mocked), `fetch_from_buf_http` gets the content uncompressed (the headers
 are fine).

 I'm not sure how difficult would be to mock `connection_buf_add_compress`.
 For now i added fixups to the non-wip PR
 (https://github.com/torproject/tor/pull/468) with the mistakes in the
 implementation. Then run a test network and retrieved the bandwidth file
 from an authority using stem:
 {{{
 >>>
 ... import stem.descriptor
 ... import stem.descriptor.remote
 ... import stem.directory
 >>> downloader = stem.descriptor.remote.DescriptorDownloader(
 ...   document_handler = stem.descriptor.DocumentHandler.DOCUMENT,
 ... )
 >>> resource = '/tor/status-vote/next/bandwidth'
 >>> query_args = {'endpoints': [('127.10.0.1', 2003)]}
 >>> bw = downloader.query(resource, **query_args)
 >>> bw.content
 b'1544811454\nversion=1.2.0\nearliest_bandwidth=2018-12-12T15:02:35\nfile_created=2018-12-15T08:09:48\ngenerator_started=2018-12-15T08:06:50\nlatest_bandwidth=2018-12-14T18:17:34\nminimum_number_eligible_relays=3820\nminimum_percent_eligible_relays=60\nnumber_consensus_relays=6366\nnumber_eligible_relays=0\npercent_eligible_relays=0\nsoftware=sbws\nsoftware_version=1.0.3-dev0\n====='
 >>> bw.compression
 ['identity']
 >>> resource = '/tor/status-vote/next/bandwidth.z'
 >>> bw = downloader.query(resource, **query_args)
 >>> bw.content
 b'1544811454\nversion=1.2.0\nearliest_bandwidth=2018-12-12T15:02:35\nfile_created=2018-12-15T08:09:48\ngenerator_started=2018-12-15T08:06:50\nlatest_bandwidth=2018-12-14T18:17:34\nminimum_number_eligible_relays=3820\nminimum_percent_eligible_relays=60\nnumber_consensus_relays=6366\nnumber_eligible_relays=0\npercent_eligible_relays=0\nsoftware=sbws\nsoftware_version=1.0.3-dev0\n====='
 >>> bw.compression
 ['gzip']
 }}}

 You think this is valid test and this can be merged?.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21377#comment:48>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online