Proposal: Expose raw bwauth votes
I'm not sure, but I think https://trac.torproject.org/projects/tor/ticket/21377 needed a proposal so I tried to write one up. -tom
On 12 Dec 2017, at 09:41, Tom Ritter <tom@ritter.vg> wrote:
I'm not sure, but I think https://trac.torproject.org/projects/tor/ticket/21377 needed a proposal so I tried to write one up.
Thanks Tom! Proposals are great for new directory authority features, particularly when we're changing the directory specification. I've quoted the proposal below with my comments:
Filename: xxx-expose-bwauth_votes.txt Title: Have Directory Authorities expose raw bwauth vote documents Author: Tom Ritter Created: 11-December-2017 Status: Open
1. Introduction
Bandwidth Authorities (bwauths) perform scanning of the Tor Network and calculate observed speeds for each relay. They produce a 'bwauth vote file' that is given to a Directory Authority. The Directory Authority uses the speed value from this file in its vote file denoting its view of the speed of the relay.
After collecting all of the votes from other Authorities, a consensus is calculated, and the consensus's view of a relay's speed is determined by choosing the low-median value [or is it high-median?]
low-median
of all the authorities' values for each relay.
Only a single metric from the bwauth vote file is exposed by a Directory Authority's vote, however the original file contains considerably more diagnostic information about how the bwauth arrives at that measurement for that relay.
2. Motivation
The bwauth vote file contains more information that is exposed in the overall vote file. This information is useful to debug anomalies in relays' utilization and suspected bugs in the (decrepit) bwauth code.
Currently, all bwauths expose the raw vote file through various (non- standard) means, and that file is downloaded (hourly) by a single person (as long as his home internet connection and home server is working) and archived (with a small amount of robustness.)
It would be preferable to have this exposed in a standard manner. Doing so would no longer require bwauths to run HTTP servers to expose the file, no longer require them to take additional manual steps to provide it, and would enable public consumption by any interested parties. We hope that Collector will begin archiving the files.
3. Specification
An authority SHOULD publish the bwauth vote used to calculate its current vote. It should make the bwauth vote file available at the
SHOULD
same time as its normal vote file.
Since the file is created by an external process, and we never store a copy of any old files, I suggest that we make it available at the URL at all times, as "the file that the authority has right now". (And return a 404 when the file does not exist, is not readable, etc.)
It should make the file available at http://<hostname>/tor/status-vote/next/bwauth.z
We shouldn't use next/ unless we're willing to cache a copy of the file we actually used to vote. If we do, we should serve it from next/ as soon as we vote using it, then serve it from current/ as soon as the consensus is created. If we don't store a copy of the file, we should use a different URL, like status-vote/now/bwauth, and recommend that the file is fetched at hh:50, when the votes are created. This would allow us to implement current/bwauth and next/bwauth in a future version. Have you thought about versioning the URL if we have multiple flavours of bwauth file? We could use bwauth-<FLAVOR> like consensuses. Also, Tor has new compression options for zstd and lzma. Given that this is an externally-controlled file, we could stream it from disk and compress it on the fly with something cheap like gzip or zstd.
It MUST NOT attempt to send its bwauth vote file in a HTTP POST to other authorities and it SHOULD NOT make bwauth vote files from other authorities available.
4. Security Implications
The raw bwauth vote file does not [really: is not believed to] expose any sensitive information. All authorities currently make this document public already, an example is at https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile
How large is the file? Maybe we should pre-compress it, to avoid CPU exhaustion attacks. If we did this, we could say that it's safe, as long as it is smaller than the full consensus or full set of descriptors.
5. Compatibility
Exposing the document presents no compatibility concerns.
The compatibility concern is with applications that want to consume the document. The bwauth vote file has no specification, and has been extended in ad-hoc ways. Applications that merely wish to archive the document (e.g. Collector) won't have a problems. Applications that want to parse it may encounter errors if a new (unexpected) field is added, or assumptions are made about the text encoding or formatting of the document.
Parsers may also encounter issues if we transition to another bwauth implementation. We could remove fields if this happens. T -- Tim / teor PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n ------------------------------------------------------------------------
Sending two replies, with an updated proposal in the second. On 11 December 2017 at 18:38, teor <teor2345@gmail.com> wrote:
It should make the file available at http://<hostname>/tor/status-vote/next/bwauth.z
We shouldn't use next/ unless we're willing to cache a copy of the file we actually used to vote. If we do, we should serve it from next/ as soon as we vote using it, then serve it from current/ as soon as the consensus is created.
If we don't store a copy of the file, we should use a different URL, like status-vote/now/bwauth, and recommend that the file is fetched at hh:50, when the votes are created. This would allow us to implement current/bwauth and next/bwauth in a future version.
Sure.
Have you thought about versioning the URL if we have multiple flavours of bwauth file? We could use bwauth-<FLAVOR> like consensuses.
For lack of a better name I'll propose bwauth-legacy?
Also, Tor has new compression options for zstd and lzma.
Given that this is an externally-controlled file, we could stream it from disk and compress it on the fly with something cheap like gzip or zstd.
I haven't seen any indicated in dir-spec how to handle those? Or how I should change the proposal to accommodate them? Should I make the url .gz and say that the DirAuth should compress it and stream it from disk?
The raw bwauth vote file does not [really: is not believed to] expose any sensitive information. All authorities currently make this document public already, an example is at https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile
How large is the file? Maybe we should pre-compress it, to avoid CPU exhaustion attacks. If we did this, we could say that it's safe, as long as it is smaller than the full consensus or full set of descriptors.
~2.6MB. See above. -tom
On 16 Jan 2018, at 09:17, Tom Ritter <tom@ritter.vg> wrote:
Also, Tor has new compression options for zstd and lzma.
Given that this is an externally-controlled file, we could stream it from disk and compress it on the fly with something cheap like gzip or zstd.
I haven't seen any indicated in dir-spec how to handle those? Or how I should change the proposal to accommodate them? Should I make the url .gz and say that the DirAuth should compress it and stream it from disk?
There is this section about gzip-compression, the extension is ".z": https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3777 I have opened this ticket to document how other compression formats work in tor-spec: https://trac.torproject.org/projects/tor/ticket/24909 T -- Tim / teor PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n ------------------------------------------------------------------------
Tom Ritter transcribed 4.8K bytes:
I'm not sure, but I think https://trac.torproject.org/projects/tor/ticket/21377 needed a proposal so I tried to write one up.
-tom
Hi tom, thanks for the proposal!
Filename: xxx-expose-bwauth_votes.txt Title: Have Directory Authorities expose raw bwauth vote documents Author: Tom Ritter Created: 11-December-2017 Status: Open
I changed things recently, you'll need a "Ticket:" field if your proposal is in state {OPEN,ACCEPTED,CLOSED,FINISHED}. [0] (Although, maybe we shouldn't require "Ticket:" for state OPEN, so as not to hinder calling it OPEN and discussing it even for those things which don't yet have tickets?)
1. Introduction
Bandwidth Authorities (bwauths) perform scanning of the Tor Network and calculate observed speeds for each relay. They produce a 'bwauth vote file' that is given to a Directory Authority. The Directory Authority uses the speed value from this file in its vote file denoting its view of the speed of the relay.
After collecting all of the votes from other Authorities, a consensus is calculated, and the consensus's view of a relay's speed is determined by choosing the low-median value [or is it high-median?] of all the authorities' values for each relay.
Only a single metric from the bwauth vote file is exposed by a Directory Authority's vote, however the original file contains considerably more diagnostic information about how the bwauth arrives at that measurement for that relay.
2. Motivation
The bwauth vote file contains more information that is exposed in the
/s/that/than/ ???
overall vote file. This information is useful to debug anomalies in relays' utilization and suspected bugs in the (decrepit) bwauth code.
Currently, all bwauths expose the raw vote file through various (non- standard) means, and that file is downloaded (hourly) by a single person (as long as his home internet connection and home server is working) and archived (with a small amount of robustness.)
It would be preferable to have this exposed in a standard manner. Doing so would no longer require bwauths to run HTTP servers to expose the file, no longer require them to take additional manual steps to provide it, and would enable public consumption by any interested parties. We hope that Collector will begin archiving the files.
3. Specification
An authority SHOULD publish the bwauth vote used to calculate its current vote. It should make the bwauth vote file available at the same time as its normal vote file. It should make the file available at http://<hostname>/tor/status-vote/next/bwauth.z
If it's "next", how is it possible to expose it at the same time as its vote which is based upon it? Maybe we should change the URL to be "current"?
It MUST NOT attempt to send its bwauth vote file in a HTTP POST to other authorities and it SHOULD NOT make bwauth vote files from other authorities available.
4. Security Implications
The raw bwauth vote file does not [really: is not believed to] expose any sensitive information. All authorities currently make this document public already, an example is at https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile
Maybe we want to think about resource exhaustion attacks if we're making a standarised interface available to it? The response after all is going likely always be much larger than the request.
5. Compatibility
Exposing the document presents no compatibility concerns.
The compatibility concern is with applications that want to consume the document. The bwauth vote file has no specification, and has been extended in ad-hoc ways. Applications that merely wish to archive the document (e.g. Collector) won't have a problems. Applications that want to parse it may encounter errors if a new (unexpected) field is added, or assumptions are made about the text encoding or formatting of the document.
A specification for the documents that BWAuths produce would be an extremely welcome contribution! but probably shouldn't be prerequisite to accepting and implementing this proposal. Thanks again for the proposal, tom! [0]: https://gitweb.torproject.org/torspec.git/commit/?id=8be6722e8d9 Best regards, -- ♥Ⓐ isis agora lovecruft _________________________________________________________ OpenPGP: 4096R/0A6A58A14B5946ABDE18E207A3ADB67A2CDB8B35 Current Keys: https://fyb.patternsinthevoid.net/isis.txt
Updated proposal attached. On 12 December 2017 at 12:44, isis agora lovecruft <isis@torproject.org> wrote:
Status: Open
I changed things recently, you'll need a "Ticket:" field if your proposal is in state {OPEN,ACCEPTED,CLOSED,FINISHED}. [0]
(Although, maybe we shouldn't require "Ticket:" for state OPEN, so as not to hinder calling it OPEN and discussing it even for those things which don't yet have tickets?)
I added Ticket: https://trac.torproject.org/projects/tor/ticket/21377
An authority SHOULD publish the bwauth vote used to calculate its current vote. It should make the bwauth vote file available at the same time as its normal vote file. It should make the file available at http://<hostname>/tor/status-vote/next/bwauth.z
If it's "next", how is it possible to expose it at the same time as its vote which is based upon it? Maybe we should change the URL to be "current"?
teor suggested 'now'? I'll make it whichever you think it should be =)
The raw bwauth vote file does not [really: is not believed to] expose any sensitive information. All authorities currently make this document public already, an example is at https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile
Maybe we want to think about resource exhaustion attacks if we're making a standarised interface available to it? The response after all is going likely always be much larger than the request.
teor suggested compressing and streaming from disk? -tom
Hi, It looks like this proposal was lost, and never committed to torspec. Sorry about that! Juga and I have updated the proposal. Please see the full text inline below. We have also written the corresponding dir-spec patch: https://github.com/torproject/torspec/pull/26/files We are tracking the spec and proposal in: https://trac.torproject.org/projects/tor/ticket/26694 The Tor implementation in: https://trac.torproject.org/projects/tor/ticket/21377 And the Collector implementation in: https://trac.torproject.org/projects/tor/ticket/21378
On 16 Jan 2018, at 08:23, Tom Ritter <tom@ritter.vg> wrote:
On 12 December 2017 at 12:44, isis agora lovecruft <isis@torproject.org> wrote:
An authority SHOULD publish the bwauth vote used to calculate its current vote. It should make the bwauth vote file available at the same time as its normal vote file. It should make the file available at http://<hostname>/tor/status-vote/next/bwauth.z
If it's "next", how is it possible to expose it at the same time as its vote which is based upon it? Maybe we should change the URL to be "current"?
teor suggested 'now'? I'll make it whichever you think it should be =)
We changed the scheme so the bandwidth file at status-vote/(next|current)/bandwidth.z is the bandwidth file used to produce: status-vote/(next|current)/authority.z We publish the bandwidth file to next/ when the vote is published to next/. And the same with current. It should be relatively easy to implement this publishing in the tor code. And to implement downloads in Collector.
The raw bwauth vote file does not [really: is not believed to] expose any sensitive information. All authorities currently make this document public already, an example is at https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile
Maybe we want to think about resource exhaustion attacks if we're making a standarised interface available to it? The response after all is going likely always be much larger than the request.
teor suggested compressing and streaming from disk?
We’ll need to take a copy of the file anyway, to avoid race conditions. We can use the same code we use for the rest of the directory documents. T Filename: xxx-expose-bandwidth-files.txt Title: Have Directory Authorities expose raw bandwidth list files Author: Tom Ritter Created: 11-December-2017 Status: Open Ticket: https://trac.torproject.org/projects/tor/ticket/21377 1. Introduction Bandwidth Authorities (bwauths) perform scanning of the Tor Network and calculate observed bandwidths for each relay. They produce a bandwidth list file that is given to a Directory Authority. The Directory Authority uses the bw (bandwidth) value from this file in its vote file denoting its view of the bandwidth of the relay. After collecting all of the votes from other Authorities, a consensus is calculated, and the consensus's view of a relay's speed is determined by choosing the low-median value of all the authorities' values for each relay. Only a single metric from the bandwidth list file is exposed by a Directory Authority's vote, however the original file contains considerably more diagnostic information about how the bwauth arrives at that measurement for that relay. For more details, see the bandwidth list file specification in bandwidth-file-spec.txt. 2. Motivation The bandwidth list file contains more information than is exposed in the overall vote file. This information is useful to debug: * anomalies in relays' utilization, * suspected bugs in the (decrepit) bwauth code, and * the transition to a replacement bwauth implementation. Currently, all bwauths expose the bandwidth list file through various (non- standard) means, and that file is downloaded (hourly) by a single person (as long as his home internet connection and home server is working) and archived (with a small amount of robustness.) It would be preferable to have this exposed in a standard manner. Doing so would no longer require bwauths to run HTTP servers to expose the file, no longer require them to take additional manual steps to provide it, and would enable public consumption by any interested parties. We hope that Collector will begin archiving the files. 3. Specification An authority SHOULD publish the bandwidth list file used to calculate its next vote. It SHOULD make the bandwidth list file available whenever the corresponding vote is available, at the corresponding URL. (See dir-spec for the exact details.) It SHOULD make the file available at http://<hostname>/tor/status-vote/next/bandwidth.z http://<hostname>/tor/status-vote/current/bandwidth.z It MUST NOT attempt to send its bandwidth list file in a HTTP POST to other authorities and it SHOULD NOT make bandwidth list files from other authorities available. Clients interested in consuming these documents should download them from each authority's: * next URL when votes are created. (In the public Tor network, this is after HH:50 during normal operation, and after HH:20 during a consensus failure.) * current URL after the valid-after time in the consensus. (After HH:00, and HH:30 during consensus failure.) 4. Security Implications The raw bandwidth list file does not [really: is not believed to] expose any sensitive information. All authorities currently make this document public already, an example is at https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile 5. Compatibility Exposing the document presents no compatibility concerns. Applications that parse the document should follow the bandwidth list file specification in bandwidth-file-spec.txt. If a new bandwidth list format version is added, the applications MAY need to upgrade to that version.
Hi, On 16/07/18 04:12, teor wrote:
It MUST NOT attempt to send its bandwidth list file in a HTTP POST to other authorities and it SHOULD NOT make bandwidth list files from other authorities available.
There is no mechanism specified that could be used to send this file via an HTTP POST. There is also no mechanism to make available bandwidth list files from other authorities specified. I wonder if this paragraph has to be here. I can also see a future where, to support more robust metrics collection, we might want to have authorities also provide mirrors of other authorities' documents. Thanks, Iain.
On 16 Jul 2018, at 18:03, Iain Learmonth <irl@torproject.org> wrote:
On 16/07/18 04:12, teor wrote:
It MUST NOT attempt to send its bandwidth list file in a HTTP POST to other authorities and it SHOULD NOT make bandwidth list files from other authorities available.
There is no mechanism specified that could be used to send this file via an HTTP POST.
But there is a mechanism for sending votes via HTTP POST: An authority SHOULD publish its vote immediately at the start of each voting period (minus VoteSeconds+DistSeconds). It does this by making it available at http://<hostname>/tor/status-vote/next/authority.z and sending it in an HTTP POST request to each other authority at the URL http://<hostname>/tor/post/vote https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1586
There is also no mechanism to make available bandwidth list files from other authorities specified.
Once an authority has a vote from another authority, it makes it available at http://<hostname>/tor/status-vote/next/<fp>.z where <fp> is the fingerprint of the other authority's identity key. And at http://<hostname>/tor/status-vote/next/d/<d>.z where <d> is the digest of the vote document. https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1597
I wonder if this paragraph has to be here.
We included this paragraph to avoid implementing unnecessary features. We wanted a minimal implementation to start with.
I can also see a future where, to support more robust metrics collection, we might want to have authorities also provide mirrors of other authorities' documents.
I think this would be a good feature, but it’s outside the scope of a minimal implementation. If someone would like to help out: * I would happily merge a spec or proposal patch * Please open tickets for the POST and GET features * See "Authorities should put a hash of the bandwidth file in their votes" https://trac.torproject.org/projects/tor/ticket/26698 T
participants (5)
-
Iain Learmonth -
isis agora lovecruft -
teor -
teor -
Tom Ritter