commit eeab56c313162130ac69afe4266cdbbf22531bdc Author: teor teor@torproject.org Date: Fri Mar 22 11:23:57 2019 +1000

bandwidth-file: Fix missing cardinalities, format versions, and headings

Also fix some typos. --- bandwidth-file-spec.txt | 63 ++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 49 insertions(+), 14 deletions(-)

diff --git a/bandwidth-file-spec.txt b/bandwidth-file-spec.txt index 1bc7667..9f573c6 100644 --- a/bandwidth-file-spec.txt +++ b/bandwidth-file-spec.txt @@ -485,7 +485,7 @@ If a parser does not recognize any extra material in a RelayLine, the extra material MUST be ignored.

- Each RelayLine MUST include the following KeyValue pairs: + Each RelayLine includes the following KeyValue pairs:

"node_id=" hexdigest

@@ -497,6 +497,13 @@ 0.3.4.1-alpha, node_id MUST NOT be at the end of the Line. These authority versions are no longer supported.

+ Current Tor versions ignore master_key_ed25519, so node_id MUST be + present in each relay Line. + + Implementations of version 1.1.0 and later SHOULD include both node_id + and master_key_ed25519. Parsers SHOULD accept Lines that contain at + least one of them. + "master_key_ed25519=" MasterKey

[Zero or one time.] @@ -505,9 +512,9 @@ without trailing "="s, to avoid ambiguity with KeyValue "=" character.

- Implementations of version 1.1.0 SHOULD include both node_id and - master_key_ed25519. - Parsers SHOULD accept Lines that contain at least one of them. + This KeyValue pair SHOULD be present, see the note under node_id. + + This KeyValue was added in version 1.1.0 of this specification.

"bw=" Bandwidth

@@ -606,7 +613,7 @@

2.4.2.1. Simple Bandwidth Scanner

- sbws RelayLines may contain these keys: + sbws RelayLines contain these keys:

"node_id=" hexdigest SP

@@ -622,12 +629,17 @@

The relay nickname.

+ Torflow also has a "nick=" KeyValue. + "rtt=" Int SP

- [Exactly once.] + [Zero or one time.]

The Round Trip Time in milliseconds to obtain 1 byte of data.

+ This KeyValue was added in version 1.1.0 of this specification. + It became optional in version 1.3.0 or 1.4.0 of this specification. + "time=" DateTime SP

[Exactly once.] @@ -635,6 +647,9 @@ The date and time timestamp in ISO 8601 format and UTC time zone when the last bandwidth was obtained.

+ This KeyValue was added in version 1.1.0 of this specification. + The Torflow equivalent is "measured_at=". + "success=" Int SP

[Zero or one time.] @@ -642,6 +657,8 @@ The number of times that the bandwidth measurements for this relay were successful.

+ This KeyValue was added in version 1.1.0 of this specification. + "error_circ=" Int SP

[Zero or one time.] @@ -649,6 +666,9 @@ The number of times that the bandwidth measurements for this relay failed because of circuit failures.

+ This KeyValue was added in version 1.1.0 of this specification. + The Torflow equivalent is "circ_fail=". + "error_stream=" Int SP

[Zero or one time.] @@ -656,6 +676,8 @@ The number of times that the bandwidth measurements for this relay failed because of stream failures.

+ This KeyValue was added in version 1.1.0 of this specification. + "error_misc=" Int SP

[Zero or one time.] @@ -663,38 +685,50 @@ The number of times that the bandwidth measurements for this relay failed because of other reasons.

+ This KeyValue was added in version 1.1.0 of this specification. + "bw_mean=" Int SP

[Zero or one time.]

The measured bandwidth mean for this relay in bytes per second.

+ This KeyValue was added in version 1.2.0 of this specification. + "bw_median=" Int SP

[Zero or one time.]

The measured bandwidth median for this relay in bytes per second.

+ This KeyValue was added in version 1.2.0 of this specification. + "desc_bw_average=" Int SP

[Zero or one time.]

The descriptor average bandwidth for this relay in bytes per second.

- "desc_bw_obs_last=" Int SP + This KeyValue was added in version 1.2.0 of this specification. + + "desc_obs_bw_last=" Int SP

[Zero or one time.]

The last descriptor observed bandwidth for this relay in bytes per second.

- "desc_bw_obs_mean=" Int SP + This KeyValue was added in version 1.2.0 of this specification. + + "desc_obs_bw_mean=" Int SP

[Zero or one time.]

The descriptor observed bandwidth mean for this relay in bytes per second.

+ This KeyValue was added in version 1.2.0 of this specification. + 2.4.2.2. Torflow

Torflow RelayLines include node_id and bw, and other KeyValue pairs [2]. @@ -703,6 +737,8 @@ References:

1. https://gitweb.torproject.org/torflow.git 2. https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/R... + The Torflow specification is outdated, and does not match the current + implementation. See section A.1. for the format produced by Torflow. 3. https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt 4. https://gitweb.torproject.org/torspec.git/tree/version-spec.txt 5. https://semver.org/ @@ -713,7 +749,7 @@ The following has not been obtained from any real measurement.

A.1. Generated by Torflow

- This an example version 1.0.0 document: +This an example version 1.0.0 document:

1523911758 node_id=$68A483E05A2ABDCA6DA5A3EF8DB5177638A27F80 bw=760 nick=Test measured_at=1523911725 updated_at=1523911725 pid_error=4.11374090719 pid_error_sum=4.11374090719 pid_bw=57136645 pid_delta=2.12168374577 circ_fail=0.2 scanner=/filepath @@ -752,7 +788,7 @@ software_version=1.0.3 bw=38000 bw_mean=1127824 bw_median=1180062 desc_avg_bw=1073741824 desc_obs_bw_last=17230879 desc_obs_bw_mean=14732306 error_circ=0 error_misc=0 error_stream=1 master_key_ed25519=YaqV4vbvPYKucElk297eVdNArDz9HtIwUoIeo0+cVIpQ nick=Test node_id=$68A483E05A2ABDCA6DA5A3EF8DB5177638A27F80 rtt=380 success=1 time=2018-05-08T16:13:26 bw=1 bw_mean=199162 bw_median=185675 desc_avg_bw=409600 desc_obs_bw_last=836165 desc_obs_bw_mean=858030 error_circ=0 error_misc=0 error_stream=0 master_key_ed25519=a6a+dZadrQBtfSbmQkP7j2ardCmLnm5NJ4ZzkvDxbo0I nick=Test2 node_id=$96C15995F30895689291F455587BD94CA427B6FC rtt=378 success=1 time=2018-05-08T16:13:36

- When there are not enough eligible measured relays: +A.3.1. When there are not enough eligible measured relays:

1540496079 version=1.2.0 @@ -786,9 +822,9 @@ percent_eligible_relays=93 scanner_country=SN software=sbws software_version=1.0.4 +=====

-A.5 Header generated by sbws version 1.1.0 and version 1.4.0 -of this specification. +A.5 Generated by sbws version 1.1.0

1523911758 version=1.4.0 @@ -813,8 +849,7 @@ recent_priority_relay_count=6243 scanner_country=SN software=sbws software_version=1.1.0 - -A.6 Lines generated by sbws version 1.1.0 and version 1.4.0 +===== bw=1 error_circ=1 error_destination=0 error_misc=0 error_second_relay=0 error_stream=0 master_key_ed25519=J3HQ24kOQWac3L1xlFLp7gY91qkb5NuKxjj1BhDi+m8 nick=snap269 node_id=$DC4D609F95A52614D1E69C752168AF1FCAE0B05F relay_recent_measurements_excluded_error_count=1 relay_recent_measurements_excluded_near_count=3 success=3 time=2019-03-16T18:20:57 unmeasured=1 vote=0 bw=1 error_circ=0 error_destination=0 error_misc=0 error_second_relay=0 error_stream=0 master_key_ed25519=h6ZB1E1yBFWIMloUm9IWwjgaPXEpL5cUbuoQDgdSDKg nick=jrelay node_id=$C4544F9E209A9A9B99591D548B3E2822236C0503 relay_recent_measurements_excluded_few_count=1 success=1 time=2019-03-17T06:50:58 unmeasured=1 vote=0