[tor-bugs] #26155 [Core Tor/Tor]: Bandwidth file Timestamp is the latest scanner result, not the file creation time

Tue May 22 11:03:25 UTC 2018

#26155: Bandwidth file Timestamp is the latest scanner result, not the file
creation time
----------------------------------+------------------------------------
 Reporter:  teor                  |          Owner:  teor
     Type:  defect                |         Status:  needs_revision
 Priority:  Medium                |      Milestone:  Tor: 0.3.5.x-final
Component:  Core Tor/Tor          |        Version:
 Severity:  Normal                |     Resolution:
 Keywords:  tor-spec, tor-bwauth  |  Actual Points:
Parent ID:                        |         Points:
 Reviewer:                        |        Sponsor:
----------------------------------+------------------------------------

Comment (by teor):

 > > > but i don't remember now if we commented that software and
 software_version should be in 3rd and 4th positions or they can be in
 arbitrary positions.
 > >
 > > We have always said they can be in arbitrary positions.
 > >
 > > Is there any reason that they need to be near the top?
 > > Otherwise, let's not have a restriction.
 >
 > i can't think of any. Initially we added them with that order in
 ``sbws``, i just forgot to change that.

 I don't understand what needs to change in sbws.

 The software and software version lines can be in any position on or after
 the 3rd line.
 (The first and second lines are reserved for timestamp and version.)
 So putting the software and software version lines in the 3rd and 4th
 positions is ok.

 Can you explain what you want to change in sbws?

 > Other question: in the examples i used ISO 8601 in the format
 "2018-05-08T16:13:26" (with "T"), though Tor uses "2018-05-08 16:13:26"
 (with space) in other files.
 >
 > I did that because if we use ISO 8601 with space in the Bandwidth Lines,
 it would break the logic of having SP as key_value separator. However we
 are not using in it in Bandwidth Lines, and in the header Lines the
 separator is new line. Which format should we use?

 The timestamp format without the space, as specified in:
 https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt#n93

 See nickm's response here:
 https://lists.torproject.org/pipermail/tor-dev/2018-May/013141.html

 The format with the space is a legacy format that is harder to parse.

 More spec questions from https://github.com/pastly/simple-bw-
 scanner/issues/169#issuecomment-390923878 :

 > When there are not any scan results (similar to the earliest_bandwith
 case https://github.com/pastly/simple-bw-
 scanner/blob/master/sbws/core/generate.py#L136), which should be the
 timestamp?.

 If there are no results in the file, then it really doesn't matter what's
 in the header.

 The timestamp is mandatory, so you can't leave it out.

 Here are your options:
 1. specify that the time must be in the past - tor will warn that the file
 is too old
 2. don't generate the file, delete any existing file - tor will warn that
 the file is missing
 3. generate an empty file, truncate any existing file - old tors will log
 stack contents, see #26007

 I suggest that we say that generators SHOULD NOT generate a file, and
 SHOULD delete any existing file, because it is the least confusing option.
 https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt#n61

 If you want, you can also say that:
 * generators SHOULD wait until enough relays are measured before
 generating the file (option 2)
 * generators MAY use a placeholder timestamp (option 1j, but the time MUST
 be at least 5 days in the past to avoid silent failures.
 * generators MUST NOT generate an empty file (option 3), because it
 triggers a bug in tor.

 Please update the spec with these changes.

 The earliest_bandwidth and latest_bandwidth are optional.
 If there is no valid value for these lines, the generator SHOULD leave
 them out.
 Please update the spec with these changes.

 > I don't understand well what do you mean by "continuously" and
 "timestamp calculation" in the trac ticket:
 >
 > > If there are scanners that do not run continuously, they SHOULD be
 excluded from the timestamp calculation

 Some torflow scanners do not run continuously. They only run when there
 are unmeasured relays.

 In a small network, if all relays are measured by the other scanners, the
 latest timestamp for the unmeasured scanner can become too old. Then the
 timestamp in the file becomes too old, and tor stops voting using the
 file. But the results are still valid, because the other scanners are
 still measuring all the relays.

 This is a bug in torflow that we won't fix.

 But we should document the issue in the spec so future scanners should not
 implement the same bug.

 Replacing "scanners" with "generators" makes this sentence even more
 confusing. See my comments in the pull request.

 > AFAIU, sbws run continuously and to generate the bandwidth file use
 results from a date period https://github.com/pastly/simple-bw-
 scanner/blob/master/sbws/core/generate.py#L134.
 > Should we in this case search for older result files than that date
 period?

 I'm not sure I understand what you mean here.
 Does "this case" mean "when the scanner has stopped producing results"?

 If the scanner stops producing results, then the generator should stop
 including stale results in the file.

 When there are not enough results in the file, the generator SHOULD delete
 the file, and tor will warn it is missing (See option 2 above.)
 If the latest result is older than Tor's voting threshold, then tor will
 stop voting using the file, and warn it is too old.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/26155#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online