[tor-bugs] #20421 [Metrics/CollecTor]: Investigate invalid descriptors in out/ and recent/ subdirectories

[Tor Bug Tracker & Wiki]

#20421: Investigate invalid descriptors in out/ and recent/ subdirectories
-----------------------------------+-----------------
     Reporter:  karsten            |      Owner:
         Type:  defect             |     Status:  new
     Priority:  High               |  Milestone:
    Component:  Metrics/CollecTor  |    Version:
     Severity:  Normal             |   Keywords:
Actual Points:                     |  Parent ID:
       Points:                     |   Reviewer:
      Sponsor:                     |
-----------------------------------+-----------------
 I just found more bad descriptors like the one mentioned in #20412 by
 looking through Onionoo logs.  This might be a bigger problem than first
 thought.

 I don't have the time to investigate this now, but it looks like we might
 not be flushing a buffer properly somewhere.  For example:

 {{{
 hidden- at uploaded-at 2016-10-19 06:53:16
 }}}

 The `hidden-` part still belongs to the previous descriptor and then we
 start writing `@uploaded-at 2016-10-19 06:53:16` as the first line of the
 next descriptor.  Note that we wouldn't include that line under normal
 circumstances.  How does it get there?

 I'll attach four files from CollecTor's `recent/` subdirectory which would
 otherwise be deleted soon.

 I'm afraid we'll not only have to fix this bug but also go through the
 tarballs from the past months and see if those contain invalid
 descriptors.  Hopefully it's a new bug.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20421>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online