[tor-bugs] #21759 [Metrics/CollecTor]: Add persistence for torperf/onionperf

[Tor Bug Tracker & Wiki]

#21759: Add persistence for torperf/onionperf
-------------------------------+---------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_review
 Priority:  Medium             |      Milestone:  CollecTor 1.3.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------

Comment (by karsten):

 Replying to [comment:3 iwakeh]:
 > Currently, Onionperf files are stored accumulated (many single
 descriptors in one file) in both recent and out folders.
 > This sort of deviates from the usual process of storing only single
 descriptor files in the out folder.
 > Should onionperf storage be changed to storing single file descriptors
 in the out directory?

 Good question! To be honest, I don't find it particularly useful that
 current tarballs contain just a single descriptor per file. This works
 okay for large descriptors like consensuses or votes, but not so much for
 server descriptors or even microdescriptors. The main reason why we're
 still doing it is that it's easy to check whether we already have a given
 descriptor if we can identify it by filename. If we had a database for
 this information and if we were to rewrite the tarball-creating code, we
 might do things differently. For example, we could put all descriptors
 starting with digest `ee` into an `ee` file. That would probably speed up
 things a lot for people using our tarballs.

 So, for this case I'd say let's leave files as they are and not split them
 up.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21759#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online