[tor-bugs] #29365 [Metrics/Onionperf]: Add digests and sizes to index.xml

[Tor Bug Tracker & Wiki]

#29365: Add digests and sizes to index.xml
-------------------------------------------------+-------------------------
 Reporter:  irl                                  |          Owner:  karsten
     Type:  enhancement                          |         Status:
                                                 |  needs_review
 Priority:  Medium                               |      Milestone:
Component:  Metrics/Onionperf                    |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  metrics-team-roadmap-2020, metrics-  |  Actual Points:  0.2
  team-roadmap-2020-june                         |
Parent ID:  #33321                               |         Points:  0.5
 Reviewer:  acute                                |        Sponsor:
                                                 |  Sponsor59-must
-------------------------------------------------+-------------------------

Comment (by acute):

 Replying to [comment:9 karsten]:
 > Even though CollecTor is currently not using this file, we should add
 more information about files being served than just the filename.
 >
 > However, I think that computing file digests is too much, because we'd
 either have to recompute these digests for all files once per day or store
 them somewhere. If we really need these digests on the consuming side,
 then let's consider adding them. But until that's the case, let's try
 something simpler.
 >
 > I wrote a patch that includes sizes and last-modified times along with
 filenames. These can be looked up really quickly while writing the XML
 file. While touching this code I also made sure that the index.xml file
 itself is not contained and I fixed the encoding issue introduced by the
 Python 3 upgrade.
 >
 > Please review
 [https://gitweb.torproject.org/user/karsten/onionperf.git/commit/?h=task-29365&id=011359d0ed6580bd8cb00f3ce8c510c4948accb7
 commit 011359d in my task-29365 branch].

 Thank you for the patch! I have read and tested it - no issues there and
 it can be merged.

 Also, I did a quick check  - it takes ~1 second to compute all sha256
 digests for one years' worth of OP data downloaded from Collector on my
 old Lenovo x220 - I don't think recomputing these is an issue on
 processors nowadays.

 Adding sha256 hashes would be a small change (have attached a patch), up
 to you if you want to go ahead with it :)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29365#comment:13>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online