[tor-bugs] #18320 [Core Tor/Tor]: Clear old entries from the key-pinning journal file

[Tor Bug Tracker & Wiki]

#18320: Clear old entries from the key-pinning journal file
----------------------------------------+----------------------------------
 Reporter:  teor                        |          Owner:  andrea
     Type:  defect                      |         Status:  assigned
 Priority:  Medium                      |      Milestone:  Tor:
Component:  Core Tor/Tor                |  0.2.9.x-final
 Severity:  Normal                      |        Version:
 Keywords:  tor-dos, TorCoreTeam201606  |     Resolution:
Parent ID:  #17293                      |  Actual Points:
 Reviewer:                              |         Points:  3
                                        |        Sponsor:  SponsorU-can
----------------------------------------+----------------------------------

Comment (by andrea):

 To prune the journal, we should remove duplicates and lines that introduce
 mappings
 that will subsequently be removed as conflicts.  The easiest approach will
 be to
 parse the journal, then re-generate it from scratch based on the in-memory
 structure,
 but this will fail to preserve the comment/reserved lines and the ordering
 of non-
 conflicting entries.  Do we care?

 If we do, then the next-easiest approach is probably an in-memory list of
 lines, and hash tables indexing it to query all lines matching particular
 keys, so when we spot a conflicting line during pruning we can find the
 conflicting previous lines and remove them, then re-emit the file from
 the list of remaining lines.

 Since there are no more than a few thousand active routers, a fully pruned
 list cannot become unreasonably large, but this ticket is concerned with
 running out of disk space due to excessive size.  I think if we remove
 earlier conflicting lines immediately while parsing to prune we'll keep it
 down to a reasonable size though.

 Has using tor_mmap_file() in keypin_load_journal() become a problem for
 space reasons too?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18320#comment:13>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online