[tor-bugs] #28877 [Core Tor/Tor]: 'GETINFO desc/fingerprints' command to get known relays (was: Paginate large controller commands like 'GETINFO desc/all-recent')

[Tor Bug Tracker & Wiki]

#28877: 'GETINFO desc/fingerprints' command to get known relays
--------------------------+--------------------------
 Reporter:  wagon         |          Owner:  (none)
     Type:  defect        |         Status:  assigned
 Priority:  Medium        |      Milestone:
Component:  Core Tor/Tor  |        Version:
 Severity:  Normal        |     Resolution:
 Keywords:                |  Actual Points:
Parent ID:                |         Points:
 Reviewer:                |        Sponsor:
--------------------------+--------------------------

Comment (by atagar):

 > Are you sure Tor makes request to hard drive each time it needs to
 provide this data?

 I could be wrong on that detail. I'd be a little surprised if tor kept the
 full server descriptors in memory since the process does not need the full
 documents and doing so would bloat tor's memory profile but I didn't check
 what the C codebase does, nor does it honestly matter for this ticket.

 I need the ability to iterate over all descriptors without getting such a
 large response. That said, I just thought of a cheap option that's far
 simpler than pagination: **add a 'GETINFO desc/fingerprints' method that
 provides the all server descirptor fingerprints tor knows about**.

 I can call that to get an enumeration of the relays tor has descriptors
 for, then query them one at a time to break up our fourteen megabyte
 response. I suspect we'd want separate methods for other descriptor types
 ('GETINFO md/fingerprints' and 'GETINFO ns/fingerprints') because the
 relays we have information for are not necessarily the same between
 document types.

 > Do you think it also explains the crash of tor-prompt --run 'GETINFO
 desc/all-recent >/dev/null command?

 No, it doesn't. Please file a separate ticket about that so we can keep
 this ticket narrowly focused. I suspect the issue is that you're using
 python3, and that tor-prompt is using print() which expect unicode. Server
 descriptors can have non-ascii content on contact lines which can cause
 the stacktrace you cited above.

 I probably need to add some escaping within tor-prompt.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28877#comment:9>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online