[tor-bugs] #19452 [Metrics/Atlas]: Make single request for Onionoo details document

[Tor Bug Tracker & Wiki]

#19452: Make single request for Onionoo details document
---------------------------+--------------------------
 Reporter:  phw            |          Owner:  irl
     Type:  enhancement    |         Status:  accepted
 Priority:  Medium         |      Milestone:
Component:  Metrics/Atlas  |        Version:
 Severity:  Normal         |     Resolution:
 Keywords:  onionoo        |  Actual Points:
Parent ID:                 |         Points:
 Reviewer:                 |        Sponsor:
---------------------------+--------------------------

Comment (by karsten):

 Please disregard my comment above starting with "In the future", which
 really belongs in another ticket, like #17938.  Let's focus on reducing
 the number of requests from Onionoo internally while keeping functionality
 unchanged.

 Here's an example for requests made by Atlas to Onionoo for displaying
 "Top 10 Relays by Consensus Weight".  Atlas first asks for the first 10
 running relays by consensus weight in decreasing order:

 {{{
 /summary?type=relay&order=-consensus_weight&limit=10&running=true
 }}}

 The result of that request is this list of relays:

 {{{
 {"version":"3.1",
 "relays_published":"2017-01-09 08:00:00",
 "relays":[
 {"n":"TorLand1","f":"E1E922A20AF608728824A620BADC6EFC8CB8C2B8","a":["37.130.227.133","[2a02:2498:e001:3c::133]"],"r":true},
 {"n":"PrivacyRepublic0001","f":"5CECC5C30ACC4B3DE462792323967087CC53D947","a":["178.32.181.96","37.187.129.166"],"r":true},
 {"n":"xshells","f":"E8E71987BCB8C24DBDF1C3BB0BF3B6C76550A108","a":["178.217.187.39","[2001:67c:2044:3226::2]"],"r":true},
 {"n":"hviv104","f":"81B75D534F91BFB7C57AB67DA10BCEF622582AE8","a":["192.42.116.16"],"r":true},
 {"n":"colosimo","f":"51939625169E2C7E0DC83D38BAE628BDE67E9A22","a":["109.236.90.209"],"r":true},
 {"n":"IPredator","f":"BC630CBBB518BE7E9F4E09712AB0269E9DC7D626","a":["197.231.221.211"],"r":true},
 {"n":"dopper","f":"D665C959571041972EA8C0DD77559EF5579BA112","a":["192.42.113.102"],"r":true},
 {"n":"0x3d001","f":"CFBBA0D858F02E40B1432A65F6D13C9BDFE7A46B","a":["91.121.23.100"],"r":true},
 {"n":"Onyx","f":"6DFEB41C04CCE846871338E85DD5ACF5CFB6C1DD","a":["192.42.115.102","[2001:610:510:115:192:42:115:102]"],"r":true},
 {"n":"radia2","f":"DE684E6C6B7773B8BE74B4D941E4178988E15E26","a":["91.121.230.212","[2001:41d0:d:22a6::1:4]"],"r":true}
 ],
 "bridges_published":"2017-01-09 06:41:04",
 "bridges":[
 ]}
 }}}

 Now it goes to fetch details documents for all contained fingerprints:

 {{{
 https://onionoo.torproject.org/details?lookup=E1E922A20AF608728824A620BADC6EFC8CB8C2B8
 https://onionoo.torproject.org/details?lookup=5CECC5C30ACC4B3DE462792323967087CC53D947
 https://onionoo.torproject.org/details?lookup=E8E71987BCB8C24DBDF1C3BB0BF3B6C76550A108
 [...]
 }}}

 What Atlas should really do is make one request that doesn't return
 summary documents but instead details documents, and with just the fields
 that will be displayed in the results table or used internally otherwise:

 {{{
 /details?type=relay&order=-consensus_weight&limit=10&running=true&fields=nickname,...
 }}}

 I didn't look at the code which fields are used, though that shouldn't be
 hard to figure out.  Also note that "Top 10 Relays by Consensus Weight" is
 just one type of request and that the search by user-provided search term
 is probably found somewhere else in the code.  Basically, Atlas should not
 request `/summary` from Onionoo anymore but instead request `/details`
 directly.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/19452#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online