[tor-bugs] #9204 [Tor Check]: Modularize check.torproject.org

[Tor Bug Tracker & Wiki]

#9204: Modularize check.torproject.org
-----------------------+----------------------------------------------------
 Reporter:  arma       |          Owner:     
     Type:  project    |         Status:  new
 Priority:  normal     |      Milestone:     
Component:  Tor Check  |        Version:     
 Keywords:             |         Parent:     
   Points:             |   Actualpoints:     
-----------------------+----------------------------------------------------
 The monolithic design of check.tp.o has given us grief for years. Here's
 my grand vision for the redesigned future of check and related components:

 A) A component that discovers the 'real' outgoing IP addresses of exits,
 and outputs a file like
 https://exitlist.torproject.org/exit-addresses from them.

 That component could start out being tordnsel. It could later be TorBEL if
 somebody fixes it. Or it could later be a simple script to parse the
 @source annotations in directory authority cached-descriptor files, drop
 instances where the source is another directory authority, and if we want
 to get fancy, aggregate results from several directory authorities.

 Note that there's no reason to run this component as or near any user-
 facing service: it just needs to somehow export the static file
 periodically so others can get it.

 B) A script that takes in this exit-addresses script, a Tor consensus
 document, and a destination port, and outputs a list of exit IPs that
 allow connections to that port.

 This is similar to Nick's 'exitlist' script from back in the day (you can
 still find it in contrib/exitlist in the Tor git), except it 1)
 incorporates the active exit addresses, 2) doesn't look at the whole exit
 policy in the descriptors, since clients don't either (I'm amenable to
 changing this part if people want), and 3) considers whether relays are
 Running (since only running relays are in the consensus).

 For users who want to consider all relays in the past n hours, this script
 should be able to handle reading multiple consensus files (ideally just by
 concatenating them before they go in).

 C) Some recommended glue code that takes in the output of 'B' and a test
 IP address, and tells you if the test IP address is in the list. In most
 languages I expect this will be short and sweet.

 D) A user-facing web service called check.torproject.org that runs the
 code from 'C' to tell users if they seem to be coming from a Tor exit. To
 be clear, the job of the check cgi is to read a static text file and tell
 you if your IP address is in it.

 E) A user-facing web service called bulkexitlist that runs the code from
 'B'. That is, it inputs the exit-addresses file, the consensus(es), and
 your destination port, and outputs a static text file of IPs that will
 exit to you. This wants to run in a separate place from check.tp.o, and
 maybe we won't even be the ones to run it.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9204>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online