[tor-bugs] #8728 [Stem]: Expand tutorial to include exit scanning

[Tor Bug Tracker & Wiki]

#8728: Expand tutorial to include exit scanning
-------------------------+--------------------------------------------------
 Reporter:  atagar       |          Owner:  atagar        
     Type:  enhancement  |         Status:  needs_revision
 Priority:  normal       |      Milestone:                
Component:  Stem         |        Version:                
 Keywords:  website      |         Parent:                
   Points:               |   Actualpoints:                
-------------------------+--------------------------------------------------
Changes (by atagar):

  * status:  needs_review => needs_revision


Comment:

 Hi ragwater, sorry about the delay. This script looks great! Feedback
 below...

 {{{
 from stem.util import term
 }}}

 The first example already demos the term util and printing bootstrap
 messages. Dropping this from later examples simplifies the scripts and
 makes them more focused on just what we're trying to show. I did this for
 another recent addition...

 https://stem.torproject.org/tutorials/to_russia_with_love.html#reading-
 twitter

 {{{
 import pycurl
 }}}

 Lets use SocksiPy instead since its usage is considerably less verbose.
 See the current tutorial page for an example.

 {{{
 import Queue
 }}}

 PEP8 (python's style guide) defines a specific order for our imports.
 Please see...

 http://www.python.org/dev/peps/pep-0008/#imports

 Running 'pep8 --ignore E251,E501,E111,E127' over your script will
 highlight these issues. For an explanation of the parts of pep8 that we're
 ignoring see...

 https://gitweb.torproject.org/stem.git/blob/HEAD:/test/settings.cfg#l95

 {{{
 STREAM_STATUS_CLOSED = "CLOSED"
 STREAM_STATUS_NEW = "NEW"
 STREAM_STATUS_DETACHED = "DETACHED"
 }}}

 Not necessary. These are part of the stem.StreamStatus enumeration. Stem's
 enums are actually strings so...

 {{{
 >>> from stem import StreamStatus
 >>> StreamStatus.DETACHED
 'DETACHED'
 }}}

 {{{
 results = {
     'SUCCESS' : 0,
     'FAILED' : 0,
     'TIMEOUT' : 0,
     'DESTROYED' : 0,}
 }}}

 Minor nitpicks but please...

 * do two space indents
 * put the colon right after the key
 * put the ending brace on a new line, since this makes copy-and-paste
 easier

 For instance in this case...

 {{{
 results = {
   'SUCCESS': 0,
   'FAILED': 0,
   'TIMEOUT': 0,
   'DESTROYED': 0,
 }
 }}}

 {{{
 return stem.process.launch_tor_with_config(config = config,
   init_msg_handler = print_bootstrap_lines)
 }}}

 Keyword based functions like launch_tor_with_config() can also benefit
 from that kind of layout...

 {{{
 return stem.process.launch_tor_with_config(
   config = config,
   init_msg_handler = print_bootstrap_lines,
 )
 }}}

 Though as mentioned above we should probably drop init_msg_handler, so
 guess there's no point in having this span multiple lines.

 > total=sum(value for value in data.values())

 Another pep8 thing but the '=' should have a space on either side of it.

 {{{
 # I was considering checking allowed ports (443) at this point by
 # getting microdescriptors for each node, we can either filter it here
 # or wait for it to happen at getting response stage (it will FAIL then).
 node_gen = controller.get_network_statuses()
 filtered_nodes = filter_nodes(controller, node_gen) # [-20:]
 circ_waiting_list = Queue.Queue()
 }}}

 Good idea about filtering based on the exit policy. Lets do this via
 server descriptors since microdescriptors make this a real pita. If you
 add "FetchUselessDescriptors 1" to the config then this should do the
 trick...

 {{{
 from stem.control import Controller

 with Controller.from_port(port = 9051) as controller:
   controller.authenticate()

   russian_exits = []

   for desc in controller.get_server_descriptors():
     if desc.exit_policy.can_exit_to(port = 443):
       # check if this exit is in russia

       exit_locale = controller.get_info('ip-to-country/%s' % desc.address)

       if exit_locale == 'ru':
         russian_exits.append(desc)

   print "Russian Exits:"

   for desc in russian_exits:
     print "  %s (%s)" % (desc.nickname, desc.address)
 }}}

 There's actually no need to honour the 'Exit' flag. That is just an
 indicator from the directory authorities that a relay allows certain ports
 and is relatively quick. Lacking the flag doesn't necessarily mean that a
 relay can't serve as an exit (that's determined by its exit policy).

 One gotcha though - there doesn't appear to be any Russian exits. *sigh*

 {{{
 $ python example.py
 Russian Exits:
 }}}

 For testing lets use another locale, then we'll note in the tutorial that
 they might need to pick someplace else.

 Cheers! -Damian

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/8728#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online