[tor-bugs] #7828 [Stem]: Run descriptor parser over all prior descriptors

[Tor Bug Tracker & Wiki]

#7828: Run descriptor parser over all prior descriptors
-------------------------+--------------------------------------------------
 Reporter:  atagar       |          Owner:  atagar
     Type:  task         |         Status:  new   
 Priority:  normal       |      Milestone:        
Component:  Stem         |        Version:        
 Keywords:  descriptors  |         Parent:        
   Points:               |   Actualpoints:        
-------------------------+--------------------------------------------------

Comment(by atagar):

 Thanks! Here's a script that should do the trick. Just fill in the
 'LOG_FILE' with the destination for the output, and provide the descriptor
 paths to the reader. The DescriptorReader's paths can be either files or
 directories.

 {{{
 #!/usr/bin/env python
 # Reads a series of files, logging issues that it comes across.

 import logging
 from stem.descriptor.reader import DescriptorReader

 LOG_FILE = "/home/atagar/Desktop/check_descriptors_log"

 LOGGER = logging.getLogger("check_descriptors")
 LOGGER.addHandler(logging.FileHandler(LOG_FILE))
 LOGGER.setLevel(logging.DEBUG)

 reader = DescriptorReader((
   "/home/atagar/Desktop/stem/test/data/cached-descriptors",
   "/home/atagar/Desktop/stem/test/data/cached-consensus",
 ))

 reader.register_read_listener(
   lambda path: LOGGER.debug("Reading %s" % path)
 )

 reader.register_skip_listener(
   lambda path, exc: LOGGER.warning("  skipped due to %s" % exc)
 )

 with reader:
   for descriptor in reader:
     unrecognized_lines = descriptor.get_unrecognized_lines()

     if unrecognized_lines:
       LOGGER.warning("  unrecognized descriptor content: %s" %
 unrecognized_lines)
 }}}

 Are the descriptors in text files or tarballs? I'm hoping for the former
 since I suspect that we still have performance concerns around tarballs,
 but there's no rush on this so as long as it finishes eventually I'm
 happy.

 Cheers! -Damian

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/7828#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online