[tor-commits] [stem/master] Example in our tutorial for writing descriptors to disk and reading them back

atagar at torproject.org atagar at torproject.org
Tue Nov 18 06:09:21 UTC 2014 

commit 23042f197de3aaed34f7c29c12359165aeba5ef3
Author: Damian Johnson <atagar at torproject.org>
Date:   Mon Nov 17 22:10:49 2014 -0800

    Example in our tutorial for writing descriptors to disk and reading them back
    
    Neat suggestion by mmcc on...
    
      https://trac.torproject.org/projects/tor/ticket/13774
---
 docs/change_log.rst                                |    4 ++
 docs/contents.rst                                  |    1 +
 docs/tutorials/double_double_toil_and_trouble.rst  |    4 ++
 docs/tutorials/examples/persisting_a_consensus.rst |   72 ++++++++++++++++++++
 docs/tutorials/mirror_mirror_on_the_wall.rst       |   55 ++++++++++++++-
 5 files changed, 134 insertions(+), 2 deletions(-)

diff --git a/docs/change_log.rst b/docs/change_log.rst
index 1724497..7fdd2bf 100644
--- a/docs/change_log.rst
+++ b/docs/change_log.rst
@@ -71,6 +71,10 @@ The following are only available within Stem's `git repository
 
   * The /info command errored for relays without contact information.
 
+ * **Website**
+
+  * Added an example for `writing descriptors to disk and reading them back <tutorials/mirror_mirror_on_the_wall.html#saving-and-loading-descriptors>`_ (:trac:`13774`)
+
 .. _version_1.2:
 
 Version 1.2
diff --git a/docs/contents.rst b/docs/contents.rst
index 15283f1..f7a49d2 100644
--- a/docs/contents.rst
+++ b/docs/contents.rst
@@ -17,6 +17,7 @@ Contents
    tutorials/examples/exit_used
    tutorials/examples/list_circuits
    tutorials/examples/outdated_relays
+   tutorials/examples/persisting_a_consensus
 
    change_log
    download
diff --git a/docs/tutorials/double_double_toil_and_trouble.rst b/docs/tutorials/double_double_toil_and_trouble.rst
index 19ff0c5..ce6c58a 100644
--- a/docs/tutorials/double_double_toil_and_trouble.rst
+++ b/docs/tutorials/double_double_toil_and_trouble.rst
@@ -103,3 +103,7 @@ Descriptors
   Provides information about the current votes from Tor's Bandwidth
   Authorities.
 
+* `Saving and Loading a Tor Consensus <examples/persisting_a_consensus.html>`_
+
+  Example for writing a Tor consensus to disk, and reading it back.
+
diff --git a/docs/tutorials/examples/persisting_a_consensus.rst b/docs/tutorials/examples/persisting_a_consensus.rst
new file mode 100644
index 0000000..d9c681b
--- /dev/null
+++ b/docs/tutorials/examples/persisting_a_consensus.rst
@@ -0,0 +1,72 @@
+Saving and Loading a Tor Consensus
+==================================
+
+.. image:: /_static/buttons/back.png
+   :target: ../double_double_toil_and_trouble.html
+
+Reading and writing a Tor consensus to disk is similar to `other descriptor
+types <../mirror_mirror_on_the_wall.html#saving-and-loading-descriptors>`_
+with one small difference.
+
+Most descriptors are just about a single relay. Server descriptors and
+microdescriptors, for instance, can be concatenated together and dumped to a
+file because they're each independent of each other.
+
+The Tor consensus, however, is a larger document containing information about
+the Tor network in addition to a little data on each of the relays.
+
+In Stem the overall document is a
+:class:`~stem.descriptor.networkstatus.NetworkStatusDocumentV3`, and the
+information on individual relays are
+:class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3` instances.
+
+Why does this matter? By default when you read a consensus Stem provides you
+**just** the :class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3`.
+This is for performance reasons, and because usually that's what developers
+want. But for writing the conssensus to disk we'll want the whole document
+instead.
+
+So how do we get it? Just tell Stem that's what you want. The
+:class:`~stem.descriptor.__init__.DocumentHandler` tells Stem how to read the
+consensus. For example, to write the consensus simply do the following...
+
+::
+
+  from stem.descriptor import DocumentHandler
+  from stem.descriptor.remote import DescriptorDownloader
+
+  downloader = DescriptorDownloader()
+  consensus = downloader.get_consensus(document_handler = DocumentHandler.DOCUMENT).run()[0]
+
+  with open('/tmp/descriptor_dump', 'w') as descriptor_file:
+    descriptor_file.write(str(consensus))
+
+Our *consensus* here is the current
+:class:`~stem.descriptor.networkstatus.NetworkStatusDocumentV3`. The
+**descriptor_dump** file now looks like...
+
+::
+
+  network-status-version 3
+  vote-status consensus
+  consensus-method 18
+  valid-after 2014-11-17 23:00:00
+  fresh-until 2014-11-18 00:00:00
+  valid-until 2014-11-18 02:00:00
+  voting-delay 300 300
+  ... etc...
+
+You can then read it back with :func:`~stem.descriptor.__init__.parse_file`...
+
+::
+
+  from stem.descriptor import DocumentHandler, parse_file
+
+  consensus = next(parse_file(
+    '/tmp/descriptor_dump',
+    descriptor_type = 'network-status-consensus-3 1.0',
+    document_handler = DocumentHandler.DOCUMENT,
+  ))
+
+  for fingerprint, relay in consensus.routers.items():
+    print "%s: %s" % (fingerprint, relay.nickname)
diff --git a/docs/tutorials/mirror_mirror_on_the_wall.rst b/docs/tutorials/mirror_mirror_on_the_wall.rst
index 8fc196b..26a158c 100644
--- a/docs/tutorials/mirror_mirror_on_the_wall.rst
+++ b/docs/tutorials/mirror_mirror_on_the_wall.rst
@@ -8,6 +8,7 @@ with what they are and where to get them then you may want to skip to the end.
 * :ref:`where-can-i-get-the-current-descriptors`
 * :ref:`where-can-i-get-past-descriptors`
 * :ref:`can-i-get-descriptors-from-the-tor-process`
+* :ref:`saving-and-loading-descriptors`
 * :ref:`putting-it-together`
 
 .. _what-is-a-descriptor:
@@ -138,8 +139,58 @@ through Tor's control socket...
 
   from stem.descriptor import parse_file
 
-  for desc in parse_file(open("/home/atagar/.tor/cached-consensus")):
-    print "found relay %s (%s)" % (desc.nickname, desc.fingerprint)
+  for desc in parse_file('/home/atagar/.tor/cached-consensus'):
+    print 'found relay %s (%s)' % (desc.nickname, desc.fingerprint)
+
+.. _saving-and-loading-descriptors:
+
+Saving and loading descriptors
+------------------------------
+
+Tor descriptors are just plaintext documents. As such, if you'd rather not use
+`Pickle <https://wiki.python.org/moin/UsingPickle>`_ you can persist a
+descriptor by simply writing it to disk, then reading it back later.
+
+::
+
+  from stem.descriptor.remote import DescriptorDownloader
+
+  downloader = DescriptorDownloader()
+  server_descriptors = downloader.get_server_descriptors().run()
+
+  with open('/tmp/descriptor_dump', 'wb') as descriptor_file:
+    descriptor_file.write(''.join(map(str, server_descriptors)))
+
+Our *server_descriptors* here is a list of
+:class:`~stem.descriptor.server_descriptor.RelayDescriptor` instances. When we
+write it to a file this looks like...
+
+::
+
+  router default 68.229.17.182 443 0 9030 
+  platform Tor 0.2.4.23 on Windows XP
+  protocols Link 1 2 Circuit 1
+  published 2014-11-17 23:42:38
+  fingerprint EE04 42C3 6DB6 6903 0816 247F 2607 382A 0783 2D5A
+  uptime 63
+  bandwidth 5242880 10485760 77824
+  extra-info-digest 1ABA9FC6B912E755483D0F4F6E9BC1B23A2B7206
+  ... etc...
+
+We can then read it back with :func:`~stem.descriptor.__init__.parse_file`
+by telling it the type of descriptors we're reading...
+
+::
+
+  from stem.descriptor import parse_file
+
+  server_descriptors = parse_file('/tmp/descriptor_dump', descriptor_type = 'server-descriptor 1.0')
+
+  for relay in server_descriptors:
+    print relay.fingerprint
+
+For an example of doing this with a consensus document `see here
+<examples/persisting_a_consensus.html>`_.
 
 .. _putting-it-together:

More information about the tor-commits mailing list