commit 23042f197de3aaed34f7c29c12359165aeba5ef3 Author: Damian Johnson atagar@torproject.org Date: Mon Nov 17 22:10:49 2014 -0800
Example in our tutorial for writing descriptors to disk and reading them back
Neat suggestion by mmcc on...
https://trac.torproject.org/projects/tor/ticket/13774 --- docs/change_log.rst | 4 ++ docs/contents.rst | 1 + docs/tutorials/double_double_toil_and_trouble.rst | 4 ++ docs/tutorials/examples/persisting_a_consensus.rst | 72 ++++++++++++++++++++ docs/tutorials/mirror_mirror_on_the_wall.rst | 55 ++++++++++++++- 5 files changed, 134 insertions(+), 2 deletions(-)
diff --git a/docs/change_log.rst b/docs/change_log.rst index 1724497..7fdd2bf 100644 --- a/docs/change_log.rst +++ b/docs/change_log.rst @@ -71,6 +71,10 @@ The following are only available within Stem's `git repository
* The /info command errored for relays without contact information.
+ * **Website** + + * Added an example for `writing descriptors to disk and reading them back <tutorials/mirror_mirror_on_the_wall.html#saving-and-loading-descriptors>`_ (:trac:`13774`) + .. _version_1.2:
Version 1.2 diff --git a/docs/contents.rst b/docs/contents.rst index 15283f1..f7a49d2 100644 --- a/docs/contents.rst +++ b/docs/contents.rst @@ -17,6 +17,7 @@ Contents tutorials/examples/exit_used tutorials/examples/list_circuits tutorials/examples/outdated_relays + tutorials/examples/persisting_a_consensus
change_log download diff --git a/docs/tutorials/double_double_toil_and_trouble.rst b/docs/tutorials/double_double_toil_and_trouble.rst index 19ff0c5..ce6c58a 100644 --- a/docs/tutorials/double_double_toil_and_trouble.rst +++ b/docs/tutorials/double_double_toil_and_trouble.rst @@ -103,3 +103,7 @@ Descriptors Provides information about the current votes from Tor's Bandwidth Authorities.
+* `Saving and Loading a Tor Consensus <examples/persisting_a_consensus.html>`_ + + Example for writing a Tor consensus to disk, and reading it back. + diff --git a/docs/tutorials/examples/persisting_a_consensus.rst b/docs/tutorials/examples/persisting_a_consensus.rst new file mode 100644 index 0000000..d9c681b --- /dev/null +++ b/docs/tutorials/examples/persisting_a_consensus.rst @@ -0,0 +1,72 @@ +Saving and Loading a Tor Consensus +================================== + +.. image:: /_static/buttons/back.png + :target: ../double_double_toil_and_trouble.html + +Reading and writing a Tor consensus to disk is similar to `other descriptor +types <../mirror_mirror_on_the_wall.html#saving-and-loading-descriptors>`_ +with one small difference. + +Most descriptors are just about a single relay. Server descriptors and +microdescriptors, for instance, can be concatenated together and dumped to a +file because they're each independent of each other. + +The Tor consensus, however, is a larger document containing information about +the Tor network in addition to a little data on each of the relays. + +In Stem the overall document is a +:class:`~stem.descriptor.networkstatus.NetworkStatusDocumentV3`, and the +information on individual relays are +:class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3` instances. + +Why does this matter? By default when you read a consensus Stem provides you +**just** the :class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3`. +This is for performance reasons, and because usually that's what developers +want. But for writing the conssensus to disk we'll want the whole document +instead. + +So how do we get it? Just tell Stem that's what you want. The +:class:`~stem.descriptor.__init__.DocumentHandler` tells Stem how to read the +consensus. For example, to write the consensus simply do the following... + +:: + + from stem.descriptor import DocumentHandler + from stem.descriptor.remote import DescriptorDownloader + + downloader = DescriptorDownloader() + consensus = downloader.get_consensus(document_handler = DocumentHandler.DOCUMENT).run()[0] + + with open('/tmp/descriptor_dump', 'w') as descriptor_file: + descriptor_file.write(str(consensus)) + +Our *consensus* here is the current +:class:`~stem.descriptor.networkstatus.NetworkStatusDocumentV3`. The +**descriptor_dump** file now looks like... + +:: + + network-status-version 3 + vote-status consensus + consensus-method 18 + valid-after 2014-11-17 23:00:00 + fresh-until 2014-11-18 00:00:00 + valid-until 2014-11-18 02:00:00 + voting-delay 300 300 + ... etc... + +You can then read it back with :func:`~stem.descriptor.__init__.parse_file`... + +:: + + from stem.descriptor import DocumentHandler, parse_file + + consensus = next(parse_file( + '/tmp/descriptor_dump', + descriptor_type = 'network-status-consensus-3 1.0', + document_handler = DocumentHandler.DOCUMENT, + )) + + for fingerprint, relay in consensus.routers.items(): + print "%s: %s" % (fingerprint, relay.nickname) diff --git a/docs/tutorials/mirror_mirror_on_the_wall.rst b/docs/tutorials/mirror_mirror_on_the_wall.rst index 8fc196b..26a158c 100644 --- a/docs/tutorials/mirror_mirror_on_the_wall.rst +++ b/docs/tutorials/mirror_mirror_on_the_wall.rst @@ -8,6 +8,7 @@ with what they are and where to get them then you may want to skip to the end. * :ref:`where-can-i-get-the-current-descriptors` * :ref:`where-can-i-get-past-descriptors` * :ref:`can-i-get-descriptors-from-the-tor-process` +* :ref:`saving-and-loading-descriptors` * :ref:`putting-it-together`
.. _what-is-a-descriptor: @@ -138,8 +139,58 @@ through Tor's control socket...
from stem.descriptor import parse_file
- for desc in parse_file(open("/home/atagar/.tor/cached-consensus")): - print "found relay %s (%s)" % (desc.nickname, desc.fingerprint) + for desc in parse_file('/home/atagar/.tor/cached-consensus'): + print 'found relay %s (%s)' % (desc.nickname, desc.fingerprint) + +.. _saving-and-loading-descriptors: + +Saving and loading descriptors +------------------------------ + +Tor descriptors are just plaintext documents. As such, if you'd rather not use +`Pickle https://wiki.python.org/moin/UsingPickle`_ you can persist a +descriptor by simply writing it to disk, then reading it back later. + +:: + + from stem.descriptor.remote import DescriptorDownloader + + downloader = DescriptorDownloader() + server_descriptors = downloader.get_server_descriptors().run() + + with open('/tmp/descriptor_dump', 'wb') as descriptor_file: + descriptor_file.write(''.join(map(str, server_descriptors))) + +Our *server_descriptors* here is a list of +:class:`~stem.descriptor.server_descriptor.RelayDescriptor` instances. When we +write it to a file this looks like... + +:: + + router default 68.229.17.182 443 0 9030 + platform Tor 0.2.4.23 on Windows XP + protocols Link 1 2 Circuit 1 + published 2014-11-17 23:42:38 + fingerprint EE04 42C3 6DB6 6903 0816 247F 2607 382A 0783 2D5A + uptime 63 + bandwidth 5242880 10485760 77824 + extra-info-digest 1ABA9FC6B912E755483D0F4F6E9BC1B23A2B7206 + ... etc... + +We can then read it back with :func:`~stem.descriptor.__init__.parse_file` +by telling it the type of descriptors we're reading... + +:: + + from stem.descriptor import parse_file + + server_descriptors = parse_file('/tmp/descriptor_dump', descriptor_type = 'server-descriptor 1.0') + + for relay in server_descriptors: + print relay.fingerprint + +For an example of doing this with a consensus document `see here +<examples/persisting_a_consensus.html>`_.
.. _putting-it-together: