commit 441dd8f4e59cc2340d15b8e8c38cb81c6f407596 Author: Damian Johnson <atagar@torproject.org> Date: Sat May 23 15:16:03 2015 -0700 Make our To Russia With Love tutorial less error prone On StackOverflow the vast majority of Stem questions relate to this tutorial, and with good reason - it's error prone. Making the following changes... * Opting for PycURL over SocksiPy. PycURL is both cleaner, and our SocksiPy example transparently routed *all* traffic over Tor. As a result when folks try to expand the example with a Tor controller connection it fails (since that control connection is going to a Tor exit). * Note common gotchas and how they can be addressed. Most questions relate to being unable to bootstrap Tor due to a lack of Russian exits, and not having PycURL's PROXYTYPE_SOCKS5_HOSTNAME. --- docs/tutorials/to_russia_with_love.rst | 134 ++++++++++++++++++++------------ 1 file changed, 85 insertions(+), 49 deletions(-) diff --git a/docs/tutorials/to_russia_with_love.rst b/docs/tutorials/to_russia_with_love.rst index 6bb4252..8a709f5 100644 --- a/docs/tutorials/to_russia_with_love.rst +++ b/docs/tutorials/to_russia_with_love.rst @@ -1,62 +1,86 @@ To Russia With Love =================== -* :ref:`using-socksipy` * :ref:`using-pycurl` +* :ref:`using-socksipy` * :ref:`reading-twitter` -.. _using-socksipy: +.. _using-pycurl: -Using SocksiPy --------------- +Using PycURL +------------ + +Say it's 1982, the height of the Cold War, and you're a journalist doing a +piece on how the Internet looks from behind the Iron Curtain. Ignoring the +minor detail that the Internet doesn't yet exist, we'll walk you through how +you could do it - no passport required! + +The Internet isn't uniform. Localization, censorship, and selective service +based on your IP's geographic location can make the Internet a very different +place depending on where you're coming from. + +Tor relays are scattered all over the world and, as such, you can pretend to be +from any place running an exit. This can be especially useful to evade pesky +geolocational restrictions, such as news sites that refuse to work while you're +traveling abroad. -Say it's 1982, the height of the Cold War, and you're a journalist doing a piece on how the Internet looks from behind the Iron Curtain. Ignoring the minor detail that the Internet doesn't yet exist, we'll walk you through how you could do it - no passport required! +Tor makes `configuring your exit locale +<https://www.torproject.org/docs/faq.html.en#ChooseEntryExit>`_ easy through +the **ExitNodes** torrc option. Note that you don't need a control port (or +even Stem) to do this, though they can be useful if you later want to do +something more elaborate. -The Internet isn't uniform. Localization, censorship, and selective service based on your IP's geographic location can make the Internet a very different place depending on where you're coming from. +In the following example we're using Stem to `start Tor +<../api/process.html>`_, then read a site through it with `PycURL +<http://pycurl.sourceforge.net/>`_. This is not always reliable (some relays +are lemons) so you may need to run this more than once. -Tor relays are scattered all over the world and, as such, you can pretend to be from any place running an exit. This can be especially useful to evade pesky geolocational restrictions, such as news sites that refuse to work while you're traveling abroad. +Having an issue? The following are some common gotchas... -Tor makes `configuring your exit locale <https://www.torproject.org/docs/faq.html.en#ChooseEntryExit>`_ easy through the **ExitNodes** torrc option. Note that you don't need a control port (or even Stem) to do this, though they can be useful if you later want to do something more elaborate. +* PycURL's **PROXYTYPE_SOCKS5_HOSTNAME** was added in v7.19.5.1. Try `upgrading + <http://tech.michaelaltfield.net/2015/02/22/pycurl-through-tor-without-leaking-dns-lookups/>`_ + if you get an AttributeError about it. -In the following example we're using Stem to `start Tor <../api/process.html>`_, then read a site through it with `SocksiPy <http://socksipy.sourceforge.net/>`_. This is not always reliable (some relays are lemons) so you may need to run this more than once. +* The following example for exiting through Russia will only work if... well, + the Tor network *has* a Russian exit. Often this isn't the case. If Tor fails + to bootstrap try dropping the line with **'ExitNodes': '{ru}'**. -**Do not rely on the following not to leak.** Though it seems to work there may be edge cases that expose your real IP. If you have a suggestion for how to improve this example then please `let me know <https://www.atagar.com/contact/>`_! +**Do not rely on the following not to leak.** Though it seems to work there may +be edge cases that expose your real IP. If you have a suggestion for how to +improve this example then please `let me know +<https://www.atagar.com/contact/>`_! :: + import pycurl import StringIO - import socket - import urllib - import socks # SocksiPy module import stem.process from stem.util import term SOCKS_PORT = 7000 - # Set socks proxy and wrap the urllib module - - socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', SOCKS_PORT) - socket.socket = socks.socksocket - - # Perform DNS resolution through the socket - - def getaddrinfo(*args): - return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (args[0], args[1]))] - - socket.getaddrinfo = getaddrinfo - def query(url): """ - Uses urllib to fetch a site using SocksiPy for Tor over the SOCKS_PORT. + Uses pycurl to fetch a site using the proxy on the SOCKS_PORT. """ + output = StringIO.StringIO() + + query = pycurl.Curl() + query.setopt(pycurl.URL, url) + query.setopt(pycurl.PROXY, 'localhost') + query.setopt(pycurl.PROXYPORT, SOCKS_PORT) + query.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5_HOSTNAME) + query.setopt(pycurl.WRITEFUNCTION, output.write) + try: - return urllib.urlopen(url).read() - except: - return "Unable to reach %s" % url + query.perform() + return output.getvalue() + except pycurl.error as exc: + return "Unable to reach %s (%s)" % (url, exc) # Start an instance of Tor configured to only exit through Russia. This prints @@ -85,43 +109,55 @@ In the following example we're using Stem to `start Tor <../api/process.html>`_, .. image:: /_static/locale_selection_output.png -.. _using-pycurl: +.. _using-socksipy: -Using PycURL ------------- +Using SocksiPy +-------------- -Besides SocksiPy, you can also use `PycURL <http://pycurl.sourceforge.net/>`_ to do the same. To do so replace the query() function above with... +Besides PycURL, you can also use `SocksiPy <http://socksipy.sourceforge.net/>`_ +to do the same. Be aware that the following example routes **all** socket +connections through Tor, so this'll break our ability to connect to Tor's +control port. To use this approach simply replace the query() function above +with... :: - import pycurl + import socks # SocksiPy module + import socket + import urllib + + # Set socks proxy and wrap the urllib module + + socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', SOCKS_PORT) + socket.socket = socks.socksocket + + # Perform DNS resolution through the socket + + def getaddrinfo(*args): + return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (args[0], args[1]))] + + socket.getaddrinfo = getaddrinfo def query(url): """ - Uses pycurl to fetch a site using the proxy on the SOCKS_PORT. + Uses urllib to fetch a site using SocksiPy for Tor over the SOCKS_PORT. """ - output = StringIO.StringIO() - - query = pycurl.Curl() - query.setopt(pycurl.URL, url) - query.setopt(pycurl.PROXY, 'localhost') - query.setopt(pycurl.PROXYPORT, SOCKS_PORT) - query.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5_HOSTNAME) - query.setopt(pycurl.WRITEFUNCTION, output.write) - try: - query.perform() - return output.getvalue() - except pycurl.error as exc: - return "Unable to reach %s (%s)" % (url, exc) + return urllib.urlopen(url).read() + except: + return "Unable to reach %s" % url .. _reading-twitter: Reading Twitter --------------- -Now lets do somthing a little more interesting, and read a Twitter feed over Tor. This can be done `using thier API <https://dev.twitter.com/rest/reference/get/statuses/user_timeline>`_, for authentication `see their instructions <https://dev.twitter.com/oauth/overview/application-owner-access-tokens>`_... +Now lets do somthing a little more interesting, and read a Twitter feed over +Tor. This can be done `using thier API +<https://dev.twitter.com/rest/reference/get/statuses/user_timeline>`_, for +authentication `see their instructions +<https://dev.twitter.com/oauth/overview/application-owner-access-tokens>`_... ::