Damian, thanks. Your summary pretty well sums up where i am trying to start. I will try this out and go from there.
On Jun 10, 2013, at 11:08 PM, Damian Johnson atagar@torproject.org wrote:
Hi Sarah. I'm not really sure what you're trying to ask in most of these questions. Assuming that your goal is simply 'I want to connect to port 9051 and dump information about all the relays I'm presently using in my circuits' then the following should do the trick.
Take the following with a grain of salt - I just wrote it, I haven't tried running it. Should be close though. ;)
from stem.control import Controller
with Controller.from_port(port = 9051) as controller: my_relay_fingerprints = [] # fingerprints of all relays in our circuits
for circ in controller.get_circuits(): my_relay_fingerprints += [fp for (fp, nickname) in circ.path]
for fingerprint in my_relay_fingerprints: desc = controller.get_network_status(fingerprint) country = controller.get_info("ip-to-country/%s" % desc.address, "unknown")
print "relay: %s:" % fingerprint print " address: %s:%s" % (desc.address, desc.or_port) print " locale: %s" % country
Did you have any other questions?
On Mon, Jun 10, 2013 at 9:43 AM, SARAH CORTES sarah@lewis.is wrote:
Damian, thanks, this is very helpful.
is there a way to do this in torrc? Else, i suppose i will need to:
- create a socket or connection to my port 9051 ; do i need/can i use
TORRC_CONTROL_SOCKET ? 2) call get_circuits() ; grab the relay fingerprints Do I need circuit = controller.get_circuit(circ_id) 3) return class:`stem.events.CircuitEvent` ; for the given circuit Not sure whether or where to use the "path" attribute 4) call controller.get_network_status() to get IP address, nickname, ORPort, Should I use: desc_by_fingerprint = controller.get_network_status(test_relay.fingerprint)
desc_by_nickname =controller.get_network_status(test_relay.nickname)
- use Maxmind- i already have the GeoIPLite DB to grab AS and country, and
onion code also from Arturo
Any guidance is appreciated
https://lists.torproject.org/pipermail/tor-commits/2012-December/051174.html
get_circuit(self, circuit_id, default = UNDEFINED):
- """
- Provides a circuit presently available from tor.
- :param int circuit_id: circuit to be fetched
- :param object default: response if the query fails
- :returns: :class:`stem.events.CircuitEvent` for the given circuit
- :raises:
* :class:`stem.ControllerError` if the call fails* ValueError if the circuit doesn't existAn exception is only raised if we weren't provided a defaultresponse.
- """
- try:
for circ in self.get_circuits():if circ.id == circuit_id:return circraise ValueError("Tor presently does not have a circuit with the idof '%s'" % circuit_id)
- except Exception, exc:
if default: return defaultelse: raise exc- def get_circuits(self): """ Provides the list of circuits Tor is currently handling.
On Jun 10, 2013, at 10:34 AM, Damian Johnson atagar@torproject.org wrote:
Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep off spam, which some of my questions at first may be, essentially Qs. But, if you think they would be of interest to tor-dev, or others could help, just let me know, and i will sign up for it.
They certainly are! If you're interested in tor and development then I would definitely suggest being on that list. Including it for this thread.
I am trying to figure out how to pull in the nodes that are actually used in my Tor circuits. They are the nodes reflected in the Network map function.
You want the get_circuits() method. As you mentioned the 'path' attribute has the relays in your present circuits...
https://stem.torproject.org/api/control.html#stem.control.Controller.get_cir...
I have created a MySql DB of some of my Tor circuits and nodes which i am analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, nickname, country) manually from my laptop's Tor network map.
That certainly sounds painful. The circuit paths will provide the relay fingerprints which you can use for get_network_status() to get the address, nickname, ORPort, etc...
https://stem.torproject.org/api/control.html#stem.control.Controller.get_net...
As for locales that would be done via get_info('ip-to-country/<address>')...
https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt#l672
... and ultimately to AS and country.
AS will require the Maxmind AS database or something else. I know that Onionoo includes the AS information so the options that come to mind are either to (a) see how it does it or (b) query Onionoo for this information.
https://onionoo.torproject.org/
And i have read much of the control-spec, don't know how much is out of date
The control-spec should be up to date. If there are any inaccuracies then please let us know!
Did you have any other questions? -Damian
On Mon, Jun 10, 2013 at 4:44 AM, SARAH CORTES sarah@lewis.is wrote:
Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep off spam, which some of my questions at first may be, essentially Qs. But, if you think they would be of interest to tor-dev, or others could help, just let me know, and i will sign up for it.
For example, I am trying to figure out how to pull in the nodes that are actually used in my Tor circuits. They are the nodes reflected in the Network map function. I have read your fine stem tutorials, it is so impressive you have dome so much work and documentation on your own initiative. It is certainly a huge help to me and otter researchers like me.
I have created a MySql DB of some of my Tor circuits and nodes which i am analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, nickname, country) manually from my laptop's Tor network map. I am analyzing their country, so I am seeking to map nickname or fingerprint of the node to IP address and ultimately to AS and country. Ultimately, I am analyzing jurisdictional arbitrage and the hostility factors of differ countries to Tor traffic, and whether it is possible to incorporate country or AS in to the Tor path algorithm.
I can see the entry guard nicknames and fingerprints in my state file and pick them up there, if necessary. Wish i could just edit the logs to display all nodes, then figure out how to map the node fingerprint or nickname to its IP address them AS and country. I understand there is a Tor atlas function but have not read it carefully yet.
I have read documentation on the contents of the consensus file, the descriptors, geoip and state files, and played around with them a bit to understand them. I grasp that a fingerprint is a hash of the node's public key, and one or both of which seem to be the most reliable ID for a node.
I can see from your documentation that variable path(nickname, fingerprint) contains the data i want. Just need to be able to get that data easily. Not sure if editing my torrc file is the way or what. If you have any suggestions, they are much appreciated.
And i have read much of the control-spec, don't know how much is out of date https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt
I have read this on modifying torrc, and played around with modifying torrc, on an elementary basis: https://www.torproject.org/docs/faq.html.en#torrc
and some tickets on modifying torrc https://trac.torproject.org/projects/tor/ticket/6147
and what I believe is your excellent documentation on stem events; https://stem.torproject.org/api/response.html#stem.response.events.Bandwidth...
On Jun 8, 2013, at 2:31 PM, Damian Johnson atagar@torproject.org wrote:
Hi Sarah. Yup, I'd be more than happy to help if you have any stem questions. Email would definitely be better since we look to keep missing each other on irc. I'd appreciate it though if we discussed it on the tor-dev@ email list...
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On Sat, Jun 8, 2013 at 3:42 AM, SARAH CORTES sarah@lewis.is wrote:
Hi Damian, I'm Sarah Cortes, a researcher. I tried you on irc on #tor-dev but was unable to reach you. We met at the tor dev meeting which i attended in March. I have some Qs for you regarding stem. Please let me know if you are available, thanks