- tor-dev - lists.torproject.org

Torsocks locking design
by David Goulet 16 Jun '13

16 Jun '13

Hi everyone, I'm posting here the small design document I've made for the Torsocks locking mechanism. I'm looking for reviews, comments, improvements, ... well anything useful! :). (Even English mistakes since it's not my primary language). Future changes will be pushed here if any are needed after this thread is done with. https://github.com/dgoulet/torsocks/blob/rewrite/doc/proposals/01-Thread-sa… Thanks people! David

2 2

TBB check sums
by Linus Nordberg 15 Jun '13

15 Jun '13

Hi, I've built TBB's using the new and shiny gitian thing. Thank you Mike Perry for putting lots of effort into this! I'm at commit f4869b0b (tor-browser-bundle.git). Check sums below. --8<---------------cut here---------------start------------->8--- user@host:~/usr/src/tor-browser-bundle/gitian$ sha256sum TorBrowserBundle-3.0-alpha-1-osx32_* tor-browser-linux* torbrowser-install-3.0-alpha-1_* 1f3265462f267c30bf9bf8a8ac41904f03166f49ce78ed1c88e16915bf644313 TorBrowserBundle-3.0-alpha-1-osx32_de.zip f588f5ffebea4fce1ecac7d4c694b94afd202e0d82fbf6a19e153c8e9c8e8f12 TorBrowserBundle-3.0-alpha-1-osx32_en-US.zip d7067a2eae9b1899e854aa1083653b452ed33137139ed94807651c208b17da33 TorBrowserBundle-3.0-alpha-1-osx32_es-ES.zip 372d9790ca6be27f617afb97ad7923d46f8b823935375ce3262abd09af6038ab TorBrowserBundle-3.0-alpha-1-osx32_fa.zip dfd614b35d00a2ac9f5d5196bd4f0064fc53a43e909e32cbc847b42121463c85 TorBrowserBundle-3.0-alpha-1-osx32_fr.zip 735bd35612a756e28d011d61c5d682be4305d33db8f48b6e8721ff40c0784e7b TorBrowserBundle-3.0-alpha-1-osx32_it.zip cf38210e03a64bec93e94a8c28bb9b3e3462f4588d816515a10a22ef39126392 TorBrowserBundle-3.0-alpha-1-osx32_ko.zip fe3f355972b5712d9084ddb3f6926bf86910a4e0613d701523d35ae1a8591d00 TorBrowserBundle-3.0-alpha-1-osx32_nl.zip 81bd77b6e446aa2eb0cf1969dbad80923ebc26fcf55f3153fb1f2fcf4a554cae TorBrowserBundle-3.0-alpha-1-osx32_pl.zip 008e1eeb8ecd6f7f0e8543943eba1d40764a7b33edff06ccf4cf87de5ac74424 TorBrowserBundle-3.0-alpha-1-osx32_pt-PT.zip df47cd6c8167e7a610df586d69c0e6e7368c904e0e42ff0186c2831932338db1 TorBrowserBundle-3.0-alpha-1-osx32_ru.zip c9af80e798824ce1154cbf32ef1ba30f1a6e7b7e682e285dc69a2aaeada7bfa2 TorBrowserBundle-3.0-alpha-1-osx32_vi.zip ae47dd903c7683db89d769b8fb26cc0df8ebede424af09aa1108d773a808bf11 TorBrowserBundle-3.0-alpha-1-osx32_zh-CN.zip c6de2a39100be02f806a820196d87c565225cfbd637754f23206463eab1b96d5 tor-browser-linux32-3.0-alpha-1_de.tar.xz d28f7e708f01fb522e4a13d987859ace98cbe0f18ec3de71391c0fd3a82a527a tor-browser-linux32-3.0-alpha-1_en-US.tar.xz 74ac8e1b3d86148d3f94fdc5c21636859b07339bec00da68bf5a15a12b4b3b45 tor-browser-linux32-3.0-alpha-1_es-ES.tar.xz e4d575f89de04c955981923e467e879cdf5f6c2ac36a0d984058d0f16bdc3cd7 tor-browser-linux32-3.0-alpha-1_fa.tar.xz 5a39818f665906b2542568329c0f859264057b9c8df21d78b0f2b34d8bdfd35e tor-browser-linux32-3.0-alpha-1_fr.tar.xz 7f59ac51056e0d933027c57b032dc4896bca61d0f464bb28f3eff1dc46c1d58c tor-browser-linux32-3.0-alpha-1_it.tar.xz f9fc3d164f79e44fd1d43d15f59ccd56175d7d18a413b18ac1abe15adbc34662 tor-browser-linux32-3.0-alpha-1_ko.tar.xz 52d9401edc032d833500bafed3e7ae277365429c6e1d9293dfa6a88132a4b0b2 tor-browser-linux32-3.0-alpha-1_nl.tar.xz 52218f2d8f0d7192ed832087ef2fa1a6cb013117e887467965875c04752d7b6f tor-browser-linux32-3.0-alpha-1_pl.tar.xz dc201b5e9c9cac0883924b8558d3f317c10136107bded636722762030316acbe tor-browser-linux32-3.0-alpha-1_pt-PT.tar.xz 4be1e6b43d55bc3ef1d4d5db360ec6b9981d6ea7e5be7c19b9d8ed81e4f72b77 tor-browser-linux32-3.0-alpha-1_ru.tar.xz 057aebd3ad07d6edd661c314102459fb97377074a06b7e68035b00be4ab948ed tor-browser-linux32-3.0-alpha-1_vi.tar.xz f79c46aa4f52000faffe32ed1a8e10b82c481861fcd131bed2eeccb5078c1df2 tor-browser-linux32-3.0-alpha-1_zh-CN.tar.xz 763a96564571c364f73c1046465c6745b2bb150f914ab2740639d779da2529cb tor-browser-linux64-3.0-alpha-1_de.tar.xz 5e5dd31612df58f2cfb870e4169c48b2578b4eb0beaee6c391189ba82e4584d5 tor-browser-linux64-3.0-alpha-1_en-US.tar.xz abc771efa43b62d3f7055528913622e1c657acb20dbdfcfebabcf25b16f20a4b tor-browser-linux64-3.0-alpha-1_es-ES.tar.xz fe9a7905073089870d7f19857d9bc1e1d7e508ea283519ddb9abd24d6b6a0d6f tor-browser-linux64-3.0-alpha-1_fa.tar.xz bc2849260e716c5e4cab7e7e157f6cf2bbea93533c5692f466bce964bff25677 tor-browser-linux64-3.0-alpha-1_fr.tar.xz 6564a05774a602557af3eb3ffbedbb542c0d966a69ee8997be56eb1ab0a02c0c tor-browser-linux64-3.0-alpha-1_it.tar.xz d92f5d527e7aa48cbe500b84e8d1c69da0a716329f2b3203fdd2d5aacb8e5856 tor-browser-linux64-3.0-alpha-1_ko.tar.xz 81dddc65e0534902d7c1a7f39f4235e9d11e449d75647555c1ca4c04625eea63 tor-browser-linux64-3.0-alpha-1_nl.tar.xz 5b678842584503a6ddd6e40c7759ff14b0d87ca9f1a6721b2c6a10355ef97e35 tor-browser-linux64-3.0-alpha-1_pl.tar.xz 3df16ccf393169d8057d11ec4d85cb2940ddfb7f65c37c2607870f9c16b28efa tor-browser-linux64-3.0-alpha-1_pt-PT.tar.xz 371777984b02897f25b6515b61db22b9fbfa0c78d253d19fc5aade754a26c74b tor-browser-linux64-3.0-alpha-1_ru.tar.xz cfc4118a8345a8226c4638ad4719a7b12e5d3efd7faf5d8575fd2b341d510956 tor-browser-linux64-3.0-alpha-1_vi.tar.xz a49a8906e0f3013e2e02c41b68bde961d25100c1a27a8a497f5cd2bcb8764c27 tor-browser-linux64-3.0-alpha-1_zh-CN.tar.xz 951ae9027cb398e7721f9fff2af357c032316b64738bf7867035dc4697da8f3a torbrowser-install-3.0-alpha-1_de.exe c0d17168b2e2c8d9d8762da62f207d3660494cb0fe4879217db0c2902a553abb torbrowser-install-3.0-alpha-1_en-US.exe ab219b4f862aa86fedeadfde723158c5c2d4d84514d91b9fa6012d3b90c95646 torbrowser-install-3.0-alpha-1_es-ES.exe f8047706446d1dc112768edd01f1af246b1b57408b307d19996b39ffeaaba452 torbrowser-install-3.0-alpha-1_fa.exe 22e2b4d76df62185f3d011bf061cb75b173a0171c8198f21baff66ccde2905e2 torbrowser-install-3.0-alpha-1_fr.exe 370aea69718e443d3ede77d772ebb1ec2a0517e3d8b9f68573fa70dd9871a208 torbrowser-install-3.0-alpha-1_it.exe ad2032d82862177fd9e07a0aa9b65720a6f4817322760de44db9fcd84d016f50 torbrowser-install-3.0-alpha-1_ko.exe 29c72de44e65a6f5f0ee83ba79e5db13d430e2d7dda244528d16f540ad1db21f torbrowser-install-3.0-alpha-1_nl.exe bd524c8dce296db2c1f9c09b024886faffc28bf4407526b2f310fdf66459faed torbrowser-install-3.0-alpha-1_pl.exe 178f87dad198a565b13a419e5d10eabd760e09feb9e892d5d0baf6259fed3f58 torbrowser-install-3.0-alpha-1_pt-PT.exe d803200c16e17f8165463e416d5d043d1edcd9a564b9734f7c83feed8c2d55d1 torbrowser-install-3.0-alpha-1_ru.exe f623e459eb4579a6de1e2014cfdfcd2059b78d16fafb6d8e3873ab26d0e8bda0 torbrowser-install-3.0-alpha-1_vi.exe c888e325784048067698b59c9db0798fba23a83b53cc645ad6d83f7b1abf3f57 torbrowser-install-3.0-alpha-1_zh-CN.exe --8<---------------cut here---------------end--------------->8---

2 1

Torsocks reengineered
by David Goulet 14 Jun '13

14 Jun '13

Hi everyone, About a week ago I've sent an email to the Torsocks maintainers to address some concerns I had about important issues of the current code base. For the reasons found below, I proposed a rewrite that I am willing to do and submit it to the community once I feel ok with it. After some back and forth, the consensus was that I should go ahead and do it. Best case scenario, I come up with a tested library that has a smaller set of problems (hopefully) and could eventually be considered to replace the current version. Worst case, I lose my time :). Now, I'm not looking to do that alone in my corner or anything, what I want is this effort to be as open as possible for everyone that want to help, contribute or/and follow it. Some help would be really appreciated for the compat. part (Windows, OS X, BSD, etc...). Here is the repository I've created and started working on the rewrite. It has been branched from upstream master at commit e0bf65b5d3ca0e28b827c08d80b0fe1841d5a149 https://github.com/dgoulet/torsocks/tree/rewrite There might be some rebasing going on in that branch in the next weeks so please don't consider it right now as "git stable" since I'm in the heavy part of getting everything together before starting a "normally growing" master branch. If you like to contribute or help, let me know and we'll make something work. Big thanks to nickm, ioerror and sysrqb for their help on all this. Cheers! David Torsocks rewrite reasons: --------------- (You can see that as either we put in 2000 lines of code to fix it in a ~3000 lines repository or we start fresh with some key aspect, maintainability and security in mind.) Code and Maintenance Issues: 1) The code needs to be easier to maintain, so that it isn't sporadically maintained. Considering the number of issues (see below), it needs more love on a regular basis right now. 2) It has never been safe from the start so a rewrite will NOT impact the current state of security. There is not much tests right now. 3) Small code base. Fix it clean before it gets too big. 4) One of the biggest technical issues is that it's not thread safe and fixing it right now would add an important impact on performance adding locks to multiple global data structures. A clean rewrite would allow to take into account this *very* important feature from the start without any ugly hacks in the current code base. 5) This is a very, and I can't stress this enough, _VERY_ intrusive library so extra care MUST be put in making it bullet proof in terms of error handling. Right now, there is multiple call sites that can fail on error. 6) There is not even close to enough tests, thus not loosing that stable portion of a project. 7) No compatibility layer is present for multiple OS and architecture. 8) FD passing through Unix socket support. * Should at least be detected and notify the user or even deny execution since the process receiving the socket is not under torsocks control. 9) Symbol name space polluted. (should all be prefixed). 10) Libc hijacked calls of torsocks don't return correct values in error path. * This is bad because for instance the caller expect errno to be set and makes decision upon that value that MUST be right once returned. There is multiple call sites that are problematic. 11) IPv6 support. ----------------

7 11

Tor over Open Garden mesh network
by Paige Peterson 13 Jun '13

13 Jun '13

Hello all, I was recommended to join this group after emailing the general help address with the following message: I am the Community Manager for Open Garden <https://opengarden.com>, a p2p mesh networking application for phones, tablets and laptops. Currently, Tor doesn't work over Open Garden and when talking to my lead developer about it a while ago, he said that Tor would need to support an upstream proxy - which he can't find any documentation on. Some users of Open Garden have requested the ability to use Tor over the Open Garden protocol and as the Community Manager, I am attempting to understand whether or not this is feasible to implement. If there is any light that can be shed from your side as to what would need to happen to support each other, then please enlighten me! If it would be more productive to have a conversation with our lead developer, then I can ask him to join the list as well. Thanks for your time! -- Paige Peterson Open Garden Community Manager forum.opengarden.com

2 2

Building better pluggable transports - GSoC 2013 project
by Chang Lan 11 Jun '13

11 Jun '13

Hello everyone! I am a Tor GSoC student who will be working on the pluggable transports this summer. My mentor is Steven and my co-mentor is George Kadianakis. It is great to be part of the Tor community! Steven already kicked off the discussion about how to build better transports. The original project proposal[1] discussed the possibility of sending data over UDP with extra efforts to guarantee reliable in-order delivery. However, as George mentioned recently[3], ScrambleSuite[2] may already solve the issue of scanning resistance. Given that ScrambleSuite is being deployed, improving protocol obfuscation will be my main focus. HTTP impersonation is really useful, since there are numerous HTTP proxy outside the censored region, while the number of bridges is quite limited. What I'm gonna be doing during the summer is implementing a good enough HTTP impersonation based on pluggable transports specification. There are still many open questions indeed. Discussions are more than welcome! I'll be keeping my GSoC log here: http://changlan.github.io/obfsproxy I'm also available #tor-dev@oftc with the handle "clan", feel free to ping me there! [1]: http://changlan.info/gsoc.html [2]: http://www.cs.kau.se/philwint/scramblesuit/ [3]: https://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/changla… Cheers, Chang

5 6

Help Wanted: Evaluate our Tool for (privacy-preserving) Automated Debugging
by Christian Grothoff 11 Jun '13

11 Jun '13

Hi! Collecting good bug reports from users is a challenge, as most users are unable (or unwilling) to inspect problems in depth. Existing tools to automatically report program failures are often problematic as they either produce superficial reports or are prone to leaking private information (i.e. by including a full snapshot of the process address space in the form of a core dump). At TUM, we are developing Monkey[1], a free software tool that automatically generates high-quality privacy-preserving bug reports (for C code) which we hope to use for GNUnet and Tor eventually. (It runs with GNUnet and Tor already, but might need more work prior to real-world use.) We have reproduced and analyzed a set of real-world bugs using the system and would now like to study how well developers can find the bugs using the extracted information. Please participate in our public "survey" (or rather, experiment) where we measure your performance at identifying bugs based on crash reports collected via different methods, including traditional stack traces, core dumps (accessed via a live gdb session) and of course Monkey. So if you can spare the time, please help our efforts by participating in our survey[1] and passing the link to your colleagues and friends. (Who finds more bugs faster?) [0]: https://gnunet.org/monkey/ [1]: https://gnunet.org/monkey/survey/index.php/943594?lang=en Happy hacking! Christian Grothoff & Markus Teich p.s.: We'll be happy to talk with you about Monkey at the Tor developer meeting in Munich; however, Markus's thesis deadline is earlier...

1 0

Re: [tor-dev] grabbing Tor circuit (node) data- Tor stem, torrc and Tor control port
by Damian Johnson 11 Jun '13

11 Jun '13

Hi Sarah. I'm not really sure what you're trying to ask in most of these questions. Assuming that your goal is simply 'I want to connect to port 9051 and dump information about all the relays I'm presently using in my circuits' then the following should do the trick. Take the following with a grain of salt - I just wrote it, I haven't tried running it. Should be close though. ;) from stem.control import Controller with Controller.from_port(port = 9051) as controller: my_relay_fingerprints = [] # fingerprints of all relays in our circuits for circ in controller.get_circuits(): my_relay_fingerprints += [fp for (fp, nickname) in circ.path] for fingerprint in my_relay_fingerprints: desc = controller.get_network_status(fingerprint) country = controller.get_info("ip-to-country/%s" % desc.address, "unknown") print "relay: %s:" % fingerprint print " address: %s:%s" % (desc.address, desc.or_port) print " locale: %s" % country Did you have any other questions? On Mon, Jun 10, 2013 at 9:43 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > Damian, thanks, this is very helpful. > > is there a way to do this in torrc? Else, i suppose i will need to: > > 1) create a socket or connection to my port 9051 ; do i need/can i use > TORRC_CONTROL_SOCKET ? > 2) call get_circuits() ; grab the relay fingerprints > Do I need circuit = controller.get_circuit(circ_id) > 3) return class:`stem.events.CircuitEvent` ; for the given circuit > Not sure whether or where to use the "path" attribute > 4) call controller.get_network_status() to get IP address, nickname, ORPort, > Should I use: > desc_by_fingerprint = controller.get_network_status(test_relay.fingerprint) > > + desc_by_nickname = > controller.get_network_status(test_relay.nickname) > > 5) use Maxmind- i already have the GeoIPLite DB to grab AS and country, and > onion code also from Arturo > > > Any guidance is appreciated > > > https://lists.torproject.org/pipermail/tor-commits/2012-December/051174.html > > get_circuit(self, circuit_id, default = UNDEFINED): > + """ > + Provides a circuit presently available from tor. > + > + :param int circuit_id: circuit to be fetched > + :param object default: response if the query fails > + > + :returns: :class:`stem.events.CircuitEvent` for the given circuit > + > + :raises: > + * :class:`stem.ControllerError` if the call fails > + * ValueError if the circuit doesn't exist > + > + An exception is only raised if we weren't provided a default > response. > + """ > + > + try: > + for circ in self.get_circuits(): > + if circ.id == circuit_id: > + return circ > + > + raise ValueError("Tor presently does not have a circuit with the id > of '%s'" % circuit_id) > + except Exception, exc: > + if default: return default > + else: raise exc > + > def get_circuits(self): > """ > Provides the list of circuits Tor is currently handling. > > > > On Jun 10, 2013, at 10:34 AM, Damian Johnson <atagar(a)torproject.org> wrote: > > Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. > > > They certainly are! If you're interested in tor and development then I > would definitely suggest being on that list. Including it for this > thread. > > I am trying to figure out how to pull in the nodes that are actually used in > my Tor circuits. They are the nodes reflected in the Network map function. > > > You want the get_circuits() method. As you mentioned the 'path' > attribute has the relays in your present circuits... > > https://stem.torproject.org/api/control.html#stem.control.Controller.get_ci… > > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. > > > That certainly sounds painful. The circuit paths will provide the > relay fingerprints which you can use for get_network_status() to get > the address, nickname, ORPort, etc... > > https://stem.torproject.org/api/control.html#stem.control.Controller.get_ne… > > As for locales that would be done via get_info('ip-to-country/<address>')... > > https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt#l672 > > ... and ultimately to AS and country. > > > AS will require the Maxmind AS database or something else. I know that > Onionoo includes the AS information so the options that come to mind > are either to (a) see how it does it or (b) query Onionoo for this > information. > > https://onionoo.torproject.org/ > > And i have read much of the control-spec, don't know how much is out of date > > > The control-spec should be up to date. If there are any inaccuracies > then please let us know! > > Did you have any other questions? -Damian > > On Mon, Jun 10, 2013 at 4:44 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > > Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. > > For example, I am trying to figure out how to pull in the nodes that are > actually used in my Tor circuits. They are the nodes reflected in the > Network map function. I have read your fine stem tutorials, it is so > impressive you have dome so much work and documentation on your own > initiative. It is certainly a huge help to me and otter researchers like me. > > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. I am analyzing > their country, so I am seeking to map nickname or fingerprint of the node to > IP address and ultimately to AS and country. Ultimately, I am analyzing > jurisdictional arbitrage and the hostility factors of differ countries to > Tor traffic, and whether it is possible to incorporate country or AS in to > the Tor path algorithm. > > I can see the entry guard nicknames and fingerprints in my state file and > pick them up there, if necessary. Wish i could just edit the logs to display > all nodes, then figure out how to map the node fingerprint or nickname to > its IP address them AS and country. I understand there is a Tor atlas > function but have not read it carefully yet. > > I have read documentation on the contents of the consensus file, the > descriptors, geoip and state files, and played around with them a bit to > understand them. I grasp that a fingerprint is a hash of the node's public > key, and one or both of which seem to be the most reliable ID for a node. > > I can see from your documentation that variable path(nickname, fingerprint) > contains the data i want. Just need to be able to get that data easily. Not > sure if editing my torrc file is the way or what. If you have any > suggestions, they are much appreciated. > > And i have read much of the control-spec, don't know how much is out of date > https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt > > I have read this on modifying torrc, and played around with modifying torrc, > on an elementary basis: > https://www.torproject.org/docs/faq.html.en#torrc > > and some tickets on modifying torrc > https://trac.torproject.org/projects/tor/ticket/6147 > > and what I believe is your excellent documentation on stem events; > https://stem.torproject.org/api/response.html#stem.response.events.Bandwidt… > > > > > > > > On Jun 8, 2013, at 2:31 PM, Damian Johnson <atagar(a)torproject.org> wrote: > > Hi Sarah. Yup, I'd be more than happy to help if you have any stem > questions. Email would definitely be better since we look to keep > missing each other on irc. I'd appreciate it though if we discussed it > on the tor-dev@ email list... > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > On Sat, Jun 8, 2013 at 3:42 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > > Hi Damian, I'm Sarah Cortes, a researcher. I tried you on irc on #tor-dev > but was unable to reach you. We met at the tor dev meeting which i attended > in March. I have some Qs for you regarding stem. Please let me know if you > are available, thanks > > >

2 1

traversing the source code of tor
by H S 10 Jun '13

10 Jun '13

Hi there, We would like to implement some specific protocol in Tor as an implementation project. Our one of pseudo codes that we have to take into account is the one mentioned on the paper under the link http://www.cypherpunks.ca/~iang/pubs/UC-OR.pdf , page 5, figure 2. I would like to know how can I connect or relate this pseudo code with the Tor source code to help us understand where we should insert our desired modification on the source code of the Tor. For instance in the first of the pseudo code line we have "upon an input (setup):". And know in which file/lines of the Tor source code we have this?

2 1

Re: [tor-dev] stem
by Damian Johnson 10 Jun '13

10 Jun '13

> Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. They certainly are! If you're interested in tor and development then I would definitely suggest being on that list. Including it for this thread. > I am trying to figure out how to pull in the nodes that are actually used in > my Tor circuits. They are the nodes reflected in the Network map function. You want the get_circuits() method. As you mentioned the 'path' attribute has the relays in your present circuits... https://stem.torproject.org/api/control.html#stem.control.Controller.get_ci… > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. That certainly sounds painful. The circuit paths will provide the relay fingerprints which you can use for get_network_status() to get the address, nickname, ORPort, etc... https://stem.torproject.org/api/control.html#stem.control.Controller.get_ne… As for locales that would be done via get_info('ip-to-country/<address>')... https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt#l672 > ... and ultimately to AS and country. AS will require the Maxmind AS database or something else. I know that Onionoo includes the AS information so the options that come to mind are either to (a) see how it does it or (b) query Onionoo for this information. https://onionoo.torproject.org/ > And i have read much of the control-spec, don't know how much is out of date The control-spec should be up to date. If there are any inaccuracies then please let us know! Did you have any other questions? -Damian On Mon, Jun 10, 2013 at 4:44 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > Hi, Damian, thanks. I am happy to discuss it on tor-dev@. But I want to keep > off spam, which some of my questions at first may be, essentially Qs. But, > if you think they would be of interest to tor-dev, or others could help, > just let me know, and i will sign up for it. > > For example, I am trying to figure out how to pull in the nodes that are > actually used in my Tor circuits. They are the nodes reflected in the > Network map function. I have read your fine stem tutorials, it is so > impressive you have dome so much work and documentation on your own > initiative. It is certainly a huge help to me and otter researchers like me. > > I have created a MySql DB of some of my Tor circuits and nodes which i am > analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address, > nickname, country) manually from my laptop's Tor network map. I am analyzing > their country, so I am seeking to map nickname or fingerprint of the node to > IP address and ultimately to AS and country. Ultimately, I am analyzing > jurisdictional arbitrage and the hostility factors of differ countries to > Tor traffic, and whether it is possible to incorporate country or AS in to > the Tor path algorithm. > > I can see the entry guard nicknames and fingerprints in my state file and > pick them up there, if necessary. Wish i could just edit the logs to display > all nodes, then figure out how to map the node fingerprint or nickname to > its IP address them AS and country. I understand there is a Tor atlas > function but have not read it carefully yet. > > I have read documentation on the contents of the consensus file, the > descriptors, geoip and state files, and played around with them a bit to > understand them. I grasp that a fingerprint is a hash of the node's public > key, and one or both of which seem to be the most reliable ID for a node. > > I can see from your documentation that variable path(nickname, fingerprint) > contains the data i want. Just need to be able to get that data easily. Not > sure if editing my torrc file is the way or what. If you have any > suggestions, they are much appreciated. > > And i have read much of the control-spec, don't know how much is out of date > https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt > > I have read this on modifying torrc, and played around with modifying torrc, > on an elementary basis: > https://www.torproject.org/docs/faq.html.en#torrc > > and some tickets on modifying torrc > https://trac.torproject.org/projects/tor/ticket/6147 > > and what I believe is your excellent documentation on stem events; > https://stem.torproject.org/api/response.html#stem.response.events.Bandwidt… > > > > > > > > On Jun 8, 2013, at 2:31 PM, Damian Johnson <atagar(a)torproject.org> wrote: > > Hi Sarah. Yup, I'd be more than happy to help if you have any stem > questions. Email would definitely be better since we look to keep > missing each other on irc. I'd appreciate it though if we discussed it > on the tor-dev@ email list... > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > On Sat, Jun 8, 2013 at 3:42 AM, SARAH CORTES <sarah(a)lewis.is> wrote: > > Hi Damian, I'm Sarah Cortes, a researcher. I tried you on irc on #tor-dev > but was unable to reach you. We met at the tor dev meeting which i attended > in March. I have some Qs for you regarding stem. Please let me know if you > are available, thanks > >

2 1

Gitian-based Deterministic Build System for TBB (Need MacOS Help!)
by Mike Perry 10 Jun '13

10 Jun '13

Over the past couple weeks I've been redoing the TBB build system to use Gitian to produce alpha TBBs using Tor Launcher instead of Vidalia. I have succeeded in producing deterministic, localized builds of TBB for Linux and Windows. This means that independent people all over the world can now easily produce their own bundles for these platforms fresh from sources, and have their bundles exactly match the bundles the Tor Project releases, down to the SHA256 hash. If we leverage this property wisely, it will allow us to defend against targeted attacks against our bundlers and their build machines, and even ultimately ensure the integrity of our bundles in the event of key compromise of the gpg keys used to sign the bundles. My plan for this is for there to be between 2-3 official signers for each bundle, where each person produces their build independently, and signs the (identical) result files. To further protect against targeted attack, in addition to these 2-3 official signers, we need some people to be "secret verifiers". Ideally these people would not be publicly affiliated with the Tor Project, but would still produce their own bundles anyway. If their SHA256 ever fails to match the signed bundles, that person should anonymously open a trac ticket (using the cypherpunks account) and attach the bundle files that differ for analysis. The differing files can be found easily enough with 'diff -r'. To ensure the existence of these "secret verifiers", I believe that the official signers should occasionally conspire to conduct "Fire Drills", where they all agree to alter the bundle in some innocuous way (such as adding whitespace to a config file or a Firefox JS file), and ensure that a verifiers anonymously report the verification failure. In future versions of Tor, we should probably add a consensus field consisting of a url to a file that lists the current recommended bundle hashes and versions, along with the current SHA256 of that file, to anchor the bundle authentication the Tor's current trust root (the 9 dirauth keys). To try out the new build system, please see the README, and let me know where the system could use clarification or improvement to make it easier to use: https://gitweb.torproject.org/builders/tor-browser-bundle.git/blob/HEAD:/gi… The build system has some quirks that are worth mentioning: 1. It requires you run it from either an Ubuntu 12.04 or above host with KVM support, *or* you run it from an Ubuntu 12.04 or above chroot/VM. The bundle scripts try to detect your current situation and suggest that you "export USE_LXC=1" from your shell if you need to, to cause the system to use LXC instead of KVM (so that you can build from an Ubuntu VM or on a machine that does not otherwise support KVM). 2. We currently have no MacOS support. To support MacOS, we need to create cross-compilers for it so that we can produce builds from the Gitian VMs (which again are Ubuntu). A few people have done this. I have sent them mail asking for instructions on how to reproduce their compiler packages: http://www.tarnyko.net/en/?q=node/9 https://launchpad.net/~flosoft/+archive/cross-apple/+packages http://wiki.freepascal.org/Cross_compiling_OSX_on_Linux Unfortunately, at least one of those URLs say that to produce a cross-compiler, you need access to an OSX SDK. Since I do not have a Mac that is currently supported by recent OSX SDKs, and since we *really* want to be sure that the cross-compilers we produce use code from a fresh known-good SDK install, I won't be doing this. Please let me know if you'd like to help tackle this problem. In the meantime, I am going to work on the rest of the "Short Term" TODO items, and produce official alpha bundles for Linux and Windows, so we can test Tor Launcher in an official alpha release. Here's the TODO file: https://gitweb.torproject.org/builders/tor-browser-bundle.git/blob/HEAD:/gi… Happy building! -- Mike Perry

3 6