- tor-dev - lists.torproject.org

Damian's Status Report - November 2014
by Damian Johnson 23 Nov '14

23 Nov '14

Hi all. Next weekend I'm returning to ye olde isle of Vashon to feast with family, so figured I might as well send this out early. I'll be in a Thanksgiving food coma this time next week, after all... My devious scheme for November was to finish rewriting arm's graph panel and, while that certainly made progress, Stem, DocTor, and internal discussions stole the spotlight this month... -------------------------------------------------------------------------------- DocTor -------------------------------------------------------------------------------- * We now notify directory authority operators directly of issues in addition to the tor-consensus-health@ list. This should reduce duration of outages and other issues with the authorities. * We replaced turtles with longclaw. Sebastian deserves major kudos for orchestrating this. [1] * Properly fixed DocTor's OOM issues by not shelling out to send notification emails. [2] * DocTor detected a burst of relays from Google App Engine. These relays lacked any contact information so we dropped them from the network as a potential Sybil attack. [3] If you're the operator of these then please let us know! We'd be delighted to add you back in once there's a proper family and contact information. -------------------------------------------------------------------------------- Stem -------------------------------------------------------------------------------- * Ossi Herrala made delightful improvements so we parse descriptors ~10% faster [4], and now read from the '.new' files in Tor's data directory [5]. Thanks Ossi! * Expanded our descriptor tutorials with an example of persisting descriptors to disk. Thanks to mmcc for the idea [6]... https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#saving… https://stem.torproject.org/tutorials/examples/persisting_a_consensus.html * Replaced quite of bit of boilerplate in Stem with a new @with_default decorator. Mmmmm, deleting code is fun... [7] * Updates to reflect changes in Tor's spec this month, including HS_DESC's new reason attribute, and validating the new consensus parameters. A couple other quick things of note is that I spruced up our volunteer page [8] and attended this month's Seattle TA3M. The later had a nice bit of serendipity in that I met Anna, a UW PhD student who's looking into our Bandwidth Authorities [8]. Best of luck Anna! That's certainly an area that could do with a lot of improvement. ;) Cheers! -Damian [1] https://trac.torproject.org/projects/tor/ticket/13296 [2] https://gitweb.torproject.org/doctor.git/commitdiff/94a2e9d [3] https://lists.torproject.org/pipermail/tor-consensus-health/2014-November/0… [4] https://trac.torproject.org/projects/tor/ticket/13821 [5] https://trac.torproject.org/projects/tor/ticket/13756 [6] https://trac.torproject.org/projects/tor/ticket/13774 [7] https://gitweb.torproject.org/stem.git/commitdiff/56a1f03 [8] https://gitweb.torproject.org/project/web/webwml.git/commitdiff/05c1b60

1 0

botnets in tor
by Enkidu Mo Shiri 22 Nov '14

22 Nov '14

Hi everyone, am new and not very sure about rules here,hope its not off-topic. im master student researcher and i am working on botnet detection. it would be appreciated if anyone can help me with : I. anyway to detect botnet trafic out of normal tor traffic? II. is this solution possible to apply? to attach a script to botnet ( so we pass reverse engineering and no mess with codes of malware) and send it back to network, let the relays and last node which is botmaster recieves the compromised botnet ,and the script report our detector machine of any time it been re-routed in tor network ( hidden service and relays and botmaster) about : ip address of receiver hosts (relays) and their computer information( os,...) and consider last hop as botmaster. after using this technic for few botnets, we can have a good view of how relays (and which relays) been used for attack bt botnet and who is botmaster : it can help tor admin to clean relays, remove that hidden service , blacklist botmaster ip address thank you *Ehsan Moshiri (Enkidu)* *Digital Forensic Student* *H/P:+96164953954 , +961124249769* *Linkedin: http://my.linkedin.com/pub/enkidu-moshiri/59/baa/90b/ <http://my.linkedin.com/pub/enkidu-moshiri/59/baa/90b/>* *Facebook: Enkidu Mo Shi Ri* *wechat: Enkidu-Moshiri* *Line: Enkidu.Moshiri*

1 0

Using Shadow to increase agility of Sponsor R deliverables
by Rob Jansen 21 Nov '14

21 Nov '14

Hi David, Roger, I think it would be great if we could show off some HS performance improvements at the next Sponsor R PI meeting in January, both as a way to show that we are making progress on the performance front and also to help demonstrate our agility and how quickly we can get results on new designs. I’m here to advocate the use of Shadow to help in this regard. So far, the list of deliverable that we could possibly show improvements is quite small: https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorR#Tickets From the list of completed tickets, the only one that stands out to me as providing a performance boost is #13211 (Allow optimistic data on connections to hidden services): https://trac.torproject.org/projects/tor/ticket/13211 This seems like a somewhat small change; despite that, do we think this may be something worth simulating to verify it works as expected and understand the extent to which it reduces time to first byte for HS downloads? Are there other HS performance improvements that we think may be ready by January? Best, Rob

2 6

Stem roadmap, and ideas for the interpreter?
by Damian Johnson 21 Nov '14

21 Nov '14

Hi Nick. While sipping my morning coffee I just realized that you might not be aware of my sinister long schemes, and that might be to the detriment of us both. The roadmap I have for myself goes something like... Step 1: Write Stem so we have a well tested, stable controller library. (Done [1]) Step 2: Write a control interpreter to greatly simplify low level controller troubleshooting. (Done [2]) Step 3: Rewrite arm, both simplifying the codebase and smoothing the rough edges that have confused users for years. (In progress, gonna keep me busy for the better part of a year) Step 4: Write a web interface similar to arm, so relay operators can have a nice monitor by going to localhost. [3] I bring this up because honestly we don't communicate much. I don't know core tor's long term plans, and I get the impression you don't know much about Stem either. My hope had been that at step #3 we'd start bundling arm and Stem with the Tor relaying bundle, providing users both a terminal monitor and the tor-prompt command. But since we axed that with the removal of Vidalia I suppose I'll need to hit step #4 before proposing that we re-introduce those bundles (... and by extension making it so Windows and OSX users can once have an easy option for relaying). Have you ever used Stem's tor-prompt before? https://stem.torproject.org/tutorials/down_the_rabbit_hole.html If not then I'd appreciate input so this can be a useful tool to you before proposing that we make it more widely available. Cheers! -Damian PS. More general ideas on directions to take Stem, the tor-prompt, and this whole space are also welcome both from you and the list! [1] https://stem.torproject.org/ [2] https://stem.torproject.org/tutorials/down_the_rabbit_hole.html [3] https://www.torproject.org/getinvolved/volunteer.html.en#relayWebPanel

1 0

Hidden Service authorization UI
by George Kadianakis 21 Nov '14

21 Nov '14

Hidden Service authorization is a pretty obscure feature of HSes, that can be quite useful for small-to-medium HSes. Basically, it allows client access control during the introduction step. If the client doesn't prove itself, the Hidden Service will not poroceed to the rendezvous step. This allows HS operators to block access in a lower level than the application-layer. It also prevents guard discovery attacks since the HS will not show up in the rendezvous. It's also a way for current HSes to hide their address and list of IPs from the HSDirs (we get this for free in rend-spec-ng.txt). In the current HS implementation there are two ways to do authorization: https://gitweb.torproject.org/torspec.git/blob/HEAD:/rend-spec.txt#l768 both have different threat models. In the future "Next Generation Hidden Services" specification there are again two ways to do authorization: https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/224-rend-spe… One way is with a password and the other is with a public key. I suspect that HS authorization is very rare in the current network, and if we believe it's a useful tool, it might be worthwhile to make it more useable by people. For example, it would be interesting if TBB would allow people to input a password/pubkey upon visiting a protected HS. Protected HSes can be recognized by looking at the "authentication-required" field of the HS descriptor. Typing your password on the browser is much more useable than editing a config file. Furthermore on the server-side, like meejah recently suggested [0], it would be nice if there was a way for HSes to be able to dynamically add/remove authorized clients using the control port. [0]: https://lists.torproject.org/pipermail/tor-dev/2014-October/007693.html

12 14

tor-0.2.5.10: weird test failure
by Thomas Klausner 21 Nov '14

21 Nov '14

Hi! I'm currently trying to update tor in pkgsrc (www.pkgsrc.org) to 0.2.5.10. When I run the self tests with 'make check', I see: .../work/tor-0.2.5.10> make check make check-TESTS check-local PASS: src/test/test make all-am ============================================================================ Testsuite summary for tor 0.2.5.10 ============================================================================ # TOTAL: 1 # PASS: 1 # SKIP: 0 # XFAIL: 0 # FAIL: 0 # XPASS: 0 # ERROR: 0 ============================================================================ python2.7 ./src/test/test_cmdline_args.py ./src/or/tor "." ............ ---------------------------------------------------------------------- Ran 12 tests in 0.299s OK python2.7 ./src/test/ntor_ref.py test-tor OK python2.7 ./src/test/ntor_ref.py self-test OK ./src/test/test-bt-cl assert | python2.7 ./src/test/bt_test.py BAD ./src/test/test-bt-cl crash | python2.7 ./src/test/bt_test.py BAD [1] Segmentation fault (core dumped) ./src/test/test-... | Done python2.7 ./src/... Running src/test/test directly works fine though: .../work/tor-0.2.5.10> ./src/test/test ... util/fgets_eagain: SKIPPED ... 222 tests ok. (1 skipped) Any ideas where to look first? Thomas

1 0

torsocks-2.0: NetBSD mmap patch
by Thomas Klausner 21 Nov '14

21 Nov '14

Hi! Please apply the attached patch to fix torsocks-2.0 on NetBSD. Thanks, Thomas

1 0

Re: [tor-dev] How to distribute Tor with other software
by David Stainton 21 Nov '14

21 Nov '14

Greetings, olde thread resurrection: Earlier Meejah pointed out that the tor control port can be used to create tor hidden services. Now that tor trac ticket #11291 (https://trac.torproject.org/projects/tor/ticket/11291) is resolved this will actually be usable? There are deployment issues. The program must run as a user who is a member of the tor group in order to read the hostname file and thus retrieve the onion address. With the current upstream master tor you can use the HiddenServiceDirGroupReadable option, like this: SETCONF hiddenservicedir=/var/lib/tor-alpha-hidden-services/hiddenService01 hiddenserviceport="80 127.0.0.1:8080" HiddenServiceDirGroupReadable=1 This should result in the hostname file being readable by the tor group. Is anyone else working on tor hidden service file sharing and or tor controller written in golang? I've implemented a golang api for creating and destroying tor hidden services using the control port... and a golang commandline tool for sharing directories via http + tor hidden service similar to carml pastebin or onionshare. If we write these tools in golang then we can use a deterministic build system much like the recent gitian golang deterministic builds for golang pluggable transports. Cheers, David

1 0

Pluggable transports meeting tomorrow (16:00UTC Wednesday 19th of November 2014)
by Yawning Angel 18 Nov '14

18 Nov '14

Hello! just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur tomorrow at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. Thanks for your attention! -- Yawning Angel

1 0

User experience issue
by Adam Shostack 15 Nov '14

15 Nov '14

Hi, I just had the fun experience of trying to walk someone who's not very technical and not familiar with Tor through downloading and installing the TBB. One element which came out was she was confused by the relationship between Tor, the Tor Browser Bundle, and Vidalia, and became frustrated by that experience. ("What's this Vidalia thing? I thought we were installing Tor!") I'm unsure if it would be appropriate/useful to file a bug, and so would like advice on if/how to tackle this. Adam

2 2