- tor-dev - lists.torproject.org

Regarding involving in tor project
by dp docs 06 Jan '15

06 Jan '15

Dear Sir, I would like to get involved in the tor project and want to start working on the "TOR Codebase Cleanup" project. My IRC nickname is "sdpan21". I have registered my nickname there. But while asking any question, I am getting an error "can not send to channel". I have tried few suggestion from google to resolve this issue but none ofo them seems to wrking. I think Either i am am not resolving it properly or there problem is different in some sense. So I request , please help me resolving this issue. And guide me how can I get involved in the project mentioned above. I have good knowledge of c/c++ java python. please respond ASAP. I appreciate your effort for resolving the issue and for giving time to guide me. Thanks Durgesh Pandey.

4 6

how to simulate TOR network through chutney?
by Mohiuddin Ebna Kawsar 06 Jan '15

06 Jan '15

HI, I need to simulate TOR network in minimal case [1 ENTRY,1 RELAY,1 EXIT] node. how can i do that use chutney? how to setup exit node on my localhost and how to make circuit with that exit node? following are my config file with generated (slightly modified) torc files i use basic-min for configure. ############## basic-min########################## Authority = Node(tag="a", authority=1, relay=1, torrc="authority.tmpl") Relay = Node(tag="r", relay=1, torrc="relay.tmpl") Client = Node(tag="c", torrc="client.tmpl") NODES = Authority.getN(1) + Relay.getN(1) + Client.getN(1) ConfigureNodes(NODES) ########################Authority-000a torc ######################### TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/000a RunAsDaemon 1 ConnLimit 60 Nickname test000a ShutdownWaitLength 0 PidFile /home/chutney-master/net/nodes/000a/pid Log notice file /home/chutney-master/net/nodes/000a/notice.log Log info file /home/chutney-master/net/nodes/000a/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 0 OrPort 5000 Address 127.0.0.1 DirPort 7000 #NOTE: Setting TestingServerConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingServerDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 # This file is named "relay.tmpl" for compatibility with previous # chutney versions # An exit relay that can exit to IPv4 & IPv6 localhost # (newer versions of tor need this to be explicitly configured) # An exit policy that allows exiting to IPv4 localhost ExitPolicy accept 127.0.0.0/8:* # An exit policy that allows exiting to IPv6 localhost ExitPolicy accept [::1]:* IPv6Exit 1 AuthoritativeDirectory 1 V3AuthoritativeDirectory 1 ContactInfo auth0(a)test.test ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:* TestingV3AuthInitialVotingInterval 300 TestingV3AuthInitialVoteDelay 20 TestingV3AuthInitialDistDelay 20 #TestingV3AuthVotingStartOffset 0 # Work around situations where the Exit and Guard flags aren't being set # These flags are set eventually, but it takes ~30 minutes # We could be more precise here, but it's easiest just to vote everything # Clients are sensible enough to filter out Exits without any exit ports, # and Guards without ORPorts # If your tor doesn't recognise TestingDirAuthVoteExit, update your chutney # to a version that includes the issue-13161-check-torrc-options features #TestingDirAuthVoteExit * #TestingDirAuthVoteGuard * #####################Relay 001r torc ############### TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/001r RunAsDaemon 1 ConnLimit 60 Nickname test001r ShutdownWaitLength 0 PidFile /home/chutney-master/net/nodes/001r/pid Log notice file /home/chutney-master/net/nodes/001r/notice.log Log info file /home/chutney-master/net/nodes/001r/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 0 OrPort 5001 Address 127.0.0.1 DirPort 7001 #NOTE: Setting TestingServerConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingServerDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 # This file is named "relay.tmpl" for compatibility with previous # chutney versions # An exit relay that can exit to IPv4 & IPv6 localhost # (newer versions of tor need this to be explicitly configured) # An exit policy that allows exiting to IPv4 localhost ExitPolicy accept 127.0.0.0/8:* # An exit policy that allows exiting to IPv6 localhost ExitPolicy accept [::1]:* IPv6Exit 1 ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:* ######################### client 002c torc ########################## TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/002c RunAsDaemon 1 ConnLimit 60 Nickname test002c ShutdownWaitLength 0 PidFile /home/raboon/Thesis/chutney-master/net/nodes/002c/pid Log notice file /home/chutney-master/net/nodes/002c/notice.log Log info file /home/chutney-master/net/nodes/002c/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 9002 #NOTE: Setting TestingClientConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingClientDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:*

1 0

test_cmdline_args.py on Windows
by Gisle Vanem 06 Jan '15

06 Jan '15

There are some problems with 'run_tor()' and line-endings. It doesn't forsee any '\r\n' from 'Popen()' and since 'lines()' only splits on '\n', most of the tests fails. With this little patch: @@ -64,7 +64,7 @@ yield fp[i:i+4] def lines(s): - out = s.split("\n") + out = s.splitlines() I only get 2 errors in that script. The others like: self.assertTrue(out_verif.endswith("Configuration was valid\n")) needs another patch which is beyond my Python knowledge. -- --gv

4 6

Easy research tasks
by spriver 06 Jan '15

06 Jan '15

Hey everyone out there, as given in the topic, are there any easy research tasks to start with? Or is somewhere some help needed? Cheers, spriver

4 4

Re: [tor-dev] Tor BSD underperformance (was [Tor-BSD] Recognizing Randomness Exhaustion)
by teor 06 Jan '15

06 Jan '15

On 1 Jan 2015, at 07:39 , Greg Troxel <gdt(a)lexort.com> wrote: > Libertas <libertas(a)mykolab.com> writes: > >> Some of the people at tor-bsd(a)lists.nycbug.org and I are trying to >> figure out why Tor relays under-perform when running on OpenBSD. Many >> such relays aren't even close to being network-bound, >> file-descriptor-bound, memory-bound, or CPU-bound, but relay at least >> 33-50% less traffic than would be expected of a Linux machine in the >> same situation. > > I'm more familiar with NetBSD, but hopefully my comments are helpful. > >> For those not familiar, a Tor relay will eventually have an open TCP >> connection for each of the other >6,000 active relays, and (if it allows >> exit traffic) must make outside TCP connections for the user's requests, >> so it's pretty file-hungry and crypto-intensive. > > It may also have something to do with TCP. A few thoughts: > > * run netstat -f inet and look and the send queues. That's not really > cleanly diagnostic, but if they are all huge, it's a clue > > * run netstat -m and vmstat -m (not sure those map from NetBSD). Look > for runnig out of mbufs and mbuf clusters. Perhaps bump up > NMBCLUSTERS in the kernel if it's not dynamic. Tor 0.2.6.2-alpha (just in the process of being released) has some changes to queuing behaviour using the KIST algorithm. The KIST algorithm keeps the queues inside tor, and makes prioritisation decisions from there, rather than writing as much as possible to the OS TCP queues. I'm not sure how functional it is on *BSDs, but Nick Mathewson should be able to comment on that. (I've cc'd tor-dev and Nick.) teor pgp 0xABFED1AC hkp://pgp.mit.edu/ https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w…

6 10

Summary of meek's costs, December 2014
by David Fifield 05 Jan '15

05 Jan '15

Here's the summary of meek's CDN fees for December 2014. Earlier reports: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html If you're just tuning in, meek is a pluggable transport introduced a few months ago. https://trac.torproject.org/projects/tor/wiki/doc/meek. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 -- total by CDN $600.63 + $917.89 + $0.00 = $1518.52 grand total Usage was up in December compared to November, but the relative increase wasn't as great as between November and October. https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… (The last few days are missing; see https://lists.torproject.org/pipermail/tor-dev/2014-December/008021.html.) Rather than attach bandwidth screenshots from the CDN control panels, this month I'll just link to the Globe pages for the different backends. A different relay is used depending on what backend you choose: meek-google: https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… meek-amazon: https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… meek-azure: https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… It's interesting that meek-google has more bandwidth (800 KB/s) but fewer clients (200), while meek-amazon has less bandwidth (550 KB/s) but more clients (340). meek-azure is behind both with 200 KB/s and 150 clients. I also attached screenshots of the bandwidth graphs from those pages. As a step toward reducing the bandwidth bills, this month I closed #12778, which reduces overhead by ensmallening HTTP headers. https://trac.torproject.org/projects/tor/ticket/12778 Headers are now less than half the size compared to before, which I estimate will save about 3% on bandwidth. We won't observe any effect, however, until there's a new release of Tor Browser. We're starting to move quite a bit of traffic. Not counting meek-azure, it's over 3.6 TB last month. (That's including an estimated 6–7% of meek-induced overhead.) == App Engine a.k.a. meek-google == Most of the App Engine bill is still using an existing credit. Of the total of $326.81, $230.00 was covered by credits. Here is how the costs broke down: 2132 GB $255.86 1419 instance hours $70.95 The attached meek-google-costs-2015-01-01.png shows the breakdown of costs between bandwidth and instance hours for the last few months. == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 105M requests $126.52 783 GB $101.09 Asia Pacific (Sydney) 183K requests $0.23 1 GB $0.18 Asia Pacific (Tokyo) 25M requests $30.28 146 GB $17.83 EU (Ireland) 57M requests $68.81 481 GB $37.23 South America (Sao Paulo) 1M requests $2.49 7 GB $1.58 US East (Northern Virginia) 19M requests $18.69 161 GB $12.39 -- total 207M requests $247.02 1579 GB $170.30* * The total from adding up subtotals is $0.01 higher than the actual bill, I think because of some rounding. == Azure a.k.a. meek-azure == I still can't figure out how to get total monthly bandwidth out of the Azure console. David Fifield

1 0

Reminder for weekly OONI dev meeting Monday 2015-01-05 18:00 UTC (19:00 CET)
by Arturo Filastò 05 Jan '15

05 Jan '15

Hello Oonitarians, This is a reminder that today there will be the weekly OONI meeting. It will happen as usual on the #ooni channel on irc.oftc.net at 18:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). Everybody is welcome to join us and bring their questions and feedback. See you later, ~ Arturo

1 0

Re: [tor-dev] Fwd: gettimeofday() Syscall Issues
by grarpamp 03 Jan '15

03 Jan '15

On Fri, Jan 2, 2015 at 12:24 PM, Konstantin Belousov <kostikbel(a)gmail.com> wrote: > On Fri, Jan 02, 2015 at 09:09:34AM -0500, grarpamp wrote: >> Some recent FreeBSD related questions in this app area. >> > What is the question ? > > As a background, I can repeat that FreeBSD implements syscall-less > gettimeofday() and clock_gettime() for x86 machines which have > usable RDTSC. The selection of the timecounter can be verified > by sysctl kern.timecounter.hardware, and enabled by default fast > gettimeofday(2) can be checked by sysctl kern.timecounter.fast_gettime. > > On some Nehalem machine, I see it doing ~30M calls/sec with enabled > fast_gettime, and ~6.25M calls/sec with disabled fast_gettime. This is > measured on 2.8GHz Core i7 930 with src/tools/tools/syscall_timing. > > Check your timecounter hardware. Since it was noted that the tests > were done in VM, check the quality of RDTSC emulation in your hypervisor. https://lists.torproject.org/pipermail/tor-dev/2015-January/thread.html http://docs.freebsd.org/mail/current/freebsd-performance.html Maybe I can just refer non subscribers out to the two lists above that way in case anyone sees anything interesting they can join/comment as desired. Background might be that Tor operators have some large relays on *BSD and were looking to validate, and ways to improve, performance there. Cheers. https://lists.torproject.org/pipermail/tor-relays/ https://lists.torproject.org/pipermail/tor-talk/

3 2

Re: [tor-dev] Tor Cloud Maintenance, revisited
by Jeremy Olexa 03 Jan '15

03 Jan '15

Hi Vlad, I agree. AWS pricing is not friendly for bandwidth users. However, it is free for the first year of use and the general population is not taking advantage of this. Which, AFACT, is one of the points of the Tor Cloud project. Once you are not free-tier eligible, I highly recommend using a cheaper service like DO, Linode, Vultr, etc. At least DO and Linode would be trivial to setup. In summary, there is no need to replace AWS, but instead to supplement it with other providers and make the Tor Cloud project more robust. -Jeremy On Fri, Jan 2, 2015 at 6:10 PM, Vlad Tsyrklevich <vlad(a)tsyrklevich.net> wrote: > Hi Jeremy, I've working on something related so I figured I'd comment. I've > been working on a TorCloud replacement using DigitalOcean's API, this has > the benefit of a simplified set-up process (one-click set-up) and the > pricing on bandwidth is a major win over AWS ($5 for 1TB U/L instead of $20 > for 40GB, though your move to t2.micro would move it down to $10.) The > back-end is there and pretty much ready to go, I've primarily been waiting > for my co-author to come back from vacation to finish the front-end. > > I haven't actually discussed sunsetting/replacing the current TorCloud with > the Tor Project so this project might as well be vaporware; however, I think > continuing to use AWS doesn't make sense for pricing and ease-of-use. All > that said, if you're taking this on, it'd be great to try to also address > TorCloud's currently susceptibility to discovery by internet-wide port > scanning as I mentioned in > https://lists.torproject.org/pipermail/tor-dev/2014-December/007957.html > > On Thu Jan 01 2015 at 1:23:49 PM Jeremy Olexa <jolexa(a)jolexa.net> wrote: >> >> Hi Everyone, Happy New Year, first post to the -dev list but I've been >> running some relays for months[1]. Overall a new user to Tor so feel >> free to point me elsewhere if I'm asking poor questions. >> >> I've noticed that the Tor Cloud project is dead in the water right >> now. The last post on this list is in June 2014[2] and the bugs have >> been neglected, especially the one I opened[3] which states that tor >> does not even start! I've seen that there is a new maintainer, but >> still don't see any [public] activity. >> >> There are a couple of issues that have opportunity to be handled >> better. A large roadblock seems to be building/publishing a new AMI so >> I've addressed that in a keeping it simple theme: >> - Use Amazon AMI instead of Ubuntu. Justification: more aligned to AWS >> and is a first rate citizen in the ecosystem, yum repos, security >> updates, etc >> - Use t2.micro instances. Justification: t1.micro (currently in use) >> are more expensive and less performant than t2.micro >> - Use cloud-init to fetch ec2-prep.sh and run the script to configure >> the instance[4]. Justification: Less likely to roll a new AMI just for >> ec2-prep.sh updates therefore more of a rolling update infrastructure. >> One example is adding a new bridge protocol, all newly launched >> instances would get the ec2-prep updates without rolling a new AMI. >> Justification 2: Less Tor Cloud maintainer work. Downside: AMI has >> dependency on gitweb.torproject.org's uptime - we could use S3 or some >> other CDN but this is starting to go against the simple theme. >> - Publishing AMI to us-east (N Virginia), us-west2 (Oregon), us-europe >> (Ireland). Justification: These are the lowest cost regions. >> - Not publishing a separate AMI for private bridges. Justification: IF >> the Tor Cloud project wants to provide configuration for private >> bridges, reusing the shared AMI makes more sense. I have an idea of >> using cloud-init's user-data field but need to test it. Justification >> 2: Simpler, less Tor Cloud maintainer work. >> - Not addressed (yet): Ubuntu's unattended upgrades concept. I have >> one idea (yum security plugin) but I haven't thought about all the >> implications. >> >> My AMI testing has proved that the above concepts work and I have ec2 >> bridges running in east and west[5]. I propose that the TOR project >> uses the above model and I'm willing to help facilitate that. I am >> willing to share the AMI with some select beta testers but I'm not >> ready to make it public yet (some changes are expected). It is my idea >> that once the final AMI is produced, the Tor Cloud project will not >> have to publish another AMI unless wanting to change the instance >> type, or other "infrastructure" changes. >> >> I look forward to hearing some feedback, thanks, >> Jeremy >> >> [1]: https://atlas.thecthulhu.com/#search/jolexa >> [2]: https://lists.torproject.org/pipermail/tor-dev/2014-June/007001.html >> [3]: https://trac.torproject.org/projects/tor/ticket/13391 >> [4]: https://github.com/jolexa/tor-cloud/blob/master/run-once.sh >> [5]: https://globe.thecthulhu.com/#/search/query=ec2bridgei >> _______________________________________________ >> tor-dev mailing list >> tor-dev(a)lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > > _______________________________________________ > tor-dev mailing list > tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >

1 0

Tor Cloud Maintenance, revisited
by Jeremy Olexa 03 Jan '15

03 Jan '15

Hi Everyone, Happy New Year, first post to the -dev list but I've been running some relays for months[1]. Overall a new user to Tor so feel free to point me elsewhere if I'm asking poor questions. I've noticed that the Tor Cloud project is dead in the water right now. The last post on this list is in June 2014[2] and the bugs have been neglected, especially the one I opened[3] which states that tor does not even start! I've seen that there is a new maintainer, but still don't see any [public] activity. There are a couple of issues that have opportunity to be handled better. A large roadblock seems to be building/publishing a new AMI so I've addressed that in a keeping it simple theme: - Use Amazon AMI instead of Ubuntu. Justification: more aligned to AWS and is a first rate citizen in the ecosystem, yum repos, security updates, etc - Use t2.micro instances. Justification: t1.micro (currently in use) are more expensive and less performant than t2.micro - Use cloud-init to fetch ec2-prep.sh and run the script to configure the instance[4]. Justification: Less likely to roll a new AMI just for ec2-prep.sh updates therefore more of a rolling update infrastructure. One example is adding a new bridge protocol, all newly launched instances would get the ec2-prep updates without rolling a new AMI. Justification 2: Less Tor Cloud maintainer work. Downside: AMI has dependency on gitweb.torproject.org's uptime - we could use S3 or some other CDN but this is starting to go against the simple theme. - Publishing AMI to us-east (N Virginia), us-west2 (Oregon), us-europe (Ireland). Justification: These are the lowest cost regions. - Not publishing a separate AMI for private bridges. Justification: IF the Tor Cloud project wants to provide configuration for private bridges, reusing the shared AMI makes more sense. I have an idea of using cloud-init's user-data field but need to test it. Justification 2: Simpler, less Tor Cloud maintainer work. - Not addressed (yet): Ubuntu's unattended upgrades concept. I have one idea (yum security plugin) but I haven't thought about all the implications. My AMI testing has proved that the above concepts work and I have ec2 bridges running in east and west[5]. I propose that the TOR project uses the above model and I'm willing to help facilitate that. I am willing to share the AMI with some select beta testers but I'm not ready to make it public yet (some changes are expected). It is my idea that once the final AMI is produced, the Tor Cloud project will not have to publish another AMI unless wanting to change the instance type, or other "infrastructure" changes. I look forward to hearing some feedback, thanks, Jeremy [1]: https://atlas.thecthulhu.com/#search/jolexa [2]: https://lists.torproject.org/pipermail/tor-dev/2014-June/007001.html [3]: https://trac.torproject.org/projects/tor/ticket/13391 [4]: https://github.com/jolexa/tor-cloud/blob/master/run-once.sh [5]: https://globe.thecthulhu.com/#/search/query=ec2bridgei

2 1

Re: [tor-dev] gettimeofday() Syscall Issues
by teor 02 Jan '15

02 Jan '15

> From: Yawning Angel <yawning(a)schwanenlied.me> > Subject: Re: [tor-dev] gettimeofday() Syscall Issues > > On Thu, 01 Jan 2015 23:42:42 -0500 > Libertas <libertas(a)mykolab.com> wrote: > >> The first two account for the bulk of the calls, as they are in the >> core data relaying logic. >> >> Ultimately, the problem seems to be that the caching is very weak. At >> most, only half of the calls to tor_gettimeofday_cached_monotonic() >> use the cache. It appears in the vomiting print statements that >> loading a single simple HTML page >> (http://www.openbsd.org/faq/ports/guide.html to be exact) will cause >>> 30 gettimeofday() syscalls. You can imagine how that would >>> accumulate for an exit carrying 800 KB/s if the caching >> doesn't improve much with additional circuits. > > So while optimization is cool and all, I'm not seeing why this > specifically is the underlying issue. > > Each cell can contain 498 bytes of user payload. Looking at things > simplistically this is 800 KiB/s -> 1644 cells/sec, leaving you with > approximately 608 microseconds of processing time per cell. > > On my i5-4250U box, gettimeofday() takes 22 ns on Linux, and 2441 ns on > FreeBSD. I'm not sure how accurate the FreeBSD results are as it was > in a VirtualBox VM (getpid() on the same VM takes 124 ns). If someone > has a OpenBSD box they should benchmark gettimeofday() and see how long > the call takes. > > Taking the FreeBSD case (since we know that tor works fine on Linux), a IPredator has complained that tor on Linux spends too much time calling time() when pushing 500Mbit/s, which is an issue for them under 3.x series kernels, but not kernel 2.6. https://ipredator.se/guide/torserver#performance > single gettimeofday() call takes approximately, 0.39% of the per-cell > processing budget. > > For reference (assuming gettimeofday() in *BSD really is this shit > performance wise), 7000 calls to gettimeofday() is 17.09 ms worth of > calls. > > The clock code in tor does need love, so I wouldn't object to cleanup, > but I'm not sure it's in the state where it's causing the massive > performance degradation that you are seeing. > Yawning/Libertas, I just reviewed my profiling of an exit relay running chutney verify with 200MB of random data. This is on OS X 10.9.5 with tor 0.2.6.2-alpha-dev running the chutney basic-min network. The three leaf functions that take the most time in the call graph are: * channel_timestamp_recv * channel_timestamp_active * time Each of these functions takes around 16% of the execution time, the next nearest function is sha1_block_data_order_avx on 4%. While I understand that OS X, BSD, and Linux syscalls aren't necessarily identical, we now have results for the following platforms suggesting that calling time() too often has a performance impact: * Linux kernel 3.x * OpenBSD * OS X 10.9 My results suggest a maximum performance improvement of 15% on OS X if we reduced the calls to time() to a reasonable number per second. teor pgp 0xABFED1AC hkp://pgp.mit.edu/ https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w…

2 2

Deadline Approaching: EECEA2015 (Electrical, Electronics, Computer Engineering and their Applications)
by Jackie Blanco 02 Jan '15

02 Jan '15

The Second International Conference on Electrical, Electronics, Computer Engineering and their Applications (EECEA2015) World Trade Center, Manila Philippines February 11, 2015 University of Perpetual Help System Dalta, Las Piñas - Manila, Philippines February 12-14, 2015 http://sdiwc.net/conferences/eecea2015/ All registered papers will be included in SDIWC Digital Library. =========================================================== The conference aims to enable researchers build connections between different digital applications. The event will be held over four days, with presentations delivered by researchers from the international community, including presentations from keynote speakers and state-of-the-art lectures. Trainings will be conducted by companies namely Microchip, Dassault Systèmes and TechSource in parallel with the paper presentations where delegates are encouraged to participate. RESEARCH TOPICS ARE NOT LIMITED TO: * Electronics Engineering * Electrical Engineering * Computer Engineering SUBMISSION GUIDELINES: - Researchers are encouraged to submit their work electronically. - Full paper must be submitted (Abstracts are not acceptable). - Submitted paper should not exceed 15 pages, including illustrations. All papers must be without page numbers. - Papers should be submitted electronically as pdf format without author(s) name. - Paper submission link: http://sdiwc.net/conferences/eecea2015/openconf/openconf.php IMPORTANT DATES: Submission Deadline: January 12, 2015 Notification of Acceptance: 2-3 weeks from the date of submission Camera Ready Submission: January 30, 2015 Registration Deadline: January 30, 2015 Seminar Date: February 11, 2015 Conference Dates: February 12-14, 2015

1 0

Re: [tor-dev] Tor BSD underperformance (was [Tor-BSD] Recognizing Randomness Exhaustion)
by teor 01 Jan '15

01 Jan '15

Libertas, As I wrote to you earlier today, IPredator on Linux complained of similar issues with excessive calls to time() under the Linux 3.x kernel series, but not 2.6. Let's track these issues in https://trac.torproject.org/projects/tor/ticket/14056 as they appear to be quite similar. I'd like to get a sense of how many calls per second this represents. (400,000 would seem to be 100 to 1000 per second, unless you were debugging for a long while.) I'd also like to know which function(s)/call stack(s) these calls are being made from. I wonder if TIME_IS_FAST is being defined as 1 in any of these Linux or BSD builds. This If TIME_IS_FAST is not defined (the default), we could change these calls to approx_time(), which is much faster. And in that case there should only be around 1 call to time() each second, which is clearly not the behaviour that either Libertas or IPredator are seeing. Libertas, can you search your tor binary or tor debug symbols for the function "approx_time"? grep will do this for you, or strings, or perhaps gdb if you feel like doing it the hard way. I'll add these questions to trac ticket #14056 as well. Libertas wrote: > I just ran ktrace/kdump (used for observing system calls) on the Tor > process of my exit node, which relays about 800 KB/s. It listed >400,000 > calls to gettimeofday(). The list was swamped with them. > > I think I remember reading somewhere that that sort of system call is > way slower in OpenBSD than Linux. Could this be related to the issue? > I've found a lot of similarly mysterious slowdowns related to *BSD > gettimeofday() on other projects' bug trackers, but nothing definitive. > > As you've likely noticed, although I started this discussion I'm very > new to system-level performance tuning. Let me know if I'm not making > sense, or if there's something else I should be focusing on. > >> <snip>

2 4

log files on windows
by Tony Willingham 01 Jan '15

01 Jan '15

Is there a trick to writing to log files in windows 7? The sample torrc file has this: Log debug file @LOCALSTATEDIR@/log/tor/debug.log I changed mine to this with no success: Log debug file C:\tor\tor_debug.log

1 0

Trac spam filter
by David Fifield 31 Dec '14

31 Dec '14

The Trac spam filter just caused me to lose a comment full of carefully researched external links. It's not the first time, and it's really frustrating. Can we whitelist usernames or something? Submission rejected as potential spam Maximum number of external links per post exceeded I tried to solve the recaptcha five times (they were really hard) and when I finally solved it, the comment wasn't posted anyway. David Fifield

1 0

Internet-wide scanning for bridges
by Vlad Tsyrklevich 31 Dec '14

31 Dec '14

Hello, while taking a looking at TorCloud I it used static OR and obfs2+obfs3 ports making them trivial to discover by scanning EC2's IP address space. This led me to consider the more general attack of internet-wide scanning to discover Tor bridges. Most Tor relays run their ORPorts on port 443 or 9001 so I began investigating there. I began by looking at Project Sonar's port 443 SSL certificate database looking for certificates matching the tor certificate pattern. In talking with Eric Wustrow about ZMap, I discovered that he and the other ZMap authors had already published about this attack in their original paper. Eric mentioned that in his discussion with Roger that the response to the attack was going to be the soon-to-be-deployed obfuscated transports which were intended to be run on random high ports; however, the eventual roll out of obfsproxy did not change the fact that bridge's ORPorts continue to be publicly accessible. While I was analyzing Internet-wide survey-data for port 443 and 9001 (helpfully provided by Project Sonar/H D Moore) I found the Onionoo data set. I was able to quantify the impact of the attack using the Onionoo data and verify it using the ZMap scans. There are 4267 bridges, of them 1819 serve their ORPort on port 443 and 383 serve on port 9001. That's 52% of tor bridges. There are 1926 pluggable-transports enabled bridges, 316 with ORPort 443 and 33 with ORPort 9001. That's 18% of Tor bridges. 203 (64%) of PT-enabled bridges with ORPort 443 were TorCloud. I was able to approximately verify these figures using the Internet-wide scans. By grabbing bridge's certificates and calculating their hashed fingerprint I realized I was also discovering a fair amount of private bridges not included in the Onionoo data set. The results aren't too awful, just under 20% of PT-enabled bridges are affected. It's unclear to me if the low number of PT-enabled bridges (and corresponding high number of vulnerable non-PT-enabled bridges) is a historical artifact or a consequence of out-of-date documentation like https://www.torproject.org/docs/bridges.html.en not setting up pluggable transports. Eliminating the ORPort entirely for PT-enabled bridges seems like the ultimate solution to this attack, but the implications of such a change are unclear to me. Another change to mitigate this issue is changing the behavior of ports specified as 'auto' and then changing defaults and documentation to use auto by default. Currently specifying 'auto' in your torrc lets the OS decide the port; however, if tor were to generate a random port on first use that it continued to use across restarts it would it possible to use this for ORPorts and pluggable transports by default (as opposed to the current situation where for bridges you want ORPort auto but constant PT ports and a constant ORPort for relays which is confusing.) I've attached a patch to warn bridge operators running with ORPort set to 443 or 9001 as a stop-gap measure. - Vladislav

8 12

stem on windows 7
by Tony Willingham 31 Dec '14

31 Dec '14

Any suggestions on how to deal with stem on Windows 7 getting stuck at: [notice] Bootstrapped 45%: Asking for relay descriptors. Occasionally it will make it to: [notice] Bootstrapped 50%: Asking for relay descriptors. If I run Vidalia in client mode only, it works fine. I see in the documentation it says: The timeout argument does not work on Windows, and relies on the global state of the signal module. I have no idea what the "global state of the signal module" means. Python 2.7 Thanks Tony

3 3

Torsocks compile errors
by grarpamp 30 Dec '14

30 Dec '14

On gcc 4.2.1 freebsd 8 (thus not really expecting a fix)... commit ab81544478eb9a0db821fd33684cc2f6aac88c7c syscall.c:213: warning: implicit declaration of function 'handle_mmap' syscall.c:283: warning: cast from pointer to integer of different size ../../src/lib/.libs/libtorsocks.so: undefined reference to `handle_mmap' *** Error code 1

1 0

Hidden service policies
by Mike Hearn 30 Dec '14

30 Dec '14

Hello, As we know, hidden services can be useful for all kinds of legitimate things (Pond's usage is particularly interesting), however they do also sometimes get used by botnets and other problematic things. Tor provides exit policies to let exit relay operators restrict traffic they consider to be unwanted or abusive. In this way a kind of international group consensus emerges about what is and is not acceptable usage of Tor. For instance, SMTP out is widely restricted. Has there been any discussion of implementing similar controls for hidden services, where relays would refuse to act as introduction points for hidden services that match certain criteria e.g. have a particular key, or whose key appears in a list downloaded occasionally via Tor itself. In this way relay operators could avoid their resources being used for establishing communication with botnet CnC servers. Obviously such a scheme would require a protocol and client upgrade to avoid nodes building circuits to relays that then refuse to introduce. The downside is additional complexity. The upside is potentially recruiting new relay operators.

6 11

ScrambleSuit's replay protection incomplete
by Philipp Winter 28 Dec '14

28 Dec '14

In short: The implementation of ScrambleSuit's replay protection is incomplete which means that an active adversary can circumvent it. All the credit for this discovery goes to Lasse Øverlier. ScrambleSuit uses Uniform Diffie-Hellman as one of its authentication mechanisms. To defend against replay attacks, a sever caches the HMAC of a client's authentication message. The attack works as follows. In the first step, an active adversary (e.g., a censor trying to detect ScrambleSuit) observes a client authenticate successfully towards a ScrambleSuit server and captures the server's Uniform Diffie-Hellman response. In the second step, the adversary replays the captured response to the very same server. Since the server did not cache the HMAC of its own response, it will interpret the replayed data as legitimate authentication message of a new client and respond with an authentication response. The adversary now successfully tricked the server into responding despite not knowing the shared secret. This creates a noteworthy distinguisher which can help identifying ScrambleSuit. Luckily, it's easy to fix this problem. Introducing message types would be one option but it would break backwards compatibility. The easiest fix which retains backwards compatibility is to make the server also cache its own HMACs which are part of the response to a client's authentication message. The downside is that it doubles the size of the replay table but that's tolerable. Note that obfs4 is not affected by this problem because a client's and a server's authentication message are different. Cheers, Philipp

1 0

Damian's Status Report - December 2014
by Damian Johnson 27 Dec '14

27 Dec '14

Hi all! Tis that time of year coal appears in all my sox so guess it's time for a status report. This month I set arm aside to polish improvements that accumulated in the Stem 1.3 release... https://blog.torproject.org/blog/stem-release-13 There were a lot of small tweaks, but the highlights for this month were... -------------------------------------------------------------------------------- Hidden Service Tutorial -------------------------------------------------------------------------------- By a bit of serendipity I came across a nice article by Jordan Wright for spinning up hidden services with Stem... https://jordan-wright.github.io/blog/2014/10/06/creating-tor-hidden-service… Federico Ceratto and Patrick O'Doherty have been improving our hidden service support, so I spruced up their contributions and wrote a tutorial of our own that demos it... https://stem.torproject.org/tutorials/over_the_river.html -------------------------------------------------------------------------------- Gentoo Support -------------------------------------------------------------------------------- Thanks to Anthony Basile Stem is now packaged for Gentoo. Installing is as simple as... % emerge stem I've been delighted at how many Gentoo people have jumped out of the woodwork to help. toralf worked with me on ticket #13904 to address testing issues, and Manuel took on Stem 1.3 packaging the same day as its release - thanks guys! -------------------------------------------------------------------------------- Couple other quick things of note is that I updated our OpenHub repository listing so it includes most of our newer projects... https://www.openhub.net/p/tor ... and trimmed membership of our internal lists. Cheers! -Damian

1 0

Drop or merge the projects wiki?
by Damian Johnson 24 Dec '14

24 Dec '14

Hi all. Quick question, any objections to us dropping the projects wiki? https://trac.torproject.org/projects/tor/wiki/org/projects It's years out of date and redundant with the volunteer page, which I continue to update on occasion... https://www.torproject.org/getinvolved/volunteer.html.en#Projects I'm asking here in case there's any strong objections, or folks want us to keep something from it. Cheers! -Damian

3 5

Stem Release 1.3
by Damian Johnson 22 Dec '14

22 Dec '14

Greetings wonderful people of the world! After months down in the engine room I'm delighted to announce the 1.3.0 release of Stem. For those who aren't familiar with it, Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write applications similar to arm and Vidalia. https://stem.torproject.org/ So what's new in this release? ------------------------------------------------------------------------------ Better Hidden Service Support ------------------------------------------------------------------------------ Now it's easier than ever to spin up hidden services! Thanks to contributions from Federico Ceratto and Patrick O'Doherty we now have a set of methods specifically for working with hidden services. Check it out in our new tutorial... https://stem.torproject.org/tutorials/over_the_river.html ------------------------------------------------------------------------------ Faster Descriptor Parsing ------------------------------------------------------------------------------ This release dramatically improves the speed at which Stem can parse decriptors. Thanks to optimizations from Nick Mathewson and Ossi Herrala we can now read descriptors 40% faster! ------------------------------------------------------------------------------ This is just the tip of the iceberg. For a full rundown on the myriad of improvements and fixes in this release see... https://stem.torproject.org/change_log.html#version-1-3 Cheers! -Damian

1 0

Thinking of raising bandwidth stats interval from 15 minutes to 4 hours
by Nick Mathewson 22 Dec '14

22 Dec '14

See https://trac.torproject.org/projects/tor/ticket/13988 . Karsten suggests that I should announce this on tor-dev before merging/deploying it, and he's probably right. Will this break anything you know about? -- Nick

2 1

Tor dev IRC meetings for the rest of this month
by Nick Mathewson 22 Dec '14

22 Dec '14

Hi, all! The regular tor patch workshop will happen on schedule, at 1800 UTC on the #tor-dev IRC channel on Tuesday the 23nd and the Tuesday 29th. I'm also planning to do a Tor dev meeting at the regular time of 1330 UTC on Wednesday the 24th and Wednesday the 31st, though of course it's likely that some folks won't make it at one of those times. (I strongly encourage taking Christmas Eve or New Years eve off, if that's your kind of thing.) Best wishes & happy holidays, -- Nick

1 0