- tor-dev - lists.torproject.org

Regarding involving in tor project
by dp docs 06 Jan '15

06 Jan '15

Dear Sir, I would like to get involved in the tor project and want to start working on the "TOR Codebase Cleanup" project. My IRC nickname is "sdpan21". I have registered my nickname there. But while asking any question, I am getting an error "can not send to channel". I have tried few suggestion from google to resolve this issue but none ofo them seems to wrking. I think Either i am am not resolving it properly or there problem is different in some sense. So I request , please help me resolving this issue. And guide me how can I get involved in the project mentioned above. I have good knowledge of c/c++ java python. please respond ASAP. I appreciate your effort for resolving the issue and for giving time to guide me. Thanks Durgesh Pandey.

4 6

how to simulate TOR network through chutney?
by Mohiuddin Ebna Kawsar 06 Jan '15

06 Jan '15

HI, I need to simulate TOR network in minimal case [1 ENTRY,1 RELAY,1 EXIT] node. how can i do that use chutney? how to setup exit node on my localhost and how to make circuit with that exit node? following are my config file with generated (slightly modified) torc files i use basic-min for configure. ############## basic-min########################## Authority = Node(tag="a", authority=1, relay=1, torrc="authority.tmpl") Relay = Node(tag="r", relay=1, torrc="relay.tmpl") Client = Node(tag="c", torrc="client.tmpl") NODES = Authority.getN(1) + Relay.getN(1) + Client.getN(1) ConfigureNodes(NODES) ########################Authority-000a torc ######################### TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/000a RunAsDaemon 1 ConnLimit 60 Nickname test000a ShutdownWaitLength 0 PidFile /home/chutney-master/net/nodes/000a/pid Log notice file /home/chutney-master/net/nodes/000a/notice.log Log info file /home/chutney-master/net/nodes/000a/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 0 OrPort 5000 Address 127.0.0.1 DirPort 7000 #NOTE: Setting TestingServerConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingServerDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 # This file is named "relay.tmpl" for compatibility with previous # chutney versions # An exit relay that can exit to IPv4 & IPv6 localhost # (newer versions of tor need this to be explicitly configured) # An exit policy that allows exiting to IPv4 localhost ExitPolicy accept 127.0.0.0/8:* # An exit policy that allows exiting to IPv6 localhost ExitPolicy accept [::1]:* IPv6Exit 1 AuthoritativeDirectory 1 V3AuthoritativeDirectory 1 ContactInfo auth0(a)test.test ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:* TestingV3AuthInitialVotingInterval 300 TestingV3AuthInitialVoteDelay 20 TestingV3AuthInitialDistDelay 20 #TestingV3AuthVotingStartOffset 0 # Work around situations where the Exit and Guard flags aren't being set # These flags are set eventually, but it takes ~30 minutes # We could be more precise here, but it's easiest just to vote everything # Clients are sensible enough to filter out Exits without any exit ports, # and Guards without ORPorts # If your tor doesn't recognise TestingDirAuthVoteExit, update your chutney # to a version that includes the issue-13161-check-torrc-options features #TestingDirAuthVoteExit * #TestingDirAuthVoteGuard * #####################Relay 001r torc ############### TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/001r RunAsDaemon 1 ConnLimit 60 Nickname test001r ShutdownWaitLength 0 PidFile /home/chutney-master/net/nodes/001r/pid Log notice file /home/chutney-master/net/nodes/001r/notice.log Log info file /home/chutney-master/net/nodes/001r/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 0 OrPort 5001 Address 127.0.0.1 DirPort 7001 #NOTE: Setting TestingServerConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingServerDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 # This file is named "relay.tmpl" for compatibility with previous # chutney versions # An exit relay that can exit to IPv4 & IPv6 localhost # (newer versions of tor need this to be explicitly configured) # An exit policy that allows exiting to IPv4 localhost ExitPolicy accept 127.0.0.0/8:* # An exit policy that allows exiting to IPv6 localhost ExitPolicy accept [::1]:* IPv6Exit 1 ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:* ######################### client 002c torc ########################## TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/002c RunAsDaemon 1 ConnLimit 60 Nickname test002c ShutdownWaitLength 0 PidFile /home/raboon/Thesis/chutney-master/net/nodes/002c/pid Log notice file /home/chutney-master/net/nodes/002c/notice.log Log info file /home/chutney-master/net/nodes/002c/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 9002 #NOTE: Setting TestingClientConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingClientDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:*

1 0

test_cmdline_args.py on Windows
by Gisle Vanem 06 Jan '15

06 Jan '15

There are some problems with 'run_tor()' and line-endings. It doesn't forsee any '\r\n' from 'Popen()' and since 'lines()' only splits on '\n', most of the tests fails. With this little patch: @@ -64,7 +64,7 @@ yield fp[i:i+4] def lines(s): - out = s.split("\n") + out = s.splitlines() I only get 2 errors in that script. The others like: self.assertTrue(out_verif.endswith("Configuration was valid\n")) needs another patch which is beyond my Python knowledge. -- --gv

4 6

Easy research tasks
by spriver 06 Jan '15

06 Jan '15

Hey everyone out there, as given in the topic, are there any easy research tasks to start with? Or is somewhere some help needed? Cheers, spriver

4 4

Re: [tor-dev] Tor BSD underperformance (was [Tor-BSD] Recognizing Randomness Exhaustion)
by teor 06 Jan '15

06 Jan '15

On 1 Jan 2015, at 07:39 , Greg Troxel <gdt(a)lexort.com> wrote: > Libertas <libertas(a)mykolab.com> writes: > >> Some of the people at tor-bsd(a)lists.nycbug.org and I are trying to >> figure out why Tor relays under-perform when running on OpenBSD. Many >> such relays aren't even close to being network-bound, >> file-descriptor-bound, memory-bound, or CPU-bound, but relay at least >> 33-50% less traffic than would be expected of a Linux machine in the >> same situation. > > I'm more familiar with NetBSD, but hopefully my comments are helpful. > >> For those not familiar, a Tor relay will eventually have an open TCP >> connection for each of the other >6,000 active relays, and (if it allows >> exit traffic) must make outside TCP connections for the user's requests, >> so it's pretty file-hungry and crypto-intensive. > > It may also have something to do with TCP. A few thoughts: > > * run netstat -f inet and look and the send queues. That's not really > cleanly diagnostic, but if they are all huge, it's a clue > > * run netstat -m and vmstat -m (not sure those map from NetBSD). Look > for runnig out of mbufs and mbuf clusters. Perhaps bump up > NMBCLUSTERS in the kernel if it's not dynamic. Tor 0.2.6.2-alpha (just in the process of being released) has some changes to queuing behaviour using the KIST algorithm. The KIST algorithm keeps the queues inside tor, and makes prioritisation decisions from there, rather than writing as much as possible to the OS TCP queues. I'm not sure how functional it is on *BSDs, but Nick Mathewson should be able to comment on that. (I've cc'd tor-dev and Nick.) teor pgp 0xABFED1AC hkp://pgp.mit.edu/ https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w…

6 10

Summary of meek's costs, December 2014
by David Fifield 05 Jan '15

05 Jan '15

Here's the summary of meek's CDN fees for December 2014. Earlier reports: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html If you're just tuning in, meek is a pluggable transport introduced a few months ago. https://trac.torproject.org/projects/tor/wiki/doc/meek. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 -- total by CDN $600.63 + $917.89 + $0.00 = $1518.52 grand total Usage was up in December compared to November, but the relative increase wasn't as great as between November and October. https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… (The last few days are missing; see https://lists.torproject.org/pipermail/tor-dev/2014-December/008021.html.) Rather than attach bandwidth screenshots from the CDN control panels, this month I'll just link to the Globe pages for the different backends. A different relay is used depending on what backend you choose: meek-google: https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… meek-amazon: https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… meek-azure: https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… It's interesting that meek-google has more bandwidth (800 KB/s) but fewer clients (200), while meek-amazon has less bandwidth (550 KB/s) but more clients (340). meek-azure is behind both with 200 KB/s and 150 clients. I also attached screenshots of the bandwidth graphs from those pages. As a step toward reducing the bandwidth bills, this month I closed #12778, which reduces overhead by ensmallening HTTP headers. https://trac.torproject.org/projects/tor/ticket/12778 Headers are now less than half the size compared to before, which I estimate will save about 3% on bandwidth. We won't observe any effect, however, until there's a new release of Tor Browser. We're starting to move quite a bit of traffic. Not counting meek-azure, it's over 3.6 TB last month. (That's including an estimated 6–7% of meek-induced overhead.) == App Engine a.k.a. meek-google == Most of the App Engine bill is still using an existing credit. Of the total of $326.81, $230.00 was covered by credits. Here is how the costs broke down: 2132 GB $255.86 1419 instance hours $70.95 The attached meek-google-costs-2015-01-01.png shows the breakdown of costs between bandwidth and instance hours for the last few months. == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 105M requests $126.52 783 GB $101.09 Asia Pacific (Sydney) 183K requests $0.23 1 GB $0.18 Asia Pacific (Tokyo) 25M requests $30.28 146 GB $17.83 EU (Ireland) 57M requests $68.81 481 GB $37.23 South America (Sao Paulo) 1M requests $2.49 7 GB $1.58 US East (Northern Virginia) 19M requests $18.69 161 GB $12.39 -- total 207M requests $247.02 1579 GB $170.30* * The total from adding up subtotals is $0.01 higher than the actual bill, I think because of some rounding. == Azure a.k.a. meek-azure == I still can't figure out how to get total monthly bandwidth out of the Azure console. David Fifield

1 0

Reminder for weekly OONI dev meeting Monday 2015-01-05 18:00 UTC (19:00 CET)
by Arturo Filastò 05 Jan '15

05 Jan '15

Hello Oonitarians, This is a reminder that today there will be the weekly OONI meeting. It will happen as usual on the #ooni channel on irc.oftc.net at 18:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). Everybody is welcome to join us and bring their questions and feedback. See you later, ~ Arturo

1 0

Re: [tor-dev] Fwd: gettimeofday() Syscall Issues
by grarpamp 03 Jan '15

03 Jan '15

On Fri, Jan 2, 2015 at 12:24 PM, Konstantin Belousov <kostikbel(a)gmail.com> wrote: > On Fri, Jan 02, 2015 at 09:09:34AM -0500, grarpamp wrote: >> Some recent FreeBSD related questions in this app area. >> > What is the question ? > > As a background, I can repeat that FreeBSD implements syscall-less > gettimeofday() and clock_gettime() for x86 machines which have > usable RDTSC. The selection of the timecounter can be verified > by sysctl kern.timecounter.hardware, and enabled by default fast > gettimeofday(2) can be checked by sysctl kern.timecounter.fast_gettime. > > On some Nehalem machine, I see it doing ~30M calls/sec with enabled > fast_gettime, and ~6.25M calls/sec with disabled fast_gettime. This is > measured on 2.8GHz Core i7 930 with src/tools/tools/syscall_timing. > > Check your timecounter hardware. Since it was noted that the tests > were done in VM, check the quality of RDTSC emulation in your hypervisor. https://lists.torproject.org/pipermail/tor-dev/2015-January/thread.html http://docs.freebsd.org/mail/current/freebsd-performance.html Maybe I can just refer non subscribers out to the two lists above that way in case anyone sees anything interesting they can join/comment as desired. Background might be that Tor operators have some large relays on *BSD and were looking to validate, and ways to improve, performance there. Cheers. https://lists.torproject.org/pipermail/tor-relays/ https://lists.torproject.org/pipermail/tor-talk/

3 2

Re: [tor-dev] Tor Cloud Maintenance, revisited
by Jeremy Olexa 03 Jan '15

03 Jan '15

Hi Vlad, I agree. AWS pricing is not friendly for bandwidth users. However, it is free for the first year of use and the general population is not taking advantage of this. Which, AFACT, is one of the points of the Tor Cloud project. Once you are not free-tier eligible, I highly recommend using a cheaper service like DO, Linode, Vultr, etc. At least DO and Linode would be trivial to setup. In summary, there is no need to replace AWS, but instead to supplement it with other providers and make the Tor Cloud project more robust. -Jeremy On Fri, Jan 2, 2015 at 6:10 PM, Vlad Tsyrklevich <vlad(a)tsyrklevich.net> wrote: > Hi Jeremy, I've working on something related so I figured I'd comment. I've > been working on a TorCloud replacement using DigitalOcean's API, this has > the benefit of a simplified set-up process (one-click set-up) and the > pricing on bandwidth is a major win over AWS ($5 for 1TB U/L instead of $20 > for 40GB, though your move to t2.micro would move it down to $10.) The > back-end is there and pretty much ready to go, I've primarily been waiting > for my co-author to come back from vacation to finish the front-end. > > I haven't actually discussed sunsetting/replacing the current TorCloud with > the Tor Project so this project might as well be vaporware; however, I think > continuing to use AWS doesn't make sense for pricing and ease-of-use. All > that said, if you're taking this on, it'd be great to try to also address > TorCloud's currently susceptibility to discovery by internet-wide port > scanning as I mentioned in > https://lists.torproject.org/pipermail/tor-dev/2014-December/007957.html > > On Thu Jan 01 2015 at 1:23:49 PM Jeremy Olexa <jolexa(a)jolexa.net> wrote: >> >> Hi Everyone, Happy New Year, first post to the -dev list but I've been >> running some relays for months[1]. Overall a new user to Tor so feel >> free to point me elsewhere if I'm asking poor questions. >> >> I've noticed that the Tor Cloud project is dead in the water right >> now. The last post on this list is in June 2014[2] and the bugs have >> been neglected, especially the one I opened[3] which states that tor >> does not even start! I've seen that there is a new maintainer, but >> still don't see any [public] activity. >> >> There are a couple of issues that have opportunity to be handled >> better. A large roadblock seems to be building/publishing a new AMI so >> I've addressed that in a keeping it simple theme: >> - Use Amazon AMI instead of Ubuntu. Justification: more aligned to AWS >> and is a first rate citizen in the ecosystem, yum repos, security >> updates, etc >> - Use t2.micro instances. Justification: t1.micro (currently in use) >> are more expensive and less performant than t2.micro >> - Use cloud-init to fetch ec2-prep.sh and run the script to configure >> the instance[4]. Justification: Less likely to roll a new AMI just for >> ec2-prep.sh updates therefore more of a rolling update infrastructure. >> One example is adding a new bridge protocol, all newly launched >> instances would get the ec2-prep updates without rolling a new AMI. >> Justification 2: Less Tor Cloud maintainer work. Downside: AMI has >> dependency on gitweb.torproject.org's uptime - we could use S3 or some >> other CDN but this is starting to go against the simple theme. >> - Publishing AMI to us-east (N Virginia), us-west2 (Oregon), us-europe >> (Ireland). Justification: These are the lowest cost regions. >> - Not publishing a separate AMI for private bridges. Justification: IF >> the Tor Cloud project wants to provide configuration for private >> bridges, reusing the shared AMI makes more sense. I have an idea of >> using cloud-init's user-data field but need to test it. Justification >> 2: Simpler, less Tor Cloud maintainer work. >> - Not addressed (yet): Ubuntu's unattended upgrades concept. I have >> one idea (yum security plugin) but I haven't thought about all the >> implications. >> >> My AMI testing has proved that the above concepts work and I have ec2 >> bridges running in east and west[5]. I propose that the TOR project >> uses the above model and I'm willing to help facilitate that. I am >> willing to share the AMI with some select beta testers but I'm not >> ready to make it public yet (some changes are expected). It is my idea >> that once the final AMI is produced, the Tor Cloud project will not >> have to publish another AMI unless wanting to change the instance >> type, or other "infrastructure" changes. >> >> I look forward to hearing some feedback, thanks, >> Jeremy >> >> [1]: https://atlas.thecthulhu.com/#search/jolexa >> [2]: https://lists.torproject.org/pipermail/tor-dev/2014-June/007001.html >> [3]: https://trac.torproject.org/projects/tor/ticket/13391 >> [4]: https://github.com/jolexa/tor-cloud/blob/master/run-once.sh >> [5]: https://globe.thecthulhu.com/#/search/query=ec2bridgei >> _______________________________________________ >> tor-dev mailing list >> tor-dev(a)lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > > _______________________________________________ > tor-dev mailing list > tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >

1 0

Tor Cloud Maintenance, revisited
by Jeremy Olexa 03 Jan '15

03 Jan '15

Hi Everyone, Happy New Year, first post to the -dev list but I've been running some relays for months[1]. Overall a new user to Tor so feel free to point me elsewhere if I'm asking poor questions. I've noticed that the Tor Cloud project is dead in the water right now. The last post on this list is in June 2014[2] and the bugs have been neglected, especially the one I opened[3] which states that tor does not even start! I've seen that there is a new maintainer, but still don't see any [public] activity. There are a couple of issues that have opportunity to be handled better. A large roadblock seems to be building/publishing a new AMI so I've addressed that in a keeping it simple theme: - Use Amazon AMI instead of Ubuntu. Justification: more aligned to AWS and is a first rate citizen in the ecosystem, yum repos, security updates, etc - Use t2.micro instances. Justification: t1.micro (currently in use) are more expensive and less performant than t2.micro - Use cloud-init to fetch ec2-prep.sh and run the script to configure the instance[4]. Justification: Less likely to roll a new AMI just for ec2-prep.sh updates therefore more of a rolling update infrastructure. One example is adding a new bridge protocol, all newly launched instances would get the ec2-prep updates without rolling a new AMI. Justification 2: Less Tor Cloud maintainer work. Downside: AMI has dependency on gitweb.torproject.org's uptime - we could use S3 or some other CDN but this is starting to go against the simple theme. - Publishing AMI to us-east (N Virginia), us-west2 (Oregon), us-europe (Ireland). Justification: These are the lowest cost regions. - Not publishing a separate AMI for private bridges. Justification: IF the Tor Cloud project wants to provide configuration for private bridges, reusing the shared AMI makes more sense. I have an idea of using cloud-init's user-data field but need to test it. Justification 2: Simpler, less Tor Cloud maintainer work. - Not addressed (yet): Ubuntu's unattended upgrades concept. I have one idea (yum security plugin) but I haven't thought about all the implications. My AMI testing has proved that the above concepts work and I have ec2 bridges running in east and west[5]. I propose that the TOR project uses the above model and I'm willing to help facilitate that. I am willing to share the AMI with some select beta testers but I'm not ready to make it public yet (some changes are expected). It is my idea that once the final AMI is produced, the Tor Cloud project will not have to publish another AMI unless wanting to change the instance type, or other "infrastructure" changes. I look forward to hearing some feedback, thanks, Jeremy [1]: https://atlas.thecthulhu.com/#search/jolexa [2]: https://lists.torproject.org/pipermail/tor-dev/2014-June/007001.html [3]: https://trac.torproject.org/projects/tor/ticket/13391 [4]: https://github.com/jolexa/tor-cloud/blob/master/run-once.sh [5]: https://globe.thecthulhu.com/#/search/query=ec2bridgei

2 1