- tor-dev - lists.torproject.org

Proposal xxx: Consensus Hash Chaining
by Andrea Shepard 10 Jan '15

10 Jan '15

Here's a proposal Nick Mathewson and I just wrote for ticket #11157. --- Begin proposal body --- Filename: xxx-consensus-hash-chaining.txt Title: Consensus Hash Chaining Author: Nick Mathewson, Andrea Shepard Created: 06-Jan-2015 Status: Draft 1. Introduction and overview To avoid some categories of attacks against directory authorities and their keys, it would be handy to have an explicit hash chain in consensuses. 2. Directory authority operation We add the following field to votes and consensuses: previous-consensus ISOTIME [SP HashName "=" Base16]* NL where HashName is any keyword. This field may occur any number of times. The date in a previous-consensus line in a vote is the valid-after date of the consensus the line refers to. The hash should be computed over the signed portion of the consensus document. A directory authority should include a previous-consensus line for a consensus using all hashes it supports for all consensuses it knows which are still valid, together with the two most recently expired ones. When this proposal is implemented, a new consensus method should be allocated for adding previous-consensus lines to the consensus. A previous-consensus line is included in the consensus if and only if a line with that date was listed by more than half of the authorities whose votes are under consideration. A hash is included in that line if the hash was listed by more than half of the authorities whose votes are under consideration. Hashes are sorted lexically with a line by hashname; dates are sorted in temporal order. If, when computing a consensus, the authorities find that any previous-consensus line is *incompatible* with another, they must issue a loud warning. Two lines are incompatible if they have the same ISOTIME, but different values for the the same HashName. The hash "sha256" is mandatory. 3. Client and cache operation All parties receiving consensus documents should validate previous-consensus lines, and complain loudly if a hash fails to match. When a party receives a consensus document, it SHOULD check all previous-consensus lines against any previous consensuses it has retained, and if a hash fails to match it SHOULD warn loudly in the log mentioning the specific hashes and valid-after times in question, and store both the new consensus containing the mismatching hashes and the old consensus being checked for later analysis. An option SHOULD be provided to disable operation as a client or as a hidden service if this occurs. All relying parties SHOULD by default retain all valid consensuses they download plus two; but see "Security considerations" below. If a hash is not mismatched, the relying party may nonetheless be unable to validate the chain: either because there is a gap in the chain itself, or because the relying party does not have any of the consensuses that the latest consensus mentions. If this happens, the relying party should log a warning stating the specific cause, the hashes and valid-after time of both the consensus containing the unverifiable previous-consensus line and the hashes and valid-after time of the line for each such line, and retain a copy of the consensus document in question. A relying party MAY provide an option to disable operation as a client or hidden service in this event, but due to the risk that breaks in the chain may occur accidentally, such an option SHOULD be disabled by default if provided. If a relying party starts up and finds only very old consensuses such that no previous-consensus lines can be verified, it should log a notice of the gap along the lines of "consensus (date, hash) is quite new. Can't chain back to old consensus (date, hash)". If it has no old consensuses at all, it should log an info-level message of the form "we got consensus (date, hash). We haven't got any older consensuses, so we won't do any hash chain verification" 4. Security Considerations: * Retaining consensus documents on clients might leak information about when the client was active if a disk is later stolen or the client compromised. This should be documented somewhere and an option to disable (but thereby also disable verifying previous-consensus hashes) should be provided. * Clients MAY offer the option to retain previous consensuses in memory only to allow for validation without the potential disk leak. --- End proposal body --- -- Andrea Shepard <andrea(a)torproject.org> PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5

4 5

[Patch] src/ext/tinytest_demo.c
by Gisle Vanem 08 Jan '15

08 Jan '15

The "compat.h" is needed due to the use of 'OP_EQ', etc: diff --git a/src/ext/tinytest_demo.c b/src/ext/tinytest_demo.c index c07f099..cfb524b 100644 --- a/src/ext/tinytest_demo.c +++ b/src/ext/tinytest_demo.c @@ -30,6 +30,7 @@ /* Make sure you include these headers. */ #include "tinytest.h" #include "tinytest_macros.h" +#include "compat.h" #include <stdio.h> #include <stdlib.h> -- --gv

1 0

Regarding involving in tor project
by dp docs 06 Jan '15

06 Jan '15

Dear Sir, I would like to get involved in the tor project and want to start working on the "TOR Codebase Cleanup" project. My IRC nickname is "sdpan21". I have registered my nickname there. But while asking any question, I am getting an error "can not send to channel". I have tried few suggestion from google to resolve this issue but none ofo them seems to wrking. I think Either i am am not resolving it properly or there problem is different in some sense. So I request , please help me resolving this issue. And guide me how can I get involved in the project mentioned above. I have good knowledge of c/c++ java python. please respond ASAP. I appreciate your effort for resolving the issue and for giving time to guide me. Thanks Durgesh Pandey.

4 6

how to simulate TOR network through chutney?
by Mohiuddin Ebna Kawsar 06 Jan '15

06 Jan '15

HI, I need to simulate TOR network in minimal case [1 ENTRY,1 RELAY,1 EXIT] node. how can i do that use chutney? how to setup exit node on my localhost and how to make circuit with that exit node? following are my config file with generated (slightly modified) torc files i use basic-min for configure. ############## basic-min########################## Authority = Node(tag="a", authority=1, relay=1, torrc="authority.tmpl") Relay = Node(tag="r", relay=1, torrc="relay.tmpl") Client = Node(tag="c", torrc="client.tmpl") NODES = Authority.getN(1) + Relay.getN(1) + Client.getN(1) ConfigureNodes(NODES) ########################Authority-000a torc ######################### TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/000a RunAsDaemon 1 ConnLimit 60 Nickname test000a ShutdownWaitLength 0 PidFile /home/chutney-master/net/nodes/000a/pid Log notice file /home/chutney-master/net/nodes/000a/notice.log Log info file /home/chutney-master/net/nodes/000a/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 0 OrPort 5000 Address 127.0.0.1 DirPort 7000 #NOTE: Setting TestingServerConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingServerDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 # This file is named "relay.tmpl" for compatibility with previous # chutney versions # An exit relay that can exit to IPv4 & IPv6 localhost # (newer versions of tor need this to be explicitly configured) # An exit policy that allows exiting to IPv4 localhost ExitPolicy accept 127.0.0.0/8:* # An exit policy that allows exiting to IPv6 localhost ExitPolicy accept [::1]:* IPv6Exit 1 AuthoritativeDirectory 1 V3AuthoritativeDirectory 1 ContactInfo auth0(a)test.test ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:* TestingV3AuthInitialVotingInterval 300 TestingV3AuthInitialVoteDelay 20 TestingV3AuthInitialDistDelay 20 #TestingV3AuthVotingStartOffset 0 # Work around situations where the Exit and Guard flags aren't being set # These flags are set eventually, but it takes ~30 minutes # We could be more precise here, but it's easiest just to vote everything # Clients are sensible enough to filter out Exits without any exit ports, # and Guards without ORPorts # If your tor doesn't recognise TestingDirAuthVoteExit, update your chutney # to a version that includes the issue-13161-check-torrc-options features #TestingDirAuthVoteExit * #TestingDirAuthVoteGuard * #####################Relay 001r torc ############### TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/001r RunAsDaemon 1 ConnLimit 60 Nickname test001r ShutdownWaitLength 0 PidFile /home/chutney-master/net/nodes/001r/pid Log notice file /home/chutney-master/net/nodes/001r/notice.log Log info file /home/chutney-master/net/nodes/001r/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 0 OrPort 5001 Address 127.0.0.1 DirPort 7001 #NOTE: Setting TestingServerConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingServerDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 # This file is named "relay.tmpl" for compatibility with previous # chutney versions # An exit relay that can exit to IPv4 & IPv6 localhost # (newer versions of tor need this to be explicitly configured) # An exit policy that allows exiting to IPv4 localhost ExitPolicy accept 127.0.0.0/8:* # An exit policy that allows exiting to IPv6 localhost ExitPolicy accept [::1]:* IPv6Exit 1 ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:* ######################### client 002c torc ########################## TestingTorNetwork 1 DataDirectory /home/chutney-master/net/nodes/002c RunAsDaemon 1 ConnLimit 60 Nickname test002c ShutdownWaitLength 0 PidFile /home/raboon/Thesis/chutney-master/net/nodes/002c/pid Log notice file /home/chutney-master/net/nodes/002c/notice.log Log info file /home/chutney-master/net/nodes/002c/info.log ProtocolWarnings 1 SafeLogging 0 DisableDebuggerAttachment 0 DirAuthority test000a orport=5000 no-v2 hs v3ident=32D2EB75473AB923F63D0D3C4B6929FC1BB658AE 127.0.0.1:7000 1DB6AEADE945A2EE1722A203CFED30478D0C8C82 SocksPort 9002 #NOTE: Setting TestingClientConsensusDownloadSchedule doesn't # help -- dl_stats.schedule is not DL_SCHED_CONSENSUS # at boostrap time. #TestingClientDownloadSchedule 10, 2, 2, 4, 4, 8, 13, 18, 25, 40, 60 ExitPolicy accept *:80 ExitPolicy accept *:443 ExitPolicy reject *:*

1 0

test_cmdline_args.py on Windows
by Gisle Vanem 06 Jan '15

06 Jan '15

There are some problems with 'run_tor()' and line-endings. It doesn't forsee any '\r\n' from 'Popen()' and since 'lines()' only splits on '\n', most of the tests fails. With this little patch: @@ -64,7 +64,7 @@ yield fp[i:i+4] def lines(s): - out = s.split("\n") + out = s.splitlines() I only get 2 errors in that script. The others like: self.assertTrue(out_verif.endswith("Configuration was valid\n")) needs another patch which is beyond my Python knowledge. -- --gv

4 6

Easy research tasks
by spriver 06 Jan '15

06 Jan '15

Hey everyone out there, as given in the topic, are there any easy research tasks to start with? Or is somewhere some help needed? Cheers, spriver

4 4

Re: [tor-dev] Tor BSD underperformance (was [Tor-BSD] Recognizing Randomness Exhaustion)
by teor 06 Jan '15

06 Jan '15

On 1 Jan 2015, at 07:39 , Greg Troxel <gdt(a)lexort.com> wrote: > Libertas <libertas(a)mykolab.com> writes: > >> Some of the people at tor-bsd(a)lists.nycbug.org and I are trying to >> figure out why Tor relays under-perform when running on OpenBSD. Many >> such relays aren't even close to being network-bound, >> file-descriptor-bound, memory-bound, or CPU-bound, but relay at least >> 33-50% less traffic than would be expected of a Linux machine in the >> same situation. > > I'm more familiar with NetBSD, but hopefully my comments are helpful. > >> For those not familiar, a Tor relay will eventually have an open TCP >> connection for each of the other >6,000 active relays, and (if it allows >> exit traffic) must make outside TCP connections for the user's requests, >> so it's pretty file-hungry and crypto-intensive. > > It may also have something to do with TCP. A few thoughts: > > * run netstat -f inet and look and the send queues. That's not really > cleanly diagnostic, but if they are all huge, it's a clue > > * run netstat -m and vmstat -m (not sure those map from NetBSD). Look > for runnig out of mbufs and mbuf clusters. Perhaps bump up > NMBCLUSTERS in the kernel if it's not dynamic. Tor 0.2.6.2-alpha (just in the process of being released) has some changes to queuing behaviour using the KIST algorithm. The KIST algorithm keeps the queues inside tor, and makes prioritisation decisions from there, rather than writing as much as possible to the OS TCP queues. I'm not sure how functional it is on *BSDs, but Nick Mathewson should be able to comment on that. (I've cc'd tor-dev and Nick.) teor pgp 0xABFED1AC hkp://pgp.mit.edu/ https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w…

6 10

Summary of meek's costs, December 2014
by David Fifield 05 Jan '15

05 Jan '15

Here's the summary of meek's CDN fees for December 2014. Earlier reports: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html https://lists.torproject.org/pipermail/tor-dev/2014-November/007716.html https://lists.torproject.org/pipermail/tor-dev/2014-December/007916.html If you're just tuning in, meek is a pluggable transport introduced a few months ago. https://trac.torproject.org/projects/tor/wiki/doc/meek. App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 November 2014 $224.67 + $362.60 + $0.00 = $587.27 December 2014 $326.81 + $417.31 + $0.00 = $744.12 -- total by CDN $600.63 + $917.89 + $0.00 = $1518.52 grand total Usage was up in December compared to November, but the relative increase wasn't as great as between November and October. https://metrics.torproject.org/userstats-bridge-transport.html?graph=userst… (The last few days are missing; see https://lists.torproject.org/pipermail/tor-dev/2014-December/008021.html.) Rather than attach bandwidth screenshots from the CDN control panels, this month I'll just link to the Globe pages for the different backends. A different relay is used depending on what backend you choose: meek-google: https://globe.torproject.org/#/bridge/88F745840F47CE0C6A4FE61D827950B06F9E4… meek-amazon: https://globe.torproject.org/#/bridge/3FD131B74D9A96190B1EE5D31E91757FADA1A… meek-azure: https://globe.torproject.org/#/bridge/AA033EEB61601B2B7312D89B62AAA23DC3ED8… It's interesting that meek-google has more bandwidth (800 KB/s) but fewer clients (200), while meek-amazon has less bandwidth (550 KB/s) but more clients (340). meek-azure is behind both with 200 KB/s and 150 clients. I also attached screenshots of the bandwidth graphs from those pages. As a step toward reducing the bandwidth bills, this month I closed #12778, which reduces overhead by ensmallening HTTP headers. https://trac.torproject.org/projects/tor/ticket/12778 Headers are now less than half the size compared to before, which I estimate will save about 3% on bandwidth. We won't observe any effect, however, until there's a new release of Tor Browser. We're starting to move quite a bit of traffic. Not counting meek-azure, it's over 3.6 TB last month. (That's including an estimated 6–7% of meek-induced overhead.) == App Engine a.k.a. meek-google == Most of the App Engine bill is still using an existing credit. Of the total of $326.81, $230.00 was covered by credits. Here is how the costs broke down: 2132 GB $255.86 1419 instance hours $70.95 The attached meek-google-costs-2015-01-01.png shows the breakdown of costs between bandwidth and instance hours for the last few months. == Amazon a.k.a. meek-amazon == Asia Pacific (Singapore) 105M requests $126.52 783 GB $101.09 Asia Pacific (Sydney) 183K requests $0.23 1 GB $0.18 Asia Pacific (Tokyo) 25M requests $30.28 146 GB $17.83 EU (Ireland) 57M requests $68.81 481 GB $37.23 South America (Sao Paulo) 1M requests $2.49 7 GB $1.58 US East (Northern Virginia) 19M requests $18.69 161 GB $12.39 -- total 207M requests $247.02 1579 GB $170.30* * The total from adding up subtotals is $0.01 higher than the actual bill, I think because of some rounding. == Azure a.k.a. meek-azure == I still can't figure out how to get total monthly bandwidth out of the Azure console. David Fifield

1 0

Reminder for weekly OONI dev meeting Monday 2015-01-05 18:00 UTC (19:00 CET)
by Arturo Filastò 05 Jan '15

05 Jan '15

Hello Oonitarians, This is a reminder that today there will be the weekly OONI meeting. It will happen as usual on the #ooni channel on irc.oftc.net at 18:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). Everybody is welcome to join us and bring their questions and feedback. See you later, ~ Arturo

1 0

Re: [tor-dev] Fwd: gettimeofday() Syscall Issues
by grarpamp 03 Jan '15

03 Jan '15

On Fri, Jan 2, 2015 at 12:24 PM, Konstantin Belousov <kostikbel(a)gmail.com> wrote: > On Fri, Jan 02, 2015 at 09:09:34AM -0500, grarpamp wrote: >> Some recent FreeBSD related questions in this app area. >> > What is the question ? > > As a background, I can repeat that FreeBSD implements syscall-less > gettimeofday() and clock_gettime() for x86 machines which have > usable RDTSC. The selection of the timecounter can be verified > by sysctl kern.timecounter.hardware, and enabled by default fast > gettimeofday(2) can be checked by sysctl kern.timecounter.fast_gettime. > > On some Nehalem machine, I see it doing ~30M calls/sec with enabled > fast_gettime, and ~6.25M calls/sec with disabled fast_gettime. This is > measured on 2.8GHz Core i7 930 with src/tools/tools/syscall_timing. > > Check your timecounter hardware. Since it was noted that the tests > were done in VM, check the quality of RDTSC emulation in your hypervisor. https://lists.torproject.org/pipermail/tor-dev/2015-January/thread.html http://docs.freebsd.org/mail/current/freebsd-performance.html Maybe I can just refer non subscribers out to the two lists above that way in case anyone sees anything interesting they can join/comment as desired. Background might be that Tor operators have some large relays on *BSD and were looking to validate, and ways to improve, performance there. Cheers. https://lists.torproject.org/pipermail/tor-relays/ https://lists.torproject.org/pipermail/tor-talk/

3 2