- tor-dev - lists.torproject.org

Proposing sbws changes
by teor 24 Apr '19

24 Apr '19

Hi all, We finished our first working version of sbws in March, and deployed it to a directory authority. We're now working on deploying it to a few more directory authorities: https://trac.torproject.org/projects/tor/ticket/29290 We're also working on archiving and analysing the bandwidth files produced by sbws and Torflow: https://trac.torproject.org/projects/tor/ticket/21378 During this work, we've discovered some missing sbws features: https://trac.torproject.org/projects/tor/ticket/30255 We need a better process for proposing and reviewing sbws changes. At the moment, I am spending a lot of time reviewing and designing sbws changes. And that's not sustainable. We need a process that works when I go on leave, or get busy with other tasks. I suggest that we use the tor proposals process: https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt We can submit small changes as diffs to the bandwidth file spec: https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt But large changes, controversial changes, and changes with multiple competing options should have their own proposal. Then, once we decide what to do, we can integrate those changes into the spec. T -- teor ----------------------------------------------------------------------

1 1

PrivCount Status
by teor 23 Apr '19

23 Apr '19

Hi, Nick asked me to send a status email about PrivCount, before I go on leave for a few weeks. Plan We want to add the following counters to a PrivCount Proof of Concept: * check counters (zero, relay count, time in seconds) * consumed bandwidth Nick also suggested adding connection counts. That seems like a good counter, but we want to make sure we do bandwidth in the first release, because it's a high-risk statistic. Status In March and April, I deferred PrivCount tasks to work on chutney for one of our other sponsors. I also delayed these tasks, because I was waiting for #29017 and #29018 to merge: * #29017 PaddingStatistics should be disabled when ExtraInfoStatistics is 0 * #29018 Make all statistics depend on ExtraInfoStatistics Tickets The top-level ticket is: PrivCount proof of concept with existing statistics https://trac.torproject.org/projects/tor/ticket/27908 I was mainly working on code for these tickets: PrivCount proof of concept: implement check counters https://trac.torproject.org/projects/tor/ticket/29004 PrivCount proof of concept: implement consumed bandwidth counters https://trac.torproject.org/projects/tor/ticket/29005 Make relays report bandwidth usage more often in test networks https://trac.torproject.org/projects/tor/ticket/29019 Code I have incomplete branches for #29004, #29005, and #29019 here: https://github.com/teor2345/tor/tree/ticket29004-wip https://github.com/teor2345/tor/tree/ticket29005 https://github.com/teor2345/tor/tree/ticket29019 I think all the necessary code is present in these branches. (But maybe it's not???) But it needs some cleanup: * rebase on to the current master, * put the commits on the right branches * make sure it does what these tickets say it should do I'm happy to do that after I come back from leave. I am also happy if Nick wants to clean up this code. See also my previous email about BridgeDB and PrivCount. Maybe we can save ourselves some effort by using PrivCount's obfuscation on BridgeDB's statistics. T -- teor ----------------------------------------------------------------------

1 0

Export BridgeDB usage statistics
by Philipp Winter 23 Apr '19

23 Apr '19

Hi Karsten, I'm working on <https://bugs.torproject.org/9316>, which will make BridgeDB export usage statistics. I would like these statistics to be public, privacy-preserving, and -- ideally -- added to Tor Metrics. I wanted to hear your thoughts on 1) what statistics we should collect, 2) how we can collect these statistics safely, and 3) what format these statistics should have. Broadly speaking, these statistics should answer the following questions: * How many requests does BridgeDB see per day? * What obfuscation protocols are the most popular? * What bridge distribution mechanisms are the most popular? * From what countries do we see the most bridge requests? * How many BridgeDB requests fail and succeed, respectively? * How many requests does BridgeDB see from Yahoo/Gmail/Riseup? * How many HTTPS requests are coming from proxies? * How many requests are suspicious, and likely issued by bots? Each request to BridgeDB carries with it some information, which allows us to answer the above questions. I suggest that we collect the following: * The distribution mechanism. Currently, this is HTTPS, email, or Moat. * The requested transport. Currently this is vanilla, fte, obfs3, obfs4, or scramblesuit. * The request's origin. For Moat and HTTPS, it's the two-letter country code, e.g., IT for Italy. For email, it's the user's email domain (Gmail, Yahoo, or Riseup). * Whether the request was successful or unsuccessful, i.e., resulted in BridgeDB handing out bridges or not. * Whether the request was issued by a user or a bot. David suggested heuristics that would allow us to estimate if a request came from a bot: <https://bugs.torproject.org/9316#comment:19> I like these suggestions but I'm not sure yet how to encode them -- it's more complex than a simple binary flag. The combination of these statistics results in ~16,800 buckets (3 mechanisms * 5 transports * ~280 ISO country codes * 2 success states * 2 bot states). We only need to export statistics with non-empty buckets. To protect users whose request is the only one in a given bucket (e.g., there may be only one user in Turkmenistan who successfully requested an FTE bridge over HTTPS on 2019-04-02), we should bin the statistics by rounding them up to the next multiple of, say, 10. We should further export statistics infrequently -- maybe once a day. Here's an example of a simple CSV format that takes into account the above: timestamp,mechanism,transport,country|domain,success,count,origin 1555977600,https,vanilla,it,successful,40,user 1555977600,https,obfs4,ca,unsuccessful,10,user 1555977600,email,vanilla,yahoo.com,successful,50,user ... What are your thoughts? Thanks, Philipp

2 1

Next Network Team Meeting Time
by teor 18 Apr '19

18 Apr '19

Hi, The next network team meeting is at 2300 UTC on Tuesday 23 April. (Our "patch party" and "first meeting of the month" time.) Here is our schedule for the next few months: April schedule: * Tuesday 23 April at 2300 UTC (at the "patch party" time) * Monday 29 April at 1700 UTC May schedule: * Monday 6 May at 1700 UTC (first May meeting, but at the normal time) * Monday 13 May at 1700 UTC * Monday 20 May at 1700 UTC * Monday 27 May at 1700 UTC June schedule: * Tuesday 4 June at 2300 UTC (first meeting of the month) * Monday 10 June at 1700 UTC T -- teor ----------------------------------------------------------------------

3 4

Support for clients using shutdown(SHUT_WR)
by Rob Jansen 16 Apr '19

16 Apr '19

Hello! I'm in the process of re-writing Shadow's traffic generation tool called tgen [0], which now depends on shutdown() for two communicating tgen nodes to inform each other that they have no more to send (i.e. to perform graceful connection shutdowns). The shutdown() functionality is useful now that tgen uses Markov models to generate more realistic Tor traffic [1]. Currently, it appears that when the tor client receives a FIN on the AP connection (because the tgen client calls shutdown(SHUT_WR)), the tor client sends a RELAY_END cell to the exit relay to instruct it to close() the exit TCP connection to the server. The behavior I expected was something more like what is described in tor-spec in Section 6.3 (search for "RELAY_FIN") [2], where the tor client sends a RELAY_FIN cell to the exit to instruct the exit to perform a shutdown(SHUT_WR) on the exit TCP connection to the server. Is there any plan to support shutdown(SHUT_WR) using RELAY_FIN cells now that Tor is itself using shutdown()? (I didn't see any tickets about it after a brief search.) Thanks! Peace, love, and positivity, Rob [0] https://github.com/shadow/tgen [1] https://tmodel-ccs2018.github.io/ [2] https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt

2 1

ICLab testing of default bridges
by David Fifield 11 Apr '19

11 Apr '19

At the anti-censorship meeting today you mentioned talking to ICLab about testing the default bridges. I believe that ICLab is already testing at least a portion of the default bridges; they may have data that you just have to ask for. OONI and ICLab test the default bridges as a result of my, Lynn's, and Qi's asking them to while we were studying default bridge blocking. The data exist but we never carefully analyzed them. You can see what little analysis we did at https://arxiv.org/abs/1709.08718 Figure 1 and Section 5.9. The data we used is in data/iclab/tor_jsons.tar.bz2 in https://repo.eecs.berkeley.edu/git-anon/users/fifield/proxy-probe-paper.git. It's a lot of JSON files that have TCP connect, TLS certificate fetch, HTTP fetch, and TCP traceroute for each destination. We sent them the list of bridges in September 2016 and they started measuring shortly after that. The most recent update to the bridge list we sent them was in August 2017 for https://bugs.torproject.org/23166.

1 0

Scripts and VMs for helping people setup a relay or bridge
by Alessandro Fiori 09 Apr '19

09 Apr '19

Hi, since i have released some Docker images for help people to run their own service or relays, i'm releasing some Virtualbox images and scripts for who want to run their relays. Virtual machines are in importable OVA format, and are based on Ubuntu Command Line Install, for a very light-weight system. I have released, for now, a Script and Virtual Machine for Tor middle relay, next days i will release bridge scripts for bridge with pluggable transports pre-installed. The Script repository is located here: https://github.com/randomtable/ChimeraOS/tree/master/Scripts The Virtual Machines repository is located here: https://github.com/randomtable/ChimeraOS/tree/master/VMs I'm writing to "dev" mailing list, because a very expert eye is always needed. I hope this can be useful for everyone. Alessandro Fiori

2 1

Re: [tor-dev] New revision: Proposal 295: Using ADL for relay cryptography (solving the crypto-tagging attack)
by Jeff Burdges 08 Apr '19

08 Apr '19

If I understand, proposal 295 looks similar to either BEAR or LION from the LIONNESS. I vaguely recall both BEAR and LION being "broken" in some setting, although I cannot site the paper. Anyone? I suppose the BEAR and LION break originates from using them for authentication while proposal 295’s separate SVer function fixes this? Jeff

1 1

Tor Directory Meta-Format + Line Wrapping?
by Iain Learmonth 03 Apr '19

03 Apr '19

Hi, Looking at the specification for the Tor directory protocol's meta-format, it has some text on line wrapping: > When line wrapping, implementations MUST wrap lines > at 64 characters. Upon decoding, implementations MUST > ignore and discard all linefeed characters. I can't work out how an implementation is meant to know if a linefeed character is meant to be for line-wrapping or if it's meant to be the end of a keyword line. Are there any implementations that actually do line wrapping? Thanks, Iain.

3 2

Re: [tor-dev] Docker images for help people to run Tor (teor) (teor)
by Alessandro Fiori 01 Apr '19

01 Apr '19

Hi, thanks for your support! I've modified Dockerfiles to properly install the right Tor version and starting two relays with both scripts. The images have been released on Docker Hub too. Next step for me is preparing script and image for easily setting up a bridge. Since all containers created are Linux containers, and since all my Dockerfiles are "one shot configurations", I will release bash scripts soon. I will contact the community very soon for more advices on relays, and promote-review the project (since the project is newborn, I want to spread for find issues and fix them). If you want to control all the released code and images, the two principal repositories are: https://github.com/randomtable/ChimeraOS https://hub.docker.com/u/alessandrofiori Many thanks Il giorno lun 1 apr 2019 alle ore 14:00 < tor-dev-request(a)lists.torproject.org> ha scritto: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Re: Docker images for help people to run Tor (teor) (teor) > 2. Next Network Team Meeting Time (teor) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 1 Apr 2019 12:57:35 +1000 > From: teor <teor(a)riseup.net> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Docker images for help people to run Tor (teor) > Message-ID: <EBC8EE49-155E-4B26-BC19-E2B0642FF2DB(a)riseup.net> > Content-Type: text/plain; charset="utf-8" > > Hi, > > > On 1 Apr 2019, at 07:55, Alessandro Fiori <alefiori3(a)gmail.com> wrote: > > > > Hi, > > I would like to share with the community some tests I've made for > setting up a "swarm" of Tor relays. > > > > I setted up 3 exit relays and 3 middle relays on the same IPv4 address, > to make some testing, and the entire swarm works like a charm. > > > > Due to Tor limitation, I have decided to build a template for preparing > the cluster, and upload it to GitHub. > > > > You can find the example mini-cluster here: > > https://github.com/randomtable/ChimeraOS/tree/master/TorSwarm > > > > I have tested the exit capabilities on another network with different > public IP address. > > > > All the relays have published their own descriptors, I have prepared the > containers as a single, separated machines. > > > > The process is different from preparing a single relay, because of > Docker build process. > > > > I have disabled caching during the cluster process, so i have been able > to modify torrc without issues. > > > > Because its very interesting, attached to this e-mail there is the > console Logs from the 6 relays. > > > > All containers are read-only, so the entire relay will destroy at the > end of session. > > > > I hope this can be useful. > > Tor clients will only see 2 of your 6 relays, because the Tor directory > authorities only publish 2 relays per IPv4 address. > > Also, please use the deb.torproject.org repositories to get the latest > version of tor: > Mar 31 19:57:08.000 [warn] Please upgrade! This version of Tor (0.3.3.9) > is not recommended, according to the directory authorities. Recommended > versions are: > 0.2.9.15,0.2.9.16,0.2.9.17,0.3.4.10,0.3.4.11,0.3.5.7,0.3.5.8,0.4.0.1-alpha,0.4.0.2-alpha,0.4.0.3-alpha > > https://trac.torproject.org/projects/tor/wiki/TorRelayGuide/DebianUbuntu > > T >

1 0