April 2019 - tor-dev - lists.torproject.org

Export BridgeDB usage statistics
by Philipp Winter 24 Apr '19

24 Apr '19

Hi Karsten, I'm working on <https://bugs.torproject.org/9316>, which will make BridgeDB export usage statistics. I would like these statistics to be public, privacy-preserving, and -- ideally -- added to Tor Metrics. I wanted to hear your thoughts on 1) what statistics we should collect, 2) how we can collect these statistics safely, and 3) what format these statistics should have. Broadly speaking, these statistics should answer the following questions: * How many requests does BridgeDB see per day? * What obfuscation protocols are the most popular? * What bridge distribution mechanisms are the most popular? * From what countries do we see the most bridge requests? * How many BridgeDB requests fail and succeed, respectively? * How many requests does BridgeDB see from Yahoo/Gmail/Riseup? * How many HTTPS requests are coming from proxies? * How many requests are suspicious, and likely issued by bots? Each request to BridgeDB carries with it some information, which allows us to answer the above questions. I suggest that we collect the following: * The distribution mechanism. Currently, this is HTTPS, email, or Moat. * The requested transport. Currently this is vanilla, fte, obfs3, obfs4, or scramblesuit. * The request's origin. For Moat and HTTPS, it's the two-letter country code, e.g., IT for Italy. For email, it's the user's email domain (Gmail, Yahoo, or Riseup). * Whether the request was successful or unsuccessful, i.e., resulted in BridgeDB handing out bridges or not. * Whether the request was issued by a user or a bot. David suggested heuristics that would allow us to estimate if a request came from a bot: <https://bugs.torproject.org/9316#comment:19> I like these suggestions but I'm not sure yet how to encode them -- it's more complex than a simple binary flag. The combination of these statistics results in ~16,800 buckets (3 mechanisms * 5 transports * ~280 ISO country codes * 2 success states * 2 bot states). We only need to export statistics with non-empty buckets. To protect users whose request is the only one in a given bucket (e.g., there may be only one user in Turkmenistan who successfully requested an FTE bridge over HTTPS on 2019-04-02), we should bin the statistics by rounding them up to the next multiple of, say, 10. We should further export statistics infrequently -- maybe once a day. Here's an example of a simple CSV format that takes into account the above: timestamp,mechanism,transport,country|domain,success,count,origin 1555977600,https,vanilla,it,successful,40,user 1555977600,https,obfs4,ca,unsuccessful,10,user 1555977600,email,vanilla,yahoo.com,successful,50,user ... What are your thoughts? Thanks, Philipp

2 1

Next Network Team Meeting Time
by teor 19 Apr '19

19 Apr '19

Hi, The next network team meeting is at 2300 UTC on Tuesday 23 April. (Our "patch party" and "first meeting of the month" time.) Here is our schedule for the next few months: April schedule: * Tuesday 23 April at 2300 UTC (at the "patch party" time) * Monday 29 April at 1700 UTC May schedule: * Monday 6 May at 1700 UTC (first May meeting, but at the normal time) * Monday 13 May at 1700 UTC * Monday 20 May at 1700 UTC * Monday 27 May at 1700 UTC June schedule: * Tuesday 4 June at 2300 UTC (first meeting of the month) * Monday 10 June at 1700 UTC T -- teor ----------------------------------------------------------------------

3 4

Support for clients using shutdown(SHUT_WR)
by Rob Jansen 16 Apr '19

16 Apr '19

Hello! I'm in the process of re-writing Shadow's traffic generation tool called tgen [0], which now depends on shutdown() for two communicating tgen nodes to inform each other that they have no more to send (i.e. to perform graceful connection shutdowns). The shutdown() functionality is useful now that tgen uses Markov models to generate more realistic Tor traffic [1]. Currently, it appears that when the tor client receives a FIN on the AP connection (because the tgen client calls shutdown(SHUT_WR)), the tor client sends a RELAY_END cell to the exit relay to instruct it to close() the exit TCP connection to the server. The behavior I expected was something more like what is described in tor-spec in Section 6.3 (search for "RELAY_FIN") [2], where the tor client sends a RELAY_FIN cell to the exit to instruct the exit to perform a shutdown(SHUT_WR) on the exit TCP connection to the server. Is there any plan to support shutdown(SHUT_WR) using RELAY_FIN cells now that Tor is itself using shutdown()? (I didn't see any tickets about it after a brief search.) Thanks! Peace, love, and positivity, Rob [0] https://github.com/shadow/tgen [1] https://tmodel-ccs2018.github.io/ [2] https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt

2 1

ICLab testing of default bridges
by David Fifield 11 Apr '19

11 Apr '19

At the anti-censorship meeting today you mentioned talking to ICLab about testing the default bridges. I believe that ICLab is already testing at least a portion of the default bridges; they may have data that you just have to ask for. OONI and ICLab test the default bridges as a result of my, Lynn's, and Qi's asking them to while we were studying default bridge blocking. The data exist but we never carefully analyzed them. You can see what little analysis we did at https://arxiv.org/abs/1709.08718 Figure 1 and Section 5.9. The data we used is in data/iclab/tor_jsons.tar.bz2 in https://repo.eecs.berkeley.edu/git-anon/users/fifield/proxy-probe-paper.git. It's a lot of JSON files that have TCP connect, TLS certificate fetch, HTTP fetch, and TCP traceroute for each destination. We sent them the list of bridges in September 2016 and they started measuring shortly after that. The most recent update to the bridge list we sent them was in August 2017 for https://bugs.torproject.org/23166.

1 0

Scripts and VMs for helping people setup a relay or bridge
by Alessandro Fiori 09 Apr '19

09 Apr '19

Hi, since i have released some Docker images for help people to run their own service or relays, i'm releasing some Virtualbox images and scripts for who want to run their relays. Virtual machines are in importable OVA format, and are based on Ubuntu Command Line Install, for a very light-weight system. I have released, for now, a Script and Virtual Machine for Tor middle relay, next days i will release bridge scripts for bridge with pluggable transports pre-installed. The Script repository is located here: https://github.com/randomtable/ChimeraOS/tree/master/Scripts The Virtual Machines repository is located here: https://github.com/randomtable/ChimeraOS/tree/master/VMs I'm writing to "dev" mailing list, because a very expert eye is always needed. I hope this can be useful for everyone. Alessandro Fiori

2 1

Re: [tor-dev] New revision: Proposal 295: Using ADL for relay cryptography (solving the crypto-tagging attack)
by Jeff Burdges 08 Apr '19

08 Apr '19

If I understand, proposal 295 looks similar to either BEAR or LION from the LIONNESS. I vaguely recall both BEAR and LION being "broken" in some setting, although I cannot site the paper. Anyone? I suppose the BEAR and LION break originates from using them for authentication while proposal 295’s separate SVer function fixes this? Jeff

1 1

Tor Directory Meta-Format + Line Wrapping?
by Iain Learmonth 03 Apr '19

03 Apr '19

Hi, Looking at the specification for the Tor directory protocol's meta-format, it has some text on line wrapping: > When line wrapping, implementations MUST wrap lines > at 64 characters. Upon decoding, implementations MUST > ignore and discard all linefeed characters. I can't work out how an implementation is meant to know if a linefeed character is meant to be for line-wrapping or if it's meant to be the end of a keyword line. Are there any implementations that actually do line wrapping? Thanks, Iain.

3 2

Re: [tor-dev] Docker images for help people to run Tor (teor) (teor)
by Alessandro Fiori 01 Apr '19

01 Apr '19

Hi, thanks for your support! I've modified Dockerfiles to properly install the right Tor version and starting two relays with both scripts. The images have been released on Docker Hub too. Next step for me is preparing script and image for easily setting up a bridge. Since all containers created are Linux containers, and since all my Dockerfiles are "one shot configurations", I will release bash scripts soon. I will contact the community very soon for more advices on relays, and promote-review the project (since the project is newborn, I want to spread for find issues and fix them). If you want to control all the released code and images, the two principal repositories are: https://github.com/randomtable/ChimeraOS https://hub.docker.com/u/alessandrofiori Many thanks Il giorno lun 1 apr 2019 alle ore 14:00 < tor-dev-request(a)lists.torproject.org> ha scritto: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Re: Docker images for help people to run Tor (teor) (teor) > 2. Next Network Team Meeting Time (teor) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 1 Apr 2019 12:57:35 +1000 > From: teor <teor(a)riseup.net> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Docker images for help people to run Tor (teor) > Message-ID: <EBC8EE49-155E-4B26-BC19-E2B0642FF2DB(a)riseup.net> > Content-Type: text/plain; charset="utf-8" > > Hi, > > > On 1 Apr 2019, at 07:55, Alessandro Fiori <alefiori3(a)gmail.com> wrote: > > > > Hi, > > I would like to share with the community some tests I've made for > setting up a "swarm" of Tor relays. > > > > I setted up 3 exit relays and 3 middle relays on the same IPv4 address, > to make some testing, and the entire swarm works like a charm. > > > > Due to Tor limitation, I have decided to build a template for preparing > the cluster, and upload it to GitHub. > > > > You can find the example mini-cluster here: > > https://github.com/randomtable/ChimeraOS/tree/master/TorSwarm > > > > I have tested the exit capabilities on another network with different > public IP address. > > > > All the relays have published their own descriptors, I have prepared the > containers as a single, separated machines. > > > > The process is different from preparing a single relay, because of > Docker build process. > > > > I have disabled caching during the cluster process, so i have been able > to modify torrc without issues. > > > > Because its very interesting, attached to this e-mail there is the > console Logs from the 6 relays. > > > > All containers are read-only, so the entire relay will destroy at the > end of session. > > > > I hope this can be useful. > > Tor clients will only see 2 of your 6 relays, because the Tor directory > authorities only publish 2 relays per IPv4 address. > > Also, please use the deb.torproject.org repositories to get the latest > version of tor: > Mar 31 19:57:08.000 [warn] Please upgrade! This version of Tor (0.3.3.9) > is not recommended, according to the directory authorities. Recommended > versions are: > 0.2.9.15,0.2.9.16,0.2.9.17,0.3.4.10,0.3.4.11,0.3.5.7,0.3.5.8,0.4.0.1-alpha,0.4.0.2-alpha,0.4.0.3-alpha > > https://trac.torproject.org/projects/tor/wiki/TorRelayGuide/DebianUbuntu > > T >

1 0

Next Network Team Meeting Time
by teor 01 Apr '19

01 Apr '19

Hi, The next network team meeting is at 2300 UTC on Tuesday 2 April. Here is our full April schedule: * Tuesday 2 April at 2300 UTC * Monday 8 April at 1700 UTC * Monday 15 April at 1700 UTC * Tuesday 23 April at 1700 UTC (Monday 22 April is the Easter Monday public holiday) * Monday 29 April at 1700 UTC Daylight saving changed recently, so at least one of these meetings will be at a different local time for you. Our 1700 UTC meeting tracks north american daylight saving time. Our 2300 UTC meeting does not change for daylight saving time. T -- teor ----------------------------------------------------------------------

1 0

Re: [tor-dev] Docker images for help people to run Tor (teor)
by Alessandro Fiori 01 Apr '19

01 Apr '19

Hi, I would like to share with the community some tests I've made for setting up a "swarm" of Tor relays. I setted up 3 exit relays and 3 middle relays on the same IPv4 address, to make some testing, and the entire swarm works like a charm. Due to Tor limitation, I have decided to build a template for preparing the cluster, and upload it to GitHub. You can find the example mini-cluster here: https://github.com/randomtable/ChimeraOS/tree/master/TorSwarm I have tested the exit capabilities on another network with different public IP address. All the relays have published their own descriptors, I have prepared the containers as a single, separated machines. The process is different from preparing a single relay, because of Docker build process. I have disabled caching during the cluster process, so i have been able to modify torrc without issues. Because its very interesting, attached to this e-mail there is the console Logs from the 6 relays. All containers are read-only, so the entire relay will destroy at the end of session. I hope this can be useful. Alessandro Fiori Il giorno sab 30 mar 2019 alle ore 13:00 < tor-dev-request(a)lists.torproject.org> ha scritto: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Re: Docker images for help people to run Tor (teor) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 30 Mar 2019 13:08:53 +1000 > From: teor <teor(a)riseup.net> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Docker images for help people to run Tor > Message-ID: <3D4905AF-1C10-47BF-8823-EB3228FF990F(a)riseup.net> > Content-Type: text/plain; charset="utf-8" > > Hi, > > (Please try to reply to the thread, not the digest!) > > > On 30 Mar 2019, at 11:51, Alessandro Fiori <alefiori3(a)gmail.com> wrote: > > > > Because the images are built separately, and Containers doesn't have > persistent data by default, i'm testing the launch of a "relay swarm", > which isn't the "Docker Swarm". > > The concept of a "Relay swarm" is to have multiple running relay > containers on the same user machine, without any conflict. > > > > The idea (i will testing it very soon) is possible, by pulling an image > and running it multiple times, creating intermetiate containers. > > > > A function like the "relay swarm" can be very useful, i don't know if > this approach can be risky, so i'm preparing a script for launching only > one type of relay swarm (middle or exit relays). > > > > If there is no security concerns, this function will be a very key > function for grow up the Network, because a user can run multiple relays at > a time. > > Tor only allows 2 relays per IPv4 address. > > T >

2 1