April 2017 - ooni-dev - lists.torproject.org

Turbo OONI report downloader
by David Fifield 15 Jun '17

15 Jun '17

I wrote a program that uses the OONI API to download reports and keep a local directory of reports up to date. It's much faster than the Wget loop I used to use and it finishes quickly when there is nothing new to download. git clone https://www.bamsoftware.com/git/ooni-sync.git For example, lately I've had to download a lot of tcp_connect reports. I run it like this: ooni-sync -xz -directory reports.tcp_connect/ test_name=tcp_connect This command downloads the index of tcp_connect reports and only downloads the ones that are not already downloaded. It compresses the downloaded files with xz. The next time I need to update, I run the same command again, and it only downloads reports that are new since the last time. You can use other query parameters supported by the API, like probe_cc, probe_asn, since, and until. For example: ooni-sync -xz -directory reports.is/ probe_cc=IS since=2017-01-01 ooni-sync -xz -directory reports.as25/ probe_asn=AS25 ooni-sync -xz -directory reports.tor-turkey/ test_name=vanilla_tor probe_cc=TR ooni-sync -xz -directory reports.web_connectivity/ test_name=web_connectivity since=2017-01-01 until=2017-01-02 I prefer to keep all the reports compressed on disk, so I always use the -xz option, but by default reports are saved unmodified.

2 3

Procedure for reporting false positives
by Khairil Yusof 15 May '17

15 May '17

I'm reviewing the Malaysian Citizen Lab test url report summary and there seem to be false positives (or possible interference that is not visible) The ones that are obviously outdated, and redirected due to domain parking or something I file for removal (usually short lived minor porn sites) Where should I report and track these? http://www.yawa.org on-line gambling site is reported as possible http- diff

3 3

Some graphs of OONI coverage (country, continent, test_name)
by David Fifield 29 Apr '17

29 Apr '17

Here are some quick graphs visualizing the number of OONI reports in various dimensions. https://people.torproject.org/~dcf/graphs/ooni-coverage-20170428/Rplots.pdf https://people.torproject.org/~dcf/graphs/ooni-coverage-20170428/ (source code) My research group was interested in knowing, for example, which countries have high OONI coverage and which don't. These graphs show total volume of reports over time, also broken down by country and continent, and the most common test_names.

2 2

Tor Trac component Metrics/Ooni
by Karsten Loesing 28 Apr '17

28 Apr '17

Hi OONI people, we briefly talked about this before (in Berlin or Amsterdam, I don't recall): I'd like to move the Metrics/Ooni subcomponent somewhere else, because it's unrelated to what Tor metrics folks are working on. I noticed that the last ticket update in that component was 3 months ago. Is it correct that you're not using Tor's Trac for tracking issues anymore? In that case, would you mind me renaming the component to Archived/Ooni? The tickets would all still be there and accessible by the same ticket numbers! Otherwise, if you still use the component, we could easily justify calling the component Ooni (or OONI) as one top-level component. I guess we should talk to a Trac admin person before doing that, but I don't expect major pushback. What do you think? All the best, Karsten

2 2

Reminder for weekly OONI IRC meeting: Monday 2017-04-24, 16:00 UTC in #ooni
by Vasilis 24 Apr '17

24 Apr '17

Hello everyone, (apologies for cross-posting ooni-dev should be the default for this announcement) This is a reminder for today's weekly OONI IRC meeting. It will happen as usual on the #ooni IRC channel (irc.oftc.net) at 16:00 UTC. Meeting times around the world: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170410T16 In case that you have no IRC client you can join via the web from: https://kiwiirc.com/client/irc.oftc.net/ooni Everybody is welcome to join us and bring their questions and feedback. Please use the meeting agenda pad if you want to propose any topics: https://pad.riseup.net/p/ooni-irc-pad You can always find previous meeting logs at: http://meetbot.debian.net/ooni/ ~Vasilis -- Fingerprint: 8FD5 CF5F 39FC 03EB B382 7470 5FBF 70B1 D126 0162 Pubkey: https://pgp.mit.edu/pks/lookup?op=get&search=0x5FBF70B1D1260162

1 0

Reminder for weekly OONI IRC meeting: Monday 2017-04-24, 16:00 UTC in #ooni
by Vasilis 24 Apr '17

24 Apr '17

Hello everyone, This is a reminder for today's weekly OONI IRC meeting. It will happen as usual on the #ooni IRC channel (irc.oftc.net) at 16:00 UTC. Meeting times around the world: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170424T16 In case that you have no IRC client you can join via the web from: https://kiwiirc.com/client/irc.oftc.net/ooni Everybody is welcome to join us and bring their questions and feedback. Please use the meeting agenda pad if you want to propose any topics: https://pad.riseup.net/p/ooni-irc-pad You can always find previous meeting logs at: http://meetbot.debian.net/ooni/ ~Vasilis -- Fingerprint: 8FD5 CF5F 39FC 03EB B382 7470 5FBF 70B1 D126 0162 Pubkey: https://pgp.mit.edu/pks/lookup?op=get&search=0x5FBF70B1D1260162

1 0

Reminder for weekly OONI IRC meeting: Monday 2017-04-10, 16:00 UTC in #ooni
by Vasilis 10 Apr '17

10 Apr '17

Hello everyone, This is a reminder for today's weekly OONI IRC meeting. It will happen as usual on the #ooni IRC channel (irc.oftc.net) at 16:00 UTC. Meeting times around the world: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170410T16 In case that you have no IRC client you can join via the web from: https://kiwiirc.com/client/irc.oftc.net/ooni Everybody is welcome to join us and bring their questions and feedback. Please use the meeting agenda pad if you want to propose any topics: https://pad.riseup.net/p/ooni-irc-pad You can always find previous meeting logs at: http://meetbot.debian.net/ooni/ ~Vasilis -- Fingerprint: 8FD5 CF5F 39FC 03EB B382 7470 5FBF 70B1 D126 0162 Pubkey: https://pgp.mit.edu/pks/lookup?op=get&search=0x5FBF70B1D1260162

1 0