# OONI Monthly Report: August 2017
The OONI team made steady progress in August.
We published a new research report on our study of internet censorship in Cuba, and we created three new test lists. We carried out research on how best to secure probe orchestration, and we made progress on our data processing pipeline and on the custom URL scheme for the mobile clients. We also created a new OONI style guide for the design of our apps and the creation of data visualizations. Our commentary about OONI was published by Project Syndicate.
## Published research report on internet censorship in Cuba
We published a new research report, titled: "Measuring Internet Censorship in Cuba's ParkNets"
You can read the report here: https://ooni.torproject.org/post/cuba-internet-censorship-2017/
We also published a summary of the report on the Tor blog: https://blog.torproject.org/blog/measuring-internet-censorship-cubas-parkne…
Our study was covered by the following:
1. Amnesty International: https://medium.com/@AmnistiaOnline/la-paradoja-de-internet-de-cuba-b3cd206a…
2. Motherboard: https://motherboard.vice.com/en_us/article/vbbjxm/here-are-the-41-websites-…
3. El Nuevo Herald: http://www.elnuevoherald.com/noticias/mundo/america-latina/cuba-es/article1…
## Style guide for design of apps and data visualizations
We created a style guide for the design of our apps and for the creation of data visualizations based on OONI data.
Our master style guide can be found here: https://openobservatory.github.io/design/
Our style guide for the data visualizations that are to be included in our reports and blogs can be found here: https://github.com/OpenObservatory/design/issues/2
It's worth noting that we created a graph based on this style guide for our Cuba report, which was re-used by Motherboard.
We also started working on a style guide for the design of our apps: https://github.com/OpenObservatory/design/issues/3
Based on these new style guides, we created some mock-ups for the new design of our website (https://github.com/OpenObservatory/design/issues/5) and mobile apps (https://github.com/OpenObservatory/design/issues/7).
## Created new test lists
In collaboration with community members, we created the following three new test lists:
1. Taiwan test list: https://github.com/citizenlab/test-lists/pull/205
2. Afghanistan test list: https://github.com/citizenlab/test-lists/pull/203
3. Angola test list: https://github.com/citizenlab/test-lists/pull/204
## Research on securing probe orchestration
We brainstormed and researched on how to best secure probe orchestration. Since this is a very critical components we believe it’s essential that it does not get compromise or if compromise happens it is mitigated.
The results of these discussions are documented here:
https://github.com/TheTorProject/proteus/issues/24https://github.com/TheTorProject/proteus/issues/26https://github.com/TheTorProject/proteus/issues/27
## Progress on custom URL scheme
A major feature request that we have received from our community members is to add support in our mobile clients for testing single URLs (independently from whether or not they are included in the Citizen Lab test lists).
This was implemented with a custom URI scheme (ex. ooni://nettest/web_connectivity?url=http://google.com) that allows people to put a badge on their website that, when clicked by a user with the ooniprobe app installed, will run an ooniprobe measurement to a site of their choosing or, if they don’t have the app installed, prompt them to install it.
This will allow people to encourage the usage of ooniprobe for testing the reachability of sites they care to monitor and/or run experiments that they believe are important.
During August, we made progress towards implementing the custom URL scheme in our mobile apps and a new version of the app supporting the URI scheme to run tests with or without custom parameter will be released soon.
More details abou this in the following tickets:
https://github.com/TheTorProject/ooniprobe-android/issues/66
## Progress on data processing pipeline
We continued to make progress on the engineering of our data processing pipeline. During August, we started the extraction of anomalies from the measurements and fixes a variety of different bugs we found in the processing of the data.
More information about this can be found here:
https://github.com/TheTorProject/ooni-pipeline/pull/62
## Project Syndicate commentary about OONI
We wrote a commentary explaining why running ooniprobe matters, which was published by Project Syndicate.
The commentary can be found here: https://www.project-syndicate.org/commentary/ooniprobe-internet-censorship-…
Our commentary was subsequently cross-published by many other publishers as well.
## Userbase
In August 2017 ooniprobe was run 78,464 times from 1,728 different vantage points across 159 countries around the world.
This information can also be found through our stats here: https://measurements.ooni.torproject.org/stats
~ The OONI team.
Here are the highlights of the Community Team's work in August 2017:
August 2017 Community Team report
Meeting notes August 2017
==================================================================
https://trac.torproject.org/projects/tor/wiki/org/teams/CommunityTeam#Curre…
Tor Meeting planning
==================================================================
We're still soliciting agenda items for the Montreal meeting. If you
haven't submitted your ideas yet, please add them to the meeting wiki
that was linked in your invitation. We'll share out a draft agenda
sometime in the next few weeks.
Phoul has also been planning a relay operators meetup for the Montreal
meeting.
Support portal/support wiki
==================================================================
We had a support wiki sprint and added tons of new material to the
support wiki:
https://trac.torproject.org/projects/tor/wiki/org/teams/CommunityTeam/Suppo…
Community documents
==================================================================
We finished the proposal period for the contributor/membership
guidelines document, which went up for a vote in September.
Library Freedom Project
==================================================================
LFP won a grant from the Institute of Museum and Library Services to
create an intensive privacy training program for librarians over the
next two years.
We're also working on a website redesign.
Speakers bureau
==================================================================
Sukhe gave a talk at the Greater Toronto Area Linux User Group (GTALUG).
Next month he'll give a number of Tor talks in India.
Phoul appeared on a privacy and security podcast:
https://privacy-training.com/podcast.html.
The community and communications teams began organizing some speaker
training sessions for members of the speakers bureau.
Tshirt requests
==================================================================
We continue to handle all the tshirt requests!
==================================================================
Next up: still more Tor Montreal meeting planning, reopening the code of
conduct proposal, finishing outstanding items on support wiki, thinking
of new places to
==================================================================
Thanks for reading!
--
Alison Macrina
Community Team Lead
The Tor Project
Hi! Our meeting log is here:
http://meetbot.debian.net/tor-dev/2017/tor-dev.2017-09-11-17.00.html
and below is the contents of our meeting update pad:
Network team meeting, 11 September 2017 (1700 UTC)
Announcements and requests:
- Can we have trac back? (It was down Sunday/Monday UTC)
[It's back up now. Folks are working to try to make it more
reliable. see thread.]
- metrics team request?
- privcount/prop280 meeting tomrrow (er, 2017-09-13 00:00Z)
People out:
-
teor (offline):
- Last week or two:
* Released (experimental) PrivCount 1.1.0: https://github.com/privcount
* Kept at it with the floating-point stuff. Now we have dev guidelines
(thanks catalyst!), and a bunch of specific issues:
https://trac.torproject.org/projects/tor/ticket/23061
* Work out how many onion addresses an HSDir should expect to see
(and intro and rend weights as well):
https://github.com/privcount/privcount/issues/397https://trac.torproject.org/projects/tor/ticket/23367
[asn: interesting stuff! definitely connected to #23126 as
well. we should think
of what to do for v3. perhaps montreal session] +1,
including thinking of useful stats with privcount.
[dgoulet: The overlap period concept has changed also re #23367]
* Revised the patch that fixes bridge clients on tor master
https://trac.torproject.org/projects/tor/ticket/23347
* Reviewed a bunch of nickm's spec patches
- This week:
* Start developing PrivCount 1.2.0 (HSDir client usage)
* Try to knock over a few of the floating-point bugs
* Merge HSv3 support into chutney (thanks asn, dgoulet!), and add a
v3 test network to tor master's make test-network-all
https://trac.torproject.org/projects/tor/ticket/22430
Nick:
* Last week
- Release 0.3.1.6-rc
- Review and merge lots of patches, cleaning up as needed.
- Run through trivial open 0.3.2.x tickets, try to solve the fast ones
- Some triage
- Get callgraph generation working again. (See
https://people.torproject.org/~nickm/tor-auto/ )
- Looked over some Rust stuff with Komlo.
* This week
- 0.3.2.x freeze: merging, coding, reviewing, triaging. Trying not to
be too ambitious.
- Triage plan: if there is anything in 0.3.1 or 0.3.2 you will not do,
and it is assigned to you, unassign it, please. If it is essential,
please talk to others about making it happen.
- Triage plan: i'm likely to remove (nearly?) all stuff in 0.3.1 or
0.3.2 that is unowned. If you will do it, pleas assign it to
yourself.
- Try to do a little 0.3.3.x planning-planning
- Think about 0.3.1.x: why so late?
- Start on a couple of proposals for 0.3.4.x and forward.
- Privcount meeting tuesday night
Isabela:
* Last week
- spent most of the week working with Linda and Steph (f2f here in
NYC) on the dev and community portals part of the website redesign
project
- the only deliverable network team had with 4 is now officially 'close'
* This week
- organize sponsor8 work
- start preparing for roadmap in Montreal with the team
- how are we with dependencies for TB team on Tor Launcher work?
moat api work and the progress bar requests from mcs
dgoulet:
* Last week:
- Most of my time was hunting down #23387 and working on a fix and unit
tests.
- Worked on KIST branch addressing pastly's review of #12541. It is now on
oniongit awaiting review and hopefully make it in 032.
- Reviewed some tickets in review-group-23.
* This week:
- Hunting down prop224 bugs and writing tests.
- Will work on more general Tor for 031 and 032 upcoming releases.
catalyst:
* Last two weeks (2017-W35, 2017-W36):
- Went down a floating point rabbit hole. Reviewed some stuff
(#23061); wrote some guidelines (#23368).
- Reviewed some patches, including the large buf API refactor (#23149)
- Looking into bootstrap reporting led to a twisty maze of periodic events
* This week (2017-W37):
- Work on control protocol extensions for bootstrap reporting. Extend
OR_CONN events to report per-connection PT progress? Or create a new
class of events?
- Maybe keep an eye on privcount stuff as relates to floating point.
- Extract bootstrap reporting from deep in the call stack and make it
the responsibility of higher level code.
- Better error message reporting to help Tor Launcher
ahf:
Last week:
Misc:
- Back from vacation tasks: read up on email threads, time sheets,
meeting logs, trac, tor.git commits, signed up for team rotation
duties, etc.
- Got travel approval and booked trip for the Montreal meeting via
Jon+Shari. Arriving the 6th of October.
- Send traveling information to OTF for the OTF meeting in Valencia.
- Continuing to keep the NG HS IRC running: found one bug
when mixing
legacy and NG HS on the same tor instance this week.
Sponsor 8:
- Working on a Tor instance where I can send events from the
Android Java interface to Tor via orbot.
- Unbricked a device that I had borked after a failed attempt at a
self-build Android image with some debugging enabled.
- Going over Nick's event list from August about timed event
sources and wrapped my head around that.
This week:
- Help out with 0.3.1 release if need be?
- Continuing with the sponsor 8 work:
- Try to flesh out a plan with some deadlines for the
tasks/results. Maybe together with Isa?
- How's it going with the Android OS dev hire we were looking for?
- Spend some time wrapping my head around Rust before the montreal
meeting.
mike:
Last week:
- Meeting in Seattle
This week:
- Page back in everything since vacation. Here's the dump from my notes:
- #23077, #23097; lower prio: #22934, #23100, #23101
- Design + document planned prop247 prototype performance experiments
asn
Last week:
- Worked on fixing more prop224 bugs that have crawled up (#23331, #23056,
#23327, #23355, #23361). We've merged most of the fixes upstream by
now. The current tor.git master offers a pretty good prop224 experience. No
blockers are currently known.
- Worked on debugging and fixing a serious reachability issue of prop224
which could cause reachability issues under natural causes: #23387. We
identified it while writing some advanced unittests for prop224 (see
#23310). The fix required both a spec patch and a code patch which has
already been merged mainline.
- Got more people to try prop224 and join our experimental ircd server. The
testing has so far been very useful since with more people we can find more
rare bugs. We've already caught #23455 this way and have learned more info
for #23457.
This week:
- Get #23457 and #23455 fixed.
- Test and fix more bugs.
- Get into review-group-23.
isis:
last week:
- more work on #15967
- trac kept going down and losing stuff i typed :'(
this week:
- finish #15967
- start #22871 (moat backend)
pastly:
Finished a 2 week vanilla/kist experiement using the KIST dgoulet
reviewed. I have graphs.
Coming up: review dgoulet's KIST work again #12541
Notes for September 7 2017 meeting:
Nick:
0) Was out most of last week; back now.
1) Released 0.3.1.6-rc; trying to time the stabilization for that series.
2) Behind on implementation and triage for 0.3.2.x; not too far, though,
I think. Mostly my fault?
Shari:
1) trying to dig out of post-vacation backlog
2) working with Giant Rabbit on end-of-year campaign; going
back-and-forth on their proposal; looks like they'll get started next week
3) trying to finalize dates for Spring 2018 Tor meeting
4) hosting Mike, Nathan, and Roger and meeting with Mozilla and Amazon
tomorrow to discuss Internet of Things and browsers
Brad:
1) Final invoice for Sponsor R will be issued today
2) Financial statement and compliance audits are continuing
3) Several new grants and contracts awarded this summer
4) Tentatively assigned sponsor codes to new grants/contracts and we're
up to Sponsor 15
Mike:
1) Coming back to consensus reality after eschaton^Wvacation. (You can
only handle so much of the undifferentiated love for all things, before
it gets boring and you really wish there was a Thunderdome nearby...)
2) Going to be buried in email while I try to dig out from a couple
weeks of backlog. (I ignored *all* of it :).
3) Going to Shari's for the meeting with Mozilla and Amazon today.
Isabela:
1) Working f2f with Linda and Steph (and Roger) on the website redesign
project - mostly focus on content mapping and organization of
dev.torproject.org and community.torproject.org
2) Working with Hiro: helping with newsletter work and survey work mostly
3) reports for invoices are out for Brad - working on sponsor4 August report
4) starting to prep sponsor8 M&E stuff
5) got nice feedback on my kanban board :)
Arturo:
1) Gave a remote presentation at a workshop held by CAIDA called IMAPS
(https://www.caida.org/workshops/imaps/1709/), they are doing very
interesting stuff
2) Most of the OONI team won the 34c3 ticket fairy game
3) Made some good progress on measurement API design and pipeline
design: https://github.com/TheTorProject/ooni-measurements/issues/32,
https://github.com/TheTorProject/ooni-pipeline/issues/76
4) We made a release candidate for ooniprobe android 1.2.0-rc.1:
https://github.com/TheTorProject/ooniprobe-android/releases/tag/v1.2.0-rc.1
5) We are evaluating a possible headless CMS
(https://headlesscms.org/about/) choice for use by OONI
Georg:
1) We released Tor Browser 7.0.5
2) Collected all the meeting ideas that are Tor Browser related and for
the team meeting day. nickm: We'll probably have a meeting with Mozilla
about what we need to do to get tor integrated into Firefox. It would be
good if one from the network team could attend to help from the
little-t-tor side
3) I started to talk to Jon about special bug bounty swag
4) Timesheet reporting question (do I as a team lead need to take care
of my team reporting timesheets via Harvest/to accounting?) <- Brad: Sue
is taking care of that
Steph:
1) working on website in NYC this week w Isa, Linda (+Roger yesterday)
2) helped publish Orfox post. Has been picked up by Threatpost, ZDNet,
Extreme Tech, Neon. Pushing it and new release on social this week
3) finalizing choice of trainer / content for speaker training in Montreal
4) working w Hiro, Isa on newsletter to integrate subscribe elements
from donate.tp. Will work with ux team to align design with newsletter
draft
5) we are live on Outreachy for user advocate internship
https://www.outreachy.org/apply/rounds/2017-december-march/
Core Tor Team August 2017 report
Work completion report:
Subobjective 1.1: Reduce Tor processing overhead for low-bandwidth
scenarios.
Activity: Improve the Directory Authority consensus part of the Tor
network in order to optimize low bandwidth users experience.
We created a plan[1] to achieve this goal, where we would into each idea
we had or could find at our issue tracker or email lists to improve the
Directory Authority consensus and implement the most promising ones.
Since all this is new at Tor, we had to start by building the tools[2,3]
to perform this measurement and these analyses to choose the best ideas
to implement for this activity.
You will find on this report the list of ideas we identified as the best
solutions for this activity goal. We also documented[4] this selection
process on our wiki.
The design changes ideas we choose gave us 45% of bandwidth reduction
for relays and 41% for clients, comparing with the releases prior to the
0.3.1 where these ideas were implemented.
A rock on our road for this project was a bug we reported in May related
to the implementation of proposals 140 and 278, that cost us some time
to figure out. This bug was leading to more bandwidth usage which is
completely the contrary of our goal with this project. The bug was fixed
and documented on ticket #22702 [5].
Design changes implemented to improve Directory Authority consensus part
of Tor network:
Idea: Consensus diffs
* Proposal 140 [6]. Ticket #13339 [7].
* Idea: Instead of fetching a consensus, clients could fetch a diff
from the most recent consensus they have.
* Status: Implemented in Tor 0.3.1.x.
Idea: Negotiate better compression algorithms
* Proposal 278 [8]. Ticket #21211 [9].
* Idea: There are other algorithms like LZMA/LZMA2 or bzip2 that
deliver better compression than zlib. We could use HTTP encoding
negotiation to select them when available.
* Status : Implemented in Tor 0.3.1.x.
Idea: Rotate onion keys less frequently
* Proposal: 274. Ticket #21641 [10]
* Idea: In order to limit the bandwidth needed for microdescriptor
listing and transmission, we reduce the onion key rotation rate from the
current value (7 days) to something closer to 28 days. Doing this will
reduce the total microdescriptor download volume by approximately 70%.
* Status: Implemented in Tor 0.3.1.x
Idea: Avoid requesting microdescriptors in small batches
* Idea: Currently when we want e.g. 21 microdescriptors, we'll probably
request them in 3 batches of 7. This makes our compression ratios worse
than they would be otherwise. But it ensures that a single cache can't
selectively deny us microdescriptors.
* Status: Ticket #23220 [11] implements a simple version of this.
[1]
https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Sponsor…
[2] https://github.com/nmathewson/consensus-diff-analysis
[3] https://gitlab.com/ahf/tor-sponsor4-compression
[4]
https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Sponsor…
[5] https://trac.torproject.org/projects/tor/ticket/22702
[6]
https://gitweb.torproject.org/torspec.git/tree/proposals/140-consensus-diff…
[7] https://trac.torproject.org/projects/tor/ticket/13339
[8]
https://gitweb.torproject.org/torspec.git/tree/proposals/278-directory-comp…
[9] https://trac.torproject.org/projects/tor/ticket/21211
[10] https://trac.torproject.org/projects/tor/ticket/21641
[11] https://trac.torproject.org/projects/tor/ticket/23220
Hi,
In August, I worked on the following:
* Tor Messenger
- Resolved build issues with the ESR52 transition related to the Windows
and macOS builds, including deterministic builds for these platforms.
- Worked on the following tickets:
#10942 #13855 #15161 #16606 #17469 #22005 #22229
This marks completion of the transition to and testing of Firefox ESR52.
The next release will be based on Tor Browser
tor-browser-52.3.0esr-7.0-1-build1, and THUNDERBIRD_52_3_0_RELEASE on
comm-central.
* TorBirdy
- Released TorBirdy 0.2.3:
https://blog.torproject.org/blog/torbirdy-023-released
* Community:
- Gave two Tor talks in Toronto, Canada
8 Aug 2017: GTALUG
9 Aug 2017: DEFCON416
(Slides: https://people.torproject.org/~sukhbir/talks/toronto-aug-17.pdf)
--
Sukhbir
Hi everyone,
Below is the monthly Transifex report for August, 2017:
### Report
Attached are daily, weekly and monthly translation graphs. The Y
axis is "source words".
13.35K source words
3,359 collaborators
154 languages
41 Project resources
9 languages at 100% completion (across all 41 project resources):
Bulgarian (bg)
Chinese (China) (zh_CN)
Chinese (Taiwan) (zh_TW)
French (fr)
French (Canada) (fr_CA)
German (de)
Norwegian Bokmål (nb)
Portuguese (Brazil) (pt_BR)
Spanish (es)
--
Colin Childs
Tor Project
https://www.torproject.org
Twitter: @Phoul
Hi all!
In August the Tor Browser team made two releases, Tor Browser 7.0.4[1]
and Tor Browser 7.5a4[2].
Both releases updated Firefox to 52.3.0esr and bumped the respective Tor
versions included. Moreover, we added a fix to avoid scary warnings when
dealing with .onion services that don't have a TLS certificate[3].
In addition to that, the alpha release provided a number of improvements
and bug fixes that needed more testing: selfrando enabled Tor Browser
bundles for 32-bit Linux users[4], regression fixes around our external
helper app dialog[5], and better W^X enforcement for Tor Browser on
Windows[6] to mention just the highlights.
Non-release work focused on remaining Sponsor4 items: we tested the
Gitian replacement we have been working on for the past months and are
switching over to it while I am writing this status report. The next
alpha release is supposed to get built with rbm/tor-browser-build to
give it a wider testing and iron out remaining issues. We documented and
fixed the first issues when building Tor Browser for 64-bit Windows.[8]
And, finally, our UX redesign of Tor Launcher got a first prototype we
can experiment with and improve upon.[9]
The full list of tickets closed by the Tor Browser team in August is
accessible using the `TorBrowserTeam201708` keyword in our bug tracker.[10]
In September we plan to get our first alpha builds out which are based
on rbm/tor-browser-build. Moreover, we finally plan to integrate
FPCentral as our Panopticlick instance into our QA system.[11]
Additional items on our ToDo list are: enabling sandboxing in Linux
bundles in the next alpha release[12], making progress with our Tor
Browser for 64-bit Windows project, getting the proposed Tor Launcher
improvements better tested and finetuned, and starting to update the
security slider by taking past year's Firefox vulnerabilities into
account[13].
All tickets on our radar for this month can be seen with the
`TorBrowserTeam201709` keyword in our bug tracker.[14]
Georg
[1] https://blog.torproject.org/blog/tor-browser-704-released
[2] https://blog.torproject.org/blog/tor-browser-75a4-released
[3] https://bugs.torproject.org/21321
[4] https://bugs.torproject.org/20848
[5] https://bugs.torproject.org/22618
[6] https://bugs.torproject.org/22563
[7] https://bugs.torproject.org/23213
[8] https://bugs.torproject.org/23229
[9] https://bugs.torproject.org/23261
[10]
https://trac.torproject.org/projects/tor/query?status=closed&keywords=~TorB…
[11] https://bugs.torproject.org/6119
[12] https://bugs.torproject.org/22692
[13] https://bugs.torproject.org/23409
[14]
https://trac.torproject.org/projects/tor/query?status=accepted&status=assig…
Hi All,
I am trying to run ricochet on MacOS.
I recently had to do a clean install of my OS so I am in the process of
reinstalling all the software.
Why is Ricochet now in spanish when I open it?
I have downloaded it from
https://ricochet.im/releases/1.1.4/ricochet-1.1.4.1-macos.dmg
Not sure what to do to get it in English again.
Kind Regards,
-- Andri Effendi <fusionman133(a)gmx.de> Organiser of The Free Software
Movement in Sydney www.freesoftware.org.au/sydney/ GPG fingerprint: 8438
138D ECDA 05E0 591F F2B4 4721 0F03 AC24 DF73 Confidentiality cannot be
guaranteed on emails sent or received unencrypted. Sent with GMX - Email
Made in Germany