Hi all,
I'll be in Canada for a destination wedding + vacation.
Isabela will be the one facilitating the UX ticket triage meeting on 8/1
and UX team meeting on 8/2, so those will still be on.
Cheers,
Linda N. Lee
Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee
GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2
Hi Tor people!
Below you can learn about what OONI has been up to in June 2017.
# OONI Monthly Report: June 2017
The OONI team made steady progress in June 2017.
We released the beta version of our measurements API and made significant progress on the re-engineering of our data processing pipeline and on probe orchestration. In collaboration with Access Now, we published a research report on recent censorship findings in Egypt. We also updated test lists in collaboration with our partners, and we continued organizing OONI's Partner Gathering.
## Progress on the re-engineering of our data processing pipeline
This month we made a lot of progress on the data processing pipeline. The new pipeline is now running on a daily basis side-by-side the existing data processing pipeline.
In particular, the measurements API is now using the data generated from the new data processing pipeline to produce the various API endpoints and data is being ingested and processed using our new compressed data format, making it much more space efficient and performant.
## Beta release of measurements API
We have released a beta version of the measurement API that is using as a backend the re-engineered data pipeline.
See: https://measurements-beta.ooni.iohttps://github.com/TheTorProject/ooni-measurements/releases/tag/v1.0.0-beta…
This version of the measurements API also adds a couple of new endpoints including:
* Support for searching individual measurements (/api/v1/measurements?<query>)
* Support for fetching an individual measurement (/api/v1/measurement/<measurement_id>)
Other improvements include:
* Better overall performance
* Some bug fixes
## Progress on probe orchestration
In the area of probe orchestration we made a series of very important improvements, namely:
* Added support for creating alert type notifications (this allows us to send messages to users in specific regions, that when clicked on redirect them to some web page)
* Switched out the push notification backend with something more stable and reliable (gorush: https://github.com/appleboy/gorush)
## Publication of research findings in Egypt
We investigated recent censorship events in Egypt and, in collaboration with Access Now, published a research report with our findings.
We were able to confirm the blocking of 10 news websites in Egypt, as well as the blocking of the Tor network, torproject.org, bridges.torproject.org, and even ooni.torproject.org in some networks.
The report can be found here: https://ooni.torproject.org/post/egypt-censors/
## Updated test lists
During June we updated the following test lists:
Zimbabwe: https://github.com/citizenlab/test-lists/pull/176
Azerbaijan: https://github.com/citizenlab/test-lists/pull/178
Palestine: https://github.com/citizenlab/test-lists/pull/180
Egypt: https://github.com/citizenlab/test-lists/pull/181
Venezuela: https://github.com/citizenlab/test-lists/pull/184
In collaboration with new partners, we also created the following test lists:
Lesotho: https://github.com/citizenlab/test-lists/pull/177
Mongolia: https://github.com/citizenlab/test-lists/pull/184
## Organizing OONI Partner Gathering
We worked on logistics pertaining to the organization of OONI's first Partner Gathering which took place on 10th & 11th July 2017 in Toronto.
### Community meeting
We hosted our monthly community meeting on https://slack.openobservatory.org/ on 28th June 2017. As part of the meeting, we discussed the following:
Resources for mapping information controls around the world
Creating an information controls database to contextualize network measurements from around the world
OONI's upcoming probe orchestration
## Userbase
In June 2017 ooniprobe was run 64 times from 1,594 different vantage points across 158 countries around the world.
This information can also be found through our stats here: https://measurements.ooni.torproject.org/stats
~ The OONI team.
Hello everyone,
This is the bi-weekly status report for Ahmia - Hidden Service Search
## Fixed broken statistics page
Statistics page has been fixed and is working at 'ahmia.fi/stats/'. [0]
## Linking structure of onions
Linking structure of onions is visualized at '/stats/link_graph/'. [1]
The data for popularity of domains according to clicks and backlinks is not
currently available. This graph will be updated once this data is
available.
Next task is to update documentation and software dependencies.
Thanks,
Pushkar Pathak
[0] :
https://github.com/ahmia/ahmia-site/commit/142a661ed638cb4ef28a4bcba3814c5f…
[1] : https://ahmia.fi/stats/link_graph
Hi everyone!
This is my report #4 for the unMessage GSoC project.
# Updates
Since the last report I continued working on #21 [0] to properly use
Twisted:
- Finished making the Peer class' methods asynchronous
- Installed the reactor in both UIs
- Removed support for the Peer class running/calling its own reactor
from a thread as now both UIs pass one to the constructor
I also started changing some other parts that should not depend on IO
to just make objects talk to other objects to simplify writing unit
tests for #33 [1].
# What's next
As #21 and #33 are complex tasks, I will continue working on them in
the next period of GSoC. This period was more focused on using Twisted
correctly (#21) and the next one will be more focused on testing (#33).
More details on these tasks can be followed in the respective
tickets [0, 1] and branches [2, 3].
Thanks,
-Felipe
[0]: https://github.com/AnemoneLabs/unmessage/issues/21
[1]: https://github.com/AnemoneLabs/unmessage/issues/33
[2]: https://github.com/felipedau/unmessage/tree/21/deferreds
[3]: https://github.com/felipedau/unmessage/tree/33/feature/test-suite
Hi everyone!
This is my bi-week status report for GSoC project "Crash Reporter for Tor
Browser".
# The work was focused on Crash Reporter server side:
- we took Mini-Breakpad-Server (MBS) [1] instead of Socorro: there were
problems with setting up Socorro and we decided to make work version with
MBS;
- splitted it into 2 services [2]: MBS-Submit (to get reports that's sent
by Crash Reporter client) and MBS-View (to view gotten reports).
# What's next
We plan consider carefully Crash Reporter data fields (that sends) and
making reproducebly build TorBrowser with enabled Crash Reporter.
Kind regards,
Nur-Magomed
[1] https://github.com/electron/mini-breakpad-server
[2]
https://github.com/nmago/mini-breakpad-server/commit/da3e4ed015a288ef7261bb…
Notes for July 20 2017 meeting:
Nick:
1) Spoke to EFF lawyers concerning licensing on specs and proposals;
will speak to Shari about what they said.
2) Helped Isabela a bit with modularization proposal
Alison:
1) finishing support wiki, would like to have blog post out next week
after support meeting
2) "support team" meeting Monday to talk about streamlining support efforts
3) Tommy and I are talking today about future funding for outreach
4) writing FOSS/Tor cultural norms doc for onboarding
5) Colin is plugging away at the RT backlog
6) Colin is also getting ready to appear on a podcast. Yay for more
people doing public speaking about Tor!
7) Colin is doing some GSoC coordination
8) asked the unconscious bias trainers for the information we wanted.
waiting for a reply.
Shari:
1) What do we need to do to move forward with Montreal meeting agenda
planning?
2) David Goulet asked about a documentary filmmaker who'd like to film
people in Montreal. Thoughts?
3) Reached out to new strategic planner.
4) Signed contract with new auditor.
5) Lots of grant proposals and reporting.
6) Scheduled conversation with lawyers to renew our trademarks on Tor
Project and onion logo.
Steph:
1) Bug bounty launched. Will be sharing throughout today and
sporadically to follow. Post on our site + at H1 (great q&a with gk!).
Has so far been picked up by ZDNet, Threatpost, VentureBeat, probably
more to follow.
2) Still working with Giant Rabbit on our newsletter. Should be on the
last step: creating a form for people to subscribe directly and feed
into CiviCRM.
3) Lining up blog content in an editorial calendar so we can always post
at least 1/week. Maintaining other posting schedules: linkedin: 1/wk.
twitter: 3-6/day. fb: 1-2day.
4) Outlined comms goals for MDF grant, helped with editing. Will map out
more in depth goals / strategy to come.
5) Creating templates. Have ppt, docx, pages. Need to create odf versions.
6) Working back through press inquiries, compiling press list and
published articles.
Georg:
1) I worked on the bug bounty program launch. Thanks to all who helped
getting this finally going.
2) Hiring process for desktop browser dev seems to move forward pretty
well. Worked on the mobile browser dev descriptions as well. Isa: What
needs to be done to move those forward? (I guess we wait on input from
Nathan and/or Mike)?
3) I spent some time looking at the GSoC project related to Tor Browser
(the crash reporter mentored by Tom)
4) Sponsor4 work
Brad:
1) Shari and I have selected new CPA firm, Lindley & Associates, to
provide audit and tax preparation services for TPI
2) Fieldwork for 2016 audit will begin in August; some audit work on
fiscal year ending June 30, 2017 will be performed concurrently
3) Working with Jon to streamline travel & expense tracking for Montreal
meeting
4) Last Friday, July 14, was the first pay date with our new US-based
payroll provider ADP. Any issues to report?
Arturo:
1) The OONI Partner gathering went very well and so did CLSI (CitizenLab
Summer Institute)
2) I am going to be away on vacation from the 26th of July until 11th of
August.
3) The beta version of the measurements API is live:
https://measurements-beta.ooni.io/ and the re-engineering pipeline has
been generating the data for it and running since a week or so. We hope
to be able to deprecated the legacy pipeline in a couple of months.
4) We are a bit late with the ooniprobe-mobile app release and are still
wrapping up some of the remaining dev work to be done for that
5) For sending push notifications and showing messages in the mobile app
of users we have setup a CMS that seems to suck less than others
(https://getgrav.org/). May be of interest to infrastructure people.
Karsten:
1) Wrote initial input for MOSS award final report.
2) Deployed new CollecTor 1.2.0 on both hosts, started working on
CollecTor 1.3.0 release.
3) Still looking into fixing some broken statistics for the last week of
June 2017.
Deadline is August 1st
Festival is October 27-29 in London
It would be great if Tor had some representation!
-tom
---------- Forwarded message ----------
From: Tom Ritter <tom(a)mozilla.com>
Date: 18 July 2017 at 08:50
Subject: Fwd: Submit a session proposal for MozFest 2017!
To: Tom Ritter <tom(a)ritter.vg>
---------- Forwarded message ----------
From: Christopher Lawrence <clawrence(a)mozillafoundation.org>
Date: Mon, Jul 17, 2017 at 1:36 PM
Subject: Submit a session proposal for MozFest 2017!
To: all-moco(a)mozilla.com
Cc: Sarah Allen <sarah(a)mozillafoundation.org>, MoFo Executive Team
<exec(a)mozillafoundation.org>
Hi everyone,
I just wanted to follow up with what Sarah Allen discussed on today’s
project call. The eighth annual MozFest will take place in London from
Friday October 27 - Sunday October 29. It’s the most ambitious
iteration of the festival yet, and a key moment for the global network
for Internet health to convene, organize and act.
Propose a MozFest session today.
The festival’s sessions, speakers and workshops are built to foster
collaboration across disciplines, borders and continents. We’re ready
to face the biggest issues of the day -- from fake news, online
harassment and global cyberattacks to inclusion, innovation and web
literacy -- together, with an eye toward practical, open source
solutions.
This year, the festival will feature six spaces -- physical and
thematic areas based around broad topics related to Internet health --
delivering a variety of talks, sessions, code sprints, artwork and
hands-on workshops. Read more about the MozFest spaces.
I’d like to personally invite you to propose a session, workshop or
demo that is hands-on, educational and encourages collaboration. As
Mozillians, your creativity, expertise, and contribution to the open
web will allow you to contribute to a growing global network.
Also we would love and appreciate your help in two additional ways:
Share the call for proposals with your personal network. We’re
striving to have a huge diversity in voices taking part in the
discussion about Internet health. You can find a sample email, along
with photos and graphics, here to share with your networks.
Tweet! Suggested Tweet: How will you keep online freedom and
opportunity alive on the web? Submit your #MozFest proposal:
https://mozillafestival.org/proposals
We look forward to reading your session ideas. The deadline for
submissions is August 1.
Chris
______________
Christopher Lawrence
V.P. Leadership Network
Mozilla Foundation
t: @chrislarry33
s: chrislarry33
e: clawrence(a)mozillafoundation.org
m: +1.718.757.0843
Hi; we had our regular meeting. Logs here:
http://meetbot.debian.net/tor-dev/2017/tor-dev.2017-07-17-17.00.html
And updates below:
==================
teor (not online):
Last week:
* Took a few days leave, because winter [wow winter!!!]
* Fixed some bugs in PrivCount's circuit and cell counting
* Got a PrivCount code review from Nick (thanks!) and made tickets for it
* Wrote some tor cell packing & unpacking code, so stem can learn to talk ORPort #18856
https://github.com/teor2345/endosome
* Found some tor bugs that might affect link padding (#22934, #22948)
This week:
* Implement some more PrivCount features
* Deploy and Run PrivCount Single Onion Service counts
* Plan and book Montreal travel
* Maybe do cell crypto in endosome
komlo (most likely won't be able to make today's meeting)
Last week:
- Submitted patches for #22905 and #22830 (thanks Isis for splitting!)
- Booked Montreal trip
This week:
- I am going to follow nickm's fuzzing guide to fuzz rust protover, as the last step before submitting this for review.
dgoulet
Last Week:
* The load and configure v3 service patch was merged!!! (#21979)
* Addressed multiple things found in #20657 by asn
* Continue the prop224 client implementation (#17242)
* Important issue found for prop224 service, asn's opened the ticket in
#22940.
This week:
* Address asn's review on #20657 so we can put it back in needs review for
nickm and thus upstream merge.
* Continue client implementation #17242.
* Roger's Defcon talk on July 28th is my hard deadline to have a working
alpha product of prop224 so we can give the world a branch to play with.
isabela:
Last Week:
- got the sponsor4 june report out
- start working with brad on info for nsf reports (because storm was down the progress report part got delayed)
This week:
- will catch up on PT discussion
- will work on tor launcher automation update (if possible)
- will be at PETS
catalyst:
Last week (2017-W28):
- helped Roger with sponsor3 reporting
- started more aggressively tagging stuff with sponsors
- laptop and phone brain transplants (Apple, Y U make iCloud Keychain "Local Items" unusable after migration?)
- helped atagar with some stem bugs #22894 #22902
- helped isis with some Travis CI stuff #22636
- .onion UX stuff
- Trac meeting
- made some progress on learning enough about chutney to make a #20532 test case
This week (2017-W29):
- packing and moving! (see earlier email message for details)
- more review of #22636
- more #20532
pastly:
- KIST code still available on gitweb. Need to talk to dgoulet sometime about reviewing
- KIST still running on my relay and dgouelet's IIRC
- Want to update tickets, but want to put in as little work as possible.
Ideally would just attach paper or something, but it wasn't accepted and
don't want its first publication to be an attachment on some ticket. Talking
to co-author.
- Attempts at measuring network latency. Last week: home-grown with RIPE Atlas. Meh.
This week: Ting.
nickm:
Last week:
- Review and merge on lots and lots of things, incl HS work
- Chase down misc bugs on odd platforms, fix them. (including zstd 1.3.0)
- Initial review and comment on the preliminary tor privcount patch series
- Finish triage on tor-unspecified, start marking some sponsors' tickets in 0.3.2-alpha milestone
This week:
- PETS
- Livetweet PETS?
- Talk with collaborators about sponsor2/anti-fingerprinting stuff
- Hack, time permitting
[WARNING: SSH HATES ME RIGHT NOW; THIS INTERNET CONNECTION IS TERRIBLE]
asn
Last week:
- Mad review of #20657. Got familiar with the code and now I'm getting
deeper with testing.
- Reviewed #22803.
- Started coding #22735. Not an easy task but very important for prop224.
- Started maintaining a pad with open prop224 tickets, so that we can
organize development/merging better as time gets closer:
https://pad.riseup.net/p/LPVY6AL_prop224_upstreaming
- (Had no real time for prop247 simulation stuff...)
This week:
- Write patches for two major open service-side issues: #22735 and #22940
- Continue review of #20657.
- More prop224 frenzy.
ahf: (traveling - will miss the meeting).
Sponsor 4:
- Extended our analyzer to include all Tor traffic such that we
can see which percentage the directory traffic amounts to.
- Started writing the outline of a blog post for blog.tpo about
the Sponsor 4 work.
Sponsor 8:
- Read up on what the proposal includes for the network team.
- Kick off meeting with Nick on Sponsor 8.
- Looked into how to do battery measurements on the Android platform
and trying to figure out how I can make it as reproducible as
possible when on a physical device and if it is possible to track
in an emulator (the wakeups only).
- Read up on using Guardian Projects orbot to get Tor onto a
device on Android as an apk.
This week:
Sponsor 4:
- Discuss results with Nick for the dir / total traffic part.
- Finish blog post about Sponsor 4 work.
Sponsor 8:
- Follow up on tasks from the Tuesday kick-off meeting: look at
Nick's results with which functions that are running in a
timer and do network $stuff.
- Get more knowledge of the Android platform work.
isis:
last week:
- made a server for BridgeDB's CAPTCHAs so that we can serve them to moat too #15976
- made a cmdline option to see the expiration date for a tor signing cert #17639
- polished up and fixed a Travis config for tor #22636
- investigated and fixed some problems with rust builds #22830 #22905
- opened some other rust build system integration tickets #22906 #22907 #22909
- reviewed #22349 (on gitlab!) and #22916
- reviewed and cleaned up #19476
- removed some dead/obsolete code in rephist.c #19871
- started moving the internship forward
this week:
- more work on getting the intern started and set up
- get review/feedback for moat CAPTCHA api #15976
- start writing moat backend server #22871
- revising farfetchd #15876
- maybe getting a service set up for farfetchd?
Freedom! Sweet GSoC-less freedom! Ok, lets ruin it.
Between newhires and new volunteers Tor is growing quite a bit right
now and a common complaint I've heard for years is that our community
makes it maddeningly hard to figure out who's who. Yes, for many of us
this is intentional but for others it's not...
"Who's that arma guy I was talking with on irc? He seemed nice. Ok,
thanks. Now which of these people is Roger? Great. And, what does he
do?". Multiply that by ninety dev meeting attendees and it's no wonder
we drive our lovely hair-pulling newcomers to early baldness.
For much of our community this anonymity is intentional and we
definitely don't want to muck with that, but I suspect some opt-in
information from those of us ok with it could make our community a lot
easier to join. As such I'd like to run the following questionnaire
among our tor-internal@ membership...
* Would you like to be listed on the 'core people' webpage [1]? If so...
* What name or alias would you like to be called?
* Provide a description for the page of what you do.
* Would it be ok to list your IRC nick? If so, what is it?
* Would it be ok to list your OpenPGP key? If so what is your public key?
* Is there a photo or image you'd like to have displayed?
Many folks will say 'heck no' to much of this and that's perfectly
fine. The last question is a bit of an experiment where I'm curious if
we can pattern ourselves after the EFF's page [2]. Maybe this won't
pan out but here's my thoughts...
* Newcomers first come to know us by cryptic irc nicks that look akin
to truncated sha256 digests. Pictures may make our community feel
friendlier and more approachable. Note this *doesn't* need to see a
photo. For example see Mark Burdett and Ben Burke on the EFF page.
* Maybe a page of smiling faces and cartoon turtles will be helpful to
Shari for her upcoming fund raiser?
Maybe this'll work, maybe it won't. My hope is that folks will have
fun coming up with a picture to represent them, but maybe too few
people will to make it work. Who knows - worth asking. ;)
Thoughts? This dovetails nicely with my role maintaining tor-internal@
so I'd be happy to keep it up to date as we continue to grow our
community.
Cheers! -Damian
[1] https://www.torproject.org/about/corepeople.html.en
[2] https://www.eff.org/about/staff