Hi,
I just had a really great conversation with some of the developers at Briar about the recent work they've done in integrating some pluggable transports into their messaging application. I thought I would summarize some key points from the conversation here. In particular, this information might prove useful for both the metrics and anti-censorship teams to know how other projects are using our tools and what they would like help with.
This summary is organized as follows:
I. How they used metrics data II. Independent reachability tests they performed III. What they did to integrate pluggable transports into Briar IV. What they want help with
I. How they used metrics data
One goal the Briar project had was to automatically detect where bridges are needed so they can be enabled automatically and for Briar to "work out of the box" with minimal developer overhead as they do not currently have dedicated anti-censorship funding. Here is the related ticket with a lot of useful discussion: https://code.briarproject.org/briar/briar/issues/1266
They wrote a collection of analytics scripts that use both OONI and Tor Project metrics data to determine where bridges are being blocked.
Source code for the scripts: https://code.briarproject.org/briar/tor-circumvention-analytics/
Results (outdated): https://grobox.de/tor/bridges.html
They rely mostly on OONI data but also use some Tor metrics to look at the ratio of bridge users in a country. Their reasoning being that while OONI shows what doesn't work, Tor metrics data shows what is currently working.
Source code for Tor metrics script: https://code.briarproject.org/briar/tor-circumvention-analytics/blob/master/...
The ticket linked above has more discussion that might be useful to the metrics team as well.
II. Independent reachability tests they performed
To validate some of the results from OONI and Tor metrics, they ran a private bridge and did some reachability tests to it from China. They found that the bridge was not blocked, however they did not perform bandwidth tests to determine whether or not obfs4 bridges are being throttled. This makes sense as briar produces probably fairly low bandwidth traffic. When I asked them about it they suggested performing the following test:
Set up obfs4 to forward connections to a local http server that servers a large, static file. Have a contact in China set up a cron job that periodically downloads the file from the bridge via obfs4.
Their original reachability test was not running for very long, so it's still possible that prolonged use would result in its discovery but the fact that it wasn't blocked suggests that China might be enumerating bridges through the distribution mechanism rather than by something identifiable from obfs4 itself. We should perform our own tests to verify this, and to check for throttling.
III. What they did to integrate pluggable transports into Briar
My understanding from the issue text and commit messages about how briar decides whether to use pluggable transports is to use the output of their reachability analysis to determine user needs by country. If Tor is not blocked, users just use vanilla Tor If it is they use obfs4. If obfs4 is blocked, then they use meek (meek lite). Right now the countries that use meek in briar are: China, Iran, Egypt, Belarus, Turkey, Syria, and Venezuela.
As far as the integration, they wrote some code that makes reproducible builds of obfs4 and meek and spits out a java/android library:
https://code.briarproject.org/briar/go-reproducer
Briar already uses Tor, so they configure these bridges in the usual way using a torrc file and a hardcoded, shipped file of bridge information (which I believe are also the default bridges used by Tor Browser). They decided they didn't want to maintain any private bridges like the one they used for their reachability tests. One of the concerns there was the ability to fingerprint Briar traffic by bridge connection and differentiate it from other Tor traffic.
IV. What they want help with
Because they do not have dedicated anti-censorship funding, they mentioned a few things that would help them maintain their pluggable transport use going forward and ease the integration of pluggable transports.
The main thing they would like on the metrics side is up-to-date information about which PT works in which country and where PTs are needed at all in order to make quick and easy decisions based on location about which transports to use. They started to work with OONI to expand their tests but it turned out to be too much work for their time/funding: https://code.briarproject.org/briar/briar/issues/1414
It's on our roadmap to work with OONI and other censorship measurement tools (like Censored Planet) to expand our tests so we should get into contact with them again once we have gotten farther with this.
On the PT development side, they expressed a desire to transfer maintenance of their reproducible builds of obfs4 and meek to someone else (that's the go-reproducer code linked above).
On 2019-02-22 17:05, Cecylia Bocovich wrote:
[...]
Thanks for summarizing that conversation here!
The main thing they would like on the metrics side is up-to-date information about which PT works in which country and where PTs are needed at all in order to make quick and easy decisions based on location about which transports to use. They started to work with OONI to expand their tests but it turned out to be too much work for their time/funding: https://code.briarproject.org/briar/briar/issues/1414
Maybe this was considered before (I didn't follow any links due to lack of time, sorry), but we do have numbers on bridge users by country and transport:
https://metrics.torproject.org/userstats-bridge-combined.html?start=2018-11-...
They could use that data as input for a tool that outputs which transport to use in which country.
Please also find the links on that page to the CSV file, the CSV data format, and how to reproduce the data. Happy to answer questions.
All the best, Karsten
Cecylia Bocovich:
Hi,
I just had a really great conversation with some of the developers at Briar about the recent work they've done in integrating some pluggable transports into their messaging application. I thought I would summarize some key points from the conversation here. In particular, this information might prove useful for both the metrics and anti-censorship teams to know how other projects are using our tools and what they would like help with.
Thanks, that's really helpful. FWIW, I think this information is not only useful for the teams you mentioned but as well for teams that actually want to implement respective user-facing changes, like the applications team.
We had a bunch of conversations in the past about how to make sure Tor Browser is helping the user to choose a proper and secure way of connecting to the Tor network depending on their location and context. Some useful tickets, showing where we are and what we think/thought, are:
https://trac.torproject.org/projects/tor/ticket/21951 https://trac.torproject.org/projects/tor/ticket/24527
Those are mostly related to desktop as we have a Tor Launcher there. I opened a ticket (#29590) to track the mobile specific part, though.
[snip]
IV. What they want help with
Because they do not have dedicated anti-censorship funding, they mentioned a few things that would help them maintain their pluggable transport use going forward and ease the integration of pluggable transports.
The main thing they would like on the metrics side is up-to-date information about which PT works in which country and where PTs are needed at all in order to make quick and easy decisions based on location about which transports to use. They started to work with OONI to expand their tests but it turned out to be too much work for their time/funding: https://code.briarproject.org/briar/briar/issues/1414
It's on our roadmap to work with OONI and other censorship measurement tools (like Censored Planet) to expand our tests so we should get into contact with them again once we have gotten farther with this.
On the PT development side, they expressed a desire to transfer maintenance of their reproducible builds of obfs4 and meek to someone else (that's the go-reproducer code linked above).
I think we should be able to provide that with our Tor Browser builds once we have all the PT pieces sorted out (which is rather soon). So, probably the easiest way would then be to just copy the respective binaries we produce over to include them in the Briar software and pointing to our build process (we could even document building those PTs as an example in our READMEs if that would be useful). Would that work, Michael? Or do you need something else here?
Georg
On 2/26/19 11:19 AM, Georg Koppen wrote:
I think we should be able to provide that with our Tor Browser builds once we have all the PT pieces sorted out (which is rather soon).
That would be nice!
So, probably the easiest way would then be to just copy the respective binaries we produce over to include them in the Briar software
I assume these binaries are reproducible?
We are currently not including these binaries directly, but publish them as a library (gradle/maven to jcenter) where Briar and other projects can get them from.
and pointing to our build process (we could even document building those PTs as an example in our READMEs if that would be useful).
Currently, we provide reproducible build and verification scripts as well as Docker containers for people to reproduce these binaries as conveniently as possible [1]. We currently produce binaries for the Android CPU architectures as well as for 64-bit Linux.
It would be nice, if there would be the same once we switch to Tor's binaries.
Kind Regards, Torsten
[1] https://code.briarproject.org/briar/go-reproducer#go-reproducer
Torsten Grote:
On 2/26/19 11:19 AM, Georg Koppen wrote:
I think we should be able to provide that with our Tor Browser builds once we have all the PT pieces sorted out (which is rather soon).
That would be nice!
So, probably the easiest way would then be to just copy the respective binaries we produce over to include them in the Briar software
I assume these binaries are reproducible?
They would be used for Tor Browser which has reproducibility as a hard requirement, so yes.
We are currently not including these binaries directly, but publish them as a library (gradle/maven to jcenter) where Briar and other projects can get them from.
Okay, good to know.
and pointing to our build process (we could even document building those PTs as an example in our READMEs if that would be useful).
Currently, we provide reproducible build and verification scripts as well as Docker containers for people to reproduce these binaries as conveniently as possible [1]. We currently produce binaries for the Android CPU architectures as well as for 64-bit Linux.
It would be nice, if there would be the same once we switch to Tor's binaries.
That should be doable. I'll open a ticket on Trac for that once we get Tor Browser for Android into stable shape and the dust settles a bit.
Georg
Kind Regards, Torsten
[1] https://code.briarproject.org/briar/go-reproducer#go-reproducer
tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On 2/27/19 4:54 AM, Georg Koppen wrote:
Torsten Grote:
On 2/26/19 11:19 AM, Georg Koppen wrote:
I think we should be able to provide that with our Tor Browser builds once we have all the PT pieces sorted out (which is rather soon).
That would be nice!
So, probably the easiest way would then be to just copy the respective binaries we produce over to include them in the Briar software
I assume these binaries are reproducible?
They would be used for Tor Browser which has reproducibility as a hard requirement, so yes.
We are currently not including these binaries directly, but publish them as a library (gradle/maven to jcenter) where Briar and other projects can get them from.
Okay, good to know.
It seems to me that Tor Browser should instead perhaps rely on the Briar build process, at least for Android. I will be switching Orbot and our AndroidPT library over to the Briar dependency in the next release.
We should also figure out who is doing the source builds, how these are published as public modules, and who is monitoring and verifying the reproducibility. This is for both PT's and the current tor-android binary project that I manage (https://github.com/n8fr8/tor-android). It would be great to have Tor to be the source for trustworthy binary builds, available through direct downloads and gradle/maven/cocoapods, etc. We have talked about this many times in the past.
Related to this, we are also building and publishing PT's as shared libraries, instead of binaries, which is eventually going to be required on Android for both tor and obfs4proxy. It isn't all quite working yet, but will be soon. At some point, we'll have to talk about that whole transition.
Our goptlib shared library build project is here: https://gitlab.com/eighthave/goptbundle which is being made available to app developers here: https://github.com/guardianproject/AndroidPluggableTransports
+n
Nathan Freitas:
On 2/27/19 4:54 AM, Georg Koppen wrote:
Torsten Grote:
On 2/26/19 11:19 AM, Georg Koppen wrote:
I think we should be able to provide that with our Tor Browser builds once we have all the PT pieces sorted out (which is rather soon).
That would be nice!
So, probably the easiest way would then be to just copy the respective binaries we produce over to include them in the Briar software
I assume these binaries are reproducible?
They would be used for Tor Browser which has reproducibility as a hard requirement, so yes.
We are currently not including these binaries directly, but publish them as a library (gradle/maven to jcenter) where Briar and other projects can get them from.
Okay, good to know.
It seems to me that Tor Browser should instead perhaps rely on the Briar build process, at least for Android. I will be switching Orbot and our AndroidPT library over to the Briar dependency in the next release.
I am not sure whether I understand your proposal. You mean we should just switch for the PTs to how Briar is doing things? Or for the whole Tor Browser for Android? (note the .apk is already being built reproducibly using the same framwork we use for desktop builds) The latter would essentially mean maintaining two different build setups which seems not like a thing we should pursue, at first glance at least.
We should also figure out who is doing the source builds, how these are published as public modules, and who is monitoring and verifying the reproducibility. This is for both PT's and the current tor-android binary project that I manage (https://github.com/n8fr8/tor-android). It would be great to have Tor to be the source for trustworthy binary builds, available through direct downloads and gradle/maven/cocoapods, etc. We have talked about this many times in the past.
Yeah, I think that's a good idea. Let's tackle this once we have Tor Browser for Android in stable shape and think about ways to improve both our build and maintenance processes.
Related to this, we are also building and publishing PT's as shared libraries, instead of binaries, which is eventually going to be required on Android for both tor and obfs4proxy. It isn't all quite working yet, but will be soon. At some point, we'll have to talk about that whole transition.
Where can we find out about the main pieces that are missing here?
Our goptlib shared library build project is here: https://gitlab.com/eighthave/goptbundle which is being made available to app developers here: https://github.com/guardianproject/AndroidPluggableTransports
Yes, we plan to use that one I think until there are hard blockers that we are not aware of yet. Shane has made a first attempt to integrate that into our build system in #28803.
Georg
-
Cecylia Bocovich -
Georg Koppen -
Karsten Loesing -
Nathan Freitas -
Torsten Grote