reproducible builds for Android tor daemon
Hey all, I'm currently working on tor for Android as part of a Guardian Project project. One key goal is making a shareable, reproducible build process for the tor daemon for Android. Then this would be published to MavenCentral as an Android AAR package to be used in all the apps that include tor (Tor Browser, Orbot, Briar, Thali, etc). I have cleaned up the existing build process a lot, so now I'm down to troubleshooting reproducible issues. First off, can anyone see any objections to switching Tor Browser, Orbot, Briar, etc. to use GPG-signed reproducible binaries via MavenCentral for the tor dameon? Second, I'd welcome pointers to the integration points in Tor Browser, Briar, etc. And third, and tips on getting a Linux shared library to build reproducibly. E.g. is faketime a hard requirement? And for those interested, the here's an overview of the whole project I'm tasked with: The basic idea is to make an Android-native TorService, which is a subclass of android.app.Service and loads tor daemon as a shared library, and starts it via JNI methods. This should be pretty close to how iOS apps use tor. Then that TorService should then plug into Orbot, Tor Browser, Briar, etc. Then based on that, I'll make a standalone, dead simple "TorServices" app that only provides Tor. No PTs, HSs, etc., just a tor daemon linked in as a shared library, providing a SOCKS and HTTP CONNECT proxy via tor. I hope to switch the ControlPort to use a UNIX domain socket. This then can also be embedded into Android OS ROMs that want to have Tor built-in as a system service, like CalyxOS. Then I'll be working on the apps choosing which tor provider to use, since we'll have a new TorServices, so apps that don't include Tor will have to figure out how to use Orbot and/or TorServices. Then that work will hopefully be extended into sharing tor between apps, e.g. letting Briar, Tor Browser, etc share the tor SOCKS proxy to other apps that want to use it. That would happen via Android mechanisms like Intents to manage the discovery of SOCKS ports. .hc -- PGP fingerprint: EE66 20C7 136B 0D2C 456C 0A4D E9E2 8DEA 00AA 5556 https://pgp.mit.edu/pks/lookup?op=vindex&search=0xE9E28DEA00AA5556
Hans-Christoph Steiner:
Hey all,
I'm currently working on tor for Android as part of a Guardian Project project. One key goal is making a shareable, reproducible build process for the tor daemon for Android. Then this would be published to MavenCentral as an Android AAR package to be used in all the apps that include tor (Tor Browser, Orbot, Briar, Thali, etc). I have cleaned up the existing build process a lot, so now I'm down to troubleshooting reproducible issues.
First off, can anyone see any objections to switching Tor Browser, Orbot, Briar, etc. to use GPG-signed reproducible binaries via MavenCentral for the tor dameon?
We want to include building tor and all its dependencies in tor-browser-build/rbm to have the latest tor for Android in our nightly builds and respective alpha and stable versions in our alpha and stable browsers. We have a ticket for that for a while now in our bug tracker but did not get to it so far.[1] The plan is to pick that work up in November after Tor Browser 9 is out. As to whether other projects would be interested in that, dunno. But I guess some at least would? Georg [1] The parent ticket for that work is: https://trac.torproject.org/projects/tor/ticket/28704.
Georg Koppen:
Hans-Christoph Steiner:
Hey all,
I'm currently working on tor for Android as part of a Guardian Project project. One key goal is making a shareable, reproducible build process for the tor daemon for Android. Then this would be published to MavenCentral as an Android AAR package to be used in all the apps that include tor (Tor Browser, Orbot, Briar, Thali, etc). I have cleaned up the existing build process a lot, so now I'm down to troubleshooting reproducible issues.
First off, can anyone see any objections to switching Tor Browser, Orbot, Briar, etc. to use GPG-signed reproducible binaries via MavenCentral for the tor dameon?
We want to include building tor and all its dependencies in tor-browser-build/rbm to have the latest tor for Android in our nightly builds and respective alpha and stable versions in our alpha and stable browsers. We have a ticket for that for a while now in our bug tracker but did not get to it so far.[1] The plan is to pick that work up in November after Tor Browser 9 is out.
As to whether other projects would be interested in that, dunno. But I guess some at least would?
Georg
[1] The parent ticket for that work is: https://trac.torproject.org/projects/tor/ticket/28704.
If building tor+libevent+openssl+libz+liblzma for Android was done reproducibly and shipped via MavenCentral, would you consider using it? Seems like we'd want this tor binary to be synced to the Tor Browser version requirements anyway, since that's the "standard configuration". .hc -- PGP fingerprint: EE66 20C7 136B 0D2C 456C 0A4D E9E2 8DEA 00AA 5556 https://pgp.mit.edu/pks/lookup?op=vindex&search=0xE9E28DEA00AA5556
Hans-Christoph Steiner:
Georg Koppen:
Hans-Christoph Steiner:
Hey all,
I'm currently working on tor for Android as part of a Guardian Project project. One key goal is making a shareable, reproducible build process for the tor daemon for Android. Then this would be published to MavenCentral as an Android AAR package to be used in all the apps that include tor (Tor Browser, Orbot, Briar, Thali, etc). I have cleaned up the existing build process a lot, so now I'm down to troubleshooting reproducible issues.
First off, can anyone see any objections to switching Tor Browser, Orbot, Briar, etc. to use GPG-signed reproducible binaries via MavenCentral for the tor dameon?
We want to include building tor and all its dependencies in tor-browser-build/rbm to have the latest tor for Android in our nightly builds and respective alpha and stable versions in our alpha and stable browsers. We have a ticket for that for a while now in our bug tracker but did not get to it so far.[1] The plan is to pick that work up in November after Tor Browser 9 is out.
As to whether other projects would be interested in that, dunno. But I guess some at least would?
Georg
[1] The parent ticket for that work is: https://trac.torproject.org/projects/tor/ticket/28704.
If building tor+libevent+openssl+libz+liblzma for Android was done reproducibly and shipped via MavenCentral, would you consider using it? Seems like we'd want this tor binary to be synced to the Tor Browser version requirements anyway, since that's the "standard configuration".
What about our nightly build requirement? Oh, and to complicate that: we build tor nightlies with Rust enabled to be able to test Rust code. And would do so for Android, too. And to further complicate matters: we plan to switch to NSS to test that part of tor in a Tor Browser context as well. (It's been long on the agenda but I finally want to get to that after Tor Browser 9 is out) And then there has been times where we actually needed to ship tor patches ourselves because they were not merged/released yet (although, luckily that's been a while ago). There might be need for such an option in the future, too. So, all in all I am skeptical that Tor Browser fits into your plans. Georg
Hi,
On 12 Sep 2019, at 20:50, Hans-Christoph Steiner <hans@guardianproject.info> wrote:
Then that work will hopefully be extended into sharing tor between apps, e.g. letting Briar, Tor Browser, etc share the tor SOCKS proxy to other apps that want to use it. That would happen via Android mechanisms like Intents to manage the discovery of SOCKS ports.
It's not always safe to have apps share Tor: a malicious website in one app can use various caches to discover activity in other apps. And there may be similar data leaks in other shared data structures or network connections. How do these data leaks affect your use cases? T
teor:
Hi,
On 12 Sep 2019, at 20:50, Hans-Christoph Steiner <hans@guardianproject.info> wrote:
Then that work will hopefully be extended into sharing tor between apps, e.g. letting Briar, Tor Browser, etc share the tor SOCKS proxy to other apps that want to use it. That would happen via Android mechanisms like Intents to manage the discovery of SOCKS ports.
It's not always safe to have apps share Tor: a malicious website in one app can use various caches to discover activity in other apps. And there may be similar data leaks in other shared data structures or network connections.
How do these data leaks affect your use cases?
Is there some documentation of these leaks somewhere so I can dive into it? Like what kind of caches? Browser caches? We're mostly talking about apps that are not browsers, like messaging, nextcloud, etc. Currently, running multiple tor daemons is really a showstopper for most mobile users in the world, both because of battery usage and bandwidth costs. I guess there was some progress towards getting tor sleeping more effectively as well as not consuming as much bandwidth in the background. So the big question is: will it be feasible to have a usable tor on mobile that aggressively sleeps to stop consuming any battery and bandwidth when not directly in use? If so, then running multiple tor daemons should be workable. If not, then we need to find a way to share the tor daemon across all apps in the device. With Orbot, all apps are already sharing one tor daemon, so this isn't a new development. .hc -- PGP fingerprint: EE66 20C7 136B 0D2C 456C 0A4D E9E2 8DEA 00AA 5556 https://pgp.mit.edu/pks/lookup?op=vindex&search=0xE9E28DEA00AA5556
On Fri, Sep 13, 2019 at 7:51 AM Hans-Christoph Steiner <hans@guardianproject.info> wrote:
teor:
Hi,
On 12 Sep 2019, at 20:50, Hans-Christoph Steiner <hans@guardianproject.info> wrote:
Then that work will hopefully be extended into sharing tor between apps, e.g. letting Briar, Tor Browser, etc share the tor SOCKS proxy to other apps that want to use it. That would happen via Android mechanisms like Intents to manage the discovery of SOCKS ports.
It's not always safe to have apps share Tor: a malicious website in one app can use various caches to discover activity in other apps. And there may be similar data leaks in other shared data structures or network connections.
How do these data leaks affect your use cases?
Is there some documentation of these leaks somewhere so I can dive into it? Like what kind of caches? Browser caches? We're mostly talking about apps that are not browsers, like messaging, nextcloud, etc.
I have a half-baked (and half-written) proposal for this. I'll try to finish it within the next few weeks. The summary of it is that in an ideal situation every application should run its own tor client. If two applications share a tor client, and they both use the control port, then they both learn information about connections requested by the other - maybe that is onion service addresses only one app should use, or it leaks user behavior. Even in the case where multiple apps don't have control port access, they can potentially use the same circuits if their proxy settings are misconfigured (which leaks timing information locally and possibly leaks information about applications and the user on the network). Instead of every tor client wasting mobile data with duplicate downloads, I'm imagining having a Tor gateway (for lack of a better word). This essentially is a local Tor bridge relay that provides a single point of entry into and exit from the device. This solves most of the problems and it can be setup today except this completely breaks the assumptions within Tor that a Bridge is an external first-hop. In a configuration like this, the Tor gateway is a local hop, so it reduces a three-hop circuit to two-hops. And, to make matters worse, every second-hop would continue to be selected randomly, so the connection into the Tor network would be a random node and not a Guard. And, third, a client loses the ability to configure an actual bridge (with or without a pluggable transport) for its connection into the network. With this being said, something like Bridge Guards [0] would solve these problems. Unfortunately, Bridge Guards are still an open question. I was hoping to look into this in the near future. This setup would probably require some synchronization between tor node if, for example, multiple applications should use the same onion service, but this can happen outside of tor. [0] https://gitweb.torproject.org/torspec.git/tree/proposals/188-bridge-guards.t...
On 9/13/19 3:51 AM, Hans-Christoph Steiner wrote:
teor:
It's not always safe to have apps share Tor: a malicious website in one app can use various caches to discover activity in other apps. And there may be similar data leaks in other shared data structures or network connections.
How do these data leaks affect your use cases?
With Orbot, all apps are already sharing one tor daemon, so this isn't a new development.
.hc
Most the use cases for Tor outside of Tor Browser and Briar tend to be related to anti-censorship, reduction of passive surveillance, and opportunistic access to onions (nytimes, DDG, facebook, etc). Also has hc said, we are talking about non-browser type applications. Since these are also applications you already have installed on your phone, they already can know a heckuva a lot about you and your device. Thus, with the threat model scope for this work, the app itself is not our adversary, just the network. +n
On Thu, 12 Sep 2019, Hans-Christoph Steiner wrote:
And third, and tips on getting a Linux shared library to build reproducibly. E.g. is faketime a hard requirement?
Usually it's not needed to use faketime. It's only useful if the toolchain has bugs that cannot easily be fixed, causing some timestamps to be inserted somewhere.
On Fri, Sep 13, 2019 at 12:32:06PM +0200, Nicolas Vigier wrote:
On Thu, 12 Sep 2019, Hans-Christoph Steiner wrote:
And third, and tips on getting a Linux shared library to build reproducibly. E.g. is faketime a hard requirement?
Usually it's not needed to use faketime. It's only useful if the toolchain has bugs that cannot easily be fixed, causing some timestamps to be inserted somewhere.
+1, using faketime so liberally would just mean there are other, bigger underlying issues (e.g., lack of support for S_D_E...) Cheers! -Santiago.
Hi all, Just saw this thread while heading out the door and wanted to mention that we already have a reproducible build setup for Tor and obfs4proxy binaries for Android and Linux. The binaries are published on JCenter. Hans-Christoph, hope this shortens your path! :-) https://code.briarproject.org/briar/tor-reproducer https://code.briarproject.org/briar/go-reproducer https://bintray.com/briarproject/org.briarproject Cheers, Michael On 13/09/2019 16:24, Santiago Torres-Arias wrote:
On Fri, Sep 13, 2019 at 12:32:06PM +0200, Nicolas Vigier wrote:
On Thu, 12 Sep 2019, Hans-Christoph Steiner wrote:
And third, and tips on getting a Linux shared library to build reproducibly. E.g. is faketime a hard requirement?
Usually it's not needed to use faketime. It's only useful if the toolchain has bugs that cannot easily be fixed, causing some timestamps to be inserted somewhere.
+1, using faketime so liberally would just mean there are other, bigger underlying issues (e.g., lack of support for S_D_E...)
Cheers! -Santiago.
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On Thu, Sep 12, 2019 at 10:50 AM Hans-Christoph Steiner <hans@guardianproject.info> wrote:
Hey all,
I'm currently working on tor for Android as part of a Guardian Project project. One key goal is making a shareable, reproducible build process for the tor daemon for Android. Then this would be published to MavenCentral as an Android AAR package to be used in all the apps that include tor (Tor Browser, Orbot, Briar, Thali, etc). I have cleaned up the existing build process a lot, so now I'm down to troubleshooting reproducible issues.
So we're clear on the expectations and goals of this, you're working on compiling Tor reproducibly without using rbm, correct? And are you imagining the Guardian Project would build and upload these packages to MavenCentral, as part of your work? I wonder if extracting tor binaries and shared libraries from tor-browser-build (or another project on top of rbm) is an easier route, in the longer term. I don't know what issue you'll find when you begin diving into the differences between builds (if there are any, at all). Overall, I think this is a great plan and I'd like to see more applications bundled with Tor (and CalyxOS is exciting). However, as Georg already mentioned, this may not be something that fits into Tor Browser in the future.
participants (8)
-
Georg Koppen -
Hans-Christoph Steiner -
Matthew Finkel -
Michael Rogers -
Nathan Freitas -
Nicolas Vigier -
Santiago Torres-Arias -
teor