Hello,
I was at the reproducible builds summit last week. During the meeting there were discussions about various topics related to reproducible builds: - reproducible builds for various languages, filesystems and distributions - bootstrapping - rebuilders and user verification - software transparency
I think the two most interesting discussions for me were about rebuilders and user verification, and software transparency.
= Rebuilders and user verification =
Some people from New York University have been working on a framework called in-toto which can be used to verify before installing a build that it has been reproduced by a number of trusted builders: https://in-toto.io/
They have implemented some prototype for Debian.
Here is how it works: - some independent organisations are running "rebuilders". Those rebuilders fetch new Debian source packages and rebuild them. When the build is finished they publish some json files containing informations about the source package, and the result of their build, and their signature. - apt is modified (using an apt-transport) so that before installing a new package, it connects to known rebuilders and fetch their build informations. If the build has been reproduced by enough rebuilders, then the package is installed.
I think it might be possible to implement something similar in Tor Browser, maybe reusing some parts of their framework.
= Software Distribution Transparency and Auditability =
Benjamin Hof is working on tools for Software Distribution Transparency and Auditability: https://arxiv.org/abs/1711.07278 https://debconf18.debconf.org/talks/104-software-transparency-package-securi...
He has implemented some prototypes to improve software distribution transparency in Debian, using a public log server which can be used to monitor the packages which have been distributed to users. I discussed with him how something similar could be done in Tor Browser.
Here is how it could work: - we add a log server which can be used to cryptographically prove: * inclusion of an update in the log * append-only operation of the log - when we release a new Tor Browser version, we add checksums of the new mar files to the log. - when the Tor Browser updater receives a new update, it checks if the update is in the log, before applying it. If the update is not in the log, then the update is not applied. - independant monitors analyse the logs. They check that all checksums published in the log correspond to a known release. They can also check that the build corresponding to those checksums is reproducible. The log is public, so anybody can monitor it.
The main improvement from this system is that it allows anyone to check that all the updates used by all users are reproductible builds, while today you can only verify that for one downloaded copy of the software.
Nicolas
On Mon, Dec 17, 2018 at 06:25:48PM +0100, Nicolas Vigier wrote:
Hello,
I was at the reproducible builds summit last week. During the meeting there were discussions about various topics related to reproducible builds:
- reproducible builds for various languages, filesystems and distributions
- bootstrapping
- rebuilders and user verification
- software transparency
I think the two most interesting discussions for me were about rebuilders and user verification, and software transparency.
Interesting, thanks!
= Software Distribution Transparency and Auditability =
Benjamin Hof is working on tools for Software Distribution Transparency and Auditability: https://arxiv.org/abs/1711.07278 https://debconf18.debconf.org/talks/104-software-transparency-package-securi...
He has implemented some prototypes to improve software distribution transparency in Debian, using a public log server which can be used to monitor the packages which have been distributed to users. I discussed with him how something similar could be done in Tor Browser.
Here is how it could work:
- we add a log server which can be used to cryptographically prove:
- inclusion of an update in the log
- append-only operation of the log
- when we release a new Tor Browser version, we add checksums of the new mar files to the log.
- when the Tor Browser updater receives a new update, it checks if the update is in the log, before applying it. If the update is not in the log, then the update is not applied.
- independant monitors analyse the logs. They check that all checksums published in the log correspond to a known release. They can also check that the build corresponding to those checksums is reproducible. The log is public, so anybody can monitor it.
The main improvement from this system is that it allows anyone to check that all the updates used by all users are reproductible builds, while today you can only verify that for one downloaded copy of the software.
This seems like it has the same goal as Mozilla's Binary Transparency log: https://wiki.mozilla.org/Security/Binary_Transparency
I'm not sure if Mozilla are actively working on this, but based on the two bug on that page, it seems like it isn't a priority right now: https://bugzilla.mozilla.org/show_bug.cgi?id=1341397 https://bugzilla.mozilla.org/show_bug.cgi?id=1341398
On Mon, 17 Dec 2018 at 18:49, Matthew Finkel matthew.finkel@gmail.com wrote:
This seems like it has the same goal as Mozilla's Binary Transparency log: https://wiki.mozilla.org/Security/Binary_Transparency
I'm not sure if Mozilla are actively working on this, but based on the two bug on that page, it seems like it isn't a priority right now: https://bugzilla.mozilla.org/show_bug.cgi?id=1341397 https://bugzilla.mozilla.org/show_bug.cgi?id=1341398
I'm playing PM for this, even though I'm not a PM. That's one of the reasons it is slow =)
BT is mostly lacking a top-down call to arms to have the disparate teams prioritize it. We've actually made good progress in the quarter, more than we have in the 3 quarters before that. We are lacking some crucial integration steps in our release pipeline though. I'm hoping we will be deploying it in none-enforcement-telemetry-make-sure-everything-is-okay mode next year.
I will note that while the goals are similar, at Mozilla we are hacking on top of the CT infrastructure rather than doing this cleanly because there isn't a production-grade BT infrastructure to work with yet.
-tom
On Mon, Dec 17, 2018 at 06:25:48PM +0100, Nicolas Vigier wrote:
= Rebuilders and user verification =
Some people from New York University have been working on a framework called in-toto which can be used to verify before installing a build that it has been reproduced by a number of trusted builders: https://in-toto.io/
Btw, I know these NYU folks, and starting in January I'll be in the same building as them sometimes. And I know they generally want to do things that make a difference in the world. So if there is something concrete they could do that would help Tor Browser make use of their work here, be sure to let them know. :)
--Roger
Nicolas Vigier:
Hello,
I was at the reproducible builds summit last week. During the meeting there were discussions about various topics related to reproducible builds:
- reproducible builds for various languages, filesystems and distributions
- bootstrapping
- rebuilders and user verification
- software transparency
I think the two most interesting discussions for me were about rebuilders and user verification, and software transparency.
= Rebuilders and user verification =
Some people from New York University have been working on a framework called in-toto which can be used to verify before installing a build that it has been reproduced by a number of trusted builders: https://in-toto.io/
They have implemented some prototype for Debian.
Here is how it works:
- some independent organisations are running "rebuilders". Those rebuilders fetch new Debian source packages and rebuild them. When the build is finished they publish some json files containing informations about the source package, and the result of their build, and their signature.
- apt is modified (using an apt-transport) so that before installing a new package, it connects to known rebuilders and fetch their build informations. If the build has been reproduced by enough rebuilders, then the package is installed.
I think it might be possible to implement something similar in Tor Browser, maybe reusing some parts of their framework.
How does that deal with content signing? Because if you look e.g. at the .dmg files we produce there is currently no direct link between the build X rebuilders reproduced and the bundle that is getting finally shipped. The same holds for Windows although dealing with the Authenticode signature is easier here.
I had the same doubts when reading, e.g. the CHAINIAC paper about proactive software-update transparency (with verified builds)[1], but did not have time to follow-up with its authors to check whether they got that part actually solved.
= Software Distribution Transparency and Auditability =
Benjamin Hof is working on tools for Software Distribution Transparency and Auditability: https://arxiv.org/abs/1711.07278 https://debconf18.debconf.org/talks/104-software-transparency-package-securi...
He has implemented some prototypes to improve software distribution transparency in Debian, using a public log server which can be used to monitor the packages which have been distributed to users. I discussed with him how something similar could be done in Tor Browser.
Here is how it could work:
- we add a log server which can be used to cryptographically prove:
- inclusion of an update in the log
- append-only operation of the log
- when we release a new Tor Browser version, we add checksums of the new mar files to the log.
- when the Tor Browser updater receives a new update, it checks if the update is in the log, before applying it. If the update is not in the log, then the update is not applied.
Well, the risk here is that this adds another necessary part for getting updates to users to the whole scheme, which in turn could get e.g. DDoSed during critical update times.
- independant monitors analyse the logs. They check that all checksums published in the log correspond to a known release. They can also check that the build corresponding to those checksums is reproducible. The log is public, so anybody can monitor it.
The main improvement from this system is that it allows anyone to check that all the updates used by all users are reproductible builds, while today you can only verify that for one downloaded copy of the software.
What do you mean by the latter? The .mar file you personally downloaded during an update?
It might be worth looking closer at those two ideas and compare them to proposals like CHAINIAC to get a better sense of where we should head to (I skimmed Benjamin's paper a bit and see they have a detailed comparison to CHAINIAC which could be a good starting point).
Georg
[1] https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-nikiti...
On Tue, Dec 18, 2018 at 07:32:00PM +0000, Georg Koppen wrote:
*SNIP* They have implemented some prototype for Debian.
Here is how it works:
- some independent organisations are running "rebuilders". Those rebuilders fetch new Debian source packages and rebuild them. When the build is finished they publish some json files containing informations about the source package, and the result of their build, and their signature.
- apt is modified (using an apt-transport) so that before installing a new package, it connects to known rebuilders and fetch their build informations. If the build has been reproduced by enough rebuilders, then the package is installed.
I think it might be possible to implement something similar in Tor Browser, maybe reusing some parts of their framework.
How does that deal with content signing? Because if you look e.g. at the .dmg files we produce there is currently no direct link between the build X rebuilders reproduced and the bundle that is getting finally shipped. The same holds for Windows although dealing with the Authenticode signature is easier here.
Hi, full disclosure, I'm part of the in-toto project linked above, and I'm also part of the people that are pushing for debian rebuilders. I'll try to be brief about how it works:
- A layout file is shipped that contains the keys of trusted rebuilder entities. This layout file is signed by a debian developer although specific namespacing can be handled on a per-application basis. - All the rebuilders create a signed json file that attests for the sources they used and the resulting hash for each artifact (e.g., the dmg you pointed out). A threshold of these files can be used to verify that the sources were adequately reproduced by a threshold number of trusted entities. There's work to be done in the sake of hardening, adaptative thresholds and so on, but this is a strong enough scaffolding to ensure compliance and security on the r-b side. - The apt-transport basically contains a map of the rebuilders and a signed layout file (the layout should be signed with people from the debian org as pointed above). When fetching a package the attestations from the rebuilders (we call these link metadata on in-toto lingo) are fetched as well and verification is run. You can think of this transport as https+threshold verification on reproducible builds (in-toto is way more powerful than this, but this is a good first step).
I had the same doubts when reading, e.g. the CHAINIAC paper about proactive software-update transparency (with verified builds)[1], but did not have time to follow-up with its authors to check whether they got that part actually solved.
Don't worry about it, CHANIAC is basically the brain-child of the in-toto team, granted it has some more ideas sprinkled on top, but the core idea stems from the same requirement of end-to-end software supply chain verification.
Let me know if this answers your questions :) -Santiago.
On Tue, 18 Dec 2018, Georg Koppen wrote:
Nicolas Vigier:
Hello,
I was at the reproducible builds summit last week. During the meeting there were discussions about various topics related to reproducible builds:
- reproducible builds for various languages, filesystems and distributions
- bootstrapping
- rebuilders and user verification
- software transparency
I think the two most interesting discussions for me were about rebuilders and user verification, and software transparency.
= Rebuilders and user verification =
Some people from New York University have been working on a framework called in-toto which can be used to verify before installing a build that it has been reproduced by a number of trusted builders: https://in-toto.io/
They have implemented some prototype for Debian.
Here is how it works:
- some independent organisations are running "rebuilders". Those rebuilders fetch new Debian source packages and rebuild them. When the build is finished they publish some json files containing informations about the source package, and the result of their build, and their signature.
- apt is modified (using an apt-transport) so that before installing a new package, it connects to known rebuilders and fetch their build informations. If the build has been reproduced by enough rebuilders, then the package is installed.
I think it might be possible to implement something similar in Tor Browser, maybe reusing some parts of their framework.
How does that deal with content signing? Because if you look e.g. at the .dmg files we produce there is currently no direct link between the build X rebuilders reproduced and the bundle that is getting finally shipped. The same holds for Windows although dealing with the Authenticode signature is easier here.
I think that in-toto has support for multiple steps between source and final package. So the main step would be the build, and I think we could add an other step to make the link between the signed and unsigned package. This will require having a tool that can deterministically remove a signature from a signed .dmg file, or add one to an unsigned .dmg file.
So the rebuilders would run both the [sources <===> unsigned package] and [unsigned package <===> signed package] steps, and clients would require both steps to be signed by enough rebuilders.
I had the same doubts when reading, e.g. the CHAINIAC paper about proactive software-update transparency (with verified builds)[1], but did not have time to follow-up with its authors to check whether they got that part actually solved.
= Software Distribution Transparency and Auditability =
Benjamin Hof is working on tools for Software Distribution Transparency and Auditability: https://arxiv.org/abs/1711.07278 https://debconf18.debconf.org/talks/104-software-transparency-package-securi...
He has implemented some prototypes to improve software distribution transparency in Debian, using a public log server which can be used to monitor the packages which have been distributed to users. I discussed with him how something similar could be done in Tor Browser.
Here is how it could work:
- we add a log server which can be used to cryptographically prove:
- inclusion of an update in the log
- append-only operation of the log
- when we release a new Tor Browser version, we add checksums of the new mar files to the log.
- when the Tor Browser updater receives a new update, it checks if the update is in the log, before applying it. If the update is not in the log, then the update is not applied.
Well, the risk here is that this adds another necessary part for getting updates to users to the whole scheme, which in turn could get e.g. DDoSed during critical update times.
I think the log can be provided as static files. So maybe that could be hosted on cdn.tpo with the .mar files.
- independant monitors analyse the logs. They check that all checksums published in the log correspond to a known release. They can also check that the build corresponding to those checksums is reproducible. The log is public, so anybody can monitor it.
The main improvement from this system is that it allows anyone to check that all the updates used by all users are reproductible builds, while today you can only verify that for one downloaded copy of the software.
What do you mean by the latter? The .mar file you personally downloaded during an update?
Yes, currently you can verify the .mar file that the update server provides you, but you cannot know if it provides the same .mar file to everybody at all times.
It might be worth looking closer at those two ideas and compare them to proposals like CHAINIAC to get a better sense of where we should head to (I skimmed Benjamin's paper a bit and see they have a detailed comparison to CHAINIAC which could be a good starting point).
Yes.
Nicolas