Tor in a safer language: Network team update from Amsterdam
Hi there tor-dev, as an update to those who didn't have the chance to meet with us in Amsterdam or those who haven't followed the efforts to rely on C less, here's what happened at the "let's not fight about Go versus Rust, but talk about how to migrate Tor to a safer language" session and what happened after. Notes from session: We didn't fight about Rust or Go or modern C++. Instead, we focused on identifying goals for migrating Tor to a memory-safe language, and how to get there. With that frame of reference, Rust emerged as a extremely strong candidate for the incremental improvement style that we considered necessary. We were strongly advised to not use cgo, by people who have used it extensively. As there are clearly a lot of unknowns with this endeavor, and a lot that we will learn/come up against along the way, we feel that Rust is a compelling option to start with, with the caveat that we will first experiment, learn from the experience, and then build on what we learn. You can also check out the session notes on the wiki (submitted, but not posted yet).[1] The real fun part started after the session. We got together to actually make a plan for an experiment and to give Rust a serious chance. We quickly got a few trivial things working like statically linking Rust into Tor, integrating with the build system to call out to cargo for the Rust build, and using Tor's allocator from Rust. We're planning to write up a blog post summarizing our experiences so far while hopefully poking the Rust developers to prioritize the missing features so we can stop using nightly Rust soon (~months, instead of years). We want to have a patch merged into tor soon so you can all play with your dev setup to help identify any challenges. We want to stress that this is an optional experiment for now, we would love feedback but nobody is paid to work on this and nobody is expected to spend more time than they have sitting around. We have committed to reviewing any patch that includes any Rust code to provide feedback, get experience to develop a style, and actually make use of this experiment. This means we're not ready to take on big patches that add lots of tricky stuff quite now, we want to take it slow and learn from this. We would like to do a session at the next dev meeting to give updates on this effort, but in the meantime, if team members would like to start learning Rust and helping us identify/implement small and well-isolated areas to begin migration, or new pieces of functionality that we can build immediately in Rust, that would be really great. So, for a TLDR: What has already been done: - Rust in Tor build - Putting together environment setup instructions and a (very small) initial draft for coding standards - Initial work to identify good candidates for migration (not tightly interdependent) What we think are next steps: - Define conventions for the API boundary between Rust and C - Add a non-trivial Rust API and deploy with a flag to optionally use (to test support with a safe fallback) - Learn from similar projects - Add automated tooling for Rust, such as linting and testing Cheers Alex, Chelsea, Sebastian [1]: Will be visible here https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam/Not...
Rust seems like the best available choice for Tor in a safer language. Rust has several issues with securely obtaining a Rust toolchain that the Tor community should be attentive to. Rust is a self hosted compiler. Building Rust requires obtaining binaries for a recent Rust compiler. The Rust toolchain is vulnerable to a "trusting trust" attack. Manish made a prototype and discussed future mitigations.[0] The Rust toolchain is built by an automated continuous integration system and distributed without human verification or intervention. Rust's build artifacts distributed by the RustUp tool are only authenticated by TLS certificates. RustUp Github issue 241 discusses a mitigation to address some of these concerns but development seems to be stalled.[1] [0] https://manishearth.github.io/blog/2016/12/02/reflections-on-rusting-trust/ [1] https://github.com/rust-lang-nursery/rustup.rs/issues/241 On Fri, Mar 31, 2017 at 2:23 PM Sebastian Hahn <sebastian@torproject.org> wrote:
Hi there tor-dev,
as an update to those who didn't have the chance to meet with us in Amsterdam or those who haven't followed the efforts to rely on C less, here's what happened at the "let's not fight about Go versus Rust, but talk about how to migrate Tor to a safer language" session and what happened after.
Notes from session:
We didn't fight about Rust or Go or modern C++. Instead, we focused on identifying goals for migrating Tor to a memory-safe language, and how to get there. With that frame of reference, Rust emerged as a extremely strong candidate for the incremental improvement style that we considered necessary. We were strongly advised to not use cgo, by people who have used it extensively.
As there are clearly a lot of unknowns with this endeavor, and a lot that we will learn/come up against along the way, we feel that Rust is a compelling option to start with, with the caveat that we will first experiment, learn from the experience, and then build on what we learn.
You can also check out the session notes on the wiki (submitted, but not posted yet).[1]
The real fun part started after the session. We got together to actually make a plan for an experiment and to give Rust a serious chance. We quickly got a few trivial things working like statically linking Rust into Tor, integrating with the build system to call out to cargo for the Rust build, and using Tor's allocator from Rust.
We're planning to write up a blog post summarizing our experiences so far while hopefully poking the Rust developers to prioritize the missing features so we can stop using nightly Rust soon (~months, instead of years).
We want to have a patch merged into tor soon so you can all play with your dev setup to help identify any challenges. We want to stress that this is an optional experiment for now, we would love feedback but nobody is paid to work on this and nobody is expected to spend more time than they have sitting around.
We have committed to reviewing any patch that includes any Rust code to provide feedback, get experience to develop a style, and actually make use of this experiment. This means we're not ready to take on big patches that add lots of tricky stuff quite now, we want to take it slow and learn from this.
We would like to do a session at the next dev meeting to give updates on this effort, but in the meantime, if team members would like to start learning Rust and helping us identify/implement small and well-isolated areas to begin migration, or new pieces of functionality that we can build immediately in Rust, that would be really great.
So, for a TLDR:
What has already been done: - Rust in Tor build - Putting together environment setup instructions and a (very small) initial draft for coding standards - Initial work to identify good candidates for migration (not tightly interdependent)
What we think are next steps: - Define conventions for the API boundary between Rust and C - Add a non-trivial Rust API and deploy with a flag to optionally use (to test support with a safe fallback) - Learn from similar projects - Add automated tooling for Rust, such as linting and testing
Cheers Alex, Chelsea, Sebastian
[1]: Will be visible here https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam/Not... _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Howdy, Thanks for making these points. They made me realize that this initiative: http://bootstrappable.org/ Might not be on folks' radars. It was an outcome of the last Reproducible Builds Summit in December [1]. It just does a good job of articulating problems and proposed some possible collaborative steps forward. peace, gunner [1] https://reproducible-builds.org/events/ On 04/01/2017 06:54 PM, zaki@manian.org wrote:
Rust seems like the best available choice for Tor in a safer language.
Rust has several issues with securely obtaining a Rust toolchain that the Tor community should be attentive to.
Rust is a self hosted compiler. Building Rust requires obtaining binaries for a recent Rust compiler. The Rust toolchain is vulnerable to a "trusting trust" attack. Manish made a prototype and discussed future mitigations.[0]
The Rust toolchain is built by an automated continuous integration system and distributed without human verification or intervention. Rust's build artifacts distributed by the RustUp tool are only authenticated by TLS certificates. RustUp Github issue 241 discusses a mitigation to address some of these concerns but development seems to be stalled.[1]
[0] https://manishearth.github.io/blog/2016/12/02/reflections-on-rusting-trust/ [1] https://github.com/rust-lang-nursery/rustup.rs/issues/241
On Fri, Mar 31, 2017 at 2:23 PM Sebastian Hahn <sebastian@torproject.org <mailto:sebastian@torproject.org>> wrote:
Hi there tor-dev,
as an update to those who didn't have the chance to meet with us in Amsterdam or those who haven't followed the efforts to rely on C less, here's what happened at the "let's not fight about Go versus Rust, but talk about how to migrate Tor to a safer language" session and what happened after.
Notes from session:
We didn't fight about Rust or Go or modern C++. Instead, we focused on identifying goals for migrating Tor to a memory-safe language, and how to get there. With that frame of reference, Rust emerged as a extremely strong candidate for the incremental improvement style that we considered necessary. We were strongly advised to not use cgo, by people who have used it extensively.
As there are clearly a lot of unknowns with this endeavor, and a lot that we will learn/come up against along the way, we feel that Rust is a compelling option to start with, with the caveat that we will first experiment, learn from the experience, and then build on what we learn.
You can also check out the session notes on the wiki (submitted, but not posted yet).[1]
The real fun part started after the session. We got together to actually make a plan for an experiment and to give Rust a serious chance. We quickly got a few trivial things working like statically linking Rust into Tor, integrating with the build system to call out to cargo for the Rust build, and using Tor's allocator from Rust.
We're planning to write up a blog post summarizing our experiences so far while hopefully poking the Rust developers to prioritize the missing features so we can stop using nightly Rust soon (~months, instead of years).
We want to have a patch merged into tor soon so you can all play with your dev setup to help identify any challenges. We want to stress that this is an optional experiment for now, we would love feedback but nobody is paid to work on this and nobody is expected to spend more time than they have sitting around.
We have committed to reviewing any patch that includes any Rust code to provide feedback, get experience to develop a style, and actually make use of this experiment. This means we're not ready to take on big patches that add lots of tricky stuff quite now, we want to take it slow and learn from this.
We would like to do a session at the next dev meeting to give updates on this effort, but in the meantime, if team members would like to start learning Rust and helping us identify/implement small and well-isolated areas to begin migration, or new pieces of functionality that we can build immediately in Rust, that would be really great.
So, for a TLDR:
What has already been done: - Rust in Tor build - Putting together environment setup instructions and a (very small) initial draft for coding standards - Initial work to identify good candidates for migration (not tightly interdependent)
What we think are next steps: - Define conventions for the API boundary between Rust and C - Add a non-trivial Rust API and deploy with a flag to optionally use (to test support with a safe fallback) - Learn from similar projects - Add automated tooling for Rust, such as linting and testing
Cheers Alex, Chelsea, Sebastian
[1]: Will be visible here https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam/Not... _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org <mailto:tor-dev@lists.torproject.org> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
-- Allen Gunn Executive Director, Aspiration +1.415.216.7252 www.aspirationtech.org Aspiration: "Better Tools for a Better World" Read our Manifesto: http://aspirationtech.org/publications/manifesto Twitter: www.twitter.com/aspirationtech --
zaki@manian.org transcribed 12K bytes:
Rust seems like the best available choice for Tor in a safer language.
Rust has several issues with securely obtaining a Rust toolchain that the Tor community should be attentive to.
Interesting development, but logical. Leaving the obvious issues (bootstrap, etc) aside: Will you stick to stable features? From a package maintainers position it is generally unacceptable (and hard) to follow (and maintain) nightly/unstable releases of a programming language. Rust stable has proven features which are expected to stick around for a reliable long time (at least that is my understanding).
Rust is a self hosted compiler. Building Rust requires obtaining binaries for a recent Rust compiler. The Rust toolchain is vulnerable to a "trusting trust" attack. Manish made a prototype and discussed future mitigations.[0]
The Rust toolchain is built by an automated continuous integration system and distributed without human verification or intervention. Rust's build artifacts distributed by the RustUp tool are only authenticated by TLS certificates. RustUp Github issue 241 discusses a mitigation to address some of these concerns but development seems to be stalled.[1]
[0] https://manishearth.github.io/blog/2016/12/02/reflections-on-rusting-trust/ [1] https://github.com/rust-lang-nursery/rustup.rs/issues/241
On Fri, Mar 31, 2017 at 2:23 PM Sebastian Hahn <sebastian@torproject.org> wrote:
Hi there tor-dev,
as an update to those who didn't have the chance to meet with us in Amsterdam or those who haven't followed the efforts to rely on C less, here's what happened at the "let's not fight about Go versus Rust, but talk about how to migrate Tor to a safer language" session and what happened after.
Notes from session:
We didn't fight about Rust or Go or modern C++. Instead, we focused on identifying goals for migrating Tor to a memory-safe language, and how to get there. With that frame of reference, Rust emerged as a extremely strong candidate for the incremental improvement style that we considered necessary. We were strongly advised to not use cgo, by people who have used it extensively.
As there are clearly a lot of unknowns with this endeavor, and a lot that we will learn/come up against along the way, we feel that Rust is a compelling option to start with, with the caveat that we will first experiment, learn from the experience, and then build on what we learn.
You can also check out the session notes on the wiki (submitted, but not posted yet).[1]
The real fun part started after the session. We got together to actually make a plan for an experiment and to give Rust a serious chance. We quickly got a few trivial things working like statically linking Rust into Tor, integrating with the build system to call out to cargo for the Rust build, and using Tor's allocator from Rust.
We're planning to write up a blog post summarizing our experiences so far while hopefully poking the Rust developers to prioritize the missing features so we can stop using nightly Rust soon (~months, instead of years).
We want to have a patch merged into tor soon so you can all play with your dev setup to help identify any challenges. We want to stress that this is an optional experiment for now, we would love feedback but nobody is paid to work on this and nobody is expected to spend more time than they have sitting around.
We have committed to reviewing any patch that includes any Rust code to provide feedback, get experience to develop a style, and actually make use of this experiment. This means we're not ready to take on big patches that add lots of tricky stuff quite now, we want to take it slow and learn from this.
We would like to do a session at the next dev meeting to give updates on this effort, but in the meantime, if team members would like to start learning Rust and helping us identify/implement small and well-isolated areas to begin migration, or new pieces of functionality that we can build immediately in Rust, that would be really great.
So, for a TLDR:
What has already been done: - Rust in Tor build - Putting together environment setup instructions and a (very small) initial draft for coding standards - Initial work to identify good candidates for migration (not tightly interdependent)
What we think are next steps: - Define conventions for the API boundary between Rust and C - Add a non-trivial Rust API and deploy with a flag to optionally use (to test support with a safe fallback) - Learn from similar projects - Add automated tooling for Rust, such as linting and testing
Cheers Alex, Chelsea, Sebastian
[1]: Will be visible here https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam/Not... _______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
participants (4)
-
Allen Gunn -
ng0 -
Sebastian Hahn -
zaki＠manian.org