[tor-project] Sponsor 38 update (Shadow simulator)
jnewsome at torproject.org
Fri Jul 24 21:52:05 UTC 2020
We've moved our NSF sponsor page from Tor's now-deprecated Trac site
into our source repository on GitHub, next to our code.
(We also have a place-holder in Tor's new GitLab instance, but it just
directs readers to the GH page).
We've continued to make progress on the new process-based architecture
(aka Phantom) https://github.com/shadow/shadow/milestone/16. In particular:
* We've migrated this work from a private repository to a new |dev|
branch in the public Shadow repo:
* This branch can now run Shadow's |phold| simulator performance
benchmark, which we've been using to iterate on performance
measurements and improvements before continuing on to add additional
syscall support to run |tor|.
Preliminary results were as much as 10x slower than the
single-process architecture in a small-scale, ad-hoc test. These
results led us to think more carefully about our design and we have
begun to implement optimizations that we expect will help close this
We've modified the |ptrace| interposition mechanism to use
|PTRACE_SYSEMU| instead of |PTRACE_SYSCALL|
giving about a 20% speedup. Adding a caching layer to address
lookups also gave a substantial speedup. Together these got us
down to roughly 3.5x slower than the single-process approach.
We have work in progress to further improve performance by
sharing memory between Shadow and its plugins
(https://github.com/shadow/shadow/issues/888) and by using
spin-locks for faster control-transfers
Since the |phold| benchmark essentially does nothing but send and
receive UDP packets, it spends most of the benchmark crossing the
new process boundaries between the Shadow process and the plugins.
We think this is a worst case scenario in which to compare the new
architecture to the current single-process architecture. We've added
a tunable CPU-load parameter to |phold| to let us evaluate how this
gets amortized, and plan on doing analysis with it soon.
We've integrated Rust into our build system such that Shadow code can be
written in Rust, and Shadow's C and Rust code can call each-other. New
code is now being being written in Rust.
We've continued to improve Shadow's test suite and continuous integration.
* We've added CI for all of the Linux distros we intend to support.
* We've started collecting test coverage measurements
https://github.com/shadow/shadow/milestone/20, and generating
differential coverage reports on PRs.
* We've moved core CI logic into bash scripts, and added an
alternative driver script to run some or all of it in local Docker
containers (vs GitHub's) for debugging.
* We've made progress on more-thoroughly testing the libc socket API.
The Shadow team
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the tor-project