Sponsor 38: https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md

Previous updates:

* 2020-02-10 https://lists.torproject.org/pipermail/tor-project/2020-February/002718.html * 2020-03-19 https://lists.torproject.org/pipermail/tor-project/2020-March/002785.html * 2020-06-01 https://lists.torproject.org/pipermail/tor-project/2020-June/002859.html

We've moved our NSF sponsor page from Tor's now-deprecated Trac site into our source repository on GitHub, next to our code. https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md. (We also have a place-holder in Tor's new GitLab instance, but it just directs readers to the GH page).

We've continued to make progress on the new process-based architecture (aka Phantom) https://github.com/shadow/shadow/milestone/16. In particular:

* We've migrated this work from a private repository to a new |dev| branch in the public Shadow repo: https://github.com/shadow/shadow/tree/dev. * This branch can now run Shadow's |phold| simulator performance benchmark, which we've been using to iterate on performance measurements and improvements before continuing on to add additional syscall support to run |tor|. https://github.com/shadow/shadow/issues/825 *

Preliminary results were as much as 10x slower than the single-process architecture in a small-scale, ad-hoc test. These results led us to think more carefully about our design and we have begun to implement optimizations that we expect will help close this performance gap.

o

We've modified the |ptrace| interposition mechanism to use |PTRACE_SYSEMU| instead of |PTRACE_SYSCALL| (https://github.com/shadow/shadow/commit/bdf9fe529ed8090bb07aa5f2683b21bc5501...), giving about a 20% speedup. Adding a caching layer to address lookups also gave a substantial speedup. Together these got us down to roughly 3.5x slower than the single-process approach.

o

We have work in progress to further improve performance by sharing memory between Shadow and its plugins (https://github.com/shadow/shadow/issues/888) and by using spin-locks for faster control-transfers (https://github.com/shadow/shadow/issues/894).

*

Since the |phold| benchmark essentially does nothing but send and receive UDP packets, it spends most of the benchmark crossing the new process boundaries between the Shadow process and the plugins. We think this is a worst case scenario in which to compare the new architecture to the current single-process architecture. We've added a tunable CPU-load parameter to |phold| to let us evaluate how this gets amortized, and plan on doing analysis with it soon. https://github.com/shadow/shadow/pull/897

We've integrated Rust into our build system such that Shadow code can be written in Rust, and Shadow's C and Rust code can call each-other. New code is now being being written in Rust. https://github.com/shadow/shadow/milestone/17

We've continued to improve Shadow's test suite and continuous integration.

* We've added CI for all of the Linux distros we intend to support. https://github.com/shadow/shadow/milestone/19 * We've started collecting test coverage measurements https://github.com/shadow/shadow/milestone/20, and generating differential coverage reports on PRs. https://codecov.io/gh/shadow/shadow/branch/dev * We've moved core CI logic into bash scripts, and added an alternative driver script to run some or all of it in local Docker containers (vs GitHub's) for debugging. https://github.com/shadow/shadow/pull/875 * We've made progress on more-thoroughly testing the libc socket API. https://github.com/shadow/shadow/milestone/21

The Shadow team (https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md#people)