<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
Sponsor 38: <a
href="https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md"
class="uri">https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md</a>
<p>Previous updates:</p>
<ul>
<li>2020-02-10 <a
href="https://lists.torproject.org/pipermail/tor-project/2020-February/002718.html"
class="uri">https://lists.torproject.org/pipermail/tor-project/2020-February/002718.html</a></li>
<li>2020-03-19 <a
href="https://lists.torproject.org/pipermail/tor-project/2020-March/002785.html"
class="uri">https://lists.torproject.org/pipermail/tor-project/2020-March/002785.html</a></li>
<li>2020-06-01 <a
href="https://lists.torproject.org/pipermail/tor-project/2020-June/002859.html"
class="uri">https://lists.torproject.org/pipermail/tor-project/2020-June/002859.html</a></li>
</ul>
<p>We've moved our NSF sponsor page from Tor's now-deprecated Trac
site into our source repository on GitHub, next to our code. <a
href="https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md"
class="uri">https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md</a>.
(We also have a place-holder in Tor's new GitLab instance, but it
just directs readers to the GH page).</p>
<p>We've continued to make progress on the new process-based
architecture (aka Phantom) <a
href="https://github.com/shadow/shadow/milestone/16" class="uri">https://github.com/shadow/shadow/milestone/16</a>.
In particular:</p>
<ul>
<li>We've migrated this work from a private repository to a new <code>dev</code>
branch in the public Shadow repo: <a
href="https://github.com/shadow/shadow/tree/dev" class="uri">https://github.com/shadow/shadow/tree/dev</a>.</li>
<li>This branch can now run Shadow's <code>phold</code> simulator
performance benchmark, which we've been using to iterate on
performance measurements and improvements before continuing on
to add additional syscall support to run <code>tor</code>. <a
href="https://github.com/shadow/shadow/issues/825" class="uri">https://github.com/shadow/shadow/issues/825</a></li>
<li>
<p>Preliminary results were as much as 10x slower than the
single-process architecture in a small-scale, ad-hoc test.
These results led us to think more carefully about our design
and we have begun to implement optimizations that we expect
will help close this performance gap.</p>
<ul>
<li>
<p>We've modified the <code>ptrace</code> interposition
mechanism to use <code>PTRACE_SYSEMU</code> instead of <code>PTRACE_SYSCALL</code>
(<a
href="https://github.com/shadow/shadow/commit/bdf9fe529ed8090bb07aa5f2683b21bc55018a32"
class="uri">https://github.com/shadow/shadow/commit/bdf9fe529ed8090bb07aa5f2683b21bc55018a32</a>),
giving about a 20% speedup. Adding a caching layer to
address lookups also gave a substantial speedup. Together
these got us down to roughly 3.5x slower than the
single-process approach.</p>
</li>
<li>
<p>We have work in progress to further improve performance
by sharing memory between Shadow and its plugins (<a
href="https://github.com/shadow/shadow/issues/888"
class="uri">https://github.com/shadow/shadow/issues/888</a>)
and by using spin-locks for faster control-transfers (<a
href="https://github.com/shadow/shadow/issues/894"
class="uri">https://github.com/shadow/shadow/issues/894</a>).</p>
</li>
</ul>
</li>
<li>
<p>Since the <code>phold</code> benchmark essentially does
nothing but send and receive UDP packets, it spends most of
the benchmark crossing the new process boundaries between the
Shadow process and the plugins. We think this is a worst case
scenario in which to compare the new architecture to the
current single-process architecture. We've added a tunable
CPU-load parameter to <code>phold</code> to let us evaluate
how this gets amortized, and plan on doing analysis with it
soon. <a href="https://github.com/shadow/shadow/pull/897"
class="uri">https://github.com/shadow/shadow/pull/897</a></p>
</li>
</ul>
<p>We've integrated Rust into our build system such that Shadow code
can be written in Rust, and Shadow's C and Rust code can call
each-other. New code is now being being written in Rust. <a
href="https://github.com/shadow/shadow/milestone/17" class="uri">https://github.com/shadow/shadow/milestone/17</a></p>
<p>We've continued to improve Shadow's test suite and continuous
integration.</p>
<ul>
<li>We've added CI for all of the Linux distros we intend to
support. <a
href="https://github.com/shadow/shadow/milestone/19"
class="uri">https://github.com/shadow/shadow/milestone/19</a></li>
<li>We've started collecting test coverage measurements <a
href="https://github.com/shadow/shadow/milestone/20"
class="uri">https://github.com/shadow/shadow/milestone/20</a>,
and generating differential coverage reports on PRs. <a
href="https://codecov.io/gh/shadow/shadow/branch/dev"
class="uri">https://codecov.io/gh/shadow/shadow/branch/dev</a></li>
<li>We've moved core CI logic into bash scripts, and added an
alternative driver script to run some or all of it in local
Docker containers (vs GitHub's) for debugging. <a
href="https://github.com/shadow/shadow/pull/875" class="uri">https://github.com/shadow/shadow/pull/875</a></li>
<li>We've made progress on more-thoroughly testing the libc socket
API. <a href="https://github.com/shadow/shadow/milestone/21"
class="uri">https://github.com/shadow/shadow/milestone/21</a></li>
</ul>
<p>The Shadow team (<a
href="https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md#people"
class="uri">https://github.com/shadow/shadow/blob/master/docs/nsf-sponsorship.md#people</a>)</p>
</body>
</html>