Mirrored from https://github.com/shadow/shadow/discussions/2428 : This is part of a series of periodic updates of development in Shadow. This work is sponsored by the [NSF](https://shadow.github.io/docs/guide/nsf_sponsorship.html). Previous update: [2022-06](https://github.com/shadow/shadow/discussions/2243). We've merged [103 non-dependabot pull requests](https://github.com/shadow/shadow/pulls?q=is%3Apr+merged%3A2022-06-27..2022-0...) and closed [12 issues](https://github.com/shadow/shadow/issues?q=closed%3A2022-06-27..2022-09-28+is...) since our previous update. Release status ============== We've released [Shadow 2.2.0](https://github.com/shadow/shadow/releases/tag/v2.2.0)! Notable user-facing changes, from the release notes: * We have removed ptrace-mode, and the associated experimental options `use-o-n-waitpid-workaround` and `--interpose-method`. ptrace-mode was an alternative to Shadow's current interposition mechanism that uses `LD_PRELOAD` and `seccomp`. This change should be transparent to most users, since it hasn't been the default for several releases, and was only accessible via experimental options. See https://github.com/shadow/shadow/issues/1945 * `dup()` and related syscalls are now supported for all file descriptors * Fixed behavior when multiple threads are blocked in `epoll_wait` on the same epoll file description. https://github.com/shadow/shadow/issues/2260 * Fixed bugs causing `timerfd_settime` to not reset the internal timer's expiration count (https://github.com/shadow/shadow/pull/2279), and not cancel previously scheduled timer-fire events (https://github.com/shadow/shadow/pull/2282). * Fixed a panic when patching the VDSO in newer kernels, such as those in Ubuntu 22.04. https://github.com/shadow/shadow/issues/2273 * Fixed the errno returned from calling `connect()` on a unix socket. This fixes a `getaddrinfo()` test failure on some systems. https://github.com/shadow/shadow/issues/2286 * Fixed minor memory leaks. https://github.com/shadow/shadow/pull/2249 Other user-facing improvements since last update ================================================ We also continue to make compatibility improvements: * Added support for escaping additional "busy loops" - ones that only use `rdtsc` without making any actual syscalls. This affected some versions of [libopenblas](https://github.com/shadow/shadow/discussions/2299#discussioncomment-3198368). https://github.com/shadow/shadow/pull/2314 * Replaced custom cmake code for locating glib with pkg-config. This makes shadow easier to compile in environments with non-standard layouts such as guix or nix. https://github.com/shadow/shadow/pull/2331 * Fixed a bug handling files opened with `O_NOFOLLOW`. https://github.com/shadow/shadow/pull/2353 * Fixed our documentation and CI to support mapping pages with `PROT_EXEC` when run under Docker, fixing [2400](https://github.com/shadow/shadow/issues/2400). When using `docker run`, you should also pass the `--tmpfs /dev/shm:rw,nosuid,nodev,exec,size=1024g`. https://github.com/shadow/shadow/pull/2402 * Made the shadow installation directory relocatable. e.g. installing to `$HOME/opt/shadow` and later moving to `$HOME/opt/shadow-old` now behaves as expected. This also makes it easier to share and reuse pre-built shadow binaries, e.g. for CI, though we still do not yet publish pre-compiled binaries. https://github.com/shadow/shadow/pull/2391 * Added support for emulating `PR_SET_DUMPABLE`, which fixes compatibility with managed programs that "harden" themselves against debuggers, including [arti](https://gitlab.torproject.org/tpo/core/arti). https://github.com/shadow/shadow/pull/2370 * Fixed a bug affecting getaddrinfo on some systems. https://github.com/shadow/shadow/pull/2292 * Partly implemented fcntl. https://github.com/shadow/shadow/pull/2259 * Made failures due to running out of tmpfs space easier to understand. https://github.com/shadow/shadow/pull/2267 Other notable changes since last update ======================================= Since the 2.2.0 release, our current focus is primarily on migrating Shadow's C code to Rust. These changes should be mostly invisible to Shadow's end users, but may be of interest to folks interested in hacking on Shadow itself. Along those lines we've: * Migrated Shadow's scheduler to Rust. This is a fairly complex and performance-sensitive component, and we're taking care to prevent performance regressions during the migration: * https://github.com/shadow/shadow/pull/2340 * https://github.com/shadow/shadow/pull/2343 * https://github.com/shadow/shadow/pull/2344 * https://github.com/shadow/shadow/pull/2352 * https://github.com/shadow/shadow/pull/2412 * https://github.com/shadow/shadow/pull/2414 * https://github.com/shadow/shadow/pull/2416 * Migrated other modules to Rust. * Event: https://github.com/shadow/shadow/pull/2327 * TokenBucket: https://github.com/shadow/shadow/pull/2306 * Manager: https://github.com/shadow/shadow/pull/2277 * Implemented a Rust mutex suitable for use in shared memory, and compatible with [rkyv](https://rkyv.org/). https://github.com/shadow/shadow/pull/2386 * Moved most of the compilation and linking of Shadow's remaining C code from cmake into Cargo build scripts. This fixes some dependency issues and will make it easier to start migrating some of our additional support libraries from C to Rust. https://github.com/shadow/shadow/pull/2404 * Improved organization of global state in Rust. * https://github.com/shadow/shadow/pull/2361 * https://github.com/shadow/shadow/pull/2363 * https://github.com/shadow/shadow/pull/2367 * https://github.com/shadow/shadow/pull/2372 * https://github.com/shadow/shadow/pull/2373 * Refactored other components to simplify migration to Rust. * Controller, Manager, Scheduler: https://github.com/shadow/shadow/pull/2255 * Thread: https://github.com/shadow/shadow/pull/2328 Shadow in Tor ============= We've been working on using Shadow to help test and evaluate [arti](https://gitlab.torproject.org/tpo/core/arti) - The Tor Projects's new Rust-based implementation of tor. This has driven many of the compatibility fixes above, and we are happy to report that arti now runs under Shadow! We plan to merge a shadow-based continuous integration test into arti itself within the next few days: https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/634 Happy simulating! The Shadow team