[tor-talk] Tor 0.4.3.1-alpha is released!
nickm at torproject.org
Thu Jan 23 01:44:29 UTC 2020
There's a new alpha Tor release! Because it's an alpha, you should
only run it if you're ready to find more bugs than usual, and report
them on trac.torproject.org.
The source code is available from the usual place on
https://www.torproject.org/download/tor/ ; if you build Tor from
source, why not give it a try? And if you don't build Tor from source,
packages should be ready over the coming days, with a Tor Browser
alpha release likely by mid-February.
Here's what's new:
Changes in version 0.4.3.1-alpha - 2020-01-22
This is the first alpha release in the 0.4.3.x series. It includes
improved support for application integration of onion services, support
for building in a client-only mode, and newly improved internal
documentation (online at https://src-ref.docs.torproject.org/tor/). It
also has numerous other small bugfixes and features, as well as
improvements to our code's internal organization that should help us
write better code in the future.
o New system requirements:
- When building Tor, you now need to have Python 3 in order to run
the integration tests. (Python 2 is officially unsupported
upstream, as of 1 Jan 2020.) Closes ticket 32608.
o Major features (build system):
- The relay code can now be disabled using the --disable-module-relay
configure option. When this option is set, we also disable the
dirauth module. Closes ticket 32123.
- When Tor is compiled --disable-module-relay, we also omit the code
used to act as a directory cache. Closes ticket 32487.
o Major features (directory authority, ed25519):
- Add support for banning a relay's ed25519 keys in the approved-
routers file. This will help us migrate away from RSA keys in the
future. Previously, only RSA keys could be banned in approved-
routers. Resolves ticket 22029. Patch by Neel Chauhan.
o Major features (onion service, controller):
- New control port commands to manage client-side onion service
authorization credentials. The ONION_CLIENT_AUTH_ADD command adds
a credential, ONION_CLIENT_AUTH_REMOVE deletes a credential, and
ONION_CLIENT_AUTH_VIEW lists the credentials. Closes ticket 30381.
o Major features (onion service, SOCKS5):
- Introduce a new SocksPort flag, ExtendedErrors, to support more
detailed error codes in information for applications that support
them. Closes ticket 30382; implements proposal 304.
o Major features (proxy):
- In addition to its current supported proxy types (HTTP CONNECT,
SOCKS4, and SOCKS5), Tor can now make its OR connections through a
HAProxy server. A new torrc option was added to specify the
address/port of the server: TCPProxy <protocol> <host>:<port>.
Currently the only supported protocol for the option is haproxy.
Closes ticket 31518. Patch done by Suphanat Chunhapanya (haxxpop).
o Major bugfixes (linux seccomp sandbox):
- Correct how we use libseccomp. Particularly, stop assuming that
rules are applied in a particular order or that more rules are
processed after the first match. Neither is the case! In
libseccomp <2.4.0 this lead to some rules having no effect.
libseccomp 2.4.0 changed how rules are generated, leading to a
different ordering, which in turn led to a fatal crash during
startup. Fixes bug 29819; bugfix on 0.2.5.1-alpha. Patch by
- Fix crash when reloading logging configuration while the
experimental sandbox is enabled. Fixes bug 32841; bugfix on
0.4.1.7. Patch by Peter Gerber.
o Major bugfixes (networking):
- Correctly handle IPv6 addresses in SOCKS5 RESOLVE_PTR requests,
and accept strings as well as binary addresses. Fixes bug 32315;
bugfix on 0.3.5.1-alpha.
o Major bugfixes (onion service):
- Report HS circuit failure back into the HS subsystem so we take
appropriate action with regards to the client introduction point
failure cache. This improves reachability of onion services, since
now clients notice failing introduction circuits properly. Fixes
bug 32020; bugfix on 0.3.2.1-alpha.
o Minor feature (configure, build system):
- Output a list of enabled/disabled features at the end of the
configure process in a pleasing way. Closes ticket 31373.
o Minor feature (heartbeat, onion service):
- Add the DoS INTRODUCE2 defenses counter to the heartbeat DoS
message. Closes ticket 31371.
o Minor features (configuration validation):
- Configuration validation can now be done by per-module callbacks,
rather than a global validation function. This will let us reduce
the size of config.c and some of its more cumbersome functions.
Closes ticket 31241.
o Minor features (configuration):
- If a configured hardware crypto accelerator in AccelName is
prefixed with "!", Tor now exits when it cannot be found. Closes
- We now use flag-driven logic to warn about obsolete configuration
fields, so that we can include their names. In 0.4.2, we used a
special type, which prevented us from generating good warnings.
Implements ticket 32404.
o Minor features (controller):
- Add stream isolation data to STREAM event. Closes ticket 19859.
- Implement a new GETINFO command to fetch microdescriptor
consensus. Closes ticket 31684.
o Minor features (debugging, directory system):
- Don't crash when we find a non-guard with a guard-fraction value
set. Instead, log a bug warning, in an attempt to figure out how
this happened. Diagnostic for ticket 32868.
o Minor features (defense in depth):
- Add additional checks around tor_vasprintf() usage, in case the
function returns an error. Patch by Tobias Stoeckmann. Fixes
o Minor features (developer tooling):
- Remove the 0.2.9.x series branches from git scripts (git-merge-
forward.sh, git-pull-all.sh, git-push-all.sh, git-setup-dirs.sh).
Closes ticket 32772.
o Minor features (developer tools):
- Add a check_cocci_parse.sh script that checks that new code is
parseable by Coccinelle. Add an exceptions file for unparseable
files, and run the script from travis CI. Closes ticket 31919.
- Call the check_cocci_parse.sh script from a 'check-cocci' Makefile
target. Closes ticket 31919.
- Add a rename_c_identifiers.py tool to rename a bunch of C
identifiers at once, and generate a well-formed commit message
describing the change. This should help with refactoring. Closes
- Add some scripts in "scripts/coccinelle" to invoke the Coccinelle
semantic patching tool with the correct flags. These flags are
fairly easy to forget, and these scripts should help us use
Coccinelle more effectively in the future. Closes ticket 31705.
o Minor features (Doxygen):
- Update Doxygen configuration file to a more recent template (from
1.8.15). Closes ticket 32110.
- "make doxygen" now works with out-of-tree builds. Closes
- Make sure that doxygen outputs documentation for all of our C
files. Previously, some were missing @file declarations, causing
them to be ignored. Closes ticket 32307.
- Our "make doxygen" target now respects --enable-fatal-warnings by
default, and does not warn about items that are missing
documentation. To warn about missing documentation, run configure
with the "--enable-missing-doc-warnings" flag: doing so suspends
fatal warnings for doxygen. Closes ticket 32385.
o Minor features (git scripts):
- Add TOR_EXTRA_CLONE_ARGS to git-setup-dirs.sh for git clone
customisation. Closes ticket 32347.
- Add git-setup-dirs.sh, which sets up an upstream git repository
and worktrees for tor maintainers. Closes ticket 29603.
- Add TOR_EXTRA_REMOTE_* to git-setup-dirs.sh for a custom extra
remote. Closes ticket 32347.
- Call the check_cocci_parse.sh script from the git commit and push
hooks. Closes ticket 31919.
- Make git-push-all.sh skip unchanged branches when pushing to
upstream. The script already skipped unchanged test branches.
Closes ticket 32216.
- Make git-setup-dirs.sh create a master symlink in the worktree
directory. Closes ticket 32347.
- Skip unmodified source files when doing some existing git hook
checks. Related to ticket 31919.
o Minor features (IPv6, client):
- Make Tor clients tell dual-stack exits that they prefer IPv6
connections. This change is equivalent to setting the PreferIPv6
flag on SOCKSPorts (and most other listener ports). Tor Browser
has been setting this flag for some time, and we want to remove a
client distinguisher at exits. Closes ticket 32637.
o Minor features (portability, android):
- When building for Android, disable some tests that depend on $HOME
and/or pwdb, which Android doesn't have. Closes ticket 32825.
Patch from Hans-Christoph Steiner.
o Minor features (relay modularity):
- Split the relay and server pluggable transport config code into
separate files in the relay module. Disable this code when the
relay module is disabled. Closes part of ticket 32213.
- When the relay module is disabled, reject attempts to set the
ORPort, DirPort, DirCache, BridgeRelay, ExtORPort, or
ServerTransport* options, rather than ignoring the values of these
options. Closes part of ticket 32213.
o Minor features (relay):
- When the relay module is disabled, change the default config so
that DirCache is 0, and ClientOnly is 1. Closes ticket 32410.
o Minor features (release tools):
- Port our ChangeLog formatting and sorting tools to Python 3.
Closes ticket 32704.
o Minor features (testing):
- Detect some common failure cases for test_parseconf.sh in
src/test/conf_failures. Closes ticket 32451.
- Allow test_parseconf.sh to test expected log outputs for successful
configs, as well as failed configs. Closes ticket 32451.
- The test_parseconf.sh script now supports result variants for any
combination of the optional libraries lzma, nss, and zstd. Closes
o Minor features (tests, Android):
- When running the unit tests on Android, create temporary files in
a subdirectory of /data/local/tmp. Closes ticket 32172. Based on a
patch from Hans-Christoph Steiner.
o Minor bugfixes (bridges):
- Lowercase the configured value of BridgeDistribution before adding
it to the descriptor. Fixes bug 32753; bugfix on 0.3.2.3-alpha.
o Minor bugfixes (build system):
- Fix "make autostyle" for out-of-tree builds. Fixes bug 32370;
bugfix on 0.4.1.2-alpha.
o Minor bugfixes (configuration handling):
- Make control_event_conf_changed() take in a config_line_t instead
of a smartlist of alternating key/value entries. Fixes bug 31531;
bugfix on 0.2.3.3-alpha. Patch by Neel Chauhan.
o Minor bugfixes (configuration):
- Check for multiplication overflow when parsing memory units inside
configuration. Fixes bug 30920; bugfix on 0.0.9rc1.
- When dumping the configuration, stop adding a trailing space after
the option name when there is no option value. This issue only
affects options that accept an empty value or list. (Most options
reject empty values, or delete the entire line from the dumped
options.) Fixes bug 32352; bugfix on 0.0.9pre6.
- Avoid changing the user's value of HardwareAccel as stored by
SAVECONF, when AccelName is set but HardwareAccel is not. Fixes
bug 32382; bugfix on 0.2.2.1-alpha.
- When creating a KeyDirectory with the same location as the
DataDirectory (not recommended), respect the DataDirectory's
group-readable setting if one has not been set for the
KeyDirectory. Fixes bug 27992; bugfix on 0.3.3.1-alpha.
o Minor bugfixes (controller):
- In routerstatus_has_changed(), check all the fields that are
output over the control port. Fixes bug 20218; bugfix
o Minor bugfixes (correctness checks):
- Use GCC/Clang's printf-checking feature to make sure that
tor_assertf() arguments are correctly typed. Fixes bug 32765;
bugfix on 0.4.1.1-alpha.
o Minor bugfixes (developer tools):
- Allow paths starting with ./ in scripts/add_c_file.py. Fixes bug
31336; bugfix on 0.4.1.2-alpha.
o Minor bugfixes (dirauth module):
- Split the dirauth config code into a separate file in the dirauth
module. Disable this code when the dirauth module is disabled.
Closes ticket 32213.
- When the dirauth module is disabled, reject attempts to set the
AuthoritativeDir option, rather than ignoring the value of the
option. Fixes bug 32213; bugfix on 0.3.4.1-alpha.
o Minor bugfixes (embedded Tor):
- When starting Tor any time after the first time in a process,
register the thread in which it is running as the main thread.
Previously, we only did this on Windows, which could lead to bugs
like 23081 on non-Windows platforms. Fixes bug 32884; bugfix
o Minor bugfixes (git scripts):
- Avoid sleeping before the last push in git-push-all.sh. Closes
- Forward all unrecognised arguments in git-push-all.sh to git push.
Closes ticket 32216.
o Minor bugfixes (hidden service v3):
- Do not rely on a "circuit established" flag for intro circuits but
instead always query the HS circuit map. This is to avoid sync
issue with that flag and the map. Fixes bug 32094; bugfix
o Minor bugfixes (logging, crash):
- Avoid a possible crash when trying to log a (fatal) assertion
failure about mismatched magic numbers in configuration objects.
Fixes bug 32771; bugfix on 0.4.2.1-alpha.
o Minor bugfixes (onion service v2):
- When sending the INTRO cell for a v2 Onion Service, look at the
failure cache alongside timeout values to check if the intro point
is marked as failed. Previously, we only looked at the relay
timeout values. Fixes bug 25568; bugfix on 0.2.7.3-rc. Patch by
o Minor bugfixes (onion services v3, client):
- Properly handle the client rendezvous circuit timeout. Previously
Tor would sometimes timeout a rendezvous circuit awaiting the
introduction ACK, and find itself unable to re-establish all
circuits because the rendezvous circuit timed out too early. Fixes
bug 32021; bugfix on 0.3.2.1-alpha.
o Minor bugfixes (onion services):
- In cancel_descriptor_fetches(), use
connection_list_by_type_purpose() instead of
connection_list_by_type_state(). Fixes bug 32639; bugfix on
0.3.2.1-alpha. Patch by Neel Chauhan.
o Minor bugfixes (scripts):
- Fix update_versions.py for out-of-tree builds. Fixes bug 32371;
bugfix on 0.4.0.1-alpha.
o Minor bugfixes (test):
- Use the same code to find the tor binary in all of our test
scripts. This change makes sure we are always using the coverage
binary when coverage is enabled. Fixes bug 32368; bugfix
o Minor bugfixes (testing):
- Stop ignoring "tor --dump-config" errors in test_parseconf.sh.
Fixes bug 32468; bugfix on 0.4.2.1-alpha.
- When TOR_DISABLE_PRACTRACKER is set, do not apply it to the
test_practracker.sh script. Doing so caused a test failure. Fixes
bug 32705; bugfix on 0.4.2.1-alpha.
- When TOR_DISABLE_PRACTRACKER is set, log a notice to stderr when
skipping practracker checks. Fixes bug 32705; bugfix
o Minor bugfixes (tests):
- Our option-validation tests no longer depend on specially
configured non-default, non-passing sets of options. Previously,
the tests had been written to assume that options would _not_ be
set to their defaults, which led to needless complexity and
verbosity. Fixes bug 32175; bugfix on 0.2.8.1-alpha.
o Minor bugfixes (windows service):
- Initialize the publish/subscribe system when running as a windows
service. Fixes bug 32778; bugfix on 0.4.1.1-alpha.
o Deprecated features:
- Deprecate the ClientAutoIPv6ORPort option. This option was not
true "Happy Eyeballs", and often failed on connections that
weren't reliably dual-stack. Closes ticket 32942. Patch by
- Provide a quickstart guide for a Circuit Padding Framework, and
documentation for researchers to implement and study circuit
padding machines. Closes ticket 28804.
- Add documentation in 'HelpfulTools.md' to describe how to build a
tag file. Closes ticket 32779.
- Create a high-level description of the long-term software
architecture goals. Closes ticket 32206.
- Describe the --dump-config command in the manual page. Closes
- Unite coding advice from this_not_that.md in torguts repo into our
coding standards document. Resolves ticket 31853.
o Removed features:
- Our Doxygen configuration no longer generates LaTeX output. The
reference manual produced by doing this was over 4000 pages long,
and generally unusable. Closes ticket 32099.
- The option "TestingEstimatedDescriptorPropagationTime" is now
marked as obsolete. It has had no effect since 0.3.0.7, when
clients stopped rejecting consensuses "from the future". Closes
- We no longer support consensus methods before method 28; these
methods were only used by authorities running versions of Tor that
are now at end-of-life. In effect, this means that clients,
relays, and authorities now assume that authorities will be
running version 0.3.5.x or later. Closes ticket 32695.
- Add more test cases for tor's UTF-8 validation function. Also,
check the arguments passed to the function for consistency. Closes
- Improve test coverage for relay and dirauth config code, focusing
on option validation and normalization. Closes ticket 32213.
- Improve the consistency of test_parseconf.sh output, and run all
the tests, even if one fails. Closes ticket 32213.
- Re-enable the Travis CI macOS Chutney build, but don't let it
prevent the Travis job from finishing. (The Travis macOS jobs are
slow, so we don't want to have it delay the whole CI process.)
Closes ticket 32629.
- Run the practracker unit tests in the pre-commit git hook. Closes
- Turn off Tor's Sandbox in Chutney jobs, and run those jobs on
Ubuntu Bionic. Turning off the Sandbox is a work-around, until we
fix the sandbox errors in 32722. Closes ticket 32240.
o Code simplification and refactoring (channel):
- Channel layer had a variable length cell handler that was not used
and thus removed. Closes ticket 32892.
o Code simplification and refactoring (configuration):
- Immutability is now implemented as a flag on individual
configuration options rather than as part of the option-transition
checking code. Closes ticket 32344.
- Instead of keeping a list of configuration options to check for
relative paths, check all the options whose type is "FILENAME".
Solves part of ticket 32339.
- Our default log (which ordinarily sends NOTICE-level messages to
standard output) is now handled in a more logical manner.
Previously, we replaced the configured log options if they were
empty. Now, we interpret an empty set of log options as meaning
"use the default log". Closes ticket 31999.
- Remove some unused arguments from the options_validate() function,
to simplify our code and tests. Closes ticket 32187.
- Simplify the options_validate() code so that it looks at the
default options directly, rather than taking default options as an
argument. This change lets us simplify its interface. Closes
- Use our new configuration architecture to move most authority-
related options to the directory authority module. Closes
- When parsing the command line, handle options that determine our
"quiet level" and our mode of operation (e.g., --dump-config and
so on) all in one table. Closes ticket 32003.
o Code simplification and refactoring (controller):
- Create a new abstraction for formatting control protocol reply
lines based on key-value pairs. Refactor some existing control
protocol code to take advantage of this. Closes ticket 30984.
- Create a helper function that can fetch network status or
microdesc consensuses. Closes ticket 31684.
o Code simplification and refactoring (dirauth modularization):
- Remove the last remaining HAVE_MODULE_DIRAUTH inside a function.
Closes ticket 32163.
- Replace some confusing identifiers in process_descs.c. Closes
- Simplify some relay and dirauth config code. Closes ticket 32213.
o Code simplification and refactoring (misc):
- Make all the structs we declare follow the same naming convention
of ending with "_t". Closes ticket 32415.
- Move and rename some configuration-related code for clarity.
Closes ticket 32304.
- Our include.am files are now broken up by subdirectory.
Previously, src/core/include.am covered all of the subdirectories
in "core", "feature", and "app". Closes ticket 32137.
- Remove underused NS*() macros from test code: they make our tests
more confusing, especially for code-formatting tools. Closes
o Code simplification and refactoring (relay modularization):
- Disable relay_periodic when the relay module is disabled. Closes
- Disable relay_sys when the relay module is disabled. Closes
o Code simplification and refactoring (tool support):
- Add numerous missing dependencies to our include files, so that
they can be included in different reasonable orders and still
compile. Addresses part of ticket 32764.
- Fix some parts of our code that were difficult for Coccinelle to
parse. Related to ticket 31705.
- Fix some small issues in our code that prevented automatic
formatting tools from working. Addresses part of ticket 32764.
o Documentation (manpage):
- Alphabetize the Client Options section of the tor manpage. Closes
- Alphabetize the General Options section of the tor manpage. Closes
- In the tor(1) manpage, reword and improve formatting of the
COMMAND-LINE OPTIONS and DESCRIPTION sections. Closes ticket
32277. Based on work by Swati Thacker as part of Google Season
- In the tor(1) manpage, reword and improve formatting of the FILES,
SEE ALSO, and BUGS sections. Closes ticket 32176. Based on work by
Swati Thacker as part of Google Season of Docs.
o Testing (circuit, EWMA):
- Add unit tests for circuitmux and EWMA subsystems. Closes
o Testing (continuous integration):
- Use zstd in our Travis Linux builds. Closes ticket 32242.
More information about the tor-talk