[tor-talk] Tor is released!

Nick Mathewson nickm at torproject.org
Fri Jan 18 15:11:20 UTC 2019

Hi, all!

There's a new alpha Tor release! Because it's an alpha, you should
only run it if you're ready to find more bugs than usual, and report
them on trac.torproject.org.

The source code is available from the usual place on the
www.torproject.org; if you build Tor from source, why not give it a
try? And if you don't build Tor from source, packages should be ready
over the coming days, with a Tor Browser alpha release likely by the
end of the month.

Here's what's new:

Changes in version - 2019-01-18
  Tor is the first release in the new 0.4.0.x series. It
  introduces improved features for power and bandwidth conservation,
  more accurate reporting of bootstrap progress for user interfaces, and
  an experimental backend for an exciting new adaptive padding feature.
  There is also the usual assortment of bugfixes and minor features, all
  described below.

  o Major features (battery management, client, dormant mode):
    - When Tor is running as a client, and it is unused for a long time,
      it can now enter a "dormant" state. When Tor is dormant, it avoids
      network and CPU activity until it is reawoken either by a user
      request or by a controller command. For more information, see the
      configuration options starting with "Dormant". Implements tickets
      2149 and 28335.
    - The client's memory of whether it is "dormant", and how long it
      has spent idle, persists across invocations. Implements
      ticket 28624.
    - There is a DormantOnFirstStartup option that integrators can use
      if they expect that in many cases, Tor will be installed but
      not used.

  o Major features (bootstrap reporting):
    - When reporting bootstrap progress, report the first connection
      uniformly, regardless of whether it's a connection for building
      application circuits. This allows finer-grained reporting of early
      progress than previously possible, with the improvements of ticket
      27169. Closes tickets 27167 and 27103. Addresses ticket 27308.
    - When reporting bootstrap progress, treat connecting to a proxy or
      pluggable transport as separate from having successfully used that
      proxy or pluggable transport to connect to a relay. Closes tickets
      27100 and 28884.

  o Major features (circuit padding):
    - Implement preliminary support for the circuit padding portion of
      Proposal 254. The implementation supports Adaptive Padding (aka
      WTF-PAD) state machines for use between experimental clients and
      relays. Support is also provided for APE-style state machines that
      use probability distributions instead of histograms to specify
      inter-packet delay. At the moment, Tor does not provide any
      padding state machines that are used in normal operation: for now,
      this feature exists solely for experimentation. Closes
      ticket 28142.

  o Major features (refactoring):
    - Tor now uses an explicit list of its own subsystems when
      initializing and shutting down. Previously, these systems were
      managed implicitly in various places throughout the codebase.
      (There may still be some subsystems using the old system.) Closes
      ticket 28330.

  o Minor features (bootstrap reporting):
    - When reporting bootstrap progress, stop distinguishing between
      situations where only internal paths are available and situations
      where external paths are available. Previously, Tor would often
      erroneously report that it had only internal paths. Closes
      ticket 27402.

  o Minor features (continuous integration):
    - Log Python version during each Travis CI job. Resolves
      issue 28551.

  o Minor features (controller):
    - Add a DROPOWNERSHIP command to undo the effects of TAKEOWNERSHIP.
      Implements ticket 28843.

  o Minor features (developer tooling):
    - Provide a git hook script to prevent "fixup!" and "squash!"
      commits from ending up in the master branch, as scripts/main/pre-
      push.git-hook. Closes ticket 27993.

  o Minor features (directory authority):
    - Directory authorities support a new consensus algorithm, under
      which the family lines in microdescriptors are encoded in a
      canonical form. This change makes family lines more compressible
      in transit, and on the client. Closes ticket 28266; implements
      proposal 298.

  o Minor features (directory authority, relay):
    - Authorities now vote on a "StaleDesc" flag to indicate that a
      relay's descriptor is so old that the relay should upload again
      soon. Relays treat this flag as a signal to upload a new
      descriptor. This flag will eventually let us remove the
      'published' date from routerstatus entries, and make our consensus
      diffs much smaller. Closes ticket 26770; implements proposal 293.

  o Minor features (fallback directory mirrors):
    - Update the fallback whitelist based on operator opt-ins and opt-
      outs. Closes ticket 24805, patch by Phoul.

  o Minor features (FreeBSD):
    - On FreeBSD-based systems, warn relay operators if the
      "net.inet.ip.random_id" sysctl (IP ID randomization) is disabled.
      Closes ticket 28518.

  o Minor features (HTTP standards compliance):
    - Stop sending the header "Content-type: application/octet-stream"
      along with transparently compressed documents: this confused
      browsers. Closes ticket 28100.

  o Minor features (IPv6):
    - We add an option ClientAutoIPv6ORPort, to make clients randomly
      prefer a node's IPv4 or IPv6 ORPort. The random preference is set
      every time a node is loaded from a new consensus or bridge config.
      We expect that this option will enable clients to bootstrap more
      quickly without having to determine whether they support IPv4,
      IPv6, or both. Closes ticket 27490. Patch by Neel Chauhan.
    - When using addrs_in_same_network_family(), avoid choosing circuit
      paths that pass through the same IPv6 subnet more than once.
      Previously, we only checked IPv4 subnets. Closes ticket 24393.
      Patch by Neel Chauhan.

  o Minor features (log messages):
    - Improve log message in v3 onion services that could print out
      negative revision counters. Closes ticket 27707. Patch
      by "ffmancera".

  o Minor features (memory usage):
    - Save memory by storing microdescriptor family lists with a more
      compact representation. Closes ticket 27359.
    - Tor clients now use mmap() to read consensus files from disk, so
      that they no longer need keep the full text of a consensus in
      memory when parsing it or applying a diff. Closes ticket 27244.

  o Minor features (parsing):
    - Directory authorities now validate that router descriptors and
      ExtraInfo documents are in a valid subset of UTF-8, and reject
      them if they are not. Closes ticket 27367.

  o Minor features (performance):
    - Cache the results of summarize_protocol_flags(), so that we don't
      have to parse the same protocol-versions string over and over.
      This should save us a huge number of malloc calls on startup, and
      may reduce memory fragmentation with some allocators. Closes
      ticket 27225.
    - Remove a needless memset() call from get_token_arguments, thereby
      speeding up the tokenization of directory objects by about 20%.
      Closes ticket 28852.
    - Replace parse_short_policy() with a faster implementation, to
      improve microdescriptor parsing time. Closes ticket 28853.
    - Speed up directory parsing a little by avoiding use of the non-
      inlined strcmp_len() function. Closes ticket 28856.
    - Speed up microdescriptor parsing by about 30%, to help improve
      startup time. Closes ticket 28839.

  o Minor features (pluggable transports):
    - Add support for emitting STATUS updates to Tor's control port from
      a pluggable transport process. Closes ticket 28846.
    - Add support for logging to Tor's logging subsystem from a
      pluggable transport process. Closes ticket 28180.

  o Minor features (process management):
    - Add a new process API for handling child processes. This new API
      allows Tor to have bi-directional communication with child
      processes on both Unix and Windows. Closes ticket 28179.
    - Use the subsystem manager to initialize and shut down the process
      module. Closes ticket 28847.

  o Minor features (relay):
    - When listing relay families, list them in canonical form including
      the relay's own identity, and try to give a more useful set of
      warnings. Part of ticket 28266 and proposal 298.

  o Minor features (required protocols):
    - Before exiting because of a missing required protocol, Tor will
      now check the publication time of the consensus, and not exit
      unless the consensus is newer than the Tor program's own release
      date. Previously, Tor would not check the consensus publication
      time, and so might exit because of a missing protocol that might
      no longer be required in a current consensus. Implements proposal
      297; closes ticket 27735.

  o Minor features (testing):
    - Allow a HeartbeatPeriod of less than 30 minutes in testing Tor
      networks. Closes ticket 28840. Patch by Rob Jansen.

  o Minor bugfixes (client, clock skew):
    - Bootstrap successfully even when Tor's clock is behind the clocks
      on the authorities. Fixes bug 28591; bugfix on
    - Select guards even if the consensus has expired, as long as the
      consensus is still reasonably live. Fixes bug 24661; bugfix

  o Minor bugfixes (compilation):
    - Compile correctly on OpenBSD; previously, we were missing some
      headers required in order to detect it properly. Fixes bug 28938;
      bugfix on Patch from Kris Katterjohn.

  o Minor bugfixes (directory clients):
    - Mark outdated dirservers when Tor only has a reasonably live
      consensus. Fixes bug 28569; bugfix on

  o Minor bugfixes (directory mirrors):
    - Even when a directory mirror's clock is behind the clocks on the
      authorities, we now allow the mirror to serve "future"
      consensuses. Fixes bug 28654; bugfix on

  o Minor bugfixes (DNS):
    - Gracefully handle an empty or absent resolve.conf file by falling
      back to using "localhost" as a DNS server (and hoping it works).
      Previously, we would just stop running as an exit. Fixes bug
      21900; bugfix on

  o Minor bugfixes (guards):
    - In count_acceptable_nodes(), the minimum number is now one bridge
      or guard node, and two non-guard nodes for a circuit. Previously,
      we had added up the sum of all nodes with a descriptor, but that
      could cause us to build failing circuits when we had either too
      many bridges or not enough guard nodes. Fixes bug 25885; bugfix on Patch by Neel Chauhan.

  o Minor bugfixes (IPv6):
    - Fix tor_ersatz_socketpair on IPv6-only systems. Previously, the
      IPv6 socket was bound using an address family of AF_INET instead
      of AF_INET6. Fixes bug 28995; bugfix on Patch from
      Kris Katterjohn.

  o Minor bugfixes (logging):
    - Rework rep_hist_log_link_protocol_counts() to iterate through all
      link protocol versions when logging incoming/outgoing connection
      counts. Tor no longer skips version 5, and we won't have to
      remember to update this function when new link protocol version is
      developed. Fixes bug 28920; bugfix on

  o Minor bugfixes (networking):
    - Introduce additional checks into tor_addr_parse() to reject
      certain incorrect inputs that previously were not detected. Fixes
      bug 23082; bugfix on

  o Minor bugfixes (onion service v3, client):
    - Stop logging a "BUG()" warning and stacktrace when we find a SOCKS
      connection waiting for a descriptor that we actually have in the
      cache. It turns out that this can actually happen, though it is
      rare. Now, tor will recover and retry the descriptor. Fixes bug
      28669; bugfix on

  o Minor bugfixes (periodic events):
    - Refrain from calling routerlist_remove_old_routers() from
      check_descriptor_callback(). Instead, create a new hourly periodic
      event. Fixes bug 27929; bugfix on

  o Minor bugfixes (pluggable transports):
    - Make sure that data is continously read from standard output and
      standard error pipes of a pluggable transport child-process, to
      avoid deadlocking when a pipe's buffer is full. Fixes bug 26360;
      bugfix on

  o Minor bugfixes (unit tests):
    - Instead of relying on hs_free_all() to clean up all onion service
      objects in test_build_descriptors(), we now deallocate them one by
      one. This lets Coverity know that we are not leaking memory there
      and fixes CID 1442277. Fixes bug 28989; bugfix on

  o Minor bugfixes (usability):
    - Stop saying "Your Guard ..." in pathbias_measure_{use,close}_rate().
      Some users took this phrasing to mean that the mentioned guard was
      under their control or responsibility, which it is not. Fixes bug
      28895; bugfix on Tor

  o Code simplification and refactoring:
    - Reimplement NETINFO cell parsing and generation to rely on
      trunnel-generated wire format handling code. Closes ticket 27325.
    - Remove unnecessary unsafe code from the Rust macro "cstr!". Closes
      ticket 28077.
    - Rework SOCKS wire format handling to rely on trunnel-generated
      parsing/generation code. Resolves ticket 27620.
    - Split out bootstrap progress reporting from control.c into a
      separate file. Part of ticket 27402.
    - The .may_include files that we use to describe our directory-by-
      directory dependency structure now describe a noncircular
      dependency graph over the directories that they cover. Our
      checkIncludes.py tool now enforces this noncircularity. Closes
      ticket 28362.

  o Documentation:
    - Mention that you cannot add a new onion service if Tor is already
      running with Sandbox enabled. Closes ticket 28560.
    - Improve ControlPort documentation. Mention that it accepts
      address:port pairs, and can be used multiple times. Closes
      ticket 28805.
    - Document the exact output of "tor --version". Closes ticket 28889.

  o Removed features:
    - Stop responding to the 'GETINFO status/version/num-concurring' and
      'GETINFO status/version/num-versioning' control port commands, as
      those were deprecated back in Also stop listing them in
      output of 'GETINFO info/names'. Resolves ticket 28757.
    - The scripts used to generate and maintain the list of fallback
      directories have been extracted into a new "fallback-scripts"
      repository. Closes ticket 27914.

  o Testing:
    - Run shellcheck for scripts in the in scripts/ directory. Closes
      ticket 28058.
    - Add unit tests for tokenize_string() and get_next_token()
      functions. Resolves ticket 27625.

  o Code simplification and refactoring (onion service v3):
    - Consolidate the authorized client descriptor cookie computation
      code from client and service into one function. Closes
      ticket 27549.

  o Code simplification and refactoring (shell scripts):
    - Cleanup scan-build.sh to silence shellcheck warnings. Closes
      ticket 28007.
    - Fix issues that shellcheck found in chutney-git-bisect.sh.
      Resolves ticket 28006.
    - Fix issues that shellcheck found in updateRustDependencies.sh.
      Resolves ticket 28012.
    - Fix shellcheck warnings in cov-diff script. Resolves issue 28009.
    - Fix shellcheck warnings in run_calltool.sh. Resolves ticket 28011.
    - Fix shellcheck warnings in run_trunnel.sh. Resolves issue 28010.
    - Fix shellcheck warnings in scripts/test/coverage. Resolves
      issue 28008.

More information about the tor-talk mailing list