[tor-talk] New alpha release: 0.3.2.1-alpha (and new release series too!)

[Nick Mathewson]

Hi, everybody!

After lots of work, we've got a tasty new alpha ready for you to find
bugs in!  Because it's an alpha, you should only run it if you're
ready to find more bugs than usual, and report them on
trac.torproject.org.

The source code is available from the usual place on
www.torproject.org ; if you build Tor from source, why not give it a
try?  And if you don't build Tor from source, packages should be ready
over the coming days, with a Tor Browser alpha release likely by the
end of the month.

(There were also new stable releases today, but they get announced on
the nice low-volume tor-announcements mailing list.)

If everything goes well, we're hoping to get this release series
stabilized by mid-December.

Here's what's new!

Changes in version 0.3.2.1-alpha - 2017-09-18
  Tor 0.3.2.1-alpha is the first release in the 0.3.2.x series. It
  includes support for our next-generation ("v3") onion service
  protocol, and adds a new circuit scheduler for more responsive
  forwarding decisions from relays. There are also numerous other small
  features and bugfixes here.

  Below are the changes since Tor 0.3.1.7.

  o Major feature (scheduler, channel):
    - Tor now uses new schedulers to decide which circuits should
      deliver cells first, in order to improve congestion at relays. The
      first type is called "KIST" ("Kernel Informed Socket Transport"),
      and is only available on Linux-like systems: it uses feedback from
      the kernel to prevent the kernel's TCP buffers from growing too
      full. The second new scheduler type is called "KISTLite": it
      behaves the same as KIST, but runs on systems without kernel
      support for inspecting TCP implementation details. The old
      scheduler is still available, under the name "Vanilla". To change
      the default scheduler preference order, use the new "Schedulers"
      option. (The default preference order is "KIST,KISTLite,Vanilla".)

      Matt Traudt implemented KIST, based on research by Rob Jansen,
      John Geddes, Christ Wacek, Micah Sherr, and Paul Syverson. For
      more information, see the design paper at
      http://www.robgjansen.com/publications/kist-sec2014.pdf and the
      followup implementation paper at https://arxiv.org/abs/1709.01044.
      Closes ticket 12541.

  o Major features (next-generation onion services):
    - Tor now supports the next-generation onion services protocol for
      clients and services! As part of this release, the core of
      proposal 224 has been implemented and is available for
      experimentation and testing by our users. This newer version of
      onion services ("v3") features many improvements over the legacy
      system, including:

      a) Better crypto (replaced SHA1/DH/RSA1024
      with SHA3/ed25519/curve25519)

      b) Improved directory protocol, leaking much less information to
      directory servers.

      c) Improved directory protocol, with smaller surface for
      targeted attacks.

      d) Better onion address security against impersonation.

      e) More extensible introduction/rendezvous protocol.

      f) A cleaner and more modular codebase.

      You can identify a next-generation onion address by its length:
      they are 56 characters long, as in
      "4acth47i6kxnvkewtm6q7ib2s3ufpo5sqbsnzjpbi7utijcltosqemad.onion".

      In the future, we will release more options and features for v3
      onion services, but we first need a testing period, so that the
      current codebase matures and becomes more robust. Planned features
      include: offline keys, advanced client authorization, improved
      guard algorithms, and statistics. For full details, see
      proposal 224.

      Legacy ("v2") onion services will still work for the foreseeable
      future, and will remain the default until this new codebase gets
      tested and hardened. Service operators who want to experiment with
      the new system can use the 'HiddenServiceVersion 3' torrc
      directive along with the regular onion service configuration
      options. We will publish a blog post about this new feature
      soon! Enjoy!

  o Major bugfixes (usability, control port):
    - Report trusted clock skew indications as bootstrap errors, so
      controllers can more easily alert users when their clocks are
      wrong. Fixes bug 23506; bugfix on 0.1.2.6-alpha.

  o Minor features (bug detection):
    - Log a warning message with a stack trace for any attempt to call
      get_options() during option validation. This pattern has caused
      subtle bugs in the past. Closes ticket 22281.

  o Minor features (client):
    - You can now use Tor as a tunneled HTTP proxy: use the new
      HTTPTunnelPort option to open a port that accepts HTTP CONNECT
      requests. Closes ticket 22407.
    - Add an extra check to make sure that we always use the newer guard
      selection code for picking our guards. Closes ticket 22779.
    - When downloading (micro)descriptors, don't split the list into
      multiple requests unless we want at least 32 descriptors.
      Previously, we split at 4, not 32, which led to significant
      overhead in HTTP request size and degradation in compression
      performance. Closes ticket 23220.

  o Minor features (command line):
    - Add a new commandline option, --key-expiration, which prints when
      the current signing key is going to expire. Implements ticket
      17639; patch by Isis Lovecruft.

  o Minor features (control port):
    - If an application tries to use the control port as an HTTP proxy,
      respond with a meaningful "This is the Tor control port" message,
      and log the event. Closes ticket 1667. Patch from Ravi
      Chandra Padmala.
    - Provide better error message for GETINFO desc/(id|name) when not
      fetching router descriptors. Closes ticket 5847. Patch by
      Kevin Butler.
    - Add GETINFO "{desc,md}/download-enabled", to inform the controller
      whether Tor will try to download router descriptors and
      microdescriptors respectively. Closes ticket 22684.
    - Added new GETINFO targets "ip-to-country/{ipv4,ipv6}-available",
      so controllers can tell whether the geoip databases are loaded.
      Closes ticket 23237.
    - Adds a timestamp field to the CIRC_BW and STREAM_BW bandwidth
      events. Closes ticket 19254. Patch by "DonnchaC".

  o Minor features (development support):
    - Developers can now generate a call-graph for Tor using the
      "calltool" python program, which post-processes object dumps. It
      should work okay on many Linux and OSX platforms, and might work
      elsewhere too. To run it, install calltool from
      https://gitweb.torproject.org/user/nickm/calltool.git and run
      "make callgraph". Closes ticket 19307.

  o Minor features (ed25519):
    - Add validation function to checks for torsion components in
      ed25519 public keys, used by prop224 client-side code. Closes
      ticket 22006. Math help by Ian Goldberg.

  o Minor features (exit relay, DNS):
    - Improve the clarity and safety of the log message from evdns when
      receiving an apparently spoofed DNS reply. Closes ticket 3056.

  o Minor features (integration, hardening):
    - Add a new NoExec option to prevent Tor from running other
      programs. When this option is set to 1, Tor will never try to run
      another program, regardless of the settings of
      PortForwardingHelper, ClientTransportPlugin, or
      ServerTransportPlugin. Once NoExec is set, it cannot be disabled
      without restarting Tor. Closes ticket 22976.

  o Minor features (logging):
    - Improve the warning message for specifying a relay by nickname.
      The previous message implied that nickname registration was still
      part of the Tor network design, which it isn't. Closes
      ticket 20488.
    - If the sandbox filter fails to load, suggest to the user that
      their kernel might not support seccomp2. Closes ticket 23090.

  o Minor features (portability):
    - Check at configure time whether uint8_t is the same type as
      unsigned char. Lots of existing code already makes this
      assumption, and there could be strict aliasing issues if the
      assumption is violated. Closes ticket 22410.

  o Minor features (relay, configuration):
    - Reject attempts to use relative file paths when RunAsDaemon is
      set. Previously, Tor would accept these, but the directory-
      changing step of RunAsDaemon would give strange and/or confusing
      results. Closes ticket 22731.

  o Minor features (startup, safety):
    - When configured to write a PID file, Tor now exits if it is unable
      to do so. Previously, it would warn and continue. Closes
      ticket 20119.

  o Minor features (static analysis):
    - The BUG() macro has been changed slightly so that Coverity no
      longer complains about dead code if the bug is impossible. Closes
      ticket 23054.

  o Minor features (testing):
    - The default chutney network tests now include tests for the v3
      hidden service design. Make sure you have the latest version of
      chutney if you want to run these. Closes ticket 22437.
    - Add a unit test to verify that we can parse a hardcoded v2 hidden
      service descriptor. Closes ticket 15554.

  o Minor bugfixes (certificate handling):
    - Fix a time handling bug in Tor certificates set to expire after
      the year 2106. Fixes bug 23055; bugfix on 0.3.0.1-alpha. Found by
      Coverity as CID 1415728.

  o Minor bugfixes (client, usability):
    - Refrain from needlessly rejecting SOCKS5-with-hostnames and
      SOCKS4a requests that contain IP address strings, even when
      SafeSocks in enabled, as this prevents user from connecting to
      known IP addresses without relying on DNS for resolving. SafeSocks
      still rejects SOCKS connections that connect to IP addresses when
      those addresses are _not_ encoded as hostnames. Fixes bug 22461;
      bugfix on Tor 0.2.6.2-alpha.

  o Minor bugfixes (code correctness):
    - Call htons() in extend_cell_format() for encoding a 16-bit value.
      Previously we used ntohs(), which happens to behave the same on
      all the platforms we support, but which isn't really correct.
      Fixes bug 23106; bugfix on 0.2.4.8-alpha.
    - For defense-in-depth, make the controller's write_escaped_data()
      function robust to extremely long inputs. Fixes bug 19281; bugfix
      on 0.1.1.1-alpha. Reported by Guido Vranken.

  o Minor bugfixes (compilation):
    - Fix unused-variable warnings in donna's Curve25519 SSE2 code.
      Fixes bug 22895; bugfix on 0.2.7.2-alpha.

  o Minor bugfixes (consensus expiry):
    - Check for adequate directory information correctly. Previously, Tor
      would reconsider whether it had sufficient directory information
      every 2 minutes. Fixes bug 23091; bugfix on 0.2.0.19-alpha.

  o Minor bugfixes (directory protocol):
    - Directory servers now include a "Date:" http header for response
      codes other than 200. Clients starting with a skewed clock and a
      recent consensus were getting "304 Not modified" responses from
      directory authorities, so without the Date header, the client
      would never hear about a wrong clock. Fixes bug 23499; bugfix
      on 0.0.8rc1.
    - Make clients wait for 6 seconds before trying to download a
      consensus from an authority. Fixes bug 17750; bugfix
      on 0.2.8.1-alpha.

  o Minor bugfixes (DoS-resistance):
    - If future code asks if there are any running bridges, without
      checking if bridges are enabled, log a BUG warning rather than
      crashing. Fixes bug 23524; bugfix on 0.3.0.1-alpha.

  o Minor bugfixes (format strictness):
    - Restrict several data formats to decimal. Previously, the
      BuildTimeHistogram entries in the state file, the "bw=" entries in
      the bandwidth authority file, and the process IDs passed to the
      __OwningControllerProcess option could all be specified in hex or
      octal as well as in decimal. This was not an intentional feature.
      Fixes bug 22802; bugfixes on 0.2.2.1-alpha, 0.2.2.2-alpha,
      and 0.2.2.28-beta.

  o Minor bugfixes (heartbeat):
    - If we fail to write a heartbeat message, schedule a retry for the
      minimum heartbeat interval number of seconds in the future. Fixes
      bug 19476; bugfix on 0.2.3.1-alpha.

  o Minor bugfixes (linux seccomp2 sandbox, logging):
    - Fix some messages on unexpected errors from the seccomp2 library.
      Fixes bug 22750; bugfix on 0.2.5.1-alpha. Patch from "cypherpunks".

  o Minor bugfixes (logging):
    - Remove duplicate log messages regarding opening non-local
      SocksPorts upon parsing config and opening listeners at startup.
      Fixes bug 4019; bugfix on 0.2.3.3-alpha.
    - Use a more comprehensible log message when telling the user
      they've excluded every running exit node. Fixes bug 7890; bugfix
      on 0.2.2.25-alpha.
    - When logging the number of descriptors we intend to download per
      directory request, do not log a number higher than then the number
      of descriptors we're fetching in total. Fixes bug 19648; bugfix
      on 0.1.1.8-alpha.
    - When warning about a directory owned by the wrong user, log the
      actual name of the user owning the directory. Previously, we'd log
      the name of the process owner twice. Fixes bug 23487; bugfix
      on 0.2.9.1-alpha.
    - The tor specification says hop counts are 1-based, so fix two log
      messages that mistakenly logged 0-based hop counts. Fixes bug
      18982; bugfix on 0.2.6.2-alpha and 0.2.4.5-alpha. Patch by teor.
      Credit to Xiaofan Li for reporting this issue.

  o Minor bugfixes (portability):
    - Stop using the PATH_MAX variable, which is not defined on GNU
      Hurd. Fixes bug 23098; bugfix on 0.3.1.1-alpha.

  o Minor bugfixes (relay):
    - When uploading our descriptor for the first time after startup,
      report the reason for uploading as "Tor just started" rather than
      leaving it blank. Fixes bug 22885; bugfix on 0.2.3.4-alpha.
    - Avoid unnecessary calls to directory_fetches_from_authorities() on
      relays, to prevent spurious address resolutions and descriptor
      rebuilds. This is a mitigation for bug 21789. Fixes bug 23470;
      bugfix on in 0.2.8.1-alpha.

  o Minor bugfixes (tests):
    - Fix a broken unit test for the OutboundAddress option: the parsing
      function was never returning an error on failure. Fixes bug 23366;
      bugfix on 0.3.0.3-alpha.
    - Fix a signed-integer overflow in the unit tests for
      dir/download_status_random_backoff, which was untriggered until we
      fixed bug 17750. Fixes bug 22924; bugfix on 0.2.9.1-alpha.

  o Minor bugfixes (usability, control port):
    - Stop making an unnecessary routerlist check in NETINFO clock skew
      detection; this was preventing clients from reporting NETINFO clock
      skew to controllers. Fixes bug 23532; bugfix on 0.2.4.4-alpha.

  o Code simplification and refactoring:
    - Extract the code for handling newly-open channels into a separate
      function from the general code to handle channel state
      transitions. This change simplifies our callgraph, reducing the
      size of the largest strongly connected component by roughly a
      factor of two. Closes ticket 22608.
    - Remove dead code for largely unused statistics on the number of
      times we've attempted various public key operations. Fixes bug
      19871; bugfix on 0.1.2.4-alpha. Fix by Isis Lovecruft.
    - Remove several now-obsolete functions for asking about old
      variants directory authority status. Closes ticket 22311; patch
      from "huyvq".
    - Remove some of the code that once supported "Named" and "Unnamed"
      routers. Authorities no longer vote for these flags. Closes
      ticket 22215.
    - Rename the obsolete malleable hybrid_encrypt functions used in TAP
      and old hidden services, to indicate that they aren't suitable for
      new protocols or formats. Closes ticket 23026.
    - Replace our STRUCT_OFFSET() macro with offsetof(). Closes ticket
      22521. Patch from Neel Chauhan.
    - Split the enormous circuit_send_next_onion_skin() function into
      multiple subfunctions. Closes ticket 22804.
    - Split the portions of the buffer.c module that handle particular
      protocols into separate modules. Part of ticket 23149.
    - Use our test macros more consistently, to produce more useful
      error messages when our unit tests fail. Add coccinelle patches to
      allow us to re-check for test macro uses. Closes ticket 22497.

  o Deprecated features:
    - Deprecate HTTPProxy/HTTPProxyAuthenticator config options. They
      only applies to direct unencrypted HTTP connections to your
      directory server, which your Tor probably isn't using. Closes
      ticket 20575.

  o Documentation:
    - Clarify in the manual that "Sandbox 1" is only supported on Linux
      kernels. Closes ticket 22677.
    - Document all values of PublishServerDescriptor in the manpage.
      Closes ticket 15645.
    - Improve the documentation for the directory port part of the
      DirAuthority line. Closes ticket 20152.
    - Restore documentation for the authorities' "approved-routers"
      file. Closes ticket 21148.

  o Removed features:
    - The AllowDotExit option has been removed as unsafe. It has been
      deprecated since 0.2.9.2-alpha. Closes ticket 23426.
    - The ClientDNSRejectInternalAddresses flag can no longer be set on
      non-testing networks. It has been deprecated since 0.2.9.2-alpha.
      Closes ticket 21031.
    - The controller API no longer includes an AUTHDIR_NEWDESCS event:
      nobody was using it any longer. Closes ticket 22377.