[tor-talk] Tor 0.3.4.1-alpha: source code now released

Nick Mathewson nickm at torproject.org
Thu May 17 13:39:59 UTC 2018


Hi, all!

There's a new alpha Tor release!  Because it's an alpha, you should
only run it if you're ready to find more bugs than usual, and report
them on trac.torproject.org.

The source code is available from the usual place on
www.torproject.org; if you build Tor from source, why not give it a
try?  And if you don't build Tor from source, packages should be ready
over the coming days, with a Tor Browser alpha release likely some
time in the coming weeks.

(And no, we haven't forgotten about the 0.3.3.x series -- there should
be a stable release out there "really soon now".)

Here's what's new since 0.3.3.5-rc!

Changes in version 0.3.4.1-alpha - 2018-05-17
  Tor 0.3.4.1-alpha is the first release in the 0.3.4.x series. It
  includes refactoring to begin reducing Tor's binary size and idle CPU
  usage on mobile, along with prep work for new bandwidth scanners,
  improvements to the experimental "vanguards" feature, and numerous
  other small features and bugfixes.

  o New system requirements:
    - Tor no longer tries to support old operating systems without
      mmap() or some local equivalent. Apparently, compilation on such
      systems has been broken for some time, without anybody noticing or
      complaining. Closes ticket 25398.

  o Major feature (directory authority, modularization):
    - The directory authority subsystem has been modularized. The code
      is now located in src/or/dirauth/, and is compiled in by default.
      To disable the module, the configure option
      --disable-module-dirauth has been added. This module may be
      disabled by default in some future release. Closes ticket 25610.

  o Major features (main loop, CPU usage):
    - When Tor is disabled (via DisableNetwork or via hibernation), it
      no longer needs to run any per-second events. This change should
      make it easier for mobile applications to disable Tor while the
      device is sleeping, or Tor is not running. Closes ticket 26063.
    - Tor no longer enables all of its periodic events by default.
      Previously, Tor would enable all possible main loop events,
      regardless of whether it needed them. Furthermore, many of these
      events are now disabled with Tor is hibernating or DisableNetwork
      is set. This is a big step towards reducing client CPU usage by
      reducing the amount of wake-ups the daemon does. Closes ticket
      25376 and 25762.
    - The bandwidth-limitation logic has been refactored so that
      bandwidth calculations are performed on-demand, rather than every
      TokenBucketRefillInterval milliseconds. This change should improve
      the granularity of our bandwidth calculations, and limit the
      number of times that the Tor process needs to wake up when it is
      idle. Closes ticket 25373.
    - Move responsibility for many operations from a once-per-second
      callback to a callback that is only scheduled as needed. Moving
      this functionality has allowed us to disable the callback when
      Tor's network is disabled. Once enough items are removed from our
      once-per-second callback, we can eliminate it entirely to conserve
      CPU when idle. The functionality removed includes: closing
      connections, circuits, and channels (ticket 25932); consensus
      voting (25937); flushing log callbacks (25951); honoring delayed
      SIGNEWNYM requests (25949); rescanning the consensus cache
      (25931); saving the state file to disk (25948); warning relay
      operators about unreachable ports (25952); and keeping track of
      Tor's uptime (26009).

  o Major bugfixes (directory authorities, security):
    - When directory authorities read a zero-byte bandwidth file, they
      would previously log a warning with the contents of an
      uninitialised buffer. They now log a warning about the empty file
      instead. Fixes bug 26007; bugfix on 0.2.2.1-alpha.

  o Major bugfixes (crash):
    - Avoid a rare assertion failure in the circuit build timeout code
      if we fail to allow any circuits to actually complete. Fixes bug
      25733; bugfix on 0.2.2.2-alpha.

  o Major bugfixes (directory authority):
    - Avoid a crash when testing router reachability on a router that
      could have an ed25519 ID, but which does not. Fixes bug 25415;
      bugfix on 0.3.3.2-alpha.

  o Major bugfixes (onion service):
    - Correctly detect when onion services get disabled after HUP. Fixes
      bug 25761; bugfix on 0.3.2.1.

  o Major bugfixes (protover, voting):
    - Revise Rust implementation of protover to use a more memory-
      efficient voting algorithm and corresponding data structures, thus
      avoiding a potential (but small impact) DoS attack where specially
      crafted protocol strings would expand to several potential
      megabytes in memory. In the process, several portions of code were
      revised to be methods on new, custom types, rather than functions
      taking interchangeable types, thus increasing type safety of the
      module. Custom error types and handling were added as well, in
      order to facilitate better error dismissal/handling in outside
      crates and avoid mistakenly passing an internal error string to C
      over the FFI boundary. Many tests were added, and some previous
      differences between the C and Rust implementations have been
      remedied. Fixes bug 24031; bugfix on 0.3.3.1-alpha.

  o Major bugfixes (relay, denial of service):
    - Impose a limit on circuit cell queue size. The limit can be
      controlled by a consensus parameter. Fixes bug 25226; bugfix
      on 0.2.4.14-alpha.

  o Minor features (accounting):
    - When Tor becomes dormant, it now uses a scheduled event to wake up
      at the right time. Previously, we would use the per-second timer
      to check whether to wake up, but we no longer have any per-second
      timers enabled when the network is disabled. Closes ticket 26064.

  o Minor features (code quality):
    - Add optional spell-checking for the Tor codebase, using the
      "misspell" program. To use this feature, run "make check-typos".
      Closes ticket 25024.

  o Minor features (compatibility):
    - Tor now detects versions of OpenSSL 1.1.0 and later compiled with
      the no-deprecated option, and builds correctly with them. Closes
      tickets 19429, 19981, and 25353.
    - Avoid some compilation warnings with recent versions of LibreSSL.
      Closes ticket 26006.

  o Minor features (compression, zstd):
    - When running with zstd, Tor now considers using advanced functions
      that the zstd maintainers have labeled as potentially unstable. To
      prevent breakage, Tor will only use this functionality when the
      runtime version of the zstd library matches the version with which
      Tor was compiled. Closes ticket 25162.

  o Minor features (configuration):
    - The "DownloadSchedule" options have been renamed to end with
      "DownloadInitialDelay". The old names are still allowed, but will
      produce a warning. Comma-separated lists are still permitted for
      these options, but all values after the first are ignored (as they
      have been since 0.2.9). Closes ticket 23354.

  o Minor features (continuous integration):
    - Our .travis.yml configuration now includes support for testing the
      results of "make distcheck". (It's not uncommon for "make check"
      to pass but "make distcheck" to fail.) Closes ticket 25814.
    - Our Travis CI configuration now integrates with the Coveralls
      coverage analysis tool. Closes ticket 25818.

  o Minor features (control port):
    - Introduce GETINFO "current-time/{local,utc}" to return the local
      and UTC times respectively in ISO format. This helps a controller
      like Tor Browser detect a time-related error. Closes ticket 25511.
      Patch by Neel Chauhan.
    - Introduce new fields to the CIRC_BW event. There are two new
      fields in each of the read and written directions. The DELIVERED
      fields report the total valid data on the circuit, as measured by
      the payload sizes of verified and error-checked relay command
      cells. The OVERHEAD fields report the total unused bytes in each
      of these cells. Closes ticket 25903.

  o Minor features (directory authority):
    - Directory authorities now open their key-pinning files as O_SYNC,
      to limit their chances of accidentally writing partial lines.
      Closes ticket 23909.

  o Minor features (directory authority, forward compatibility):
    - Make the lines of the measured bandwidth file able to contain
      their entries in any order. Previously, the node_id entry needed
      to come first. Closes ticket 26004.

  o Minor features (entry guards):
    - Introduce a new torrc option NumPrimaryGuards for controlling the
      number of primary guards. Closes ticket 25843.

  o Minor features (geoip):
    - Update geoip and geoip6 to the May 1 2018 Maxmind GeoLite2 Country
      database. Closes ticket 26104.

  o Minor features (performance):
    - Avoid a needless call to malloc() when processing an incoming
      relay cell. Closes ticket 24914.
    - Make our timing-wheel code run a tiny bit faster on 32-bit
      platforms, by preferring 32-bit math to 64-bit. Closes
      ticket 24688.
    - Avoid a needless malloc()/free() pair every time we handle an ntor
      handshake. Closes ticket 25150.

  o Minor features (testing):
    - Add a unit test for voting_schedule_get_start_of_next_interval().
      Closes ticket 26014, and helps make unit test coverage
      more deterministic.
    - A new unittests module specifically for testing the functions in
      the (new-ish) bridges.c module has been created with new
      unittests, raising the code coverage percentages. Closes 25425.
    - We now have improved testing for addressmap_get_virtual_address()
      function. This should improve our test coverage, and make our test
      coverage more deterministic. Closes ticket 25993.

  o Minor features (timekeeping, circuit scheduling):
    - When keeping track of how busy each circuit have been recently on
      a given connection, use coarse-grained monotonic timers rather
      than gettimeofday(). This change should marginally increase
      accuracy and performance. Implements part of ticket 25927.

  o Minor bugfixes (bandwidth management):
    - Consider ourselves "low on write bandwidth" if we have exhausted
      our write bandwidth some time in the last second. This was the
      documented behavior before, but the actual behavior was to change
      this value every TokenBucketRefillInterval. Fixes bug 25828;
      bugfix on 0.2.3.5-alpha.

  o Minor bugfixes (C correctness):
    - Add a missing lock acquisition in the shutdown code of the control
      subsystem. Fixes bug 25675; bugfix on 0.2.7.3-rc. Found by
      Coverity; this is CID 1433643.

  o Minor bugfixes (circuit path selection):
    - Don't count path selection failures as circuit build failures.
      This change should eliminate cases where Tor blames its guard or
      the network for situations like insufficient microdescriptors
      and/or overly restrictive torrc settings. Fixes bug 25705; bugfix
      on 0.3.3.1-alpha.

  o Minor bugfixes (client):
    - Don't consider Tor running as a client if the ControlPort is open,
      but no actual client ports are open. Fixes bug 26062; bugfix
      on 0.2.9.4-alpha.

  o Minor bugfixes (code style):
    - Fixed multiple includes of transports.h in src/or/connection.c
      Fixes bug 25261; bugfix on 0.2.5.1-alpha.
    - Remove the unused variable n_possible from the function
      channel_get_for_extend(). Fixes bug 25645; bugfix on 0.2.4.4-alpha

  o Minor bugfixes (control interface):
    - Respond with more human-readable error messages to GETINFO exit-
      policy/* requests. Also, let controller know if an error is
      transient (response code 551) or not (response code 552). Fixes
      bug 25852; bugfix on 0.2.8.1-alpha.

  o Minor bugfixes (controller):
    - Make CIRC_BW event reflect the total of all data sent on a
      circuit, including padding and dropped cells. Also fix a mis-
      counting bug when STREAM_BW events were enabled. Fixes bug 25400;
      bugfix on 0.2.5.2-alpha.

  o Minor bugfixes (directory client):
    - When unverified-consensus is verified, rename it to cached-
      consenus. Fixes bug 4187; bugfix on 0.2.0.3-alpha.
    - Fixed launching a certificate fetch always during the scheduled
      periodic consensus fetch by fetching only in those cases when
      consensus are waiting for certs. Fixes bug 24740; bugfix
      on 0.2.9.1-alpha.

  o Minor bugfixes (documentation):
    - Stop saying in the manual that clients cache ipv4 dns answers from
      exit relays. We haven't used them since 0.2.6.3-alpha, and in
      ticket 24050 we stopped even caching them as of 0.3.2.6-alpha, but
      we forgot to say so in the man page. Fixes bug 26052; bugfix
      on 0.3.2.6-alpha.

  o Minor bugfixes (error reporting):
    - Improve tolerance for directory authorities with skewed clocks.
      Previously, an authority with a clock more than 60 seconds ahead
      could cause a client with a correct clock to warn that the
      client's clock was behind. Now the clocks of a majority of
      directory authorities have to be ahead of the client before this
      warning will occur. Fixes bug 25756; bugfix on 0.2.2.25-alpha.

  o Minor bugfixes (Linux seccomp2 sandbox):
    - Allow the nanosleep() system call, which glibc uses to implement
      sleep() and usleep(). Fixes bug 24969; bugfix on 0.2.5.1-alpha.

  o Minor bugfixes (onion service):
    - Fix a memory leak when a v3 onion service is configured and gets a
      SIGHUP signal. Fixes bug 25901; bugfix on 0.3.2.1-alpha.
    - When parsing the descriptor signature, look for the token plus an
      extra white-space at the end. This is more correct but also will
      allow us to support new fields that might start with "signature".
      Fixes bug 26069; bugfix on 0.3.0.1-alpha.

  o Minor bugfixes (path selection):
    - Only select relays when they have the descriptors we prefer to use
      for them. This change fixes a bug where we could select a relay
      because it had _some_ descriptor, but reject it later with a
      nonfatal assertion error because it didn't have the exact one we
      wanted. Fixes bugs 25691 and 25692; bugfix on 0.3.3.4-alpha.

  o Minor bugfixes (portability):
    - Do not align mmap length, as it is not required by POSIX, and the
      getpagesize function is deprecated. Fixes bug 25399; bugfix
      on 0.1.1.23.

  o Minor bugfixes (portability, FreeBSD):
    - In have_enough_mem_for_dircache(), the variable DIRCACHE_MIN_MEM_MB
      does not stringify on FreeBSD, so we switch to tor_asprintf().
      Fixes bug 20887; bugfix on 0.2.8.1-alpha. Patch by Neel Chauhan.

  o Minor bugfixes (relay statistics):
    - When a relay is collecting internal statistics about how many
      create cell requests it has seen of each type, accurately count
      the requests from relays that temporarily fall out of the
      consensus. (To be extra conservative, we were already ignoring
      requests from clients in our counts, and we continue ignoring them
      here.) Fixes bug 24910; bugfix on 0.2.4.17-rc.

  o Minor bugfixes (relay, crash):
    - Avoid a crash when running with DirPort set but ORPort tuned off.
      Fixes a case of bug 23693; bugfix on 0.3.1.1-alpha.

  o Minor bugfixes (restart-in-process):
    - When shutting down, Tor now clears all the flags in the control.c
      module. This should prevent a bug where authentication cookies are
      not generated on restart. Fixes bug 25512; bugfix on 0.3.3.1-alpha.

  o Minor bugfixes (testing):
    - When testing workqueue event-cancellation, make sure that we
      actually cancel an event, and that cancel each event with equal
      probability. (It was previously possible, though extremely
      unlikely, for our event-canceling test not to cancel any events.)
      Fixes bug 26008; bugfix on 0.2.6.3-alpha.
    - Repeat part of the test in test_client_pick_intro() a number of
      times, to give it consistent coverage. Fixes bug 25996; bugfix
      on 0.3.2.1-alpha.
    - Remove randomness from the hs_common/responsible_hsdirs test, so
      that it always takes the same path through the function it tests.
      Fixes bug 25997; bugfix on 0.3.2.1-alpha.
    - Change the behavior of the "channel/outbound" test so that it
      never causes a 10-second rollover for the EWMA circuitmux code.
      Previously, this behavior would happen randomly, and result in
      fluctuating test coverage. Fixes bug 25994; bugfix
      on 0.3.3.1-alpha.
    - Use X509_new() to allocate certificates that will be freed later
      with X509_free(). Previously, some parts of the unit tests had
      used tor_malloc_zero(), which is incorrect, and which caused test
      failures on Windows when they were built with extra hardening.
      Fixes bugs 25943 and 25944; bugfix on 0.2.8.1-alpha. Patch by
      Marcin Cieślak.
    - While running the circuit_timeout test, fix the PRNG to a
      deterministic AES stream, so that the test coverage from this test
      will itself be deterministic. Fixes bug 25995; bugfix
      on 0.2.2.2-alpha.

  o Minor bugfixes (vanguards):
    - Allow the last hop in a vanguard circuit to be the same as our
      first, to prevent the adversary from influencing guard node choice
      by choice of last hop. Also prevent the creation of A - B - A
      paths, or A - A paths, which are forbidden by relays. Fixes bug
      25870; bugfix on 0.3.3.1-alpha.

  o Code simplification and refactoring:
    - Remove duplicate code in parse_{c,s}method_line and bootstrap
      their functionalities into a single function. Fixes bug 6236;
      bugfix on 0.2.3.6-alpha.
    - We remove the PortForwsrding and PortForwardingHelper options,
      related functions, and the port_forwarding tests. These options
      were used by the now-deprecated Vidalia to help ordinary users
      become Tor relays or bridges. Closes ticket 25409. Patch by
      Neel Chauhan.
    - In order to make the OR and dir checking function in router.c less
      confusing we renamed some functions and
      consider_testing_reachability() has been split into
      router_should_check_reachability() and
      router_do_reachability_checks(). Also we improved the documentation
      in some functions. Closes ticket 18918.
    - Initial work to isolate Libevent usage to a handful of modules in
      our codebase, to simplify our call structure, and so that we can
      more easily change event loops in the future if needed. Closes
      ticket 23750.
    - Introduce a function to call getsockname() and return tor_addr_t,
      to save a little complexity throughout the codebase. Closes
      ticket 18105.
    - Make hsdir_index in node_t a hsdir_index_t rather than a pointer
      as hsdir_index is always present. Also, we move hsdir_index_t into
      or.h. Closes ticket 23094. Patch by Neel Chauhan.
    - Merge functions used for describing nodes and suppress the
      functions that do not allocate memory for the output buffer
      string. NODE_DESC_BUF_LEN constant and format_node_description()
      function cannot be used externally from router.c module anymore.
      Closes ticket 25432. Patch by valentecaio.
    - Our main loop has been simplified so that all important operations
      happen inside events. Previously, some operations had to happen
      outside the event loop, to prevent infinite sequences of event
      activations. Closes ticket 25374.
    - Put a SHA1 public key digest in hs_service_intro_point_t, and use
      it in register_intro_circ() and service_intro_point_new(). This
      prevents the digest from being re-calculated each time. Closes
      ticket 23107. Patch by Neel Chauhan.
    - Refactor token-bucket implementations to use a common backend.
      Closes ticket 25766.
    - Remove extern declaration of stats_n_seconds_working variable from
      main, protecting its accesses with get_uptime() and reset_uptime()
      functions. Closes ticket 25081, patch by “valentecaio”.
    - Remove our previous logic for "cached gettimeofday()" -- our
      coarse monotonic timers are fast enough for this purpose, and far
      less error-prone. Implements part of ticket 25927.
    - Remove the return value for fascist_firewall_choose_address_base(),
      and sister functions such as fascist_firewall_choose_address_node()
      and fascist_firewall_choose_address_rs(). Also, while we're here,
      initialize the ap argument as leaving it uninitialized can pose a
      security hazard. Closes ticket 24734. Patch by Neel Chauhan.
    - Rename two fields of connection_t struct. timestamp_lastwritten is
      renamed to timestamp_last_write_allowed and timestamp_lastread is
      renamed to timestamp_last_read_allowed. Closes ticket 24714, patch
      by "valentecaio".
    - Since Tor requires C99, remove our old workaround code for libc
      implementations where free(NULL) doesn't work. Closes ticket 24484.
    - Use our standard rate-limiting code to deal with excessive
      libevent failures, rather than the hand-rolled logic we had
      before. Closes ticket 26016.
    - We remove the return value of node_get_prim_orport() and
      node_get_prim_dirport(), and introduce node_get_prim_orport() in
      node_ipv6_or_preferred() and node_ipv6_dir_preferred() in order to
      check for a null address. Closes ticket 23873. Patch by
      Neel Chauhan.
    - We switch to should_record_bridge_info() in
      geoip_note_client_seen() and options_need_geoip_info() instead of
      accessing the configuration values directly. Fixes bug 25290;
      bugfix on 0.2.1.6-alpha. Patch by Neel Chauhan.

  o Deprecated features:
    - As we are not recommending 0.2.5 anymore, we require relays that
      once had an ed25519 key associated with their RSA key to always
      have that key, instead of allowing them to drop back to a version
      that didn't support ed25519. This means they need to use a new RSA
      key if the want to downgrade to an older version of tor without
      ed25519. Closes ticket 20522.

  o Documentation:
    - Correct an IPv6 error in the documentation for ExitPolicy. Closes
      ticket 25857. Patch from "CTassisF".

  o Removed features:
    - Directory authorities will no longer support voting according to
      any consensus method before consensus method 25. This keeps
      authorities compatible with all authorities running 0.2.9.8 and
      later, and does not break any clients or relays. Implements ticket
      24378 and proposal 290.
    - The PortForwarding and PortForwardingHelper features have been
      removed. The reasoning is, given that implementations of NAT
      traversal protocols within common consumer grade routers are
      frequently buggy, and that the target audience for a NAT punching
      feature is a perhaps less-technically-inclined relay operator,
      when the helper fails to setup traversal the problems are usually
      deep, ugly, and very router specific, making them horrendously
      impossible for technical support to reliable assist with, and thus
      resulting in frustration all around. Unfortunately, relay
      operators who would like to run relays behind NATs will need to
      become more familiar with the port forwarding configurations on
      their local router. Closes 25409.
    - The TestingEnableTbEmptyEvent option has been removed. It was used
      in testing simulations to measure how often connection buckets
      were emptied, in order to improve our scheduling, but it has not
      been actively used in years. Closes ticket 25760.
    - The old "round-robin" circuit multiplexer (circuitmux)
      implementation has been removed, along with a fairly large set of
      code that existed to support it. It has not been the default
      circuitmux since we introduced the "EWMA" circuitmux in 0.2.4.x,
      but it still required an unreasonable amount of memory and CPU.
      Closes ticket 25268.


More information about the tor-talk mailing list