Proposal 162: Publish the consensus in multiple flavors

[Roger Dingledine]

On Fri, May 15, 2009 at 01:05:41PM -0400, Nick Mathewson wrote:
> Filename: 162-consensus-flavors.txt
> Title: Publish the consensus in multiple flavors

Looks good. We should do it.

>    Our past approach to cases like this has been to shovel all of
>    the data into the consensus document.  But this is rather poor
>    for bandwidth.  Adding a single SHA256 hash to a consensus for
>    each router increases the compressed consensus size by 47%.  In
>    comparison, replacing a single SHA1 hash with a SHA256 hash for
>    each listed router increases the consensus size by only 18%.

SHA256's are still huge. It's a real shame there aren't accepted hash
functions that use only 20 bytes.

>    In addition to the consensus currently served at
>    /tor/status-vote/(current|next)/consensus.z ,

Are we expecting to make this url deprecated at some point? I guess not
until 0.2.1.x is obsolete, which is a long time from now.

In the mean time, do we really want to force caches to mirror the
old-school consensus URL, and also the "ns" consensus flavor URL, even
though they'll always be the same document? Seems to me that we could
start out with only one consensus flavor, the "microdesc" one.

>     The algname part of a signature describes what algorithm was
>     used to hash the identity and signing keys, and to compute the
>     signature.  The algorithm "sha256" MUST be recognized;
>     signatures with unrecognized algorithms MUST be ignored.
>     (See below).

I'm still curious about Sebastian's question here. I guess the
plan is to use sha256 sigs for now, and if we decide we don't like
that hash function, we a) make up a new consensus flavor that hashes
microdescriptors with whirlpool++, b) get all the authorities to upgrade
to a consensus method that agrees to build that flavor and list the
new flavor in the consensus-index file, hashed by whirlpool++ as well
as sha256, and c) teach clients to fetch the new flavor and make sure
the ... make sure the what? Is there anything in the consensus itself
that shows what digest the authority used when signing? It seems like we
want to make sure the "Signature*" lines from the consensus-index get
used in the appropriate consensus flavor too.

More generally, at some point we should specify our upgrade path plan,
so we're less likely to learn later that it has holes.

Also, in that case, caches that haven't upgraded can still be tricked
by an authority into fetching a colliding consensus flavor document,
since they'd only be checking the sha256. That's not so bad since clients
should be able to check the hashes themselves. In theory it's a DoS worry,
since an attacker could make it hard to find a copy with the right valid
hashes and signatures; but in practice I don't see an authority actually
being able to pull off that attack without somebody noticing and fixing
it out of band. And we're only vulnerable to an authority doing it,
right (assuming honest but not-upgraded relays, and nobody mitm'ing the
connection from relay to authority)?

>     The consensus index is made available at
>        /tor/status-vote/(current|next)/consensus-index.z.

You should be aware that due to a bug in the current code, that url is
valid already:
128.31.0.34:9031/tor/status-vote/current/consensus-index

Not a huge problem though, I think.

>     Caches should fetch this document so they can check the
>     correctness of the different consensus documents they fetch.
>     They do not need to check anything about an unrecognized
>     consensus document beyond its digest.

and length.

This proposal should definitely be targeted for 0.2.2, so caches can
start caching flavors.

Thanks!
--Roger