On 5/7/12 7:49 PM, Beck Chen wrote:

On Mon, May 7, 2012 at 5:13 PM, Karsten Loesing <karsten@torproject.org>wrote:

...

On 5/6/12 3:36 AM, Damian Johnson wrote:

...

First I'd like to make sure that I'm clear on what we're trying to do. The javadocs for VerifyDescriptors [1] says that it...

...

Verify server descriptors using the contained signing key. Verify that 1) a contained fingerprint is actually a hash of the signing key and 2) a router signature was created using the signing key.

Verify consensuses using the separate certs. Verify that 1) the fingerprint in a cert is actually a hash of the identity key, 2) a cert was signed using the identity key, 3) a consensus was signed using the signing key from the cert.

Honestly I'm not yet sure what most of this means. The first #2 is simply checking that the descriptor content can be verified using the router-signature and signing-key, right?

Yup. But you also need the first #1, or metrics-db could modify server descriptors at will, put in its own key, and re-sign the descriptor with that key, and stem would think everything's valid.

Then is it possible that metrics-db modifies the fingerprint at the same time so that the first #1 would pass?

Right, but when the fingerprint changes, we consider the descriptor as coming from a different relay. If metrics-db wanted to fake descriptors of a known relay and stem only checked #2, stem would be fooled here. With #1 it wouldn't. Note that for most analyses you'd start with the consensus and only look at referenced descriptors by their SHA digest, which rules out modifications like the one discussed above. But if there's a method in the server descriptor class to check whether it is valid it should check #1 and #2. Robert may have a more complete list of things that need to be checked in descriptors of all sorts. I just asked him in #2768 for such a list.

Any why would metrics-db try to do this at all?

Well, it wouldn't. But why trust it when you can check yourself? Best, Karsten

On 5/8/12, Beck Chen <csybeck@gmail.com> wrote:

According to the outline, the long-term identity key should be different from the signing key, which changes every 3-12 months. Then why should the signing key become the identity key in the descriptor format, and fingerprint become the hash of the identity key?

The ‘relay identity key’ is not the same as the ‘authority identity key’. The ‘relay identity key’ might also be different from the ‘directory signing key’; I'm not sure about that. Descriptors contain and are signed with the ‘relay identity key’, and the fingerprint in a descriptor is the hash of the relay identity key. Robert Ransom

Hi Karsten & Damian, I've been trying to port function determineKeyHash() from VerifyDescriptors.java to stem, and it turned out to be more complicated than I thought due to my unfamiliarity to many cryptography terms and standards. I finally figured out a way to do it by using a light-weighted library called Python-RSA, and it solved my problem in no more than 5 lines of code. I realized that I should get better understanding of some basic stuff before I move on. Here's the python code that verifies the key hash:

...

import rsa, hashlib pubkey = rsa.PublicKey.load_pkcs1(desc.signing_key) der_encoded = pubkey.save_pkcs1(format = "DER") hashlib.sha1(der_encoded).hexdigest() == desc.fingerprint.lower() True

1. Specs says signing key is "a public key in PEM format" [1], but what standard does it use? I tried to use M2Crypto, a Python wrapper for OpenSSL, to import this key, but failed. Then I found out that OpenSSL uses X.509 for its public keys, thus M2Crypto only supports X.509 public keys. Then I looked at the Java code for determineKeyHash() and it suggests that the key uses PKCS standard. It seems that PKCS#1 is the final answer, but the javadoc page of PEMReader in BouncyCastle says it can read "OpenSSL PEM encoded streams containing X509 certificates, PKCS8 encoded keys and PKCS7 objects" [2]. So is it PKCS#1 or PKCS#8? And what's the difference? 2. Specs says fingerprint is "a HASH_LEN-byte of asn1 encoded public key, encoded in hex" [1]. But to me, it seems to be "a SHA1 digest of DER encoded public key, encoded in hex". Specifically, is it necessary to specify the length of fingerprint? And ASN1 is a standard with several possible encoding rules, do we always use DER? I know that's a lot of questions...thanks! Best, Beck PS. Are you guys getting email updates from ticket #5810? I'm doubting that you two are not actually CCed since no one is replying... PSS. Do we need to keep Ravi in this conversation? PSSS. Should I wrap up my email to 78 words per line? [1]. https://gitweb.torproject.org/torspec.git?a=blob_plain;hb=HEAD;f=dir-spec.tx... [2]. http://www.bouncycastle.org/docs/pkixdocs1.5on/org/bouncycastle/openssl/PEMR...

Hi Beck. Unfortunately at this point you know far more about descriptor crypto than me so I won't be much help. A quick search in apt for 'pkcs' only came up with one python result, PyKCS11 (http://www.bit4id.org/trac/pykcs11), which looks to be mostly related to smartcards. Hopefully Karsten will have some suggestions.

PS. Are you guys getting email updates from ticket #5810? I'm doubting that you two are not actually CCed since no one is replying...

Yes. Both Karsten and I are subscribed to tor-bugs@ so we see all trac updates.

PSS. Do we need to keep Ravi in this conversation?

Nope, everyone on this thread is on tor-dev@ already. If you aren't on this list yourself then I'd suggest it. :)

PSSS. Should I wrap up my email to 78 words per line?

Really doesn't matter. Some people care strongly about it, others don't. I'm in the later camp but do it anyway since I think it looks a little nicer and gmail defaults to 80-character wrap when you chose plaintext. Cheers! -Damian

It seems that PKCS#1 is the final answer, but

Quick question, if it is PKCS1 that we need then will the pycrypto package do the trick? It looks like PKCS1 support is pretty new for it, but the package itself is generally available by default. See the last comment on... https://bugs.launchpad.net/pycrypto/+bug/502291 Cheers! -Damian

On 5/14/12 2:30 AM, Damian Johnson wrote:

Hi Karsten, hi Beck. Just pushed stem support for extrainfo descriptors (ye gods they have a lot of attributes)...

* Implementation https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/descriptor/extrainfo_...

* Unit Tests https://gitweb.torproject.org/stem.git/blob/HEAD:/test/unit/descriptor/extra...

* Integ Tests https://gitweb.torproject.org/stem.git/blob/HEAD:/test/integ/descriptor/extr...

Barring distractions I plan to next implement network status entities, then start porting over Onionoo. That said, between going oncall for work and the upcoming release of Diablo 3 it might be a little while. :P

Pretty cool! Very much looking forward to the Python Onionoo! Let me know when/if you want to discuss the design there. Best, Karsten

Hi Beck, I don't have good answers to your questions. To be honest, when I implemented the Java verification code for #2768, I looked for hints in an old Java version of Tor, rewrote that code, updated it for current BouncyCastle versions using their JavaDocs and examples, and tweaked everything until it finally worked. :) On 5/13/12 9:04 AM, Beck Chen wrote:

1. Specs says signing key is "a public key in PEM format" [1], but what standard does it use? I tried to use M2Crypto, a Python wrapper for OpenSSL, to import this key, but failed. Then I found out that OpenSSL uses X.509 for its public keys, thus M2Crypto only supports X.509 public keys. Then I looked at the Java code for determineKeyHash() and it suggests that the key uses PKCS standard. It seems that PKCS#1 is the final answer, but the javadoc page of PEMReader in BouncyCastle says it can read "OpenSSL PEM encoded streams containing X509 certificates, PKCS8 encoded keys and PKCS7 objects" [2]. So is it PKCS#1 or PKCS#8? And what's the difference?

2. Specs says fingerprint is "a HASH_LEN-byte of asn1 encoded public key, encoded in hex" [1]. But to me, it seems to be "a SHA1 digest of DER encoded public key, encoded in hex". Specifically, is it necessary to specify the length of fingerprint? And ASN1 is a standard with several possible encoding rules, do we always use DER?

I'm cc'ing Nick who would have much better answers to your questions. Thanks! Karsten

On Mon, May 14, 2012 at 2:04 AM, Karsten Loesing <karsten@torproject.org> wrote:

Hi Beck,

I don't have good answers to your questions.  To be honest, when I implemented the Java verification code for #2768, I looked for hints in an old Java version of Tor, rewrote that code, updated it for current BouncyCastle versions using their JavaDocs and examples, and tweaked everything until it finally worked. :)

On 5/13/12 9:04 AM, Beck Chen wrote:

...

1. Specs says signing key is "a public key in PEM format" [1], but what standard does it use? I tried to use M2Crypto, a Python wrapper for OpenSSL, to import this key, but failed. Then I found out that OpenSSL uses X.509 for its public keys, thus M2Crypto only supports X.509 public keys. Then I looked at the Java code for determineKeyHash() and it suggests that the key uses PKCS standard. It seems that PKCS#1 is the final answer, but the javadoc page of PEMReader in BouncyCastle says it can read "OpenSSL PEM encoded streams containing X509 certificates, PKCS8 encoded keys and PKCS7 objects" [2]. So is it PKCS#1 or PKCS#8?

In crypto_pk_write_to_string_impl, it's generated via PEM_write_bio_RSAPublicKey, so (according the manpage) that's a PKCS#1 RSAPublicKey structure. Let's hope the mangpage is right.

...

And what's the difference?

I'd have to read the standards to find out; if the standards don't make that clear, let me know and I can give it a try myself.

...

2. Specs says fingerprint is "a HASH_LEN-byte of asn1 encoded public key, encoded in hex" [1]. But to me, it seems to be "a SHA1 digest of DER encoded public key, encoded in hex". Specifically, is it necessary to specify the length of fingerprint?

That should indeed say say "a HASH_LEN-byte SHA1 digest of".... "digest" is the part that's strictly necessary to specify, since I think we say that everything's SHA1 unless somebody says otherwise.

...

And ASN1 is a standard with several possible encoding rules, do we always use DER?

I believe so. -- Nick

Hi Damian, I was actually on a short trip these days, and just arrived home. I'll restart working on it tomorrow, and I believe there is more I can do. That said I might join Ravi's work on the general controller class a bit later after I've finished my work on the descriptor crypto. Best, Beck