Hi Damian, hi devs,
I'm planning to make microdescriptor tarballs available on the metrics website that contain both microdescriptor consensuses and microdescriptors.
Some background: Recent Tor clients don't download the network status consensus and full server descriptors anymore, but instead download the microdescriptor consensus and the microdescriptors referenced from it. We didn't provide these formats on the metrics website yet, because they are derived from the formats we already provide and don't contain anything novel. But having the new formats will, for example, make it easier for developers to analyze the directory protocol and for researchers to understand what information is available to clients to make path selection decisions. If you need more background, see #2785 and search for "microdesc" in dir-spec.txt.
Here's a sample tarball:
https://people.torproject.org/~karsten/microdescs-2014-01.tar.bz2
Damian, can you try to parse these descriptors using stem, to see if the descriptor annotations are correct and if stem can parse them without issues?
If all goes well, microdescriptor tarballs will start to be available on the metrics website before the end of the month.
All the best, Karsten