Erik and I have finished the proc.py integration tests
Great! I'm looking forward to checking them out.
Tomorrow, we will be starting the next Stem task regarding the Tor Export project. Do you have any recommendations on where we should start exploring in the Stem documentation?
Hmm, this concerns translating Descriptor subclasses [1] into a csv (and I suppose loading them back up). First step would be to google around to see if there's a builtin function what'll do this or help with it. If not then this might simply include dumping out their __dict__...
class Foo:
... def __init__(self): ... self.foo = 5 ... self.bar = "hello" ... def set_bar(self, value): ... self.bar = value ...
f = Foo() f.__dict__
{'foo': 5, 'bar': 'hello'}
",".join([str(v) for v in f.__dict__.values()])
'5,hello'
You probably want to sort the keys and use that order, but besides that this should do the trick. Just add a unit test and you're done. :)
If we want to convert csvs *back* into Descriptors then that's a bit more work.
Also, what do you anticipate being necessary when it comes to parsing consensus entries?
A NetworkStatusEntry class (and its tests) would be similar to the ServerDescriptor [2] and ExtraInfoDescriptor [3], but a fair bit easier (it only has six entries). To implement this you'd...
1. Read over the ServerDescriptor and ExtraInfoDescriptor to figure out how we're tackling the parsing and verification in python. 2. Read over Karsten's NetworkStatusEntryImpl class [4] to see how he's handling this in metrics-lib. 3. Read the spec for these entries [5]. Do we need both a NetworkStatusEntryV2 and NetworkStatusEntryV3 class? Exactly how should we model this? Karsten: thoughts? 4. Go to 'https://metrics.torproject.org/data.html' to get some test data, so you see what the things that you'll be parsing looks like. 5. Write a NetworkStatusEntry class with the goal of "figure out each and every way that data might be malformed". A large part of this class is to verify that the data we're being given is perfectly valid according to the spec, so read the spec and implement exactly what it says. If there's some ambiguity in the spec or you see data that doesn't conform to the spec then let us know. That's a tor bug. 6. Write unit and integ tests similar to what the other descriptors have... - unit tests are the majority of the work, and exercise all the use cases that you can think of against mock objects - integ tests are pretty short, and just run the parser against some test data from the metrics archive and the cached consensus
Let me know if you have any questions! -Damian
PS. Karsten: Do we want to call this "NetworkStatusEntry" or something else? It seems like "ConsensusEntry" would be more intuitive, but maybe this would just spawn confusion.
[1] https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/descriptor/__init__.p... [2] https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/descriptor/server_des... [3] https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/descriptor/extrainfo_... [4] https://gitweb.torproject.org/metrics-lib.git/blob/HEAD:/src/org/torproject/... [5] https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt#l1334