[tor-dev] Guardiness: Yet another external dirauth script

[Sebastian Hahn]

On 16 Sep 2014, at 16:15, George Kadianakis <desnacked at riseup.net> wrote:
> ====How guardiness works====
> The idea was that the guardiness script will be an external script
> that is run by Tor in a similar fashion to the bandwidth auth
> scripts. We chose that because we could write the script in a
> high-level language and because it could be modular and we could
> change the algorithm in the future if we wanted. Unfortunately, it
> seems that external scripts in dirauths are a PITA to maintain as can
> be seen by the lack of bwauth operators.

The problem isn't so much that this is an external script, the problem
is that there are never dedicated maintainers for these things (tho
it being an external script is additionally problematic). If they
are in Tor proper, we have someone who cares about them when new stuff
gets introduced, and it gets updated with the rest of Tor, etc.

> Summary files are designed to be quick to parse (even with Python) and
> contain all the necessary information for guardiness. For example,
> parsing 2000 summary files in my laptop takes about 10 seconds.

Does this scale linearly? 9 months would be ~6500 files.

> FWIW, the guardiness scripts are ready for review and can be found here:
> https://gitweb.torproject.org/user/asn/hax.git/shortlog/refs/heads/guardiness


> 
> ====How the guardiness script will be deployed====
> 
> The idea is that dirauths will add another script to their crontab
> that is called every hour (before or after the bwauth scripts).

Only 4/9 have such scripts, if at all - it is possible to run a
bwauth on a different host, and scp the file over. I don't know
if any of the dirauth ops actually do this currently.

> The script first calls the summarizer script, which goes to the
> consensus/ directory and summarizes all consensuses it finds and puts
> them in the summary/ directory. The summarizer script then deletes all
> the consensuses that got summarized.

You must not delete files which you did not create. It's not cool
to delete consensuses which tor decided to put somewhere if you're
not tor. This is relevant to the plan to use mvdan's script.

> Then the script calls the the guardiness script, which goes to the
> summary/ directory, parses all summary files it finds, and outputs a
> guardiness output file that gets parsed by the dirauth prior to voting.
> 
> That should be all. Easy, eh? :)

What are the failure modes? Are there version strings included, does
Tor notice if the guardiness file is outdated, etc? What happens when
different dirauths use different versions of the guardiness file
generation script?

Cheers
Sebastian