[tor-dev] Using the consensus-health web page to debug the Tor network
Karsten Loesing
karsten at torproject.org
Thu Sep 19 12:44:10 UTC 2013
Hi Damian,
Roger and I discussed the consensus-health web page that the Java
version of DocTor produces [0] but your Python version does not. Roger
says he uses that page to debug the Tor network. In particular, he says
he scans the huge relay flags table to spot differences between relay
flags assigned by the directory authorities. I'm pasting the IRC log
below, but we should move this discussion to email, because it seems
unlikely that we're all present in #tor-dev at the same time to discuss
this.
I see four ways how we can make most people mostly happy:
1.) We add votes documents to Onionoo (#9778) and include relay flags
contained in votes in relay details pages of Atlas/Globe, so that people
can find out what the different directory authorities think about their
relay. We write a new Onionoo client that fetches this information for
all running relays and produces a long table of relay flags. Before we
do all this we make sure that there's nothing else on the
consensus-health web page that Roger secretly hoped to keep. Then we
shut down the Java DocTor.
2.) We rewrite the website-generating part of DocTor in a language we're
happier to maintain long-term. This could be part of your DocTor, or of
a new tool. Then we shut down the Java DocTor.
3.) We keep the Java DocTor running to generate the website, but disable
the part that sends notifications to the mailing list and to the IRC
bot. Or rather, I'll want to move the page away from metrics.tpo to,
say, consensus-health.tpo or consensus-explorer.tpo, because it has
always been a hack to serve it on metrics.tpo. Then we hope the Java
DocTor never breaks.
4.) We do something much smarter which I didn't think of yet.
What do you think?
All the best,
Karsten
[0] https://metrics.torproject.org/consensus-health.html
08:04:49 < karsten> armadev: about removing per-relay votes from
https://metrics.torproject.org/consensus-health.html. that was
intentional, because the page took so long to
load and it seemed nobody cared.
08:05:14 < karsten> armadev: but it seems the next step will be to
kill that web page entirely..
08:05:34 < karsten> the reason is that atagar's consensus-health
checker will take over very soon.
08:05:56 < karsten> so, what information from the consensus-health
page do you still need, either the current one or
even the one with per-relay votes?
08:06:10 < karsten> we should try to rescue that information then.
08:07:16 < armadev> karsten: is atagar's consensus health checker up
yet?
08:07:22 < karsten> yes, it is.
08:07:26 < armadev> url?
08:07:33 < karsten> it's sending mail to tor-consensus-health@ and to
#tor-bots.
08:07:36 < karsten> it has no website
08:07:48 < karsten> it's sending out warnings if something goes wrong.
08:07:56 < karsten> when* :)
08:08:54 < armadev> karsten: in this case, what i wanted from the
consensus health page was the set of votes per
authority for 000000000000myTOR2
08:08:56 < karsten> right now, both consensus-health things send mail
to that list. the ones at :03 are from mine, the
other ones from atagar's.
08:09:11 < armadev> i pieced it together manually from moria1's
v3-status-votes, but i'm special so i can do that
08:09:16 < karsten> right
08:09:25 < karsten> I thought about this a while ago. we could
include votes in onionoo.
08:09:34 < karsten> not sure how yatei will like that though.
08:09:50 < karsten> but it's the votes you care about, ok. anything
else from the consensus-health page?
08:10:16 < karsten> wfn: ^^ we should talk about this.
08:11:05 < armadev> s/votes/votes about flags/ here
08:11:23 < karsten> right.
08:37:19 < armadev> it seems i used consensus-health as
consensus-explorer
10:11:35 < armadev> karsten: can you fwd me a sample mail from the
health checker mailing list?
10:12:07 < karsten> armadev: they're archived:
https://lists.torproject.org/pipermail/tor-consensus-health/2013-September/003422.html
10:13:43 < armadev> oh good
10:13:56 < armadev> that's a very short mail.
10:14:02 < armadev> it has, like, nothing from the consensus health
checker
10:14:20 < karsten> that's the things that need fixing to make it
quiet again.
10:14:42 < armadev> why is "it takes a long time to load" a bug in
consensus-health.html ?
10:15:37 < karsten> well, the page is huge, and firefox can't handle
that very well. that's not a bug, but an
inconvenience.
10:16:09 < armadev> sure
10:16:25 < karsten> I could have split the page into two.
10:16:34 < karsten> but anyway, the entire page is going away soon.
10:16:39 < armadev> if you were planning to drop it entirely, you
could just leave it alone?
10:17:12 < karsten> as unmaintained service?
10:17:36 < karsten> I'd rather take the pieces we like and integrate
them in maintained services.
10:17:37 < armadev> i guess. or write a howto for how to take votes
in and generate the html, and i'll run it on moria
10:17:53 < armadev> though i think it needs more than votes as input
10:18:11 < armadev> i've used most parts of this page over the past
few months
10:18:13 < karsten> it downloads votes and consensuses.
10:18:19 < karsten> I see.
10:18:42 < armadev> i have votes and a consensus on moria
10:18:49 < armadev> if that's the input, i can just cron it to run on
moria
10:18:58 < armadev> based on the votes i see and the consensus i see
10:19:09 < armadev> unless i guess it requires a massive tomcat
engine too :)
10:19:15 < karsten> it doesn't.
10:19:17 < karsten> it requires java.
10:19:29 < karsten> and produces a static html that you can service
with whatever.
10:19:55 < karsten> well, if you need it, it's easier to keep it
running on yatei.
10:19:57 < armadev> i don't have any java. i guess i could add it. or
we could stick it on people.tp.o or something
10:19:59 < armadev> right
10:20:09 < karsten> and just turn off the email notifications,
because atagar's thing will send those.
10:20:11 < armadev> i use it for debugging the network
10:20:32 < karsten> ok
10:20:58 < karsten> would #9778 solve the problem with the missing
relay flags table?
10:20:59 -zwiebelbot:#tor-dev- [tor#9778: Adding votes documents to
Onionoo]
10:21:08 < karsten> assuming atlas or globe would show those?
10:21:32 < armadev> do i have to load each relay one page at a time?
10:22:05 < karsten> probably.
10:22:31 < karsten> so, you're looking through the table to spot
problems?
10:22:41 < karsten> or, have been looking, when the table was there.
10:22:41 < armadev> yes. and to get a sense of trends.
10:22:44 < armadev> right
10:22:57 < armadev> i guess in this most recent case i just wanted
one relay
10:23:11 < armadev> but after that, i realized that i wanted to know
how many relays had a Running flag from those two
authorities and no Running flag from any others
10:23:33 < karsten> ok
10:23:42 < armadev> (i still want to know :)
10:24:06 < karsten> let me re-enable the table and think about a
better long-term solution.. :)
10:24:07 < armadev> (related to
https://trac.torproject.org/projects/tor/ticket/9775 )
10:24:08 -zwiebelbot:#tor-dev- [tor#9775: Authorities should report
when they don't vote Running but some addresses are still
reachable]
10:24:48 < armadev> we are dropping an unknown number of relays from
the consensus because they enable ipv6 but screw
up the reachability
10:25:16 < armadev> hopefully every relay operator who knows what
ipv6 is also knows how to check relay-search for
themselves
10:28:21 < karsten> the next consensus-health.html will contain the
relay flags table again. in roughly 35 mins.
10:29:51 < armadev> thanks. sorry for the troubles. and i understand
the drive to get rid of extra things. i wish we
had more things that were actually extra.
10:32:10 < karsten> yeah, we have a lot of stuff. and some of it is
really old and hasn't seen love for too long.
More information about the tor-dev
mailing list