[metrics-team] Bringing back Tor Weather - volunteering developer

[Eran Sandler]

On Fri, 2018-07-20 at 17:11 +0100, Iain Learmonth wrote:

> Sorry it's taken a while to reply to this thread. I have been busy on
> another task.

That's fine :-) I have patience and I don't run around writing code
just like that :-Pp

> 
> First, I think that it's great that you'd like to volunteer to help
> bring back Tor Weather. Unfortunately we cannot be sure how much time
> we
> could commit to helping you with this, or whether we would be able to
> run the service once it is finished.

I would love any help that you or other can give but its not mandatory.
I am able to assemble this service up and running on my own as well as
maintain it.


> 
> As this would not at least at the start be considered an official
> project, we would ask that you don't call the new service Tor
> Weather.

Do you have any other name suggestion?

nodestatus maybe? You know that naming stuff is probably the hardest
part of a project. We can also just nickname it for now and change it
to an "official" Tor Weather if and when everyone deems it so.

I'll figure something out.


> 
> We have previously thought about what a new Tor Weather service would
> look like when roadmapping. Some thoughts:
> 
> * It should use Onionoo as a data source. I know nusenu has raised
> concerns about the delay, but I don't believe that 2 hours will make
> a
> huge difference. This greatly reduces code complexity and increases
> the
> chances of it being adopted as an official project.

I agree that Onionoo is probably the best data source. Is there a way
to reduce the 2 hours delay to something shorter? 

I know that the previous Tor Weather service would alert me after a few
days that the node is down, which was a bit annoying. If you are paying
for a VPS you want it service stuff, not sitting idle doing nothing.

2 hours means that a new weather system would be calling Onionoo every
1 hour (to make sure we don't miss the 2 hours update window too much)
or even every 30min.

I would love to know why there is a 2 hours delay and see if I can try
and help reduce that (maybe have newer Tor clients push notifications
instead of pulling and discovering alone - but I guess that's a
discussion for another place). 

> 
> * It should allow operators to subscribe by email to all relays, for
> a
> relay's family or for a single relay.

Agree.

> 
> * It should not, at least initially, attempt to parse contactinfo. At
> a
> later time this feature could be added but initially it would be best
> if
> we were not sending emails to relay operators that they are not
> expecting. This is especially the case during early development where
> we
> may inadvertently spam operators with garbage messages.

I think that actually Onionoo is the right place to parse the contact
info and serve it as part of its meta data. This will also reduce
additional complexity from the weather system and pass it to the right
place.

The reason I mentioned using the contactinfo and verifying via that is
that I was thinking of a set of services that are just for relay
operators - for example, I have the domain torexitnode.net - it would
be great if I could provide anyone with a subdomain on that (such as
the name of the relay or something similar) - then the relay operator
could make sure the reverse DNS lookup (if its hosting provider allows
it) point back to mynode.torexitnode.net - this makes it even clearer
that this IP address is a Tor Exit node. This system would benefit from
a way to authenticate a relay operator. If the operator can change
ContactInfo information it means they control the system so it would
fit. I thought that a system like that will allow reducing the
complexity for other such system - including weather - but I guess that
its not really needed for weather.

> 
> * The volume of email being sent should be quite low. We would
> probably
> fit into the SendGrid free plan. This would further reduce code
> complexity by offloading email functionality from the new service.
> 

Yeah, either SendGrid or Mailgun or whoever will give a reasonable free
plan. Later we can ask to get some larger free tiers if we can show
what's its used for. I really don't think one can really run an SMTP
server these days and have Email actually get delivered.

All of these system assumes there is a domain for that. I'll get one
(or use some domain I already have).

> * E-mail addresses should be verified before we sent anything other
> than
> a verification email. We should rate-limit verification emails.
> 

Of course - The same way any such system will work.

> * All events could be stored in a log and presented via an RSS feed
> that
> you could subscribe to. This could be a feed for all relays, for a
> relay's family or for a single relay.

Having a feed is a good alternative for using an email as it will allow
people to get the data without giving out information at all. 

That kind of makes me think of a system that will keep a log of node
changes. For example:

* An event for when the node first appeared
* An event for when a certain flag was added to a node
* An event for when a node went down
* An event for when a node goes back up

I don't know if this data is stored today but it can be generated from
a certain point onwards from Onionoo if one listens to it all the time.


> 
> * Personally I am comfortable with Python, Java and Go. If you pick
> one
> of these languages I am far more likely to be able to assist you.

I'm comfortable with these languages as well. I think I would do it in
either Python or Go. I already wrote an Onionoo wrapper in Go a while
back - https://github.com/erans/gonionoo - (not that its super complex
to work with it anyway). I also think its the most effective language
to run in a small time VPS and still be able to give all the
information it needs :-)

> 
> * nusenu has listed a lot of additional emails that might be sent,
> but I
> think we should first have a minimum viable product before expanding
> beyond that. The fact that more events might be added in the future
> should be kept in mind though in the implementation, so that it's not
> more work than is needed to have them later.

Agree.

> 
> You are welcome to come along to our next meeting at 14:30 UTC on
> Thursday 26th in #tor-meeting (irc.oftc.net) to discuss this project.
> I'll also try to be more responsive replying to this thread.

I'm on PST time so will do my best to see if I can join.


Eran
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: This is a digitally signed message part
URL: <http://lists.torproject.org/pipermail/metrics-team/attachments/20180720/8dc30750/attachment.sig>