[tor-relays] new ansible-tor features: automatic instance configuration + automatic MyFamily generation (PATCH)

[Moritz Bartl]

On 02/17/2015 06:29 PM, Nusenu wrote:
> Arzhel and Moritz, could you comment on whether you prefer to manually
> create/specify tor instances (via 'proc_instances') as you do here
> [1], or whether you would make use of an instance auto configuration
> (two instances per IP - see example here [2])?

Automate away please!! :-)

In my ideal world, someone would write a "wizard" as wrapper around the
ansible stuff (or whatever really), so people who run it can either
specify attributes in a clear CLI language that feels "more native" to
the domain, or answer questions asked by the wizard. The knowledge
spectrum is "Moritz wants to use this", and "Uh. I have zero knowledge
of computers, but I want to run a Tor relay. I have this cheap $5 VPS,
please turn it into something useful. I don't know if it should be a
bridge, or a relay. I don't even really want to know.".

> If you prefer manual configuration: Do your manual steps follow any
> specific design that could be automated as well or are these steps
> unpredictable? :)

My algorithm. Hm. This all sucks because Tor still doesn't scale across
multiple cores.

I understand that many people want "fast relays" (ie. processes as fast
as possible) merely for psychological reasons. People look at the top
#10 fastest relays, and from a marketing perspective, we should make
sure we have all top #10 spots (phrased differently: People want to see
their relay(s) at the top!). At the beginning of Torservers.net, when
torstatus was still the thing to use, I took screenshots from the top
#10-20 and marked in red which relays were "ours". This brought us a lot
of positive attention and a lot of donations.

We still lack the "gamification" Relay Challenge website that Virgil was
talking about. It would just sum up all relays of a family, and then it
really does not matter any more. Until then, I think the perfect
automation would take that into account.

This is roughly my "algorithm":

Traffic limit? Line speed? => Calculate mean total speed across a month.
I like relays that are up for at least 2 weeks per month. Take
RELAY_REQUIRED_MIN_BANDWIDTH/BRIDGE_REQUIRED_MIN_BANDWIDTH into account.
( https://trac.torproject.org/projects/tor/ticket/13822 )

A modern CPU without AES-NI can do ~100 Mbit/s per core, with AES-NI up
to 400 Mbit/s.

=> from that, derive lower limit of how many relay processes we need.
Then, monitor closely to see if the CPU (core) can actually handle the
amount of traffic. If a core reaches 90% use, split off another relay
process.

If we have more IPs, I try to spread the processes across IPs and
various well-known ports. I keep some IPs clean and unused as "	backup".
Ideally, you don't want to expose SSH at all (portknock, whitelist
allowed source IPs etc), but if there's enough IPs you can at least put
it on a separate IP.

In practice and for simplicity, I wouldn't mind if you completely ignore
the "make processes handle as much traffic as possible". The default
should be "as automated as possible, even if suboptimal". As long as you
expose the "expert options for tuning", that's totally fine with me. :-)

Does this help?


Moritz

> 
> [1]
> https://github.com/XioNoX/moz-tor-relays/blob/master/host_vars/tor-relay1
> [2]
> https://lists.torproject.org/pipermail/tor-relays/2015-February/006414.html
> 
> 
> David Stainton wrote:
>>> - - the directory layout change is more cosmetic but your current
>>> way (everything in /etc) is rather unusual and as an example
>>> would require custom logrotate configuration that wouldn't be
>>> required otherwise
> 
>> OK... I don't have a strong opinion... and I think the parent 
>> directory for all this should be configuration via a role variable
>> so that the user can specify.
> 
> The user is free to specify the vars in a flexible way. Defaults in
> the patch are:
> tor_ConfDir: /etc/tor
> tor_PidDir: /var/run/tor
> tor_LogDir: /var/log/tor
> tor_DataDir: /var/lib/tor
> 
>> However I initially created this Ansible role to help Moritz of
>> torservers.net and those people that may be working for him;
>> therefore pull requests and feedback helps; for instance Moritz
>> specified several features it should have... and an engineer 
>> working for Mozilla chatted with me about the features they
>> needed; then he sent me a pull request on github.
> 
> I'm surprised that Moritz didn't ask for automatic MyFamily generation ;)
> 
> 
>>>> I'd be much more likely to merge your patches if they were one
>>>> feature per patch... instead of this monolithic patch with many
>>>> features.
>>>
>>> Yes, that is what I expected, but then I thought that the two
>>> main changes code wise (autoconfig + directory structure) are
>>> dependent on each other anyway. Merging  autoconfig without the
>>> directory restructuring (or vice versa) wouldn't be much fun
>>> since these modifications always touch overlapping areas. If you
>>> want to add it as additional option, including it as a separate 
>>> yml in tasks/main.yml + separate torrc is also a possibility -
>>> but probably not the nicest way (duplicate code, multiple
>>> torrc's).
>>>
> 
>> OK... I agree with you... but let's make this a seperate yml task 
>> file; your use is quite different than most of the entities
>> currently using this ansible role. So let's add these as a new task
>> file instead of modifying the existing task file.
> 
> That is fine with me.
> 
> regards,
> Nusenu
> 

-- 
Moritz Bartl
https://www.torservers.net/