[tor-relays] MaxMemInQueues - per host, or per instance?

[David Goulet]

On 22 Dec (20:37:37), r1610091651 wrote:
> I'm wondering if it is necessary to have a lot of ram assigned to queues?
> Is there some rule of thumb to determine the proper sizing? Based on number
> of circuits maybe?

So there are probably many different answers to this or ways to look at
it but I can speak on how "tor" is built and why it is important to have
this memory limit assigned to queues.

A tor relay gets cells in and most of the time will relay them so send
them outbound. But for every cell that comes in, we need to do some
processing on them that is mostly decryption work.

So we get them, process then put them on a circuit queue. Then tor does
its best to dequeue a "not too big amount of cells" from a circuit and
puts them on the outbound connection buffers which, when the socket is
writable, will be flushed onto the network (write to the socket).

The MaxMemInQueues parameter basically tells the tor OOM handler when it
is time to start cleaning up allocated memories. But here is the catch,
it only handles cells on circuit queues, not connection's buffer (it
actually handles other things but the majority of allocated data is in
cells usually).

For that reason, we are better off for now to keep relays with a sane
value for MaxMemInQueues so the OOM is actually triggered before the
load goes out of control.

If that MaxMemInQueues value is not set in your torrc, tor will pick 3/4
of the total memory of your system. Usually, this is fine for most use
cases but if you machine has 16GB of RAM but only 4GB are available,
problem. So when setting it, it is not that easy to come up with a good
value but a rule of thumb for now is look at how much memory you have
available normally and estimate around it. It is also important to not
go to low, a fast relay limited to 1GB for instance will start to
degrade performance by killing cicuits more often if it sees 20MB/s
(imperically speaking).

I think we could do a better job at estimating it when not set, we could
do a better job with the OOM, we could do lot more but unfortunately for
now, this is the state of thing we need to deal with. We'll be trying to
work on more DoS resistance feature hopefully in the near future.

Hope this help!

Cheers!
David

> 
> Do the wise-minds have a guidance on this one?
> 
> On Fri, 22 Dec 2017 at 21:08 Igor Mitrofanov <igor.n.mitrofanov at gmail.com>
> wrote:
> 
> > Thanks. I do have the IP space.
> >
> > It is a pity multiple instances cannot watch the overall RAM
> > remaining. I have quite a bit of RAM left, but there are large
> > discrepancies in terms of how much RAM different relays are using (>3
> > GB for some, <1 GB for others), so it will be tricky to set
> > MaxMemInQueues without making it too conservative.
> >
> > On Fri, Dec 22, 2017 at 11:46 AM, r1610091651 <r1610091651 at telenet.be>
> > wrote:
> > > It would expect it to be per instance. Instances are independent of each
> > > other. Further one can only run 2 instances max / ip.
> > >
> > > On Fri, 22 Dec 2017 at 20:40 Igor Mitrofanov <
> > igor.n.mitrofanov at gmail.com>
> > > wrote:
> > >>
> > >> Hi,
> > >>
> > >> Is MaxMemInQueues parameter per-host (global) or per-instance?
> > >> Say, there are 10 relays on the same 24 GB host. Should I set
> > >> MaxMemInQueues to 20 GB, or 2 GB in each torrc?
> > >>
> > >> Thanks,
> > >> Igor
> > >> _______________________________________________
> > >> tor-relays mailing list
> > >> tor-relays at lists.torproject.org
> > >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
> > >
> > >
> > > _______________________________________________
> > > tor-relays mailing list
> > > tor-relays at lists.torproject.org
> > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
> > >
> > _______________________________________________
> > tor-relays mailing list
> > tor-relays at lists.torproject.org
> > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
> >

> _______________________________________________
> tor-relays mailing list
> tor-relays at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays


-- 
tPcuU+9hl1BRjXh3xHhFgg22HULt2edIxY5kAKLBPPA=
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 585 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20171222/d3e444ce/attachment.sig>