The default system one should if large blocks are allocated and deallocated at once.<br><br><div><span class="gmail_quote">On 5/11/06, <b class="gmail_sendername">Ben Wilhelm</b> <<a href="mailto:zorba@pavlovian.net">zorba@pavlovian.net
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>Does your allocator actually return memory to the OS? Many don't, and in
<br>my (admittedly brief) look through the source, I don't remember seeing a<br>custom allocator.<br><br>If it doesn't return memory to the OS, it'll just sit at its maximum<br>allocated size for all eternity, despite not using much of this memory.
<br>(Although your buffer-shrinking would help reduce that<br>maximum-allocated-size.)<br><br>-Ben<br><br>Nick Mathewson wrote:<br>> On Thu, May 11, 2006 at 09:21:05PM +0200, Joerg Maschtaler wrote:<br>>> Hi,<br>
>><br>>> if it possible to add an option which allows to shrink memory buffers<br>>> immediatly if they are not full? [1]<br>><br>> I don't think you would want that; the CPU usage would be *insanely*
<br>> high. Every time you transmitted any information at all, you'd need<br>> to shrink the buffer, and then immediately re-grow the buffer the<br>> buffer when you had more data to transmit.<br>><br>> Right now, Tor shrinks buffers ever 60 seconds, down to the next
<br>> largest power of two above the largest amount of the buffer at any<br>> time in the last 60 seconds. A 60-second lag here probably does no<br>> harm memory-wise, but the power-of-two thing will, on average, make
<br>> 25% of your buffer space unused.<br>><br>> The only thing that would actually help trade cpu for RAM here won't<br>> be a more frequent shrinking; instead, we'd have to switch off the<br>> power-of-two buffers implementation. But if we're going to do *that*,
<br>> we may as well move to an mbuf/skbuff-style implementation, and get<br>> improved RAM usage and improved CPU usage at the same time. (That<br>> approach will make our SSL frame-size uniformity code a little
<br>> trickier, but I think we can handle that.)<br>><br>>> I run an Tor server on a virtual server in which the amount of RAM is<br>>> the bottleneck of this system.<br>>> Since ~ 3 weeks i measure the "resident" memory allocation and the
<br>>> corresponding traffic of the Tor server and the thing i realize is that<br>>> the allocation only shrinks if i shutdown (and restart) the Tor<br>>> server. [2]<br>><br>> Hm. I should look at a breakdown of buffer size; I'll try to do that
<br>> later tonight, once I've had my server running for a bit. It's<br>> probably important to know whether our real problem is wedged<br>> connections whose buffers get impossibly large, or buffers whose<br>> capacities are larger than they have to be.
<br>><br>> yrs,<br><br></blockquote></div><br><br clear="all"><br>-- <br>"Those who would give up Essential Liberty to purchase a little Temporary Safety deserve neither Liberty nor Safety."<br>-- Benjamin Franklin