Re: How to manage growing memory with Rainbows!

From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
To: Rainbows! list <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: How to manage growing memory with Rainbows!
Date: Thu, 14 Feb 2013 07:15:12 +0000	[thread overview]
Message-ID: <20130214071512.GA10890@dcvr.yhbt.net> (raw)
In-Reply-To: <B3ED3231-62E8-4FF5-A65A-859AFD790C2B-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

Claudio Poli <claudio-3HQ/CcOImoi171pxa8y+qA@public.gmane.org> wrote:
> Il giorno 12/feb/2013, alle ore 06:00, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> ha scritto:
> 
> Using LD_PRELOAD with libtcmalloc

Last I checked, tcmalloc never releases memory to the OS, so that
could be a problem.  (Giving memory back to the kernel and then
getting it back soon afterwards is slow because the kernel must
clear that memory, first, so most malloc implementations (including
glibc) will try to keep memory for the process)

> > You can try MALLOC_ARENA_MAX_=1 to limit the number of arenas if you
> > want.  That might reduce fragmentation since the GVL in MRI means
> > it's unlikely to hit malloc lock contention (glibc uses multiple
> > malloc arenas to avoid contention by default).
> 
> I didn't knew about this setting, might be worth a try, thanks.

You might also want to try MALLOC_MMAP_THRESHOLD_=131072 to disable the
sliding sbrk/mmap window (you can try larger/smaller values).  The
latest Linux man-pages releases have better mallopt(3) documentation
(but the arenas stuff was only documented on Dreppers blog, AFAIK (and
glibc.git/malloc/*.[ch] comments)

> > However, you can safely send SIGQUIT to any Rainbows! worker (bypassing
> > master) whenever you feel memory usage is high, master will restart it.
> 
> Will Rainbows! wait after the last request before restarting?

Yes, SIGQUIT => graceful shutdown will always work for normal apps.
The only exception is the new, optional rack.hijack (in rainbows.git).

> What Rainbows! strategy would you run on 1.9.3 given that some API call might take 800ms/1200ms (uncached) and the number of requests is fairly high?

It depends on your app.  For standard Ruby API (Net::*),
ThreadSpawn/Pool should be the most compatible and work well enough.
These will work great if you're using Celluloid, too.

EventMachine can scale very well but it requires your network-capable
gems/libraries to use EM to benefit.  Same issue with cool.io.

I also like XEpollThread{Pool/Spawn}, but they're Linux-only and
less-used.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying