* memory usage
@ 2013-10-22 8:28 Alexandre Riveira
[not found] ` <52663744.6040600-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
0 siblings, 1 reply; 6+ messages in thread
From: Alexandre Riveira @ 2013-10-22 8:28 UTC (permalink / raw)
To: Rainbows! list
I usually use XEpollThreadPool however persebi a large memory usage for
sites that have a lot of content and access reaching over 1 GB. Changed
to ThreadPool and memory fell not from 4x 256MB. However
XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
configuration. I tested also with CoolioThreadPool speed was good but
the memory consumption and large too.
Tanks
Alexandre Riveira
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 6+ messages in thread[parent not found: <52663744.6040600-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>]
* Re: memory usage [not found] ` <52663744.6040600-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> @ 2013-10-22 16:09 ` Eric Wong [not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> 0 siblings, 1 reply; 6+ messages in thread From: Eric Wong @ 2013-10-22 16:09 UTC (permalink / raw) To: Rainbows! list Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote: > I usually use XEpollThreadPool however persebi a large memory usage > for sites that have a lot of content and access reaching over 1 GB. > Changed to ThreadPool and memory fell not from 4x 256MB. However > XEpollThreadPool is faster than ThreadPool. Accepted suggestions for > configuration. I tested also with CoolioThreadPool speed was good > but the memory consumption and large too. Wait, ThreadPool uses significantly less than XEpollThreadPool? Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these are mainly documented by Red Hat, but in all recent-ish versions of glibc/eglibc support it. Then increase the values if you see performance gains (unlikely unless your app releases the GVL a lot or don't have one (rbx)). _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>]
* Re: memory usage [not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> @ 2013-10-22 18:08 ` Alexandre Riveira 2013-10-27 11:35 ` Alexandre Riveira 1 sibling, 0 replies; 6+ messages in thread From: Alexandre Riveira @ 2013-10-22 18:08 UTC (permalink / raw) To: Rainbows! list Yes, XEpollThreadPool 4 times more memory than ThreadPool will do tests as you indicated and verifying systems that have other glibc Tanks Alexandre Riveira On 22-10-2013 16:09, Eric Wong wrote: > Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote: >> I usually use XEpollThreadPool however persebi a large memory usage >> for sites that have a lot of content and access reaching over 1 GB. >> Changed to ThreadPool and memory fell not from 4x 256MB. However >> XEpollThreadPool is faster than ThreadPool. Accepted suggestions for >> configuration. I tested also with CoolioThreadPool speed was good >> but the memory consumption and large too. > Wait, ThreadPool uses significantly less than XEpollThreadPool? > > Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and > MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these > are mainly documented by Red Hat, but in all recent-ish versions of > glibc/eglibc support it. > > Then increase the values if you see performance gains (unlikely unless > your app releases the GVL a lot or don't have one (rbx)). > _______________________________________________ > Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org > http://rubyforge.org/mailman/listinfo/rainbows-talk > Do not quote signatures (like this one) or top post when replying > _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: memory usage [not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> 2013-10-22 18:08 ` Alexandre Riveira @ 2013-10-27 11:35 ` Alexandre Riveira [not found] ` <526CFA8C.7000300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> 1 sibling, 1 reply; 6+ messages in thread From: Alexandre Riveira @ 2013-10-27 11:35 UTC (permalink / raw) To: Rainbows! list Hello Eric, I discovered that when you restart the server (linux 2.3.50, glibc 2.15) memory stopped growing indiscriminately. 2 questions: 1) The POOL_SIZE XEpollThreadPool series of the number of connections accepted epoll so'd have one for each thread 2) I saw the gem 'mall' and she seems to adjust as you indicted by glibc environment variables are correct? If yes in config / initializers added a file.rb with the following: Mall.opt (Mall :: ARENA_MAX, 1) Mall.opt (Mall :: ARENA_TEST, 1) Tanks, Alexandre Riveira On 22-10-2013 16:09, Eric Wong wrote: > Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote: >> I usually use XEpollThreadPool however persebi a large memory usage >> for sites that have a lot of content and access reaching over 1 GB. >> Changed to ThreadPool and memory fell not from 4x 256MB. However >> XEpollThreadPool is faster than ThreadPool. Accepted suggestions for >> configuration. I tested also with CoolioThreadPool speed was good >> but the memory consumption and large too. > Wait, ThreadPool uses significantly less than XEpollThreadPool? > > Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and > MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these > are mainly documented by Red Hat, but in all recent-ish versions of > glibc/eglibc support it. > > Then increase the values if you see performance gains (unlikely unless > your app releases the GVL a lot or don't have one (rbx)). > _______________________________________________ > Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org > http://rubyforge.org/mailman/listinfo/rainbows-talk > Do not quote signatures (like this one) or top post when replying > _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <526CFA8C.7000300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>]
* Re: memory usage [not found] ` <526CFA8C.7000300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> @ 2013-10-28 0:37 ` Eric Wong [not found] ` <20131028003755.GA6569-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> 0 siblings, 1 reply; 6+ messages in thread From: Eric Wong @ 2013-10-28 0:37 UTC (permalink / raw) To: Rainbows! list Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote: > Hello Eric, > > I discovered that when you restart the server (linux 2.3.50, glibc > 2.15) memory stopped growing indiscriminately. Linux 2.3.50? Huh? Assuming that's 3.2.50. Anyways, user-visible memory usage is only dependent on glibc version. > 2 questions: > > 1) The POOL_SIZE XEpollThreadPool series of the number of > connections accepted epoll so'd have one for each thread No, pool_size for XEpollThreadPool is the number of worker threads capable of dispatching the application. It can hold many more clients. > 2) I saw the gem 'mall' and she seems to adjust as you indicted by > glibc environment variables are correct? If yes in config / > initializers added a file.rb with the following: > Mall.opt (Mall :: ARENA_MAX, 1) > Mall.opt (Mall :: ARENA_TEST, 1) Yes, mall works, too, and you can tune a running app with some of the knobs. However, for arena settings, I think they must be set early on (before many threads are spawned) to be effective. So environment variables (set before starting Ruby) are probably the best bet, since Ruby (MRI) spawns a background timer thread right away. _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <20131028003755.GA6569-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>]
* Re: memory usage [not found] ` <20131028003755.GA6569-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> @ 2013-10-28 18:40 ` Alexandre Riveira 0 siblings, 0 replies; 6+ messages in thread From: Alexandre Riveira @ 2013-10-28 18:40 UTC (permalink / raw) To: Rainbows! list Hello Eric, My linux is 3.2.50, sorry my error, environment variables work fine now. Tanks for your help Alexandre Riveira On 28-10-2013 00:37, Eric Wong wrote: > Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote: >> Hello Eric, >> >> I discovered that when you restart the server (linux 2.3.50, glibc >> 2.15) memory stopped growing indiscriminately. > Linux 2.3.50? Huh? Assuming that's 3.2.50. Anyways, user-visible > memory usage is only dependent on glibc version. > >> 2 questions: >> >> 1) The POOL_SIZE XEpollThreadPool series of the number of >> connections accepted epoll so'd have one for each thread > No, pool_size for XEpollThreadPool is the number of worker threads > capable of dispatching the application. It can hold many more clients. > >> 2) I saw the gem 'mall' and she seems to adjust as you indicted by >> glibc environment variables are correct? If yes in config / >> initializers added a file.rb with the following: >> Mall.opt (Mall :: ARENA_MAX, 1) >> Mall.opt (Mall :: ARENA_TEST, 1) > Yes, mall works, too, and you can tune a running app with some of the > knobs. However, for arena settings, I think they must be set early on > (before many threads are spawned) to be effective. So environment > variables (set before starting Ruby) are probably the best bet, since > Ruby (MRI) spawns a background timer thread right away. > _______________________________________________ > Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org > http://rubyforge.org/mailman/listinfo/rainbows-talk > Do not quote signatures (like this one) or top post when replying > _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-10-28 20:40 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-22 8:28 memory usage Alexandre Riveira
[not found] ` <52663744.6040600-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2013-10-22 16:09 ` Eric Wong
[not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-10-22 18:08 ` Alexandre Riveira
2013-10-27 11:35 ` Alexandre Riveira
[not found] ` <526CFA8C.7000300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2013-10-28 0:37 ` Eric Wong
[not found] ` <20131028003755.GA6569-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-10-28 18:40 ` Alexandre Riveira
Code repositories for project(s) associated with this public inbox https://yhbt.net/rainbows.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).