* memory usage
@ 2013-10-22 8:28 Alexandre Riveira
[not found] ` <52663744.6040600-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
0 siblings, 1 reply; 6+ messages in thread
From: Alexandre Riveira @ 2013-10-22 8:28 UTC (permalink / raw)
To: Rainbows! list
I usually use XEpollThreadPool however persebi a large memory usage for
sites that have a lot of content and access reaching over 1 GB. Changed
to ThreadPool and memory fell not from 4x 256MB. However
XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
configuration. I tested also with CoolioThreadPool speed was good but
the memory consumption and large too.
Tanks
Alexandre Riveira
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: memory usage
[not found] ` <52663744.6040600-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2013-10-22 16:09 ` Eric Wong
[not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
0 siblings, 1 reply; 6+ messages in thread
From: Eric Wong @ 2013-10-22 16:09 UTC (permalink / raw)
To: Rainbows! list
Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> I usually use XEpollThreadPool however persebi a large memory usage
> for sites that have a lot of content and access reaching over 1 GB.
> Changed to ThreadPool and memory fell not from 4x 256MB. However
> XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
> configuration. I tested also with CoolioThreadPool speed was good
> but the memory consumption and large too.
Wait, ThreadPool uses significantly less than XEpollThreadPool?
Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and
MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these
are mainly documented by Red Hat, but in all recent-ish versions of
glibc/eglibc support it.
Then increase the values if you see performance gains (unlikely unless
your app releases the GVL a lot or don't have one (rbx)).
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: memory usage
[not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2013-10-22 18:08 ` Alexandre Riveira
2013-10-27 11:35 ` Alexandre Riveira
1 sibling, 0 replies; 6+ messages in thread
From: Alexandre Riveira @ 2013-10-22 18:08 UTC (permalink / raw)
To: Rainbows! list
Yes, XEpollThreadPool 4 times more memory than ThreadPool will do tests
as you indicated and verifying systems that have other glibc
Tanks
Alexandre Riveira
On 22-10-2013 16:09, Eric Wong wrote:
> Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
>> I usually use XEpollThreadPool however persebi a large memory usage
>> for sites that have a lot of content and access reaching over 1 GB.
>> Changed to ThreadPool and memory fell not from 4x 256MB. However
>> XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
>> configuration. I tested also with CoolioThreadPool speed was good
>> but the memory consumption and large too.
> Wait, ThreadPool uses significantly less than XEpollThreadPool?
>
> Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and
> MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these
> are mainly documented by Red Hat, but in all recent-ish versions of
> glibc/eglibc support it.
>
> Then increase the values if you see performance gains (unlikely unless
> your app releases the GVL a lot or don't have one (rbx)).
> _______________________________________________
> Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
> http://rubyforge.org/mailman/listinfo/rainbows-talk
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: memory usage
[not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-10-22 18:08 ` Alexandre Riveira
@ 2013-10-27 11:35 ` Alexandre Riveira
[not found] ` <526CFA8C.7000300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
1 sibling, 1 reply; 6+ messages in thread
From: Alexandre Riveira @ 2013-10-27 11:35 UTC (permalink / raw)
To: Rainbows! list
Hello Eric,
I discovered that when you restart the server (linux 2.3.50, glibc 2.15)
memory stopped growing indiscriminately.
2 questions:
1) The POOL_SIZE XEpollThreadPool series of the number of connections
accepted epoll so'd have one for each thread
2) I saw the gem 'mall' and she seems to adjust as you indicted by glibc
environment variables are correct? If yes in config / initializers added
a file.rb with the following:
Mall.opt (Mall :: ARENA_MAX, 1)
Mall.opt (Mall :: ARENA_TEST, 1)
Tanks,
Alexandre Riveira
On 22-10-2013 16:09, Eric Wong wrote:
> Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
>> I usually use XEpollThreadPool however persebi a large memory usage
>> for sites that have a lot of content and access reaching over 1 GB.
>> Changed to ThreadPool and memory fell not from 4x 256MB. However
>> XEpollThreadPool is faster than ThreadPool. Accepted suggestions for
>> configuration. I tested also with CoolioThreadPool speed was good
>> but the memory consumption and large too.
> Wait, ThreadPool uses significantly less than XEpollThreadPool?
>
> Since this is likely glibc/eglibc, try setting MALLOC_ARENA_MAX=1 and
> MALLOC_ARENA_TEST=1 in the env before starting Rainbows! AFAIK, these
> are mainly documented by Red Hat, but in all recent-ish versions of
> glibc/eglibc support it.
>
> Then increase the values if you see performance gains (unlikely unless
> your app releases the GVL a lot or don't have one (rbx)).
> _______________________________________________
> Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
> http://rubyforge.org/mailman/listinfo/rainbows-talk
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: memory usage
[not found] ` <526CFA8C.7000300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2013-10-28 0:37 ` Eric Wong
[not found] ` <20131028003755.GA6569-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
0 siblings, 1 reply; 6+ messages in thread
From: Eric Wong @ 2013-10-28 0:37 UTC (permalink / raw)
To: Rainbows! list
Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Hello Eric,
>
> I discovered that when you restart the server (linux 2.3.50, glibc
> 2.15) memory stopped growing indiscriminately.
Linux 2.3.50? Huh? Assuming that's 3.2.50. Anyways, user-visible
memory usage is only dependent on glibc version.
> 2 questions:
>
> 1) The POOL_SIZE XEpollThreadPool series of the number of
> connections accepted epoll so'd have one for each thread
No, pool_size for XEpollThreadPool is the number of worker threads
capable of dispatching the application. It can hold many more clients.
> 2) I saw the gem 'mall' and she seems to adjust as you indicted by
> glibc environment variables are correct? If yes in config /
> initializers added a file.rb with the following:
> Mall.opt (Mall :: ARENA_MAX, 1)
> Mall.opt (Mall :: ARENA_TEST, 1)
Yes, mall works, too, and you can tune a running app with some of the
knobs. However, for arena settings, I think they must be set early on
(before many threads are spawned) to be effective. So environment
variables (set before starting Ruby) are probably the best bet, since
Ruby (MRI) spawns a background timer thread right away.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: memory usage
[not found] ` <20131028003755.GA6569-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2013-10-28 18:40 ` Alexandre Riveira
0 siblings, 0 replies; 6+ messages in thread
From: Alexandre Riveira @ 2013-10-28 18:40 UTC (permalink / raw)
To: Rainbows! list
Hello Eric,
My linux is 3.2.50, sorry my error, environment variables work fine now.
Tanks for your help
Alexandre Riveira
On 28-10-2013 00:37, Eric Wong wrote:
> Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
>> Hello Eric,
>>
>> I discovered that when you restart the server (linux 2.3.50, glibc
>> 2.15) memory stopped growing indiscriminately.
> Linux 2.3.50? Huh? Assuming that's 3.2.50. Anyways, user-visible
> memory usage is only dependent on glibc version.
>
>> 2 questions:
>>
>> 1) The POOL_SIZE XEpollThreadPool series of the number of
>> connections accepted epoll so'd have one for each thread
> No, pool_size for XEpollThreadPool is the number of worker threads
> capable of dispatching the application. It can hold many more clients.
>
>> 2) I saw the gem 'mall' and she seems to adjust as you indicted by
>> glibc environment variables are correct? If yes in config /
>> initializers added a file.rb with the following:
>> Mall.opt (Mall :: ARENA_MAX, 1)
>> Mall.opt (Mall :: ARENA_TEST, 1)
> Yes, mall works, too, and you can tune a running app with some of the
> knobs. However, for arena settings, I think they must be set early on
> (before many threads are spawned) to be effective. So environment
> variables (set before starting Ruby) are probably the best bet, since
> Ruby (MRI) spawns a background timer thread right away.
> _______________________________________________
> Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
> http://rubyforge.org/mailman/listinfo/rainbows-talk
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-10-28 20:40 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-22 8:28 memory usage Alexandre Riveira
[not found] ` <52663744.6040600-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2013-10-22 16:09 ` Eric Wong
[not found] ` <20131022160930.GA3020-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-10-22 18:08 ` Alexandre Riveira
2013-10-27 11:35 ` Alexandre Riveira
[not found] ` <526CFA8C.7000300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2013-10-28 0:37 ` Eric Wong
[not found] ` <20131028003755.GA6569-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-10-28 18:40 ` Alexandre Riveira
Code repositories for project(s) associated with this public inbox
https://yhbt.net/rainbows.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).