Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
To: Rainbows! list <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: Rainbows! + EventMachine + Sinatra::Synchrony == pegged CPU when idle?
Date: Mon, 23 Jul 2012 17:21:40 -0700	[thread overview]
Message-ID: <20120724002140.GA25177@dcvr.yhbt.net> (raw)
In-Reply-To: <CAAsVqN91_KiexPc4u9byuVstyknj7TgT4V7evs1x65W+6izzkQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

Ilya Maykov <ivmaykov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi Eric,
> 
> Sorry for the delayed response. I've added inline answers to your
> questions. We've since resolved this issue by disabling keepalive in
> our rainbows config. So, this probably had to do with the keepalive
> implementation either in Rainbows itself or in the base Unicorn code.
> Answers to your other questions are inlined below.

Thank you very much for the follow-up (I wish more folks would do this
:)

If Rainbows! is using EventMachine, it'll use the
EM.set_comm_inactivity_timeout method in EventMachine.

> On Tue, Jun 19, 2012 at 10:54 AM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> > Ilya Maykov <ivmaykov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > Can you also try "keepalive_timeout 0" to disable keepalive?  (EM
> > handles it internally, but I'm not sure how well)
> >
> 
> This turned out to be the problem. Disabling keepalive got rid of the
> CPU pegging. Surprisingly, it also made our average latency drop from
> about 50 ms to about 20 ms per request, even though every request now
> has to negotiate a connection handshake. So, we're just going to keep
> it disabled for now. The bug may be inside the keepalive code in
> Rainbows or Unicorn (not familiar with the codebase so not sure where
> that code lives).

Are you setting Content-Length or "Transfer-Encoding: chunked" in
responses?  Rack::ContentLength or Rack::Chunked middleware might need
to be loaded if your framework doesn't already include it.

Lack of these headers may confuse clients, even...  Where did you
measure the 50 -> 20ms latency drop, from the client?

About the latency drop:

  Was :tcp_cork enabled in your listen directive?  I wonder if there's
  some bad interaction with :tcp_cork + EM which might explain the
  latency (but not the CPU usage).  Disabling keepalive would force data
  out immediately and avoid any bad effects of :tcp_cork.j

  Can you also try listen() with :tcp_defer_accept => 0?  That might
  help if you're accept()-ing many connections at once.


Anyways, the TCP connection handshake is very fast on LANs/localhost,
but hurts on high-latency connections (I seem to remember mainstream web
browsers double simultaneous requests to compensate for lack of
keepalive).

Which version of EM are you using?  The keepalive implementation for
EM+Rainbows! is entirely handled by EM.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


      parent reply	other threads:[~2012-07-24  0:21 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-06-19  9:06 Rainbows! + EventMachine + Sinatra::Synchrony == pegged CPU when idle? Ilya Maykov
     [not found] ` <CAAsVqN8=HxMbbQkfonojsoCHgJAPsDhC7hqqcBK9GnYsO2ww7g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-06-19 17:54   ` Eric Wong
     [not found]     ` <20120619175409.GA27303-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-07-23 23:34       ` Ilya Maykov
     [not found]         ` <CAAsVqN91_KiexPc4u9byuVstyknj7TgT4V7evs1x65W+6izzkQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-07-24  0:21           ` Eric Wong [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://yhbt.net/rainbows/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20120724002140.GA25177@dcvr.yhbt.net \
    --to=normalperson-rmlxzr9ms24@public.gmane.org \
    --cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhbt.net/rainbows.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).