Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
To: Rainbows! list <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: EventMachine with thread pool and thread spawn
Date: Thu, 5 Sep 2013 23:03:06 +0000	[thread overview]
Message-ID: <20130905230305.GA5823@dcvr.yhbt.net> (raw)
In-Reply-To: <CAA2_N1utfNGSUcaNv9oLAHVzdO1MbC3wH0ar+wkfMHeCmPjkOQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

"Lin Jen-Shin (godfat)" <godfat-hOE/xeEBYYIdnm+yROfE0A@public.gmane.org> wrote:
> On Mon, Aug 26, 2013 at 5:57 AM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> > A multi-threaded HTTP server must implement its own synchronization
> > on top of ET/LT epoll/kqueue to prevent serving the same client
> > from multiple threads.
> 
> I searched and read a bit. I am not sure if I understand correctly,
> but I have a feeling that both LT and ET epoll are not designed to
> be used in a multithreaded server (i.e. handle I/O in epoll and dispatch
> tasks to threaded handlers), or say it would be quite hard to get it right.

I came to the same conclusion as you a while back.

The epoll_ctl() API isn't very efficient for single-threaded servers
(this is probably the biggest complaint of existing single-threaded
epoll users).  The kevent() API is better for single-threaded operation,
but I'm not sure if it's optimal for multithreaded.

I might propose a new epoll_xchg syscall which behaves closer to kevent.
It would only be a tiny improvement for multi-threaded epoll, but
possibly big for single-threaded epoll (but would require rewriting
existing single-threaded apps).  I don't think it's worth it since
there's lower hanging fruit for improving MT epoll performance.

> That is, oneshot would be the way to go in this scenario?
> I also heard that libev is merely a thin wrapper around epoll...

Yes.  Fwiw, I've been working on yet another Ruby server :)
It's not Rack-specific and the design is based on cmogstored
(a oneshot + multithreaded storage server for MogileFS).

> >> I wonder if I should just go with XEpollThreadPool then? My concern
> >> is that as Heroku does not buffer the entire request and response,
> >> we need something which would do this for us. Not sure if XEpollThreadPool
> >> would be sufficient?
> >
> > Probably using XEpollThread* is a good choice, but neither buffer (but
> > you get more I/O concurrency anyways, so maybe it's not so bad)
> 
> After benching mark against Heroku, I feel it does have some kind of
> buffers. It's hard to tell though........ but yeah, maybe we don't really
> need full buffering at application server level.

If your responses are all small, you'll never need/hit output buffering
in userspace (the kernel always buffers some).  I think any proxy
(Heroku included) does full header buffering, it's just few proxies can
do full request body (PUT/POST) buffering like nginx.

> > Plain XEpoll does do full buffering, and you can probably add a
> > TryDefer-like middleware on top of that, even...
> 
> Could you please explain to me why there's difference?

XEpoll is similar to the EventMachine use in Rainbows!.  It just won't
disconnect if a client goes overboard with pipelining, and there's no EM
API.  It reads all of the input before processing, and will buffer if
writes are blocked.

> Is it because it might not make too much sense to buffer in
> XEpollThread*, or implementation-wise, it's easier to do it this way?

Yeah, it was easier.  (Lazy) buffering with multiple threads helps,
too, but not as much as buffering with a single thread.  Threads
are fairly cheap nowadays.

> > It could be a librelist problem, too, but I just sent out a message
> > considering a move away from librelist and it went through fine:
> > http://mid.gmane.org/20130825212946.GA32430-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org
> 
> What's happening was that I sent it to librelist, and received an
> email from librelist for confirming if I am subscribing it. I replied
> the confirmation, wondering if my previous mail would show up
> or not?
> 
> I was worried about sending it twice, thus didn't send again.

Ah, the message says to send it again, I think.  Anyways, I don't
mind receiving too much email as long as it's plain-text.

> It seems after the confirmation everything works normally.
> Not sure if librelist is a good choice, but at least it seems
> working for me now.

Yeah, I don't think I'll use librelist for new projects (especially
since I favor reply-all more nowadays).  Not sure if it's worth the
migration hassle for small projects like s.p, though...

I'll probably go with Rubyforge or savannah.nongnu.org for new projects
for now.  Maybe using something completely decentralized and friendly to
command-line users will be suitable in the future.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


  parent reply	other threads:[~2013-09-05 23:04 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-23 21:22 EventMachine with thread pool and thread spawn Lin Jen-Shin (godfat)
     [not found] ` <CAA2_N1uiz7Razb5J6wYCnD0w8sXrbCRp6LnLC+hTg2+Oipfrrw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-08-23 22:51   ` Eric Wong
     [not found]     ` <20130823225114.GA5691-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-08-25 12:34       ` Lin Jen-Shin (godfat)
     [not found]         ` <CAA2_N1sqvUap-97EjpiKyLicXt3J5zeNSws3O4CAJ3VKUvgVcg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-08-25 21:57           ` Eric Wong
     [not found]             ` <20130825215701.GA31966-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-09-05 19:59               ` Lin Jen-Shin (godfat)
     [not found]                 ` <CAA2_N1utfNGSUcaNv9oLAHVzdO1MbC3wH0ar+wkfMHeCmPjkOQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-09-05 23:03                   ` Eric Wong [this message]
     [not found]                     ` <20130905230305.GA5823-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-09-26 17:59                       ` Lin Jen-Shin (godfat)
2013-09-06  6:52                   ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://yhbt.net/rainbows/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130905230305.GA5823@dcvr.yhbt.net \
    --to=normalperson-rmlxzr9ms24@public.gmane.org \
    --cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhbt.net/rainbows.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).