Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
From: "Lin Jen-Shin (godfat)" <godfat-hOE/xeEBYYIdnm+yROfE0A@public.gmane.org>
To: "Rainbows! list" <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: EventMachine with thread pool and thread spawn
Date: Sun, 25 Aug 2013 20:34:44 +0800	[thread overview]
Message-ID: <CAA2_N1sqvUap-97EjpiKyLicXt3J5zeNSws3O4CAJ3VKUvgVcg@mail.gmail.com> (raw)
In-Reply-To: <20130823225114.GA5691-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>

On Sat, Aug 24, 2013 at 6:51 AM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> "Lin Jen-Shin (godfat)" <godfat-hOE/xeEBYYIdnm+yROfE0A@public.gmane.org> wrote:
>> This way, it might be easier for me to "rebase" on master.
>> However, I guess it might be time to give up this approach.
>> All tests passed except for t0106-rack-input-keepalive.sh,
>> but I don't find a good way to make it pass.
>
>> The key might be that pause/resume don't seem to work in
>> EventMachine? And using tempfile to buffer the request
>> might not be realistic.
>
> Yeah, t0106 is a tough one given the EM interface.
>
> Btw, has anybody sent a patch to the EM guys to allow this?
> (I don't do C++)

I just searched on Github and didn't see a relevant patch.
There are some related tickets, and it looks like pause/resume
should just work. I guess it might have some bugs somewhere.

I know some of C++, but I know little about I/O and system
programming. A naive patch that returns early in the event
dispatcher when it detects the connection has been paused,
didn't seem to make t0106 pass.

Since how pause/resume is implemented in EventMachine is
merely a flag telling it's pausing or not, I believe this
kind of implementation could be quite brittle as every place
would need to look into this flag to implement pause/resume
properly... I won't be too surprised this might not work well.

EM is also not actively maintained. A bunch of patches were
not reviewed (or rejected) and just sit there for several years...
This also discourages me putting more effort on it.

>> Another way would be... simply mark this model as not
>> suitable for large chunk pipelined requests. At least it seems
>> working fine on our production site.
>>
>> What do you think?
>
> That's probably fine.

Great! Then I'll try to make some patches for this.

>> Thanks for all your listening.
>>
>> (Maybe it's really time to move forward to celluloid-io,
>> not sure if I would get the chance to work on and finish it though..)
>
> Fwiw, nowadays my personal take these days is to avoid EM/libev*-style
> wrapper library unless they expose (or only have :P) a oneshot interface
> (EPOLLONESHOT/EV_ONESHOT).  This is *especially* the case (for me) when
> mixing epoll/kqueue with threads.
>
> Standard event-triggered and level-triggered interfaces are both too
> confusing to me.  Maybe it's just me, though :x

Since I don't know much about I/O, I don't know what they are :x
I wonder if I should just go with XEpollThreadPool then? My concern
is that as Heroku does not buffer the entire request and response,
we need something which would do this for us. Not sure if XEpollThreadPool
would be sufficient?

Another issue is that most of us develop on a Mac, so Mac support
would be desirable. Because of this I looked into sleepy_penguin,
realizing that it now has kqueue support. I just sent a patch to
its mailing list, trying to make it work on Mac. At least all
tests passed on my computer with that patch.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


  parent reply	other threads:[~2013-08-25 12:37 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-23 21:22 EventMachine with thread pool and thread spawn Lin Jen-Shin (godfat)
     [not found] ` <CAA2_N1uiz7Razb5J6wYCnD0w8sXrbCRp6LnLC+hTg2+Oipfrrw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-08-23 22:51   ` Eric Wong
     [not found]     ` <20130823225114.GA5691-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-08-25 12:34       ` Lin Jen-Shin (godfat) [this message]
     [not found]         ` <CAA2_N1sqvUap-97EjpiKyLicXt3J5zeNSws3O4CAJ3VKUvgVcg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-08-25 21:57           ` Eric Wong
     [not found]             ` <20130825215701.GA31966-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-09-05 19:59               ` Lin Jen-Shin (godfat)
     [not found]                 ` <CAA2_N1utfNGSUcaNv9oLAHVzdO1MbC3wH0ar+wkfMHeCmPjkOQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-09-05 23:03                   ` Eric Wong
     [not found]                     ` <20130905230305.GA5823-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-09-26 17:59                       ` Lin Jen-Shin (godfat)
2013-09-06  6:52                   ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://yhbt.net/rainbows/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAA2_N1sqvUap-97EjpiKyLicXt3J5zeNSws3O4CAJ3VKUvgVcg@mail.gmail.com \
    --to=godfat-hoe/xeebyyidnm+yrofe0a@public.gmane.org \
    --cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhbt.net/rainbows.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).