From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=1.3 required=3.0 tests=AWL,MSGID_FROM_MTA_HEADER, RDNS_NONE shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: "Lin Jen-Shin (godfat)" Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: EventMachine with thread pool and thread spawn Date: Sun, 25 Aug 2013 20:34:44 +0800 Message-ID: References: <20130823225114.GA5691@dcvr.yhbt.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1377434259 28283 80.91.229.3 (25 Aug 2013 12:37:39 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 25 Aug 2013 12:37:39 +0000 (UTC) To: "Rainbows! list" Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Sun Aug 25 14:37:43 2013 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=godfat.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=OageNnEPZtaRrIvEmg9dkxJGuXL9EBoZcv7g65l10C8=; b=nto937TyOP8qEKF8kDQE5dwK3YAcxcDQ9xOIaqJLr/veRjRQ7Oba2kouWdYgSkGqeK bE4j+oAuS6Hfzw2yEHzGT1tUYBowxjQp2WYSZUdzvZqZCajlh5WgMf0erPU5MXJVTUC6 xUuMFWBHyNwNNdLXIXr3kum50T/ezOvDAmnTw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=OageNnEPZtaRrIvEmg9dkxJGuXL9EBoZcv7g65l10C8=; b=XA2HXk6oSFrhIt/7S96r3KwPILqMHHJF6N2wCcT0AqT7a3/tHQzokTEzux8Nrk58mH Tnagjj5R1eqYSY0hNFAH/XuWQho5rRWQmoHV6o93cO1uk3Gg/JAKMHOtJFgdUZmkGeF3 7ooR6ILE7YGrFx4A3qSF2bSYPLmzsEMTyeLmQvY6G0ybKkfYN3uvdqqfUgxaOqjd8p67 EtAz2lYEmqiXb8dZFlwyCBf9RPpZkEc0zfbnvImWY9cCaJjgzRfHzX3GvomV9yJcVLXr Lqg6NFmxXq7kImY7qHFGbG4JdoAYQcY3TR07AKA908se/Itb0ho+B8AKd5a1AehV0hPw llSg== X-Gm-Message-State: ALoCoQlODwl29LZubIuQ5neyf8hqL93iyxbimJx0FAzgaw4fpmAiUaiI9CwO714wAOEJSVt//ApU X-Received: by 10.15.67.131 with SMTP id u3mr16443518eex.34.1377434115013; Sun, 25 Aug 2013 05:35:15 -0700 (PDT) In-Reply-To: <20130823225114.GA5691-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Broken-Reverse-DNS: no host name found for IP address 50.56.192.79 Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:524 Archived-At: Received: from [50.56.192.79] (helo=rubyforge.org) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VDZZd-0002ek-SX for gclrrg-rainbows-talk@m.gmane.org; Sun, 25 Aug 2013 14:37:42 +0200 Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 6473D2E190; Sun, 25 Aug 2013 12:37:41 +0000 (UTC) Received: from mail-ee0-f42.google.com (mail-ee0-f42.google.com [74.125.83.42]) by rubyforge.org (Postfix) with ESMTP id BAAEC2E153 for ; Sun, 25 Aug 2013 12:35:16 +0000 (UTC) Received: by mail-ee0-f42.google.com with SMTP id b45so1119988eek.1 for ; Sun, 25 Aug 2013 05:35:15 -0700 (PDT) Received: by 10.223.172.69 with HTTP; Sun, 25 Aug 2013 05:34:44 -0700 (PDT) On Sat, Aug 24, 2013 at 6:51 AM, Eric Wong wrote: > "Lin Jen-Shin (godfat)" wrote: >> This way, it might be easier for me to "rebase" on master. >> However, I guess it might be time to give up this approach. >> All tests passed except for t0106-rack-input-keepalive.sh, >> but I don't find a good way to make it pass. > >> The key might be that pause/resume don't seem to work in >> EventMachine? And using tempfile to buffer the request >> might not be realistic. > > Yeah, t0106 is a tough one given the EM interface. > > Btw, has anybody sent a patch to the EM guys to allow this? > (I don't do C++) I just searched on Github and didn't see a relevant patch. There are some related tickets, and it looks like pause/resume should just work. I guess it might have some bugs somewhere. I know some of C++, but I know little about I/O and system programming. A naive patch that returns early in the event dispatcher when it detects the connection has been paused, didn't seem to make t0106 pass. Since how pause/resume is implemented in EventMachine is merely a flag telling it's pausing or not, I believe this kind of implementation could be quite brittle as every place would need to look into this flag to implement pause/resume properly... I won't be too surprised this might not work well. EM is also not actively maintained. A bunch of patches were not reviewed (or rejected) and just sit there for several years... This also discourages me putting more effort on it. >> Another way would be... simply mark this model as not >> suitable for large chunk pipelined requests. At least it seems >> working fine on our production site. >> >> What do you think? > > That's probably fine. Great! Then I'll try to make some patches for this. >> Thanks for all your listening. >> >> (Maybe it's really time to move forward to celluloid-io, >> not sure if I would get the chance to work on and finish it though..) > > Fwiw, nowadays my personal take these days is to avoid EM/libev*-style > wrapper library unless they expose (or only have :P) a oneshot interface > (EPOLLONESHOT/EV_ONESHOT). This is *especially* the case (for me) when > mixing epoll/kqueue with threads. > > Standard event-triggered and level-triggered interfaces are both too > confusing to me. Maybe it's just me, though :x Since I don't know much about I/O, I don't know what they are :x I wonder if I should just go with XEpollThreadPool then? My concern is that as Heroku does not buffer the entire request and response, we need something which would do this for us. Not sure if XEpollThreadPool would be sufficient? Another issue is that most of us develop on a Mac, so Mac support would be desirable. Because of this I looked into sleepy_penguin, realizing that it now has kqueue support. I just sent a patch to its mailing list, trying to make it work on Mac. At least all tests passed on my computer with that patch. _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying