From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=1.7 required=3.0 tests=AWL,HK_RANDOM_FROM, MSGID_FROM_MTA_HEADER,RDNS_NONE shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: EventMachine with thread pool and thread spawn Date: Sun, 25 Aug 2013 21:57:01 +0000 Message-ID: <20130825215701.GA31966@dcvr.yhbt.net> References: <20130823225114.GA5691@dcvr.yhbt.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1377467824 560 80.91.229.3 (25 Aug 2013 21:57:04 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 25 Aug 2013 21:57:04 +0000 (UTC) To: Rainbows! list Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Sun Aug 25 23:57:08 2013 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Broken-Reverse-DNS: no host name found for IP address 50.56.192.79 Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:526 Archived-At: Received: from [50.56.192.79] (helo=rubyforge.org) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VDiJ2-0002ks-3M for gclrrg-rainbows-talk@m.gmane.org; Sun, 25 Aug 2013 23:57:08 +0200 Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 3B74C2E182; Sun, 25 Aug 2013 21:57:08 +0000 (UTC) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 45D7D2E17D for ; Sun, 25 Aug 2013 21:57:03 +0000 (UTC) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 438C31F70D; Sun, 25 Aug 2013 21:57:01 +0000 (UTC) "Lin Jen-Shin (godfat)" wrote: > On Sat, Aug 24, 2013 at 6:51 AM, Eric Wong wrote: > > "Lin Jen-Shin (godfat)" wrote: > EM is also not actively maintained. A bunch of patches were > not reviewed (or rejected) and just sit there for several years... > This also discourages me putting more effort on it. Yeah. I get that feeling, too. (And also what I've said about being hooked on ONESHOT nowadays :) > > Fwiw, nowadays my personal take these days is to avoid EM/libev*-style > > wrapper library unless they expose (or only have :P) a oneshot interface > > (EPOLLONESHOT/EV_ONESHOT). This is *especially* the case (for me) when > > mixing epoll/kqueue with threads. > > > > Standard event-triggered and level-triggered interfaces are both too > > confusing to me. Maybe it's just me, though :x > > Since I don't know much about I/O, I don't know what they are :x kqueue is a queue, it's in the name. I haven't looked too hard at the internal details, but the name says much about it :) I've studied epoll internals a bit and know that epoll is also a queue, it's just not in the name. Oneshot behavior basically makes epoll/kqueue behave like a traditional queue; once you pull a file/socket off the queue, it won't magically reappear unless you ask epoll/kqueue to watch for it again. This is significant for multithreaded servers, because it allows the kernel to handle synchronization (just like accept()). Level/edge-triggered epoll/kqueue is kind of like a tracklist in a music player with repeat=on + random=on. It's fine if you're playing one track-a-time (like a single-threaded HTTP server), but if you try to play multiple tracks in different threads, you could end up playing the _same_ track in two or more threads. A multi-threaded HTTP server must implement its own synchronization on top of ET/LT epoll/kqueue to prevent serving the same client from multiple threads. > I wonder if I should just go with XEpollThreadPool then? My concern > is that as Heroku does not buffer the entire request and response, > we need something which would do this for us. Not sure if XEpollThreadPool > would be sufficient? Probably using XEpollThread* is a good choice, but neither buffer (but you get more I/O concurrency anyways, so maybe it's not so bad) Plain XEpoll does do full buffering, and you can probably add a TryDefer-like middleware on top of that, even... Anyways, it's still infuriating to me that Heroku cannot do full buffering and still claims to "support" unicorn. > Another issue is that most of us develop on a Mac, so Mac support > would be desirable. I can't officially support non-Free systems, but I can accept non-intrusive patches. I'm not nearly as experienced with kqueue, so maybe you really found a bug in my kqueue API. Anyways, I've considered unifying a X{Epoll,Kqueue}Thread* interface, which is why I added kqueue support to sleepy_penguin before I forgot about it :x > Because of this I looked into sleepy_penguin, > realizing that it now has kqueue support. I just sent a patch to > its mailing list, trying to make it work on Mac. At least all > tests passed on my computer with that patch. I didn't get it on the list and can't find it on the archives, either. http://librelist.com/browser/sleepy.penguin/ Are you sure it went out? Feel free to Bcc me or send it here. (Librelist doesn't like plain Cc :<) It could be a librelist problem, too, but I just sent out a message considering a move away from librelist and it went through fine: http://mid.gmane.org/20130825212946.GA32430-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying