From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS14383 205.234.109.0/24 X-Spam-Status: No, score=1.0 required=3.0 tests=AWL,HK_RANDOM_FROM, MIME_QP_LONG_LINE,MSGID_FROM_MTA_HEADER shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: rails 2 and slow external services Date: Tue, 14 Dec 2010 06:35:52 +0000 Message-ID: <20101214063552.GA12020@dcvr.yhbt.net> References: <20101213103936.GA8440@dcvr.yhbt.net> <20101214045720.GC5051@dcvr.yhbt.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Trace: dough.gmane.org 1292309352 11713 80.91.229.12 (14 Dec 2010 06:49:12 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 14 Dec 2010 06:49:12 +0000 (UTC) To: Rainbows! list Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Tue Dec 14 07:49:07 2010 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:171 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1PSOh9-00014I-48 for gclrrg-rainbows-talk@m.gmane.org; Tue, 14 Dec 2010 07:49:07 +0100 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 733151858356; Tue, 14 Dec 2010 01:49:06 -0500 (EST) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 941181858346 for ; Tue, 14 Dec 2010 01:35:53 -0500 (EST) Received: from localhost (unknown [127.0.2.5]) by dcvr.yhbt.net (Postfix) with ESMTP id 150A31F7EA; Tue, 14 Dec 2010 06:35:53 +0000 (UTC) ghazel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org wrote: > On Mon, Dec 13, 2010 at 8:57 PM, Eric Wong wrote: > > ghazel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org wrote: > >> > ghazel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org wrote: > >> >> Some of my page loads (currently serviced by Unicorn) spend a great > >> >> deal of time waiting for external services (OpenID, OAuth, etc over > >> >> Net::HTTP and curb), so I'm looking at Rainbows!. I use Rails 2.3.1= 0. > >> > >> This is Ruby 1.8.7 (REE). Is there any interesting difference between > >> ThreadPool and ThreadSpawn in this environment? > > > > ThreadPool is generally more predictable, but ThreadSpawn has lower > > memory usage if your traffic spikes are intermittent or low. > > > > ThreadSpawn is much like the original Mongrel and ThreadPool was an > > experiment with Ruby 1.9 in mind. =A01.9 has more expensive (but slight= ly > > more concurrent) threading. =A0If your bottlenecks are external HTTP > > requests on 1.8, but first instinct would be to use ThreadSpawn. > > > > Ruby 1.9 + ThreadPool would probably be well-suited for large file > > serving to LAN clients with many slowish disks as it can use sendfile > > via IO.copy_stream), but if you can afford the constant memory overhead, > > it could be good in 1.8, too. > = > I don't mind constant memory overhead - actually I prefer it to > spikey memory usage with an unknown peak. Otherwise they should be the > same? Having idle threads with ThreadPool would affect GC performance, actually. > I'm running a bit of my traffic through some Rainbows! right now, but I g= ot: > = > 2010/12/14 02:30:24 [error] 3183#0: *9229917 upstream timed out (110: > Connection timed out) while reading response header from upstream, > client: 1.2.3.4, server: mysite.com, request: "GET /blah HTTP/1.1", > upstream: "http://unix:/tmp/rainbows.sock:/blah", host: "mysite.com", > referrer: "https://foofoo.com" > 2010/12/14 04:28:25 [error] 3182#0: *9440717 upstream timed out (110: > Connection timed out) while reading response header from upstream, > client: 5.6.7.8, server: mysite.com, request: "GET /blah HTTP/1.1", > upstream: "http://unix:/tmp/rainbows.sock:/blah", host: "mysite.com" > = > From nginx in the error log. My proxy_read_timeout is 300, and my > listen backlog is 2048 (for now...). Basically my Rainbows! config is > identical to my Unicorn config, where I have not seen that happen, > except I added "Rainbows! { use :ThreadPool; worker_connections 100 > }". Was your app hitting the Unicorn kill -9 timeout frequently before? In Rainbows!, the kill -9 timeout only kicks in when the entire interpreter/VM is stuck due to a broken C extension or bug in Ruby. If it's some component of your app taking a long time (and relying on the Unicorn kill -9 timeout), you can try the Rainbows::ThreadTimeout middleware: http://rainbows.rubyforge.org/Rainbows/ThreadTimeout.html -- = Eric Wong _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying