From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=1.0 required=3.0 tests=AWL,HK_RANDOM_FROM, MSGID_FROM_MTA_HEADER,TVD_RCVD_IP shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: negative timeout in Rainbows::Fiber::Base Date: Sat, 22 Sep 2012 19:42:22 +0000 Message-ID: <20120922194222.GA6839@dcvr.yhbt.net> References: <20120825024556.GA25977@dcvr.yhbt.net> <20120829211707.GA22726@dcvr.yhbt.net> <20120831013731.GA16613@dcvr.yhbt.net> <20120905232739.GA25153@dcvr.yhbt.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1348342954 1280 80.91.229.3 (22 Sep 2012 19:42:34 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 22 Sep 2012 19:42:34 +0000 (UTC) To: Rainbows! list Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Sat Sep 22 21:42:37 2012 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:407 Archived-At: Received: from 50-56-192-79.static.cloud-ips.com ([50.56.192.79] helo=rubyforge.org) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1TFVb1-0008JT-5C for gclrrg-rainbows-talk@m.gmane.org; Sat, 22 Sep 2012 21:42:35 +0200 Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 8E5F22E071; Sat, 22 Sep 2012 19:42:29 +0000 (UTC) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id D635B2E06D for ; Sat, 22 Sep 2012 19:42:23 +0000 (UTC) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id B1F411F42D; Sat, 22 Sep 2012 19:42:22 +0000 (UTC) "Lin Jen-Shin (godfat)" wrote: > Here's a problem troubles me for a long time, and I failed to figure out > what's going on and cannot find a proper fix. I didn't mention this before, > because I don't think it's Zbatery/Rainbows' issue, but it turns out that > it might be related, and perhaps you would be interested to provide > some hints, I would much appreciate it. > > So we're running all our apps on http://www.heroku.com > It's a hosting service that you simply `git push` to deploy your app, > worrying not all the complexity regarding deployment. > > There's a very weird issue running Zbatery/Rainbows on it. > I don't remember if Unicorn would be suffering from that > since I don't run Unicorn directly for a while now. > > The problem comes with Rails 3 and its "assets pipeline" feature. > It's basically a feature that let your assets (e.g. images, javascripts, > css, etc) no longer simple assets which could be served directly > via nginx or some other static web server. > > Like, developers could name their coffeescripts files as: > > application.coffee.js > > And when a client is requesting "application.js", that "assets pipeline" > thing would compile the coffeescript file into a javascript file on the fly, > and return it to the client. > > We can also "precompile" those assets before deploying to production. > Heroku would do this if it detects that the application is deploying is > built with Rails 3. > > If this precompile thing is done successfully, then there's no problems, > at least in some of the cases.... But what if precompile failed? In my cases, > it would usually cause system stack overflow whenever it tries to compile > it on the fly. > > Oh, suddenly I think maybe it's because I would wrap a fiber around > each request in some cases... and it just used too many method calls. > I am not sure about that... might check it next time I saw it. > > Somehow I feel this might be related to persistent connections or > pipelined requests, because last time I didn't write the client class > correctly to handle pipelined requests as you pointed out, it would > also cause some issues regarding "assets pipeline" on Heroku. > > I believe Heroku is running nginx in front off Zbatery, and if I didn't > get pipelined requests correctly, it would result timeout for many of > assets the client is asking about. I feel Heroku is using pipelined > requests if the client is using this. > > And the most weird thing I can't understand is, I am trying to convince > one of my friend to try out Zbatery, but using ThreadSpawn model > would usually cause serving assets timeout, just like in my case > where I didn't get pipelined requests correct. Switching to EventMachine > solved the problem! Huh? > > Moreover, once there are some assets timeout issues on EventMachine, > too. When I tried to debug this, I put some traces into Rainbows, > realizing that sometimes EventMachine didn't call `receive_data' > when receiving some pipelined requests. Could it be an eventmachine bug!? It could be the front-end proxy (incorrectly) detected the Rainbows! instance was down and stopped sending traffic to it. Does this information get logged? > Sorry that I wrote so much and it's unclear what's going on, since > I don't understand so I can't remember correctly... Yeah, it's a bit much to understand. Can you reproduce it consistently? With the serving timeout for Zbatery+ThreadSpawn, can you ensure Content-Length/Transfer-Encoding:chunked is set in the response headers? Since you mentioned stack overflows in response generation, perhaps whatever proxy Heroku is using doesn't handle crashed servers during the response correctly... Can you get stderr logs from Heroku? > If you have any idea for a specific case, I can try to provide all the > information I could collect. I can't get the nginx configuration Heroku > is using though :( I highly doubt nginx will pipeline requests, but we're not sure if they're really using nginx, yet. With the problems you've described, it doesn't sound like they are, or they're using some broken version of it. _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying