From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=0.0 required=3.0 tests=AWL,MSGID_FROM_MTA_HEADER, TVD_RCVD_IP shortcircuit=no autolearn=unavailable version=3.3.2 Path: news.gmane.org!not-for-mail From: "Lin Jen-Shin (godfat)" Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: negative timeout in Rainbows::Fiber::Base Date: Fri, 31 Aug 2012 05:33:52 +0800 Message-ID: References: <20120825024556.GA25977@dcvr.yhbt.net> <20120829211707.GA22726@dcvr.yhbt.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1346362485 16533 80.91.229.3 (30 Aug 2012 21:34:45 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 30 Aug 2012 21:34:45 +0000 (UTC) To: "Rainbows! list" Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Thu Aug 30 23:34:45 2012 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=godfat.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=15THXKMw5wnsv/lyE2vSXJXGUg2NTkvq/hTAPwExQnY=; b=KOFFA/fQUlz2jU5vzbrq485cQDzWF2cof2oHq2N3OXdRcWgvN9BaLyje+tYEYYiDae 7UGaHXWru6oB0JVXwLOwvpDh5qlex7B70H3qcneB9GAQcFJ2i4YIe2EA2DCl6Hww2Nxh E1wH2mn+2od5/+TinuGAFX5Ehv9dPmewZ4A7I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:x-gm-message-state; bh=15THXKMw5wnsv/lyE2vSXJXGUg2NTkvq/hTAPwExQnY=; b=cwZ2PnfkIS0ZuFSi/hineELD9wEVKopecJ1w84Dtv7FCDC8/w57A3dqfKZYF5awZGC Vcp76RpYMgTwZEpRVLvqoPM8FPuA46zJlXQ/Lq7hmtbY0pX19k5Fv0tsaYjeIcMcrjmy yGNlavVWf1zTxbLlXjirCR6eYk69jmm5D5INnprSePFso+kG1mGWP1oNAEyw+wHsCF3U C2n+bdrkxXHR4KAVNacSwiZ8PMJKvLBAjuS/eVN7V/Wh07b1g+21jX5N8B9wE4U7CPvK 9gHg2WW+ZA7aHbRva4GVgWY0N04dIP9nKK3H5Ei5+dLjCZhMIKiXpGz518bWEne2jfD+ 5hSw== In-Reply-To: <20120829211707.GA22726-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> X-Gm-Message-State: ALoCoQlj9S+S3SGhbtAUYjjyYyhxQJtQbyFpTdDkqSmSU0Dbsd+8hu4uK/O4cS2URPm2OjLNhPee X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:400 Archived-At: Received: from 50-56-192-79.static.cloud-ips.com ([50.56.192.79] helo=rubyforge.org) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1T7CNw-0002ym-21 for gclrrg-rainbows-talk@m.gmane.org; Thu, 30 Aug 2012 23:34:44 +0200 Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id B24522E062; Thu, 30 Aug 2012 21:34:39 +0000 (UTC) Received: from mail-lpp01m010-f50.google.com (mail-lpp01m010-f50.google.com [209.85.215.50]) by rubyforge.org (Postfix) with ESMTP id 157662E060 for ; Thu, 30 Aug 2012 21:34:23 +0000 (UTC) Received: by lago2 with SMTP id o2so2236243lag.23 for ; Thu, 30 Aug 2012 14:34:22 -0700 (PDT) Received: by 10.112.83.35 with SMTP id n3mr1803899lby.68.1346362462786; Thu, 30 Aug 2012 14:34:22 -0700 (PDT) Received: by 10.114.71.44 with HTTP; Thu, 30 Aug 2012 14:33:52 -0700 (PDT) On Thu, Aug 30, 2012 at 5:17 AM, Eric Wong wrote: > Oops :x I encourage unsubscribed senders to remind repliers > to Cc: them for this reason. > > Unfortunately ruby-talk has conditioned the Ruby community to favor > subscriber-only lists and the Mailman config for rainbows-talk avoids > Cc-by-default (even though we'll still accept unsubscribed senders). That's ok, since most of time I would check the archive regularly whenever I am expecting responses. I forgot this time because I was quite busy recently. And now I subscribed the list because I guess there won't have too many mails anyway and I would be definitely interested in discussing Rainbows! :) > Anyways, I've pushed my patch to master as > commit e794a40049959a23ba311c572e518bb7c2861812 > > Will probably release v4.4.1 with that tonight unless there's something > else... Yeah, I saw it, thank you. > I haven't followed celluloid* closely since it was originally announced. > Maybe it's worth it to offer celluloid-io as an option and I will accept > patches for it. Cool. Then I might want to try it some other time. I don't have confidence though. >> https://github.com/godfat/ruby-server-exp/blob/9d44f8387739f5395bf97aff1689f17615cc4c7e/config/rainbows-em-thread-pool.rb#L21-25 >> class REMTPoolClient < Rainbows::EventMachine::Client >> def app_call input >> EM.defer{ super } >> end >> end > > I seem to recall this didn't work well with some corner cases > (large static files/socket-proxying in responses). Sorry that I have no idea about those corner cases. (not sure if they are corner enough?) I am not surprised if this approach won't work for all cases. I would be interested to try out though. Not sure what's socket-proxying, but I'll try large static files, and see why it's not working properly. Hope it would be easy to reproduce. >> > > https://github.com/cardinalblue/rest-more/blob/master/example/rainbows.rb#L15-19 >> > > class REMFClient < Rainbows::EventMachine::Client >> > > def app_call input >> > > Fiber.new{ super }.resume >> > > end >> > > end >> > Does it actually show benefits compared to regular EM? >> > I suppose the Rack application still needs to be made Fiber-aware >> > to reap benefits of Fibers >> >> The Rack application doesn't need to be fiber-aware, but if >> the libraries are fiber-aware, then it would be beneficial. >> The rest of program doesn't have to know the existence of >> fibers, because we don't throw :async. > > OK. But are clients served concurrently by Rainbows! in this case? > > I'm not sure if I'm following this correctly (haven't thought about > Fibers in a long time), but control never goes back to Rainbows! itself > in your above case, does it? It does as far as I know. I am not good at explaining in natural languages, and English is not my first language. Let me show this concept in codes. It's something like this: ( Also in gist: https://gist.github.com/3540749 ) # Wrap a fiber around app_call Fiber.new{ # [...] # Below is modified from event_machine/client.rb, in def app_call input, # but let's forget about :async for now. # In APP.call, it would call Fiber.yield whenever we're waiting for data. status, headers, body = APP.call(@env.merge!(RACK_DEFAULTS)) # The response is fully ready at this point. ev_write_response(status, headers, body, @hp.next?) }.resume # * * * * * # Here's the app APP = lambda{ |env| # First we remember where we need to jump back f = Fiber.current # Wait for 5 seconds to emulate waiting for data EM.add_timer(5){ # Here then we have data available, let's resume back. # But we also wrap resuming in a next_tick block, # giving EM some chances to cleanup it's internal states. EM.next_tick{ f.resume('OK') } } # Here we immediately yield and give control back to Rainbows! # and Rainbows! would then go back to EventMachine's regular loop, # buffering requests, making requests, etc. body = Fiber.yield # So now body is referring the data resumed from above. # Rainbows! should handle the response at this point. [200, {}, [body]] } Not sure if this is clear enough, please let me know if I didn't make it clear, or there's anything wrong in my assumption or the over simplified codes. Thanks! > Interesting. Are you on 32 or 64-bit and are you constrained by VMSize > or RSS? I think it's an x86_64 VM. Not sure what's VMSize, but we're observing memory usage in RSS. They claim we have about 500M for one process, but sometimes the app cannot allocate more memory even below 500M. (Well, I don't quite understand how the app would grow to 500M? It's about 160M on average) > If it's VMSize and you're not dealing with deep data structures _and_ > your code doesn't use recursion much; lowering pthreads stack size to > 64K in thread_pthread.c should help with memory usage. Not sure if you > can supply your own Ruby builds for that hosting service, though. There's a chance to do that on Heroku, but I guess it's too much effort to do so. I don't feel there's a standard way to compile stuffs on it. How do I tell the current stack size for a thread in pthread? It seems I cannot run ulimit on Heroku. On the other hand, I am not sure if it's significant enough to reduce thread stack size. I guess we can also reduce the concurrent level (worker_connections?) to reduce memory usage, too? > If you have time, you could also try making something like GNU pth or > npth work with Ruby 1.9. I suspect Fibers will become unusable > with *pth, though... Cool, might be interesting. Could then threads be as lightweight as fibers? :P Though I really doubt if threads are really that heavy comparing to fibers. At least in some simple tests, threads are fine and efficient enough. EventMachine is still a lot faster than regular sockets (net/http) though, so I'll still keep EventMachine for a while even if I switched to threads. > I'm actually surprised unicorn got the attention it has. > > As project leader, my preference to maintain a low public profile and my > refusal to endorse/associate with for-profit interests certainly hurts > adoption. > > If somebody else (like you) wants to market these projects, then more > power to them :) I'm certainly never going to speak at any conference. > > Rainbows! just overwhelms potential users with too many choices, > so it's unlikely to ever be widely-adopted. Umm... I don't know the reason why you want to stay away from them, but please let me know if you don't want someone to market them. I'll then try to be as natural as possible when talking about Unicorns. I guess Github's post certainly attracted at lot of people's attention. I don't remember where I did see Unicorn in the beginning, but certainly I became more interested because of that post. I guess that's also one of the reasons that people didn't know about Rainbows! (they didn't mention it) Or you might be right, a lot of people just don't try Rainbows! because there are too many choices. Or, use it wrongly, I guess... _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying