Rainbows! Rack HTTP server user/dev discussion
 help / color / Atom feed
* Weird random Zbatery exception
@ 2011-12-02 19:39 Damian Janowski
       [not found] ` <CABAijKG5VwHN9p9p_4R4oSruFVH7j3TY0Z6twxLSLpKoxcO8jw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 7+ messages in thread
From: Damian Janowski @ 2011-12-02 19:39 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw

Hello everyone,

I'm giving Zbatery a try, currently testing with ThreadPool.

By the way, this instance is sitting behind Varnish.

I can't figure out precise steps to trigger this exception, but it
happens after a few GET/POST requests:

/usr/local/lib/ruby/gems/1.9.1/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:361:in
`awaken_master': undefined method `kgio_trywrite' for nil:NilClass
(NoMethodError)
        from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:130:in
`block in start'
        from /usr/local/lib/ruby/gems/1.9.1/gems/rainbows-4.3.1/lib/rainbows/thread_pool.rb:32:in
`call'
        from /usr/local/lib/ruby/gems/1.9.1/gems/rainbows-4.3.1/lib/rainbows/thread_pool.rb:32:in
`join'
        from /usr/local/lib/ruby/gems/1.9.1/gems/rainbows-4.3.1/lib/rainbows/thread_pool.rb:32:in
`block in worker_loop'
        from /usr/local/lib/ruby/gems/1.9.1/gems/rainbows-4.3.1/lib/rainbows/thread_pool.rb:30:in
`each'
        from /usr/local/lib/ruby/gems/1.9.1/gems/rainbows-4.3.1/lib/rainbows/thread_pool.rb:30:in
`worker_loop'
        from /usr/local/lib/ruby/gems/1.9.1/gems/zbatery-4.1.1/lib/zbatery.rb:100:in
`join'
        from /usr/local/lib/ruby/gems/1.9.1/gems/zbatery-4.1.1/bin/zbatery:121:in
`<top (required)>'
        from /usr/local/bin/zbatery:19:in `load'
        from /usr/local/bin/zbatery:19:in `<main>'

(Needless to say, I googled already without much luck).

Any ideas?

D.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Weird random Zbatery exception
       [not found] ` <CABAijKG5VwHN9p9p_4R4oSruFVH7j3TY0Z6twxLSLpKoxcO8jw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2011-12-02 22:47   ` Eric Wong
       [not found]     ` <20111202224711.GA23422-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Wong @ 2011-12-02 22:47 UTC (permalink / raw)
  To: Rainbows! list

Damian Janowski <jano-eWlILCNyKRBBDgjK7y7TUQ@public.gmane.org> wrote:
> I'm giving Zbatery a try, currently testing with ThreadPool.

Thanks for giving it a try! :)

> I can't figure out precise steps to trigger this exception, but it
> happens after a few GET/POST requests:

> /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:361:in
> `awaken_master': undefined method `kgio_trywrite' for nil:NilClass
> (NoMethodError)

Does your application fork/exec other processes?  (`` or "system()",
popen all included).

> (Needless to say, I googled already without much luck).

I think you found a new bug, does the following patch fix it?

diff --git a/lib/zbatery.rb b/lib/zbatery.rb
index 735b1e6..17d7c06 100644
--- a/lib/zbatery.rb
+++ b/lib/zbatery.rb
@@ -82,6 +82,7 @@ module Rainbows
       trap(:USR1) { Thread.new { reopen_logs } }
       trap(:USR2) { Thread.new { reexec } }
       trap(:HUP) { Thread.new { reexec; stop } }
+      trap(:CHLD, "IGNORE")
 
       # technically feasible in some cases, just not sanely supportable:
       %w(TTIN TTOU WINCH).each do |sig|
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Weird random Zbatery exception
       [not found]     ` <20111202224711.GA23422-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2011-12-02 23:40       ` Eric Wong
       [not found]         ` <20111202234003.GA27418-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  2011-12-05 18:02       ` Damian Janowski
  1 sibling, 1 reply; 7+ messages in thread
From: Eric Wong @ 2011-12-02 23:40 UTC (permalink / raw)
  To: Rainbows! list

Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> diff --git a/lib/zbatery.rb b/lib/zbatery.rb
> index 735b1e6..17d7c06 100644
> --- a/lib/zbatery.rb
> +++ b/lib/zbatery.rb
> @@ -82,6 +82,7 @@ module Rainbows
>        trap(:USR1) { Thread.new { reopen_logs } }
>        trap(:USR2) { Thread.new { reexec } }
>        trap(:HUP) { Thread.new { reexec; stop } }
> +      trap(:CHLD, "IGNORE")

On second thought, "IGNORE" will probably break existing code.
trap(:CHLD, "DEFAULT") should be better.

Btw, this issue shouldn't affect Rainbows!, only Zbatery.
Can you confirm?
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Weird random Zbatery exception
       [not found]         ` <20111202234003.GA27418-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2011-12-03  1:46           ` Eric Wong
       [not found]             ` <20111203014632.GA3639-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  2011-12-05 18:02           ` Damian Janowski
  1 sibling, 1 reply; 7+ messages in thread
From: Eric Wong @ 2011-12-03  1:46 UTC (permalink / raw)
  To: Rainbows! list

Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> > +      trap(:CHLD, "IGNORE")
> 
> On second thought, "IGNORE" will probably break existing code.
> trap(:CHLD, "DEFAULT") should be better.

I just pushed out Zbatery 4.1.1.2.g3693 (prerelease) to RubyGems.org
with trap(:CHLD,"DEFAULT") + test cases.

You can install it with: gem install --pre zbatery

Let me know if it fixes your problem since I've been meaning to release
for 90d1d769b9af83d6d1 anyways (unlink oldbin pid file during graceful
shutdown).

(I also found https://redmine.ruby-lang.org/issues/5700 while working
 on this :o)
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Weird random Zbatery exception
       [not found]     ` <20111202224711.GA23422-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  2011-12-02 23:40       ` Eric Wong
@ 2011-12-05 18:02       ` Damian Janowski
  1 sibling, 0 replies; 7+ messages in thread
From: Damian Janowski @ 2011-12-05 18:02 UTC (permalink / raw)
  To: Rainbows! list

On Fri, Dec 2, 2011 at 7:47 PM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> Thanks for giving it a try! :)

Thank you, Eric, for your prompt reply!

> Does your application fork/exec other processes?  (`` or "system()",
> popen all included).

Not in my application code, and I'm pretty sure not in any of the few
libraries I'm using.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Weird random Zbatery exception
       [not found]         ` <20111202234003.GA27418-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  2011-12-03  1:46           ` Eric Wong
@ 2011-12-05 18:02           ` Damian Janowski
  1 sibling, 0 replies; 7+ messages in thread
From: Damian Janowski @ 2011-12-05 18:02 UTC (permalink / raw)
  To: Rainbows! list

On Fri, Dec 2, 2011 at 8:40 PM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> Btw, this issue shouldn't affect Rainbows!, only Zbatery.
> Can you confirm?

I can confirm Rainbows has been stable for a few days (whereas Zbatery
crashed after a few requests).

I'll test the pre release now and let you know, thank you again.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Weird random Zbatery exception
       [not found]             ` <20111203014632.GA3639-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2011-12-05 18:29               ` Damian Janowski
  0 siblings, 0 replies; 7+ messages in thread
From: Damian Janowski @ 2011-12-05 18:29 UTC (permalink / raw)
  To: Rainbows! list

On Fri, Dec 2, 2011 at 10:46 PM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> I just pushed out Zbatery 4.1.1.2.g3693 (prerelease) to RubyGems.org
> with trap(:CHLD,"DEFAULT") + test cases.
>
> You can install it with: gem install --pre zbatery
>
> Let me know if it fixes your problem since I've been meaning to release
> for 90d1d769b9af83d6d1 anyways (unlink oldbin pid file during graceful
> shutdown).

It does seem to fix the problem.

Thank you very much!
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-12-02 19:39 Weird random Zbatery exception Damian Janowski
     [not found] ` <CABAijKG5VwHN9p9p_4R4oSruFVH7j3TY0Z6twxLSLpKoxcO8jw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-12-02 22:47   ` Eric Wong
     [not found]     ` <20111202224711.GA23422-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2011-12-02 23:40       ` Eric Wong
     [not found]         ` <20111202234003.GA27418-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2011-12-03  1:46           ` Eric Wong
     [not found]             ` <20111203014632.GA3639-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2011-12-05 18:29               ` Damian Janowski
2011-12-05 18:02           ` Damian Janowski
2011-12-05 18:02       ` Damian Janowski

Rainbows! Rack HTTP server user/dev discussion

Archives are clonable:
	git clone --mirror http://bogomips.org/rainbows-public
	git clone --mirror http://ou63pmih66umazou.onion/rainbows-public

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.lang.ruby.rainbows
	nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.rainbows

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git