From: Eric Wong <normalperson@yhbt.net>
To: unicorn list <mongrel-unicorn@rubyforge.org>
Cc: Cedric Maion <cedric@maion.com>
Subject: Re: [PATCH] `kill -SIGTRAP <worker pid>`
Date: Mon, 25 Jun 2012 03:59:37 +0000 [thread overview]
Message-ID: <20120625035937.GA22367@dcvr.yhbt.net> (raw)
In-Reply-To: <4FE6F491.6070108@maion.com>
Cedric Maion <cedric@maion.com> wrote:
> Eric Wong <normalperson@yhbt.net> wrote:
> > SIGKILL timeout is only a last line of defense when the Ruby VM itself
> > is completely broken. Handling SIGTRAP implies the worker can still
> > respond (and /can/ be rescued), so your SIGTRAP handler is worthless if
> > SIGKILL is required to kill a process.
> Sure. But if the VM is responding, being able to get a backtrace is nice.
> And if it's stuck, you won't get anything indeed, but that's still an
> information (in that case, one may eventually want to get a gdb
> backtrace too). No?
Sure it's nice. But the point is you should've had something around to
handle it in your app anyways if your worker was capable of responding
to SIGTRAP at all. The SIGKILL logic only exists in the master because
it must run outside of the worker.
> > See http://unicorn.bogomips.org/Application_Timeouts.html
> Yes, I'm well aware of this. However, when you still get rare unicorn
> timeouts, debugging them is not obvious.
> In my case, a server in a loadbalanced farm sometimes sees all it's
> unicorn workers timeout in the same minute (approx once a day at what
> seems a random time) -- other servers are fine. Couldn't correlate this
> with any specific network/disk/misc system/user activity yet.
I might even crank the unicorn timeout sky high and have something
else (per-worker) handling timeouts + debugging/dumping in this case.
I recall some mailing list threads on similar topics over the years,
gmane has excellent archives and I'd start there (and not the Rubyforge
archives): gmane.org/gmane.comp.lang.ruby.unicorn.general
The Rainbows::ThreadTimeout could be used as a starting point for a Rack
middleware to debug with.
git clone git://bogomips.org/rainbows
cat lib/rainbows/thread_timeout.rb
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
prev parent reply other threads:[~2012-06-25 4:15 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-23 16:12 `kill -SIGTRAP <worker pid>` to get a live ruby backtrace + generate backtrace when murdering worker due to timeout Cedric Maion
2012-06-23 16:12 ` [PATCH] `kill -SIGTRAP <worker pid>` to get a live ruby backtrace + generate backtrace when murdered " Cedric Maion
2012-06-23 18:55 ` [PATCH] `kill -SIGTRAP <worker pid>` Eric Wong
2012-06-24 11:05 ` Cedric Maion
2012-06-25 3:59 ` Eric Wong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/unicorn/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120625035937.GA22367@dcvr.yhbt.net \
--to=normalperson@yhbt.net \
--cc=cedric@maion.com \
--cc=mongrel-unicorn@rubyforge.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).