From: Sam Saffron <sam.saffron@gmail.com>
To: Eric Wong <e@80x24.org>
Cc: unicorn-public <unicorn-public@bogomips.org>
Subject: Re: Master wait time metric
Date: Tue, 1 Nov 2016 19:21:31 +1100 [thread overview]
Message-ID: <CAAtdryOYrXQfnZtxUousnZrATvcfPsoQPFAzrJKU3WqN86=eLQ@mail.gmail.com> (raw)
In-Reply-To: <20161031233646.GA10820@dcvr>
Brilliant, this will make it super easy to write a Prometheus
exporter, will post here once I am done.
On Tue, Nov 1, 2016 at 10:36 AM, Eric Wong <e@80x24.org> wrote:
> Sam Saffron <sam.saffron@gmail.com> wrote:
>> Hi Eric / everyone :)
>>
>> I would like to start graphing how long our master process spends
>> waiting for worker processes to be available.
>
> Fwiw, the master doesn't wait for workers to become available for
> processing requests. But I think I know what you mean to ask :>
>
> Rather, the connection request sits in the listen queue (a kernel
> object) shared by all workers, and instrumenting this is always
> kernel-dependent (because unicorn avoids doing stuff in userspace).
>
>> This metric will allow us to quickly tell if a unicorn is being
>> overloaded and allow us to quickly remediate.
>>
>> Once a minute I want to ask the master process how long it spent
>> waiting for child processes to become available.
>>
>> How would I go about getting that metric?
>
> Linux-only, but you can probably look at Raindrops::LastDataRecv
>
> https://bogomips.org/raindrops/Raindrops/LastDataRecv.html
>
> Raindrops::Middleware can give you how big the listen queue is,
> too. Ideally, this should never exceed 1.
>
> https://bogomips.org/raindrops/Raindrops/Middleware.html
>
>
> You can probably get the same metrics directly from the kernel
> via systemtap, dtrace, or similar, too.
prev parent reply other threads:[~2016-11-01 8:21 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-31 23:08 Master wait time metric Sam Saffron
2016-10-31 23:36 ` Eric Wong
2016-11-01 8:21 ` Sam Saffron [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/unicorn/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAAtdryOYrXQfnZtxUousnZrATvcfPsoQPFAzrJKU3WqN86=eLQ@mail.gmail.com' \
--to=sam.saffron@gmail.com \
--cc=e@80x24.org \
--cc=unicorn-public@bogomips.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).