graceful shutdown

graceful shutdown

[Mike Perham]

I've got a multithreaded system, some threads are responding to web
requests, others are performing background/async tasks.  Is there any
way to implement graceful shutdown with Rainbows?  Currently I have a
blocking SIGQUIT shutdown hook for my async task library:

https://github.com/mperham/girl_friday/blob/master/lib/girl_friday.rb#L55

Will this work? Is this all that's really needed?
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: graceful shutdown

[Eric Wong]

Mike Perham <mperham-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> I've got a multithreaded system, some threads are responding to web
> requests, others are performing background/async tasks.  Is there any
> way to implement graceful shutdown with Rainbows?  Currently I have a
> blocking SIGQUIT shutdown hook for my async task library:
> 
> https://github.com/mperham/girl_friday/blob/master/lib/girl_friday.rb#L55
> 
> Will this work? Is this all that's really needed?

No, install an at_exit block instead.

Rainbows! itself relies on SIGQUIT for graceful shutdown, so your trap
would conflict with that.

There are some caveats since Rainbows! won't wait forever for graceful
shutdowns, either.

The Unicorn/Rainbows! master process defaults the "timeout"[1] parameter
to 60s.  If an at_exit handler takes a long time, increase the "timeout"
directive.

The "timeout" directive affects how long the master process will wait
for a worker process to gracefully exit after a SIQQUIT is sent.  If the
timeout expires after SIGQUIT, the master will assume the worker is
non-responsive/stuck and send a SIGKILL to finish the worker off.

By definition of a graceful shutdown, Rainbows! will also try to wait
until all currently running requests are complete, too[2].  Having a
long keepalive_timeout (default is only 5s) also affects the time it
takes to shutdown[3].


[1] http://unicorn.bogomips.org/Unicorn/Configurator.html#method-i-timeout
[2] this is a huge pain for infinite endpoints like
    http://raindrops-demo.bogomips.org/tail/0.0.0.0%3A80.txt :<
[3] It's not consistent across concurrency options right now and
    not easy to fix consistently.

-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying