Timeout on Rainbows!

Timeout on Rainbows!

[Lin Jen-Shin (godfat)]

Hi,

I am reworking on thread pool for EventMachine, and just realized that
Rainbows doesn't try to put a timeout for each request. Previously,
I thought the timeout setup in Unicorn would be used for each request.

Then I found Rainbows::ThreadTimeout is probably what I want.
I wonder, does that make sense to do something like this, wrap the
application with Rainbows::ThreadTimeout with the same timeout
time in Unicorn automatically?

  def worker_loop worker
    build_app! # This is needed for preload_app=false
    server = Rainbows.server
    server.app = Rainbows::ThreadTimeout.new(server.app,
                   :timeout => server.config.set[:timeout])
    super
  end

I did this in my attempt for EventMachineThreadPool:
https://github.com/godfat/rainbows-emtp/blob/rainbows-emtp-0.5.0/lib/rainbows/event_machine_thread_pool.rb#L14-L20

Because I saw this in EventMachine model:

  def worker_loop(worker) # :nodoc:
    init_worker_process(worker)
    server = Rainbows.server
    server.app.respond_to?(:deferred?) and
      server.app = TryDefer.new(server.app)
  [...]

and I think in most cases we would want this timeout for each
request. The down side might be that then the users cannot
use their own Rainbows::ThreadTimeout (maybe?) or setup a custom
threshold.

Probably then we could introduce another timeout/threshold config
in Configurator? I feel it would be common enough.

Thoughts? Thanks!
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: Timeout on Rainbows!

[Eric Wong]

"Lin Jen-Shin (godfat)" <godfat-hOE/xeEBYYIdnm+yROfE0A@public.gmane.org> wrote:
> I am reworking on thread pool for EventMachine, and just realized that
> Rainbows doesn't try to put a timeout for each request. Previously,
> I thought the timeout setup in Unicorn would be used for each request.

The normal timeout is only a last resort (it works when the VM is hosed
on a worker).  ThreadTimeout isn't nearly as reliable because it works
on the same VM/process, and it's still always preferable to rely on
"local" timeouts exposed by every library.  (see
http://unicorn.bogomips.org/Application_Timeouts)

> Then I found Rainbows::ThreadTimeout is probably what I want.
> I wonder, does that make sense to do something like this, wrap the
> application with Rainbows::ThreadTimeout with the same timeout
> time in Unicorn automatically?

<snip>

> Because I saw this in EventMachine model:
> 
>   def worker_loop(worker) # :nodoc:
>     init_worker_process(worker)
>     server = Rainbows.server
>     server.app.respond_to?(:deferred?) and
>       server.app = TryDefer.new(server.app)
>   [...]

That's only because other servers (Thin) support this non-standard
extension by default.  Otherwise, I've always been strongly against
adding non-standard extensions/features into the Rack middleware stack.

> and I think in most cases we would want this timeout for each
> request. The down side might be that then the users cannot
> use their own Rainbows::ThreadTimeout (maybe?) or setup a custom
> threshold.

Right.  And some users (e.g. myself) don't ever use either timeout
feature.  We audit every single line of code and make sure all
potentially long-running method calls have timeouts set appropriately.
(IMNSHO, this is the _only_ way to write applications for a
multi-threaded/evented platform)

> Probably then we could introduce another timeout/threshold config
> in Configurator? I feel it would be common enough.

Not worth it.  It's not reliable enough nor generally useful enough
compared to the regular timeout (which again, is a last resort).
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: Timeout on Rainbows!

[Lin Jen-Shin (godfat)]

On Sat, Aug 24, 2013 at 3:35 AM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> "Lin Jen-Shin (godfat)" <godfat-hOE/xeEBYYIdnm+yROfE0A@public.gmane.org> wrote:
>> I am reworking on thread pool for EventMachine, and just realized that
>> Rainbows doesn't try to put a timeout for each request. Previously,
>> I thought the timeout setup in Unicorn would be used for each request.
>
> The normal timeout is only a last resort (it works when the VM is hosed
> on a worker).  ThreadTimeout isn't nearly as reliable because it works
> on the same VM/process, and it's still always preferable to rely on
> "local" timeouts exposed by every library.  (see
> http://unicorn.bogomips.org/Application_Timeouts)
[...]
>> and I think in most cases we would want this timeout for each
>> request. The down side might be that then the users cannot
>> use their own Rainbows::ThreadTimeout (maybe?) or setup a custom
>> threshold.
>
> Right.  And some users (e.g. myself) don't ever use either timeout
> feature.  We audit every single line of code and make sure all
> potentially long-running method calls have timeouts set appropriately.
> (IMNSHO, this is the _only_ way to write applications for a
> multi-threaded/evented platform)
>
>> Probably then we could introduce another timeout/threshold config
>> in Configurator? I feel it would be common enough.
>
> Not worth it.  It's not reliable enough nor generally useful enough
> compared to the regular timeout (which again, is a last resort).

I see, thank you very much. I'll take out ThreadTimeout in
EventMachineThreadPool and add it into our application stack
as a temporary workaround for now.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying