Rainbows! Rack HTTP server user/dev discussion
 help / color / Atom feed
* background threads
@ 2012-06-18 18:32 Alexandre Riveira
       [not found] ` <4FDF7420.7030300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
  0 siblings, 1 reply; 7+ messages in thread
From: Alexandre Riveira @ 2012-06-18 18:32 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw

Hello !

I'm trying to put in background threads in Rainbows. I created a 
config/initializers/background_thread.rb, where I

Thread.new do
# my background job
end

With that, I have problems to "reload" in rainbows

Thank you all for your help.


Alexandre Riveira
Brazil
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: background threads
       [not found]     ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2012-06-18 19:08       ` Alexandre Riveira
       [not found]         ` <4FDF7CA0.6030103-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
  2012-06-18 19:26       ` Alexandre Riveira
  1 sibling, 1 reply; 7+ messages in thread
From: Alexandre Riveira @ 2012-06-18 19:08 UTC (permalink / raw)
  To: Eric Wong; +Cc: Rainbows! list

Em 18-06-2012 21:55, Eric Wong escreveu:

What kind of problems?

The application consumes 100% cpu after reload


Is preload_app true or false?  (default is false)
preload_app is true


I changed  my code  as below  and got  a result in  small servers
  Thread.new(log) do |rotina_log|
     loop do
        log.info("rotina automatica") {"waiting process before end"}
        sleep(5)
        break unless File.exists? Dir.pwd + '/tmp/pids/rainbows.pid.oldbin'
     end

     loop do
       #rotina_log.info("rotina automatica") { "waiting 60 secs #{Process.pid}" }
       sleep(60)
       if $0.include? "old"
         rotina_log.info("rotina automatica") { "exit..... #{Process.pid}" }
         Thread.exit
       end
       begin
         #rotina_log.info("rotina automatica") { "exec #{Process.pid}" }
         empresas = Empresa::Base.find_all_by_cancelado(false)
         #sleep(180)
         empresas.invoke(:rotinas_automaticas, :usuario_id =>  Auth::Configuracao.instance.usuario_sistema_id)
         #rotina_log.info("rotina automatica") { "exec ok #{Process.pid}" }
       rescue Exception =>  e
         rotina_log.fatal("rotina automatica") { e.to_s + e.backtrace.inspect }
       end
     end
   end


Tanks,


Alexandre Riveira
Brazil

_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: background threads
       [not found]     ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  2012-06-18 19:08       ` Alexandre Riveira
@ 2012-06-18 19:26       ` Alexandre Riveira
  1 sibling, 0 replies; 7+ messages in thread
From: Alexandre Riveira @ 2012-06-18 19:26 UTC (permalink / raw)
  To: Eric Wong; +Cc: Rainbows! list

Em 18-06-2012 21:55, Eric Wong escreveu:
> So for background threads, I prefer to start it in the app stack
> on the first request (use a mutex to protect against double-starts).
> For rails, you can probably do it in a before_filter.
Eric, como você garante a aplicação não criar uma thread em cada worker

Tanks

Alexandre Riveira
brazil
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: background threads
       [not found] ` <4FDF7420.7030300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 21:55   ` Eric Wong
       [not found]     ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
       [not found]     ` <4FDF81A5.6080605@objectdata.com.br>
  0 siblings, 2 replies; 7+ messages in thread
From: Eric Wong @ 2012-06-18 21:55 UTC (permalink / raw)
  To: Rainbows! list; +Cc: Alexandre Riveira

Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Hello !
> 
> I'm trying to put in background threads in Rainbows. I created a
> config/initializers/background_thread.rb, where I
> 
> Thread.new do
> # my background job
> end
> 
> With that, I have problems to "reload" in rainbows

What kind of problems?

Is preload_app true or false?  (default is false)

preload_app being true will cause that background thread to get
started in the master process and not the workers.  You can
workaround it by starting th background thread in an after_fork
hook in the Rainbows!/unicorn config file:

http://unicorn.bogomips.org/Unicorn/Configurator.html#method-i-after_fork

> Thank you all for your help.

I'm often lazy when it comes to setting up hooks myself :x

So for background threads, I prefer to start it in the app stack
on the first request (use a mutex to protect against double-starts).
For rails, you can probably do it in a before_filter.

Raindrops::Watcher is a Rack app (no other framework, just rack)
which starts a background thread in the #call method:

	git clone git://bogomips.org/raindrops raindrops
	cat raindrops/lib/watcher.rb
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: background threads
       [not found]         ` <4FDF7CA0.6030103-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 22:21           ` Eric Wong
  0 siblings, 0 replies; 7+ messages in thread
From: Eric Wong @ 2012-06-18 22:21 UTC (permalink / raw)
  To: Rainbows! list

Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Em 18-06-2012 21:55, Eric Wong escreveu:
> 
> What kind of problems?
> 
> The application consumes 100% cpu after reload
> 
> 
> Is preload_app true or false?  (default is false)
> preload_app is true

With preload_app true, the application worker won't be able to
communicate with the background thread after reload.

I'm not sure if your setup relies on threads communicating
directly with the application.

> I changed  my code  as below  and got  a result in  small servers
>  Thread.new(log) do |rotina_log|
>     loop do
>        log.info("rotina automatica") {"waiting process before end"}
>        sleep(5)
>        break unless File.exists? Dir.pwd + '/tmp/pids/rainbows.pid.oldbin'
>     end
> 
>     loop do
>       #rotina_log.info("rotina automatica") { "waiting 60 secs #{Process.pid}" }
>       sleep(60)
>       if $0.include? "old"
>         rotina_log.info("rotina automatica") { "exit..... #{Process.pid}" }
>         Thread.exit
>       end
>       begin
>         #rotina_log.info("rotina automatica") { "exec #{Process.pid}" }
>         empresas = Empresa::Base.find_all_by_cancelado(false)
>         #sleep(180)
>         empresas.invoke(:rotinas_automaticas, :usuario_id =>  Auth::Configuracao.instance.usuario_sistema_id)
>         #rotina_log.info("rotina automatica") { "exec ok #{Process.pid}" }
>       rescue Exception =>  e
>         rotina_log.fatal("rotina automatica") { e.to_s + e.backtrace.inspect }
>       end
>     end
>   end

Reading the code above (but not knowing your original code), I'm not
seeing any communcation between the threads.  It could probably be done
better as a standalone daemon or even a cronjob.  There's more setup
overhead, sure, but it should also be easier to debug/troubleshoot.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: background threads
       [not found]       ` <4FDF81A5.6080605-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 22:42         ` Eric Wong
       [not found]           ` <4FDF85C2.1030900@objectdata.com.br>
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Wong @ 2012-06-18 22:42 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw; +Cc: Alexandre Riveira

Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Em 18-06-2012 21:55, Eric Wong escreveu:
> >So for background threads, I prefer to start it in the app stack
> >on the first request (use a mutex to protect against double-starts).
> >For rails, you can probably do it in a before_filter.
> 
> Eric, how do you ensure the application does not create a thread for
> each worker

In the Raindrops::Watcher case, I don't.  Watcher works as a single
process and doesn't need/care about multiple background threads.

If you only want one background thread for all workers, it probably is
best to just use a separate daemon independent of the Rainbows! server.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: background threads
       [not found]             ` <4FDF85C2.1030900-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 22:53               ` Eric Wong
  0 siblings, 0 replies; 7+ messages in thread
From: Eric Wong @ 2012-06-18 22:53 UTC (permalink / raw)
  To: Alexandre Riveira; +Cc: rainbows-talk

Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Em 18-06-2012 22:42, Eric Wong escreveu:
> >If you only want one background thread for all workers, it
> >probably is best to just use a separate daemon independent of the
> >Rainbows! server.
> I want only one worker to run a background thread. I would like to
> ensure that only one worker thread will be in the background. Sure
> ... using rainbows :)

It's a bad idea, I think...

You can use process-shared locks Ruby provides File#flock.
It's also possible to get fcntl()-based locking to work with pack().

Maybe there's extensions to share locks with multiple processes
(e.g. POSIX semaphores, database-based locks in Redis/MySQL/Postgres).

...Or even use a really ugly method of using TCPServer to bind to a
pre-determined port (and bailing on Errno::EADDRINUSE) (no need to
accept connections on the TCPServer, just keep the object around
so it doesn't get close(2)-ed on GC.

But really, I think this type of background thread is a bad idea
for Rainbows! (or any HTTP server).
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-06-18 18:32 background threads Alexandre Riveira
     [not found] ` <4FDF7420.7030300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 21:55   ` Eric Wong
     [not found]     ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-06-18 19:08       ` Alexandre Riveira
     [not found]         ` <4FDF7CA0.6030103-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 22:21           ` Eric Wong
2012-06-18 19:26       ` Alexandre Riveira
     [not found]     ` <4FDF81A5.6080605@objectdata.com.br>
     [not found]       ` <4FDF81A5.6080605-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 22:42         ` Eric Wong
     [not found]           ` <4FDF85C2.1030900@objectdata.com.br>
     [not found]             ` <4FDF85C2.1030900-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 22:53               ` Eric Wong

Rainbows! Rack HTTP server user/dev discussion

Archives are clonable:
	git clone --mirror http://bogomips.org/rainbows-public
	git clone --mirror http://ou63pmih66umazou.onion/rainbows-public

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.lang.ruby.rainbows
	nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.rainbows

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git