Re: Forking off the unicorn master process to create a background worker

[Eric Wong <normalperson@yhbt.net>]

Russell Branca <chewbranca@gmail.com> wrote:
> Hello Eric,
> 
> Sorry for the delayed response, with the combination of being sick and
> heading out of town for a while, this project got put on the
> backburner. I really appreciate your response and think its a clean
> solution for what I'm trying to do. I've started back in getting the
> job queue working this week, and will hopefully have a working
> solution in the next day or two. A little more information about what
> I'm doing, I'm trying to create a centralized resque job queue server
> that each of the different applications can queue work into, so I'll
> be using redis behind resque for storing jobs and what not, which
> brings me an area I'm not sure of the best approach on. So when we hit
> the job queue endpoint in the rack app, it spawns the new worker, and
> then immediately returns the 200 ok started background job message,
> which cuts off communication back to the job queue. My plan is to save
> a status message of the result of the background task into redis, and
> have resque check that to verify the task was successful. Is there a
> better approach for returning the resulting status code with unicorn,
> or is this a reasonable approach? Thanks again for your help.

Hi Russell, please don't top post, thanks.

If you already have a queue server (and presumably a standalone app
processing the queue), I would probably forgo the background Unicorn
worker entirely.

Based on my ancient (mid-2000s) knowledge of user-facing web
applications: the application should queue the job, return 200, and have
HTML meta refresh to constantly reload the page every few seconds.

Hitting the reload endpoint would check the database (Redis in this
case) for completion, and return a new HTML page to stop the meta
refresh loop.

This means you're no longer keeping a single Unicorn worker idle and
wasting it.  Nowadays you could do it with long-polling on
Rainbows!/Thin/Zbatery, too, but long-polling is less reliable for
people switching between WiFi access points.  The meta refresh method
can be a waste of power/bandwidth on the client side if the background
job takes a long time, though.

I'm familiar at all with Resque or Redis, but I suspect other folks
on this mailing list should be able to help you flesh out the details.

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying