Resque: COUNT= n not working for spawning multiple workers

Created on 22 Dec 2009 · 3Comments · Source: resque/resque

Hello all,
This is my first post to the abode of Git :).

I'm trying to use resque for backend job processing in my Rails app ( v 2.0.2). I've installed rescue from the git as Rails plugin. But even if I mention the count variable only one worker is spawned.

COUNT= x QUEUE=request_queue rake environment resque:work

This I'm verifying from the rescue-web interface.

Can somebody help me out on this?

Another query is how can I use god module that comes with Resque. There is not much information available on this.

Thanks all in advance
dg

Source

dibyajyoti

Most helpful comment

You are using the wrong rake task to start multiple workers. Try "resque:workers" instead of "resque:work".

For god configuration, see http://github.com/defunkt/resque/tree/master/examples/god/. I had to tweak it a bit to work for me, and it still seems to have problems, but it's a decent starting point. Note, it won't actually stop the workers when you ask it to. You'll need to write a script for that or kill them manually (via kill -QUIT $PID).

nevans on 22 Dec 2009

👍3

All 3 comments

You are using the wrong rake task to start multiple workers. Try "resque:workers" instead of "resque:work".

nevans on 22 Dec 2009

👍3

Thanks a lot for Nevans fro your help. Multiple workers creation is working.

Does every worker runs on a thread spawned by the Rails App(rescue plugin) or is it
a separate process altogether?
According to the online documenation - With Resque's parent / child architecture you can tell the parent to forcefully kill the child then immediately start processing more jobs.
Is the Rails App process here being mentioned as the parent and the workers as the child or is it that I am missing a point here?
Is there some form of architecture or design document available on Resque?

cheers
dg

dibyajyoti on 23 Dec 2009

processes are used, not threads.
The source is fairly simple and explains all of this:
- The parent worker processes are started via "rake resque:work". See http://github.com/defunkt/resque/blob/master/lib/resque/tasks.rb.
- The child worker processes are forked from the parent, inheriting the parent's environment. See http://github.com/defunkt/resque/blob/27dbbdb9ec940c99d32a8e0bcc91e72bdae88667/lib/resque/worker.rb#L85.
- In its current form "rake resque:workers" merely runs "rake resque:work" multiple times, with no monitoring of the workers whatsoever. It should only be used in development (use god with "rake resque:work" in production).
- "rake resque:work" will only autoload your rails environment if you tell it to, by setting "environment" to be a prereq for "resque:setup".
Read the source, read the README, maybe there are some helpful blog posts out there by now.