Resque is a lightweight, fast, and a very powerful message
queuing system used to run Ruby jobs in the background, asynchronously. It’s generally used when the system asks for
better scalability and quick response.
A Resque job is any Ruby class or module with a perform
class method.
class MyDesign
@queue = :myqueue
#Jobs will be placed and picked up from this queue.
# call to queue processing in Resque until later
def self.perform(seconds)
sleep(seconds)
end
end
end
How the queuing of Redis works:
The real ability of resque comes from its queue, Redis. Redis is a NoSQL key-value store.
Unlike other NoSQL key-value stores, which takes only strings as keys/values,
Redis can process lists, sets, hashes, and sorted sets as values, and operates
on them atomically. Resque works on the
top of Redis list datatype, where the queue name behaves as a key and list as
values.
Jobs of resque are en-queued on the Redis queue, and the
designated workers de-queues the same for processing. All these operations are
atomic, hence the en-queuers and workers do not bother about the locking and synchronous
access. Every element of a list (set, hashes etc.) must be string, and so the
data structures are not nested in Redis.
Redis is a very fast, in-memory dataset, and can persist to
disk (configurable by time or number of operations), or save operations to a
log file for recovery after a re-start, and supports master-slave replication.
Redis owns the command set to read and process the keys and
it does not need SQL to inspect its data.
How Queuing with Resque works
Resque stores the job queue in Redis list named “resque:queue:name”,
where each element of the list is a hash. Redis also maintains its own
management structures, with an additional
“failed” job list.
It keeps all the the operation light. We don’t need to pass a lot of data to the job. Instead passing
the references to other records, files, etc. is sufficient for the workers to
pop and perform the jobs.
When a job is popped from the queue, Resque instantiates an
object of its parent class and calls it’s perform method, passing the
additional parameters.
Calling external systems with Resque
There are ports of Resque to other languages such as python,
C, Java, .NET, node, PHP and Clojure. If your external system is written in one
of these languages, then you can start workers listening to their queues. Since
you are not talking to a ruby class with arguments, you can set up a
placeholder class with the proper queue name. This will allow Resque plugins to
fire on enqueue.
