RQSCHEDULER INSTALL

Create a new pyenv environment for playing with python-rq and redisįrom datetime import datetime, timedelta, timezone from redis import Redis from rq_scheduler import Scheduler from work import fib def schedule_job (): # Get a scheduler for the 'default' queue on the default redis server scheduler = Scheduler ( connection = Redis ()) now = time_to_str ( datetime.Start redis with the default settings in a new terminal window % redis-server.

RQSCHEDULER INSTALL

Install redis with % brew install redis.

It is instructive to get this all going outside of django. I can even create a custom worker classes if I need different behavior from rq worker. I then run the command line script rq worker that pops jobs from queues, fork()'s and executes them.

I then enqueue a job onto the queue with the Queue.enqueue() API. I create a queue instance with the Queue() API passing it the redis connection. Workers are python processes that run jobs by popping an id from the list, unpickling the function, fork()ing, and then executing the function. Jobs are python function that are pickled, given an id, and stored in a redis hash. Python-rq implements a simple API to queue jobs and run them using workers. It has data structures like lists and hashes, and commands like rpush, lpop, blpop that make it an excellent choice for implementing queues. Redis is a rock solid in-memory data structure store. For me python-rq hits the sweet spot between complexity and flexibility. I'm going to use a combination of redis, python-rq, and rq-scheduler. It can get the job done but I'm not going to use it either. It uses a relational database to store the jobs and cron as the scheduler. django-background-tasks is a simpler option. I need a task queue.Ĭelery is the big dog. My django app needs to periodically fetch and process data from a 3rd party API.

Rqscheduler