I'm trying to come up with a sane architecture to delay a task for 24 hours. So far I've thought of:
- unix cron job, but this will not be able to take the load we give it, since we will have many thousands of delayed tasks per day
- Use RabbitMQ with dead letter exchange, however, this is a hack and being a hack is riddled with problems - impossible to monitor without mutilating the messages, etc.
- Other solutions I've come up with are poll based, e.g., insert task into MySQL and have a poller responsible
- Amazon SQS only supports up to 15 minute delays. I could write a punt loop for # of delays, btu this is already reinventing the wheel way too much. Plus using SQS this would get expensive as we pay per message.
- Redis possibly...? have not heavily investigated this. I know Memcached is not appropriate as it won't really guarantee persistence that long.
What existing technology is good for this? I don't want to reinvent the wheel here, given that I'm pretty sure I'm not working on the first software product that requires performing a task tomorrow. Thus far I've been database driving things.