Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to create a web service that executes every hour. It will be used to review data in a database and add alerts to a table in the same database if certain conditions are met/not met. What we currently have is:

We have end devices that use Python to report to an Amazon Web Services (AWS) virtual server. The AWS server takes that information and stores it in a MySQL database. The AWS server is Linux running Django and Apache. I need to be able to have some python code run every hour that verifies the data that has been stored by the end devices. If certain conditions are not met then a record will be added to the alerts table in the database.

We originally contracted to have the above setup created. I am new to Python, Django, and Apache. However, I have already made several changes to the Python code that sends and also receives the data from the end devices. I am a coder that is breaking into web programming.

Does anyone have any recommendations on how I can do this?

share|improve this question

closed as off-topic by MichaelT, Scant Roger, Ixrec, Brian, Dan Pichelman Dec 31 '15 at 16:57

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions asking us to find or recommend tools, libraries, programming languages, resources (including books, blogs, tutorials, and examples), or projects to undertake are off-topic on Programmers as they attract opinionated answers that won't have lasting value to others. You may be able to get help in The Whiteboard, our chat room." – Scant Roger, Ixrec, Brian
If this question can be reworded to fit the rules in the help center, please edit the question.

    
Use Quartz-Scheduler. Following Link will be helpfull. quartz-scheduler.org Tutorial quartz-scheduler.org/documentation/quartz-2.1.x/quick-start – user69878 Oct 24 '12 at 16:27
    
Quartz looks to be designed for Java. I am not using Java at all. I am using Python/Django. – Linger Oct 24 '12 at 16:33
up vote 31 down vote accepted

How about making a cronjob, assuming you have shell access?

The cron daemon exists on virtually any UNIX-like system and schedules commands to run based on a description in a file called the crontab.

Each line of the file contains a set of fields to indicate the timepoints when a command shall be executed.

Your task could be either a standalone program that does the task you wish to accomplish or as another answer suggests, an invocation of a HTTP client like wget, curl or fetch to access a web resource that will perform the action.

If you've got limits for how long a request may take to serve, you might have to move the task into an offline script or program that doesn't run inside your web framework/server.

share|improve this answer
    
Cron definitely was what I ended up using. Thanks for the answer. For those who don't want to work with Cron but want the same functionality, I did find two online services that perform jobs like Cron: SetCronJob and Cronless. – Linger Oct 30 '12 at 15:13

With Django on AWS, I'd look into Celery.

Celery adds asynchronous tasks and includes a scheduler, and on AWS you can configure Celery to use the Amazon Simple Queue Service as the broker (see Celery with Amazon SQS on Stack Overflow and this blog post on the subject).

You set up a Celery periodic task schedule and it'll run a configured task according to that schedule.

Advantage is that you can use the whole setup to run any asynchronous task, offloading heavy tasks from your web server to the Celery workers.

The light-weight alternative is to just set up a crontab job; you could even configure a route in your Django application to be called using curl or wget:

* 5 * * * curl http://username:password@hostname/route_to_job
share|improve this answer
1  
+1 A much more refined solution than my simple cron task, and specific to the OP's implementation. – msanford Oct 24 '12 at 14:40

Since your AWS instance runs Linux, you can probably accomplish this as a cron job.

You could take what I would term Drupal's cron approach which, in the case of Django, involves creating a controller to respond to a URL and then perform the action you want.

You then configure a cron task to curl the controller's URL, triggering your script.

This has the added advantage of being easily callable at any time from a URL: maybe an impatient manager wants a report generated from data from last 20 minutes.

share|improve this answer

You may want to look into APScheduler. This is a quartz-like scheduler (although not as extensive) for Python.

This can be a far better alternative to externally run cron scripts for long-running applications (e.g. web applications), as it is platform neutral and can directly access your application’s variables and functions.

You can find a description for the most recent release here: http://pypi.python.org/pypi/APScheduler/2.0.3

There is some documentation for installation/implementation here: https://apscheduler.readthedocs.org/en/latest/

Here are some of the features:

  • No (hard) external dependencies
  • Thread-safe API
  • Excellent test coverage (tested on CPython 2.5 - 2.7, 3.3, Jython 2.5.3, PyPy 1.9)
  • Configurable scheduling mechanisms (triggers):
    • Cron-like scheduling
    • Delayed scheduling of single run jobs (like the UNIX “at” command)
    • Interval-based (run a job at specified time intervals)
  • Multiple, simultaneously active job stores:
    • RAM
    • File-based simple database (shelve)
    • SQLAlchemy (any supported RDBMS works)
    • MongoDB
    • Redis
share|improve this answer

I think, you can use some stuff like a django-extensions

Django-Extensions Website

There is a module - jobs. For me this is a very good tool to control your cronjobs.

Second option is use a Fabric and create a function for this.

And I see third way. Use your imagination and knowledge and create own function with subprocess and sh.

share|improve this answer

The most likely the answer here is provided that you are running crontab with the default setting which means it is displaying the crontab SPOOL RATHER THAN THE FILES.

Meanwhile, it can be run also in AWS as it comes with cron pre-installed and configured, that allows for set up of a task that should be run hourly, daily, weekly or monthly as well as any other time period by putting files in a /etc/cron.xxxxxx directory as explained here.

Setting up a job to run hourly, daily, weekly or monthly is very quick. Since the question is asking how to execute code every hour so in AWS Linux you can create a file in /etc/cron.hourly.

Here is the step when you have Log in to your instance via the SSH client.

$ echo "/usr/bin/python -q /path/to/your/file" > application
$ sudo mv application /etc/cron.hourly/
$ sudo chown -R root /etc/cron.hourly
$ sudo chmod 2755 /etc/cron.hourly/application
$ sudo /etc/init.d/crond restart

In the example above the file saved and named 'application'. The name doesn’t really matter so long as it is unique. This is the log report by running: $ sudo vim /var/log/cron

Dec 28 19:01:01 ip-xxx-xx-xx-xx CROND[20243]: (root) CMD (run-parts /etc/cron.hourly)
Dec 28 19:01:01 ip-xxx-xx-xx-xx run-parts(/etc/cron.hourly)[20243]: starting 0anacron
Dec 28 19:01:01 ip-xxx-xx-xx-xx run-parts(/etc/cron.hourly)[20261]: finished 0anacron
Dec 28 19:01:01 ip-xxx-xx-xx-xx run-parts(/etc/cron.hourly)[20243]: starting application
Dec 28 19:01:02 ip-xxx-xx-xx-xx run-parts(/etc/cron.hourly)[20323]: finished application

As shown in the log, at hourly basis it will start to run anacron that performs periodic command scheduling which is traditionally done by cron, and then call every other files in the directory and run the commands in each files.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.