Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I need to create a web service that executes every hour. It will be used to review data in a database and add alerts to a table in the same database if certain conditions are met/not met. What we currently have is:

We have end devices that use Python to report to an Amazon Web Services (AWS) virtual server. The AWS server takes that information and stores it in a MySQL database. The AWS server is Linux running Django and Apache. I need to be able to have some python code run every hour that verifies the data that has been stored by the end devices. If certain conditions are not met then a record will be added to the alerts table in the database.

We originally contracted to have the above setup created. I am new to Python, Django, and Apache. However, I have already made several changes to the Python code that sends and also receives the data from the end devices. I am a coder that is breaking into web programming.

Does anyone have any recommendations on how I can do this?

share|improve this question
add comment

6 Answers

up vote 29 down vote accepted

How about making a cronjob, assuming you have shell access?

The cron daemon exists on virtually any UNIX-like system and schedules commands to run based on a description in a file called the crontab.

Each line of the file contains a set of fields to indicate the timepoints when a command shall be executed.

Your task could be either a standalone program that does the task you wish to accomplish or as another answer suggests, an invocation of a HTTP client like wget, curl or fetch to access a web resource that will perform the action.

If you've got limits for how long a request may take to serve, you might have to move the task into an offline script or program that doesn't run inside your web framework/server.

share|improve this answer
    
Cron definitely was what I ended up using. Thanks for the answer. For those who don't want to work with Cron but want the same functionality, I did find two online services that perform jobs like Cron: SetCronJob and Cronless. –  Linger Oct 30 '12 at 15:13
add comment

With Django on AWS, I'd look into Celery.

Celery adds asynchronous tasks and includes a scheduler, and on AWS you can configure Celery to use the Amazon Simple Queue Service as the broker (see Celery with Amazon SQS on Stack Overflow and this blog post on the subject).

You set up a Celery periodic task schedule and it'll run a configured task according to that schedule.

Advantage is that you can use the whole setup to run any asynchronous task, offloading heavy tasks from your web server to the Celery workers.

The light-weight alternative is to just set up a crontab job; you could even configure a route in your Django application to be called using curl or wget:

* 5 * * * curl http://username:password@hostname/route_to_job
share|improve this answer
1  
+1 A much more refined solution than my simple cron task, and specific to the OP's implementation. –  msanford Oct 24 '12 at 14:40
add comment

Since your AWS instance runs Linux, you can probably accomplish this as a cron job.

You could take what I would term Drupal's cron approach which, in the case of Django, involves creating a controller to respond to a URL and then perform the action you want.

You then configure a cron task to curl the controller's URL, triggering your script.

This has the added advantage of being easily callable at any time from a URL: maybe an impatient manager wants a report generated from data from last 20 minutes.

share|improve this answer
add comment

You may want to look into APScheduler. This is a quartz-like scheduler (although not as extensive) for Python.

This can be a far better alternative to externally run cron scripts for long-running applications (e.g. web applications), as it is platform neutral and can directly access your application’s variables and functions.

You can find a description for the most recent release here: http://pypi.python.org/pypi/APScheduler/2.0.3

There is some documentation for installation/implementation here: https://apscheduler.readthedocs.org/en/latest/

Here are some of the features:

  • No (hard) external dependencies
  • Thread-safe API
  • Excellent test coverage (tested on CPython 2.5 - 2.7, 3.3, Jython 2.5.3, PyPy 1.9)
  • Configurable scheduling mechanisms (triggers):
    • Cron-like scheduling
    • Delayed scheduling of single run jobs (like the UNIX “at” command)
    • Interval-based (run a job at specified time intervals)
  • Multiple, simultaneously active job stores:
    • RAM
    • File-based simple database (shelve)
    • SQLAlchemy (any supported RDBMS works)
    • MongoDB
    • Redis
share|improve this answer
add comment

I think, you can use some stuff like a django-extensions

Django-Extensions Website

There is a module - jobs. For me this is a very good tool to control your cronjobs.

Second option is use a Fabric and create a function for this.

And I see third way. Use your imagination and knowledge and create own function with subprocess and sh.

share|improve this answer
add comment

Use Quartz-Scheduler. Following Link will be helpfull.

http://quartz-scheduler.org/

Tutorial

http://quartz-scheduler.org/documentation/quartz-2.1.x/quick-start

share|improve this answer

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

1  
Quartz looks to be designed for Java. I am not using Java at all. I am using Python/Django. –  Linger Oct 24 '12 at 16:33
2  
Answers that contain links without some kind of explanation or context (such as relevant quotations, summaries, or personal experiences) are not considered to be good answers. Please edit this post to provide more details. –  Thomas Owens Oct 24 '12 at 16:49
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.