Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

With normal form submission I use the pattern Post / Redirect / Get, when processing the forms.

I have a database application built with Django. I want to allow the users to select a number of items from the database, then launch a computationally intensive task based on those items.

I expect the task to take between 10 minutes and 2 hours to complete.

Is there a standard approach to dealing with requests like this (i.e. that don't return immediately)? Ideally there would be some way to display the progress.

share|improve this question
2  
Ah, hours. My first thought was a spinny animated gif icon. That probably wouldn't seal the deal on the UX front. –  Erik Reppen Oct 17 '13 at 6:41

5 Answers 5

I would trigger a job (e.g. a worker-thread) for this which is able to report the progress and finish the current request. And for displaying the process, i would poll the progress of this job via ajax or similar.

share|improve this answer
    
For a more robust approach, you may even want to spawn a different process. That way if it crashes it won't bring down the whole web server process. And since we are talking django (and python), I'd like to point out that multithread tend to be a bit inefficient in python due to the GIL . –  Laurent Bourgault-Roy Oct 16 '13 at 15:53

Celery is a setup for django which is designed for exactly this type of situation. You can launch a background task and let it run asynchronously as its own worker thread.

Someone has also made an AJAX loading bar to show the progress of the job.

share|improve this answer
    
Also here's a tutorial outlining the basics of how Celery works. –  Ashok Fernandez Oct 16 '13 at 11:31
3  
The question says Is there a standard approach to dealing with requests like this (i.e. that don't return immediately)? Ideally there would be some way to display the progress. Celery is a standard way to handle such a problem when using django (as specified by the question) - just look at the list of sites which use it if you don't believe me. It also happens to support showing a progress bar (also another requirement specified by the question). What better way to learn a design pattern which solves the problem than by a well accepted example? –  Ashok Fernandez Oct 16 '13 at 11:49
    
Maybe My question wasn't so clear. I had kind of guessed I needed to launch a job in the background, but what is the standard way to deal with the web page response? A small box saying "waiting for response from htp://webserver.com/mypage" in the bottom left of the browser for 2 hours isn't very satisfactory to me. –  wobbily_col Oct 16 '13 at 12:37
1  
Another solution would be to start up the job using celery and then load a page which tells the user they will receive and email or something when the job is finished. This seems like it might be a little easier to get running if you are struggling with the AJAX progress bar and would kind of makes sense if the job is going to take upwards of 2 hours to execute. –  Ashok Fernandez Oct 16 '13 at 13:36
1  
@wobbily_col Perhaps if you refocus the question here to the more technical side, to match the answers, then post the frontend half of the question on UX.SE, you'll get answers more along the lines of what you expect –  Izkata Oct 17 '13 at 1:52

HTTP 202 Accepted

The 202 response is intentionally non-committal. Its purpose is to allow a server to accept a request [...] without requiring that the user agent's connection to the server persist until the process is completed. The entity returned with this response SHOULD include an indication of the request's current status and either a pointer to a status monitor or some estimate of when the user can expect the request to be fulfilled.

The Web is resource-oriented, and so the "pointer to a status monitor" would be a URI to a resource that you've made available. In some specific sense, you might have a database table of jobs with associated status information that the client can GET. The other answers contain useful information about how you might implement the jobs, but I feel your question is about the web - and the answer is yes, HTTP has reasonably well thought out status codes for conveying this kind of information.

share|improve this answer

The actual thread doing the work must be separate from its association with the session (although you can link it to the user performing the work). Have a application-scoped class (lets call it JobQuery) which receives job requests and answers queries about current jobs.

Your web application therefore becomes stateless once again. Whenever you want to know the status of a job, the web application performs an inquiry using JobQuery.

In the case of error, a database component comes in handy. If the web server crashes or other unexpected problems occur, your JobQuery, in addition to starting jobs, should also keep tabs by writing the status to the database. If something were to happen, JobQuery can indicate the error status of incompleted jobs or even pick up where it left off.

What's important here is that the session-scoped side of your web application is free to continue responding to the user and continues to be stateless while ensuring that 2 hour jobs are not lost in the void in the case of problems.

share|improve this answer

You probably want some sort of job queue here. The exact form of such a queue can vary. But basically your server side script will add a task into a queue. The tasks in the queue are handled by a completely different process, and mark them complete when finished. You can then use something like websockets or long polling to 'watch' the queue, and report on task status.

For example, you might make a database table that will function as a queue. When a request comes in to your server, you insert the necessary task information into that table. A separate process will poll that table every X seconds/minutes, and when it finds a new entry, it executes the requested job. When the job is finished, it marks the task as complete in the database. Meanwhile, inside the browser, you have opened up a connection to your server using either websockets or long polling. That will poll the queue for the status of the task, and send it back to the end user who can then see what the status of the task is.

Thats a simple table-based solution. There are of course other ways to handle the specifics (message queues).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.