Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to create a queue processing script that's purpose is to find particular files and apply actions based on specific parameters, with the result being to call a PHP script that serves to process a list of message files that we generated and passed to it.

One of the things I need to specify in a config file the maximum number of concurrent calls to the PHP processor that can be running at a given time, and if max number of calls is >= then loop until <=

Also need to specify a timeout variable 'tOut' in seconds...

So basically it looks for the oldest file in the directory, locates a variable in that filename 'chCode', searches for the next oldest files in the directory with the same chCode and reprefix them and output file a list of those reprefixed paths.

Then using Popen or Pcall the script with run 'php /path/to/filepathlist' and if: it returns output '00' then continue... it returns any text other than 00 then reprefix and mv.. it returns nothing for tOut secs then reprefix and mv.

The counter is the part I'm really needing help with, as this needs to be proper accurate and high performance with batches of hundreds of thousands of outgoing text messages being sorted using this. I can confirm the counter from the PHP script but as said it needs to be accurate.

Using Python 2.7 on CentOS6.

share|improve this question
up vote 2 down vote accepted

If I understand your question correctly, the calls to PHP are asynchronous but you need to throttle them, right?

Use a semaphore.

throttle = threading.Semaphore(NUMBER_OF_CONCURRENT_EXECUTIONS)

Then when you call PHP, do it via

with throttle :
    ... call php, wait for timeout, etc ...
share|improve this answer
Thanks. Exactly what I needed! – 264nm Jan 13 '14 at 7:03
@264nm Could you mark it as the accepted answer? ;-) – Ryan Yoosefi Aug 17 '14 at 12:12

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.