Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My PHP application needs to send product expiry notifications to customers everyday. I am thinking of:

  1. A one-time cron job for a php script that identifies the customers from a table of 10000 rows then send an email to each one using the SwiftMailer library. Each email will have unique content so I am not planning to do "batch send". *Con: * Too slow and the server may kill it.

  2. A recurring five-minute cron job that would send a definite number of emails (eg. 20) for every execution. *Con: * All customers may not receive emails since the the queue may not finish within 24 hours.

Please give alternatives and suggestions. TIA.

share

migration rejected from stackoverflow.com Feb 21 at 14:11

This question came from our site for professional and enthusiast programmers. Votes, comments, and answers are locked due to the question being closed here, but it may be eligible for editing and reopening on the site where it originated.

closed as unclear what you're asking by MichaelT, TZHX, gnat, Bart van Ingen Schenau, enderland Feb 21 at 14:11

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question.If this question can be reworded to fit the rules in the help center, please edit the question.

Why would the server kill long running processes? Sure, Apache will kill long running HTTP requests but if you start a PHP script directly from the command line or cron it'll run until it exits or crashes.

I would do 1, but with a small refinement: record somehow when you send a email to a customer, so if it crashes half way through you can pick up and not send multiple emails to 1 user.

If you do want to do 2, change it so instead of X emails it has an internal timer. After each email it checks the timer, and if more than 4 mins has passed it exits. Then you don't have to set some magic number as a limit that depends en your server anyway and may change.

share
    
Agree. And add sleep(1) after each email to spare CPU. – OZ_ May 15 '11 at 8:42
1  
I am currently sending out anywhere from 2000 to 20,000 opt-in emails a night using a PHP job run off a cron job. Ilt logs each email as suggested by James, and it crashes a couple of times a year where I have to restart in the middle. The emails are sent out in batches of 1000, delaying 50 ms between each call to mail. Then another 1000 emails are obtained from a remote server -- so there is an additional delay every 1000 emails. – tcrosley May 20 '11 at 22:00

I'd use a 5 min cron, that sends ceil(count(*)) / (24 * 20) emails.

Whichever you do, don't forget to lock rows in case of cron overlaps.

share

I faced exactly this problem that by using cron with an appropriate interval, it took over a day to send all the newsletters, by which time a second batch of newsletters was due to be sent.

So I created The Fat Controller which is a daemon that can run instances of a script (PHP, Python, whatever) in parallel. By using exit statuses, the scripts can tell The Fat Controller whether more data is still in the queue and if so it can restart the script and add another script running in parallel, up to a maximum number of parallel instances.

Using The Fat Controller we were able to reduce the time taken to send newsletters from over a day to just a few hours.

There's more information and use cases detailing this and similar scenarios on the website:

http://www.4pmp.com/fatcontroller/

share

It really doesnt matter how you send them. You just may have to turn off PHP's time limit so that the script will run as long as needed.

That said, what I'd recommend doing is generating the email text and sending the email seperately. Make a generic 'mail' table, for example, that will hold the recipient, subject, and text for each email, as well as flag it to be sent. Have a mailing script query that and after each send, flag it as sent or delete the email. This way if the process crashes 90% through, you dont have to re-run it and send duplicates.

At the one dot-com I worked for a while back, we had php based mailer script running 24x7. Every minute or two it'd check the mail table for any new mail to send. That worked well, and it handled all the application's email to be sent. It wasnt the most graceful thing going, but it was very reliable.

share

I use unix based deamon for probleme like your. Live monitoring expired account and immediately sent and email. http://phpdaemon.net/ is good for this

share