Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote -1 down vote favorite

Is PHP suitable for downloading large amounts of data, usually via cron? Should I worry about the memory footprint when I am using PHP for downloading multiple zip files?

share|improve this question
7  
Answers on efficiency will be completely speculative, unless you describe exactly what your doing. Why are you using PHP in the first place? For most common scenarios that are related to what your doing, PHP would be a very weird (to say the least) choice. If you've managed to work out cron, chances are you are familiar with wget and curl. – Yannis Rizos♦ Nov 18 '11 at 20:02

closed as not a real question by MichaelT, gnat, Walter, thorsten müller, Mark Booth Mar 7 at 18:13

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, see the FAQ.

3 Answers

up vote 5 down vote

While you can use PHP for this, it is hardly the most efficient tool, nor the easiest to use. Specialized command-line downloaders such as wget or curl are much better suited; you can drive them directly from cron, or you can write a simple shell script to encapsulate the required parameters. Another advantage of using shell script is that it is fairly easy to do multiple downloads in parallel (by chaining commands together with &).

share|improve this answer
up vote 1 down vote

As tdammers pointed out, PHP can do this, but there are far more efficient tools for this task.

One of the reasons for this is, as you pointed out, the extra memory footprint required to instantiate a PHP script.

Also, while PHP can run perfectly fine in a command line (or continuous) environment, it is not specifically designed to do so. I have seen many professionals neglect memory management in these circumstances, because they're used to the PHP "we'll clean it up at the end of the script" methodology.

As a final point, why write something that already exists? Assuming that you don't need any special logic, simple curl statement would be just fine to download the zips you require. Also, depending on what you will be doing after this, you could pipe them through zip command and decompress them automatically after download.

share|improve this answer
up vote 1 down vote

Should I worry about the memory footprint when I am using PHP for downloading multiple zip files?

Generally speaking, no, you probably dont have to be worried about the footprint of a PHP script. Its just not that large. So unless you are running a thousand concurrent instances of your script or something, the php footprint isnt likely to be an issue.

As others have pointed out, there are command line means of downloading files that dont require a PHP script if all you are doing is simply retrieving them. Though PHP or other scripting languages might certainly be useful if you are doing something to the data as its downloaded.

Conclusion: If PHP does the trick for you, knock yourself out.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.