Is PHP suitable for downloading large amounts of data, usually via cron? Should I worry about the memory footprint when I am using PHP for downloading multiple zip files?
closed as not a real question by MichaelT, gnat, Walter, thorsten müller, Mark Booth Mar 7 at 18:13
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, see the FAQ.
While you can use PHP for this, it is hardly the most efficient tool, nor the easiest to use. Specialized command-line downloaders such as
As tdammers pointed out, PHP can do this, but there are far more efficient tools for this task.
One of the reasons for this is, as you pointed out, the extra memory footprint required to instantiate a PHP script.
Also, while PHP can run perfectly fine in a command line (or continuous) environment, it is not specifically designed to do so. I have seen many professionals neglect memory management in these circumstances, because they're used to the PHP "we'll clean it up at the end of the script" methodology.
As a final point, why write something that already exists? Assuming that you don't need any special logic, simple curl statement would be just fine to download the zips you require. Also, depending on what you will be doing after this, you could pipe them through zip command and decompress them automatically after download.
Generally speaking, no, you probably dont have to be worried about the footprint of a PHP script. Its just not that large. So unless you are running a thousand concurrent instances of your script or something, the php footprint isnt likely to be an issue.
As others have pointed out, there are command line means of downloading files that dont require a PHP script if all you are doing is simply retrieving them. Though PHP or other scripting languages might certainly be useful if you are doing something to the data as its downloaded.
Conclusion: If PHP does the trick for you, knock yourself out.