In my application, I'm getting product information files from an FTP server. They are text files that have been ZIP-compressed.
At first, I thought uncompressed the file on the server then reading the text over the web would work, but the uncompressed files are ~900 MB, making it slow and inefficient while causing out-of-memory exceptions.
How should I handle this type of problem? Should I download the compressed files, uncompress and process them locally, then re-upload them? Or is there another, more efficient way to do this?