Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I want to create a service where people can upload files. However, since file storage costs money, I want to compress the files so they take less space. I would want to write my own compression algorithm, however, PHP doesn't have good ways to handle binary data (which is needed for many compression algorithms). So I wondered, what would be a better language to create such a website in?

I have knowledge of PHP (and Javascript, HTML and CSS) but no experience with other things like Ruby, Perl, Python, and other web development languages.

share|improve this question
8  
Compression algorithms are a seriously researched area. Do you really think you are going to build a better one than the dozen or so out there? –  Matthew Flynn Nov 27 '11 at 22:45
1  
No, I don't. However, I would love to try writing one myself. :) –  Qqwy Nov 27 '11 at 22:47
1  
C/C++ is pretty awesome for this –  Raynos Nov 27 '11 at 23:25
1  
Designing and implementing a compression algorithm is a very different project from a storage service. The best way would be to write your compressor in C and your service app in whatever you know best (PHP in your case). That way you use the best tool for each one, and the success/failure of one isn't tied to the other. –  Javier Nov 28 '11 at 3:53
2  
BTW, keep in mind that the vast majority of space-hogging user files (images, music, video) are already compressed, so you'd be hard pressed to improve on that by even a few percent. Specially since most of these are content-specific lossy algorithms. IMHO, adding compression to the service would only be worth if your target demographic suggest lots of non-compressed data or huge opportunities for deduplication (for example, you're likely to get lots of direct copies from a single popular DVD movie. but if theyr'e recompressed, forget about it.) –  Javier Nov 28 '11 at 3:58
add comment

4 Answers

up vote 6 down vote accepted

PHP supports a variety of compression algorithms and popular compression libraries, like Bzip2, LZF and Rar, and for most common scenarios it provides excellent support for working with binary data. There's at least one working PHP implementation of a compression algorithm out there, proving that it's possible. PHP and any other interpreted language may prove inefficient for the task, but efficiency issues highly depend on the specifics of the implementation and the actual usage of your library, for which you don't provide enough information.

I would advise you to not try and create your own compression algorithm and use an established library instead, there's no point in reinventing the wheel. You don't have to use PHP if you don't want to, every web oriented language out there has support for popular compression libraries. You can use perl's Gzip and Zlib core modules, or Python's gzip or bzip2 modules. I don't know what kind of support for compression Ruby has, but this StackOverflow question can help you, if you choose Ruby.

But if for any reason you really want to write your own original compression algorithm, you should probably choose C. C is extremely portable* and if you manage to actually built a compression library then you can use it from PHP via an extension or in Python via a module. Both PHP extensions and Python modules are also commonly written in C, and it would be fairly easy to write one of those if you coped with writing a compression algorithm. And there's always the old school option of C web programming via CGI, if you don't mind diving into a world of hurt.

* There are C compilers for almost every imaginable platform and ways to use C libraries from almost every other language.

share|improve this answer
1  
+1 for no wheel-reinvention. Compression is a VERY well studied field and whatever you might roll of your own just isn't going to perform as well as something you'll get from somebody else. Plus it's probably free and already has bindings to your language of choice. –  Dan Ray Nov 28 '11 at 13:17
add comment

Any language that allows you to shuffle bits and bytes can be used to write a compressor. I'm pretty sure PHP lets you shuffle bits and bytes but it's been a while since I wrote anything in it. Various javascript environments like Node.js also have support for binary data so it's not a matter of which web-oriented language would work best with binary data but how much of a performance penalty you're willing to take by writing a compressor in one.

share|improve this answer
add comment

In truth you best bet it to use a pre existing compression package. Really you are not going to do better than gzip or bzip2 or the like.

Also remember if they are uploading images, sound or video then the files are already compressed so you will not be able to compress them farther.

That being said if you want to work with Binary data check out Erlang. It is an amazing langauge and it has really cool syntax for working with bits.

Disclaimer: I wrote a book on Erlang

share|improve this answer
add comment

Haskell, Ruby, Python, Java and other JVM languages, and C# and other .NET languages all have good support for both web programming and programming against binary data.

I personally would find Ruby and Python to be the simplest to get into.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.