Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I work on a team of three - five developers that work on an web application remotely. We currently run a full local database from a recent backup on all of our machines during development. The current backup, compressed, is about 18 GB.

I'm looking to see if there's an easier way to keep all of our local copies relatively fresh without each of us individually downloading the 18 GB file over HTTP from our web server on a regular basis. I guess FTP is an option, but it won't speed the process up at all.

I'm familiar with torrents and the thought keeps hitting me that something like that would be effective, but I'm unsure of the security or the process.

share|improve this question
Wouldn't cloning it on a centralized server be an option? We have a testdatabase with about 120 GB that we use that way when we need the complete database. For most other development tasks I prefer a far smaller test database with a few "typical" entries. – thorsten müller Nov 7 '12 at 15:14
The main problem with that is that all of our servers are behind a firewall and only allow HTTP traffic and VPN for RDP. Having the entire database locally allows offline development as well. – MattGWagner Nov 7 '12 at 15:16
If all you want to do is speed up transfers of the DB so that you're all using the same copy each day, you could torrent it. Put a tracker on the server and each dev/test has a client running. – Steve Evers Nov 7 '12 at 15:33
Might be worth to invest in a server located closer to the developers. – johannes Nov 7 '12 at 16:25
Other tools you might want to research are rsync/deltacopy which only transmits the delta, or ant/nant tasks to download it at night/weekend. – james Nov 7 '12 at 20:32

Does your database support incremental backups or log shipping? If so, you might try one of those, and just update the database instead of doing a complete restore. This won't give much advantage if your DB structure changes a lot, but if it's mostly data changes then this could save you a good deal of time (and network bandwidth).

share|improve this answer

rsync or some other copy-only-differences system. Deltas between two subsequent db dumps ought to be pretty similar.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.