Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm currently developing a replication system to keep data in-synch between an arbitrary number of servers.

Some of these servers exist in one cluster on one LAN. Others exist somewhere else in the world.

I'm wondering what are the pros/cons of different paths that we choose to flow replicated data on between servers?

In other words, what are the different strategies to load balance the replication process ?

share|improve this question
add comment

migrated from stackoverflow.com May 21 '11 at 22:18

This question came from our site for professional and enthusiast programmers.

2 Answers

This would totally depend on your replication requirements.

  1. Do you want immediate consistency across all boxes? (If so, the choice is simple - replicate rt and deal with failures / slow speed).
  2. Do you want guaranteed tolerance against datacenter failures?
  3. How much data can you afford to lose in such a case? This would determine how often you move data across wan.
  4. Do you have local datacenters that can help alleviate such data center level failures? (This would still get into trouble if the whole city blacks out due to an earth quake let's say). But the advantages here are very good - you can have a local fiber ring and replicate quick if you are ok with such failures.

Overall long-term eventual consistency via scheduled backups work for many folks. Some others choose short-term eventual consistency by replicating in real time. Some others go for even more "Online" setups where you have to commit across at least two datacenters. There are systems that have location awareness built into their servers, so these could ensure you have to commit across locations. And there is the other end of replication - immediate hard consistency, everywhere.

share|improve this answer
add comment

Paxos is an algorithm that can be used for active-active replication between multiple masters. In the (patented) version we use, it has been enhanced to work well in a WAN environment.

A nice feature is the entirely of failure conditions are contained within the algorithm, so the issues of data loss or packet loss are not a threat to data safety. The amount of data you can afford to lose can be zero. (Assuming the entire universe isn't blown up at once.)

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.