I have a series of systems that need to be distributed to a variable number of nodes (at least two, but more likely 8 - 10). For performance reasons, any one piece of state needs to be maintained in memory and replicated to at least two nodes although as demand increases this may be replicated to more nodes as dictated by the manager. Unfortunately, we cannot buy commercial data grids which are already very good at this.
The question: Assuming the state is immutable, what are some patterns and/or algorithms I can use to facilitate this process? I've supplied a sample to facilitate the discussion, but I'm not tied to this. Really!... I'm wide open on picking a new direction to meet these requirements. If you could point me to pictures or code samples that would be much appreciated.
As it stands, one of the nodes will self-promote and will orchestrate who should replicate and when (i.e. the manager).
- List item
- Systems will communicate via TCP/IP and Multicast
- No databases
- No file transfer... just byte arrays
- All components are currently C#, and I can change almost anything
- Eventual consistency will be fine
Note: I realize this sample does not handle fail cases. At this point I thought it would over-complicate things