Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

This is long question I know. I have a few options for what I want. I don't want to know "this is obviously the best," but maybe any pros or cons I may be missing about the options, as well as any options I'm unaware of.

We have a java application seperately run as multuple daemons, each a seperate JVM. Right now SQL is used to store data and effecticvely our 'shared memory', but luckily not much is dynamic once in memory.

We will soon be dynamically creating and destroying connections and that will mean dynamic changes happening semi-fast and need something better then a SQL database. We think all the dynamic stuff will happen in only one daemon, but we aren't certain.

We will need a faster method of saving and loading this state data, going to disk each time is just too slow. The current database needs an overhaul for this change anyways, so we're open to modification or tossing of it as well. We can re-learn all the state we keep in memory; but it's a slow process. Persisting some or all of the data to a DB between runs to save time learning state at bootup is nice, if we can be certain it gets persisted right.

Ability to easily query a 'database' to see current state for debugging is a major plus.

The solutions I have considered, and some of the pros & cons are:

1) some sort of hibernate solution combined with an in-memory sql database. I haven't used hibernate yet (really want to!), but I think it can address persistance and avoid stale objects between JVM's for us and sql will still be close to the speed of other options if in memory? I don't know how much the overhead for this is or how it scales to multuple JVM's and possible (semi?) fast changes to data. Presumable this is slower then all the other options, but think it's fast enough for our needs?

2) Have each JVM store it's own internal state without any shared system. Use reliable intercommunication (probably JVM) to try to keep state consistent. Most important state is in one daemon, so there won't be that much sharing of dynamic state, but still need to address inconsistent state and dataraces when it does occure. I'm uncertain of how we would want to record persistant state to allow fast reboot even if daemon dies unexpectedly. Also, this solution would require implementing some quick messages to allow fetching a description of memory state to help debugging. It would offer the highest performance once we get it right, but probably the most time consuming to implement (I anticipate obscure bugs due to data-race and inconsistent data?).

3) Use a system to allow some shared memory between JVMs. Quick research suggested MappedByteBuffer and Ecache as possible options, there could be more. Only looked into this a little so far so don't fully know pro's and cons yet. Slightly afraid of creating a single point of failure if our memory is all shared.

4) Some sort of alternative I don't know about or possible merging of two of these idea.

share|improve this question
1  
You can create an in-memory file and use that. It's probably the fastest idea. I'd like some idea about how much and what type of data you need to share. –  Benjamin Gruenbaum Mar 1 '13 at 18:41
add comment

3 Answers

up vote 2 down vote accepted

The easiest and most stable method would probably to use an in-memory database (SQL or not). I'm not sure what your data structure looks like, but a relatively competent DBA (or someone with a bit of Google Fu) should be able to configure the system to store the relevant tables in memory, assuming they fit.

share|improve this answer
1  
Even if the existing database doesn't have a store-in-memory mode, putting its data files on a RAM disk can make a significant difference. –  Blrfl Mar 1 '13 at 17:36
add comment

Have you looked at the Hazelcast technology? We're using it internally with vert.x - goes without a hitch (for now).

share|improve this answer
add comment

How much data are you saving/retrieving? Is it simply configuration information, or do you want to store business data?

If the latter, then the only thing that makes sense is an on-disk database. Unless you're OK with losing all of your data due to a system crash, and don't mind dirty reads. And I wouldn't rule out SQL-Server based on performance until you've actually tested it.

If all you want to do is share a (relatively small amount of) configuration data, then take a look at Apache Zookeeper: http://zookeeper.apache.org/

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.