I've asked this question before and I got a few answers/idea but I'm not sure how to implement them.
I'm building a telecom messaging solution, currently I'm using Database to save my transaction/messages for the network stack I've built, and as you know it's slower than using data structure (hash, linkedlist, etc...), my problem is that the data can be really huge and it won't fit in the memory.
I was thinking of saving the records in a file and the a key and line number in a hash, then if I want to access some record then I can get the line number from the hash and get it from the file. I don't know how efficient is this, I think the DB is doing way better job than this on my behalf.
Please share whatever you have in mind.