Favoring Immutability in Database Design

Question

One of the items in Joshua Bloch's Effective Java is the notion that classes should allow mutation of instances as little as possible, and preferably not at all.

Oftentimes, the data of an object is persisted to a database of some form. This has led me to thinking about the idea of immutability within a database, especially for those tables that represent a single entity within a larger system.

Something I have been experimenting with recently is the idea of trying to minimize the updates I do to table rows representing these objects, and trying to perform inserts instead as much as I can.

A concrete example of something I was experimenting with recently. If I know I might append a record with additional data later on, I'll create another table to represent that, sort of like the following two table definitions:

create table myObj (id integer, ...other_data... not null);
create table myObjSuppliment (id integer, myObjId integer, ...more_data... not null);

It is hopefully obvious that these names are not verbatim, but just to demonstrate the idea.

Is this a reasonable approach to data persistence modeling? Is it worth trying to limit updates performed on a table, especially for filling in nulls for data that might not exist when the record is originally created? Are there times when an approach like this might cause severe pain later on?

Answer 1

The primary purpose of immutability is to ensure that there's no instant in time when the data in memory is in an invalid state. (The other is because mathematical notations are mostly static, and so immutable things are easier to conceptualize and model mathematically.) In memory, if another thread tries to read or write data while it's being worked with, it might end up going corrupt, or it might itself be in a corrupt state. If you have multiple assignment operations to an object's fields, in a multithreaded application, another thread might try to work with it sometime in between -- which could be bad.

Immutability remedies this by first writing all the changes to a new place in memory, and then doing the final assignment as one fell-swoop step of rewriting the pointer to the object to point to the new object -- which on all CPUs is an atomic operation.

Databases do the same thing using atomic transactions: when you start a transaction, it writes all the new updates to a new place on disk. When you finish the transaction, it changes the pointer on disk to where the new updates are -- which it does in a short instant during which other processes can't touch it.

This is also the exact same thing as your idea of creating new tables, except more automatic and more flexible.

So to answer your question, yes, immutability is good in databases, but no, you don't need to make separate tables just for that purpose; you can just use whatever atomic transaction commands are available for your database system.

Answer 2

It depends on what benefits you expect to gain from immutability. Rei Miyasaka's answer addressed one (avoidance of invalid intermediate states), but here's another.

Mutation is sometimes called destructive update: when you mutate an object, the old state is lost (unless you take additional steps to explicitly preserve it somehow). In contrast, with immutable data, it's trivial to simultaneously represent the state both before and after some operation, or to represent multiple successor states. Imagine trying to implement a breadth-first search by mutating a single state object.

This probably shows up in the database world most often as temporal data. Say last month you were on the Basic plan, but on the 16th you switched to the Premium plan. If we just overwrote some field that indicated what plan you're on, we might have difficulties getting billing right. We might also miss out on the ability to analyze trends. (Hey, look what this local ad campaign did!)

That's what comes to my mind when you say "immutability in database design", anyway.

Answer 3

The idea of avoiding updates, and preferring inserts, is one of the thoughts behind building your data storage as an Event Source, an idea you will often find used together with CQRS. In an event source model, there is no update: an aggregate is represented as the sequence of its "transformation" (events), and as a result the storage is append-only.
This site contains interesting discussions on CQRS and event sourcing, if you are curious about it!

Answer 4

Another possible reason for having an immutable database would be to support better parallel processing. Updates happening out of order can mess up data permanently, so locking has to occur to prevent that, destroying parallel performance. Lots of inserts of events can go in any order, and the state will at least be eventually right as long as all events are eventually processed. However this is so hard to work with in practice compared to doing database updates that you would have to really need a lot of parallelism to consider doing things this way - I am not recommending it.

Answer 5

Disclaimer: I am pretty much a newby in DB :p

That being said, this approach of satellizing data has an immediate impact on performance:

• Good less traffic on the primary table
• Good smaller rows on the primary table
• Bad requiring the satellite data means another look-up is necessary
• Bad more space occupied if all objects exist in both tables

depending on your requirements, you may either welcome this, or not, but it is certainly a point to consider.

Answer 6

This bears a very close relationship with what are known as "Slowly Changing Dimensions" in hte data warehousing world, and , "Temporal" or "Bi-Temporal" tables in other doamains.

The basoc construct is:-

1. Always use a generated surrogate key as the primary key.
2. The unique identifier of whatever you are describing becomes the "logical key".
3. Each row should have at least a "ValidFrom" timestamp and optionally a "ValidTo" timestamp and even more optionally a "Latest Version" flag.
4. On the "creation" of a logical entity you In sert a new row with a "Valid From" of current timestamp. The optional ValidTo set to "forever" (9999-12-31 23:59:59) and Last Version to "True.
5. On a subsequent update of the logical entity. You at least insert a new row as above. You may also need to adjust the ValidTo on the previous version to "now() - 1 second" and the Lastest Version to "False"
   1. On logical deletion (this only works with the ValidTo timestamp!) you set the ValidTo flag in the current row to "now() -1 second".

The advantages of this scheme is that you can recreate the "state" of your logical entity at any point in time, you have a history of your entity over time and you minimize contention if your "logical entity" is heavily used.

The disadvantages are you store a lot more data, and, you need to maintain more indexes ( at the very least on Logical Key + ValidFrom + ValidTo. An index on Logical Key + Latest Version greatly speeds up most queries. It also complicates your SQL!

Whether this is worth doing unless you really need to maintain a history and have a requirement to recreate the state of your entities at a given point in time is up to you.