Our MVC based framework does not allow us to cache only part of our output. Ideally we want to cache all static and semi-static parts, and run dynamic parts. In addition, we need to consider data caching that reacts to database changes.
The concept I came up with was to represent a page as a tree of XML fragment objects. Some of the fragments are dynamic, and can pull their data directly from models or other sources, but most of the fragments are static scaffolding.
If a subtree of fragments is completely static, then I imagine that they could unfold into XML that would then be cached as the text representation of their parent element.
This process would ideally continue until we are left with a root element that contains all of the static XML, and has a couple of dynamic XML fragments that are resolved and attached to the relevant nodes of the XML tree just before the page is displayed.
In addition to separating content into dynamic and static fragments, some fragments could be dynamic and cached. A simple expiry time which propagates up through the XML fragment tree would indicate that a specific fragment should periodically be refreshed. A newspaper section or front page does not need to be updated each second. Minutes or sometimes even longer is sufficient.
Other fragments would be dynamic and uncached. Typically too many articles are viewed for them to be cached - the cache would overflow. Some individual articles may be cached if they are extremely popular.
The folding mechanism could be to be smart enough to judge when it would be more profitable to fold a dynamic cached fragment and propagate the expiry date to the parent fragment, or to keep it separate and simply attach to the XML tree when resolving the page.
If some dynamic cached fragments are associated to database objects through mechanisms like a globally unique content id, then changes to the database could trigger changes to the output cache. If fragments store the identifiers of parent fragments, then they could trigger a refolding process that would then include the updated data.
A set of pure XML with an ordered array of fragment objects (that each store the identifying information of the node to which they should be attached), can be resolved in a fairly simple way by walking the XML tree, and merging the data from the fragments.
Because it is not necessary to parse and construct the entire tree in memory before attaching nodes, processing should be fairly fast.
The identifiers of each fragment would be a combination of relevant identity data and the type of fragment object. Cached parent fragments would contain references to these identifiers, in order to then either pull them from the fragment cache, or to run their code.
The controller's responsibility is reduced to making changes to the database, and telling the root XML fragment object to render itself.
I would just like to make it clear that while I say XML, what I really mean is XHTML. The design generates and folds XHTML from the very start, without trying to add an additional step to transform the XML to XHTML. It's just functionally equivalent to XML.
The design is still essentially MVC. Traditionally in Web applications, controllers make changes to models directly, then load all the data from models, and send it to the view.
The problem with this is that it's hard to do partial page caching.
Most web applications spend most of their time in the DB. If the controller has to load the data from models each time, then caching the view has no point, unless you cache the whole view and do no database queries.
However, in Smalltalk, where the MVC pattern originally came from, views pulled data from models directly. "The view gets its own data from the model." - MVC on Wikipedia
If views pull their data from the models, and views are composed of other views, then it becomes trivial to cache the output of views. If it does not need loading again, then it doesn't load, and it doesn't contact the DB.
I am treating view output as XML/XHTML because we have a couple of tools that allow us to manipulate XML quite well, making the interweaving of static and dynamic fragments fairly simple.
My question has two parts:
Is this a good design? Are there any obvious flaws I'm missing?
Has somebody else thought of this before? References?
Is there an existing alternative that I should consider? A cool templating engine maybe?