Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

My problem

Our MVC based framework does not allow us to cache only part of our output. Ideally we want to cache all static and semi-static parts, and run dynamic parts. In addition, we need to consider data caching that reacts to database changes.

My idea

The concept I came up with was to represent a page as a tree of XML fragment objects. Some of the fragments are dynamic, and can pull their data directly from models or other sources, but most of the fragments are static scaffolding.

If a subtree of fragments is completely static, then I imagine that they could unfold into XML that would then be cached as the text representation of their parent element.

This process would ideally continue until we are left with a root element that contains all of the static XML, and has a couple of dynamic XML fragments that are resolved and attached to the relevant nodes of the XML tree just before the page is displayed.

In addition to separating content into dynamic and static fragments, some fragments could be dynamic and cached. A simple expiry time which propagates up through the XML fragment tree would indicate that a specific fragment should periodically be refreshed. A newspaper section or front page does not need to be updated each second. Minutes or sometimes even longer is sufficient.

Other fragments would be dynamic and uncached. Typically too many articles are viewed for them to be cached - the cache would overflow. Some individual articles may be cached if they are extremely popular.

Functional notes

The folding mechanism could be to be smart enough to judge when it would be more profitable to fold a dynamic cached fragment and propagate the expiry date to the parent fragment, or to keep it separate and simply attach to the XML tree when resolving the page.

If some dynamic cached fragments are associated to database objects through mechanisms like a globally unique content id, then changes to the database could trigger changes to the output cache. If fragments store the identifiers of parent fragments, then they could trigger a refolding process that would then include the updated data.

A set of pure XML with an ordered array of fragment objects (that each store the identifying information of the node to which they should be attached), can be resolved in a fairly simple way by walking the XML tree, and merging the data from the fragments.

Because it is not necessary to parse and construct the entire tree in memory before attaching nodes, processing should be fairly fast.

The identifiers of each fragment would be a combination of relevant identity data and the type of fragment object. Cached parent fragments would contain references to these identifiers, in order to then either pull them from the fragment cache, or to run their code.

The controller's responsibility is reduced to making changes to the database, and telling the root XML fragment object to render itself.

I would just like to make it clear that while I say XML, what I really mean is XHTML. The design generates and folds XHTML from the very start, without trying to add an additional step to transform the XML to XHTML. It's just functionally equivalent to XML.

UPDATE:

Modified MVC

The design is still essentially MVC. Traditionally in Web applications, controllers make changes to models directly, then load all the data from models, and send it to the view.

The problem with this is that it's hard to do partial page caching.

Most web applications spend most of their time in the DB. If the controller has to load the data from models each time, then caching the view has no point, unless you cache the whole view and do no database queries.

However, in Smalltalk, where the MVC pattern originally came from, views pulled data from models directly. "The view gets its own data from the model." - MVC on Wikipedia

If views pull their data from the models, and views are composed of other views, then it becomes trivial to cache the output of views. If it does not need loading again, then it doesn't load, and it doesn't contact the DB.

I am treating view output as XML/XHTML because we have a couple of tools that allow us to manipulate XML quite well, making the interweaving of static and dynamic fragments fairly simple.

The Question

My question has two parts:

  1. Is this a good design? Are there any obvious flaws I'm missing?

  2. Has somebody else thought of this before? References?

  3. Is there an existing alternative that I should consider? A cool templating engine maybe?

share|improve this question
1  
This caching, is done on the server? Or do you expect browsers to somehow "cache" this folded XML? –  Raynos Nov 11 '11 at 14:22
1  
It would be cached on the server in a key-value store (like MongoDB or Memcache). I expect that a consistent key can be created from the fragment type and identification. When resolving fragments, a pull attempt is made from the store, if that fails, the code is run to generate the fragment, and optionally inserted into the fragment cache. –  Gustav Bertram Nov 11 '11 at 14:26
1  
The only downside I can think of, is how easy will it be to write XHTML in your system? How easy/hard will it be to have full control over what your page looks like, will all these XML snippets be all over the place? I also don't see the point compared to generic caching templating languages –  Raynos Nov 11 '11 at 14:30
1  
I expect that it will be no more difficult than writing a set of views that load their own data, and also load subviews. The expected advantage is that static, or semi-static elements are cached, and dynamic elements can be interspersed as desired. Indeed, cache times could also be customized for each fragment. Cached elements associated with data objects will only update when the underlying data is updated. If you know of a templating engine that caches static output, but still handles dynamic, please put it in an answer. –  Gustav Bertram Nov 11 '11 at 14:36
1  
I can't think of any templating engines that do that, they must exist and I'm halfway through writing a templating engine that caches static content –  Raynos Nov 11 '11 at 14:40

1 Answer 1

up vote 3 down vote accepted
+50

This sounds suspiciously like you're trying to invent output caching. I definitely wouldn't agree that it's hard to do partial page caching in MVC, it's just that you would need to use partial Views with their own Controller actions to do it as opposed to having the View itself call its sub-Views, which can be slightly counter-intuitive. (So in ASP.net MVC you'd want to use Html.RenderAction as opposed to Html.RenderPartial). There's a name for this particular pattern that is currently escaping my recollection.

I would suggest that the main flaw with your design is that Views will have to know things about the architecture of the site. So they'll have to know about where to get their data from AND how to get it, they'll have to know about where to cache it, when to cache it, when not to cache it, etc.

Realistically you should be trying to separate layers away from knowledge of other layers, as the less knowledge each layer has of another the easier it is to change a layer (i.e. switch DB, add a transparent data caching layer, move a DB call across to a web service, etc.).

I would suggest that if you're going to implement such an idea, and there isn't a native output caching system in your MVC-framework-of-choice, that you add the caching layer to the Controller actions. Unless you've got some VERY heavyweight Views that need to do large amounts of recursive rendering of Models (something that's very rare) the actual HTML generation time is miniscule compared to DB calls and client network latency, so take a more pragmatic approach and cache where you need to cache, i.e. in the application layer.

If you really need custom output caching then you probably want to just slip in a caching layer above your Controller actions with either a wrapping class (if you're using a dynamic language) or a different implementation of an interface (if in a static language) that can hijack the calls to the Action and react accordingly.

As for caching that reacts to DB changes, you'd be better off with a caching layer that takes into account both loading and saving in your repository class, with each save call flushing the cache (or part of the cached set) and each load being gracefully degraded from cache to DB when needed (i.e. use the cache when the cached data is available). That way you keep the database driven behaviour close to the database.

share|improve this answer
    
CodeIgniter's output caching exists, but is monolithic, which lead me on this navel-gazing journey. I believe the pattern you are referring to is the Hierarchical Model View Controller, also known as Presentation-Abstraction-Control. It's better than my idea, so I'll use it instead. The only caveat is that one has to be careful not to fragment your code into so many tiny triads that the separation of layers inside them becomes pointless. Tiny triad trees! –  Gustav Bertram Nov 16 '11 at 13:35
    
That was indeed the pattern I was referring to! I agree about the fragmentation issue, but if you're careful about it you shouldn't have many issues unless your project gets really large and is full of edge-cases that eliminate code reuse. Anyway, that's why you should use a "service" layer as well as a controller layer and repository layer, so you can share functionality between actions easily, and actions are really only there to prepare data for display. –  Ed Woodcock Nov 16 '11 at 14:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.