I frequently encounter a situation where I need to write a program which reads data-objects from a file or system, does a transformation on the data, convert the data to another datamodel and then push the data in another file, or system. Do you know of a object-model which facilitates these type of programs in such a way that it is pluggable and scalable?
Though it is mainly for multimedia domain, and i don't know what is yours; however, here is what you can gain from the overall architecture -
There are many more stuff - i am not trying the exhaustive about what gstreamer does but only suggesting that this could be directly useful to you or at least can inspire you a lot.
Some people believe that gstreamer is derived out of DirectX. That may be true or false but doesn't bother me. I am not comparing.
There is a website related to it here that contains a wealth of inter-application processing design patterns for you to browse. In particular, I'd draw your attention to these patterns:
Messaging How can I integrate multiple applications so that they work together and can exchange information?
Message Channel How does one application communicate with another using messaging?
Message Router How can you decouple individual processing steps so that messages can be passed to different filters depending on a set of conditions?
I've got a copy of the book myself and it's packed with detail about inter-application and inter-system design patterns.
here's a link to an article on an alleged Extract/Transform/Load (ETL) design pattern but it sounds like what you want is a framework