Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I believe I have a fundamental misunderstanding when it comes to workflow engines which I would appreciate if you could help me sort out. I'm not sure if my misunderstanding is specific to the workflow engine I'm using, or if it's a general misunderstanding. I happen to use Windows Workflow Foundation (WWF).

TLDR-version

WWF allows you to implement business processes in long-running workflows (think months or even years). When started, the workflows can't be changed. But what business process can't change at any time? And if a business process changes, wouldn't you want your software to reflect this change for already started 'instances' of the business process? What am I missing?

Background

In WWF you define a workflow by combining a set of activites. There are different types of activities - some of them are for flow control, such as the IfElseActivity and the WhileActivty while others allows you to perform actual tasks, such as the CodeActivity wich allows you to run .NET code and the InvokeWebServiceActivity which allows you to call web services. The activites are combined to a workflow using a visual designer. You pretty much drag-and-drop activities from a toolbox to a designer area and connect the activites to each other. The workflow and activities have input paramters, output parameters and variables.

We have a single workflow which sometimes runs in a matter of a few days, but it may run for 5-6 months. WWF takes care of persisting the workflow state (what activity are we currently executing, what are the variable values and so on).

So far I think WWF makes sense. Some people will prefer to implement a software representation of a business process using a visual designer over writing all of it in code.

So what's the issue then?

What I don't really get is the following:

WWF is designed to take care of long-running workflows. But at the same time, WWF has no built-in functionality which allows you to modify the running workflows. So if you model a business process using a workflow and run that for 6 months, you better hope that the business process does not change. Because if it do, you'll have to have multiple versions of the workflow executing at the same time. This seems like a fundamental design mistake to me, but at the same time it seems more likely that I've misunderstood something.

For us, this has had some real-world effects:

  • We release new versions every month, but some workflows may run for a year. This means that we have several versions of the workflow running in parallell, in other words several versions of the business logics. This is the same as having many differnt versions of your code running in production in the same system at the same time, which becomes a bit hard to understand for users. (depending on on whether they clicked a 'Start' button 9 or 10 months ago, the software will behave differently)
  • Our workflow refers to different types of entities and since WWF now has persisted and serialized these we can't really refactor the entities since then existing workflows can't be resumed (deserialization will fail

We've received some suggestions on how to handle this

  • When we create a new version of the workflow, cancel all running workflows and create new ones. But in our workflows there's a lot of manual work involved and if we start from scratch a lot of people has to re-do their work.
  • Track what has been done in the workflow and when you create a new one skip activites which have already been executed. I feel that this alternative may work for simple workflows, but it becomes hairy to automatically figure out what activities to skip if there's major refactoring done to a workflow.
  • When we create a new version of the workflow, upgrade old versions using the new WWF 4.5 functionality for upgrading workflows. But then we would have to skip using the visual designer and write code to inject activities in the right places in the workflow. According to MSDN, this upgrade functionality is only intended for minor bug fixes and not larger changes.

What am I missing?

share|improve this question

2 Answers 2

tl;dr: Long-running workflows are not expected to change during their lifetime. If there's a possibility that they might change, break them into smaller and shorter workflows, and chain them together to make the larger workflow.

Changing Running Logic is Hard

Why does your computer system sometimes require a reboot? Because you can't change out device drivers while they're in use. How do you get around this? By making things immutable and writing functional-style programs.

The Erlang programming language is specifically designed to be upgraded while programs are running. How does it do this? By maintaining immutable instances of functions in memory. When you hot-patch a program in Erlang, Erlang simply maintains the existing functional instances until they finish executing. Any new instances of the functions will use the new version of the function, and because the functions are immutable, stability is assured during the transition.

Note that this would not help you with your workflows, because you still wouldn't be able to hot-patch an existing workflow while it is running. Hot patching refers to entire systems of functions, not individual instances of running functions. Your workflow (essentially a long-running function) would still have to finish executing first.

As you've already pointed out, changes to running workflows are intended for minor tweaks, not wholesale changes to the workflow.

share|improve this answer

Track what has been done in the workflow and when you create a new one skip activites which have already been executed. I feel that this alternative may work for simple workflows, but it becomes hairy to automatically figure out what activities to skip if there's major refactoring done to a workflow.

The problem is, it can become hairy even manually to to figure out what activities to skip. For example, what if the current activity in the old workflow has no pendant in the new one? So I suggest you should take a combined strategy:

  • use the "track & skip" approach where easily possible
  • use the "cancel & restart" where not possible
  • for special cases you could try to create sub-workflows to avoid a complete restart, specificially designed for continuing an already started (old) workflow at a specific activity, but without the need to restart from the complete beginning
  • try to keep the number of "transition points" small (the activities where the actors have to change from the old workflow version to the new ones) - make sure your actors reach one of those points until the old workflow gets replaced by the new version. The transition points should be always having a pendant in the new version. This will help in all of the above cases.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.