# When decomposing a large function, how can I avoid the complexity from the extra subfunctions?

Say I have a large function like the following:

function do_lots_of_stuff(){

{ //subpart 1
...
}

...

{ //subpart N
...
}
}

a common pattern is to decompose it into subfunctions

function do_lots_of_stuff(){
subpart_1(...)
subpart_2(...)
...
subpart_N(...)
}

I usually find that decomposition has two main advantages:

1. The decomposed function becomes much smaller. This can help people read it without getting lost in the details.
2. Parameters have to be explicitly passed to the underlying subfunctions, instead of being implicitly available by just being in scope. This can help readability and modularity in some situations.

However, I also find that decomposition has some disadvantages:

1. There are no guarantees that the subfunctions "belong" to do_lots_of_stuff so there is nothing stopping someone from accidentally calling them from a wrong place.
2. A module's complexity grows quadratically with the number of functions we add to it. (There are more possible ways for things to call each other)

Therefore:

Are there useful convention or coding styles that help me balance the pros and cons of function decomposition or should I just use an editor with code folding and call it a day?

EDIT: This problem also applies to functional code (although in a less pressing manner). For example, in a functional setting we would have the subparts be returning values that are combined in the end and the decomposition problem of having lots of subfunctions being able to use each other is still present.

We can't always assume that the problem domain will be able to be modeled on just some small simple types with just a few highly orthogonal functions. There will always be complicated algorithms or long lists of business rules that we still want to correctly be able to deal with.

function do_lots_of_stuff(){
p1 = subpart_1()
p2 = subpart_2()
pN = subpart_N()
return assembleStuff(p1, p2, ..., pN)
}
-
Can you use a class that has a public function do_lots_of_stuff and private functions: subpart_1 subpart_2 etc.? – MasterMastic Jun 20 '12 at 18:34
Your peer tells you after reviewing the code whether your balance is right or not – gnat Jun 20 '12 at 18:43
– user281377 Jun 20 '12 at 20:08
Bleh, none of the answers really deal with complex data... Repeating the Functional Programming Mantra doesn't really help the OP... – Paul Jun 20 '12 at 20:27
@Paul: I disagree. It's quite likely at the root of the problem, or then the OP should provide us with the code so we can have a closer look. I'd attack a similar problem in my code exactly with the angle I give below. The FP approach does help. Of course, you'll need to entangle internal states and coupled APIs, which can be difficult if the state of code rot is already advanced and rigor mortis has already set in, but that's still the way to go. You take it apart brick by brick, block by block. – haylem Jun 20 '12 at 20:39

Keep each function as simple as possible.

Think of it in simple terms, the way a function is meant to be:

• gets 0 to N inputs,
• returns 0 or 1 result (possible a composite or collection),
• and isn't tied to state.

When you stick to functional programming idioms, you get rid of most of these questions you're asking yourself. Sure, your class will get bigger in terms of number of functions. But if the methods are not tied to each by internal state changes, they get easier to understand, manage and compose to achieve an end result.

Also, try to give them appropriate accesses based on the above design decisions. Helpers will commonly easily be declared as statics (and if they don't seem to need to be private or can be reused, they could extracted to an helper class), which gives a strong hint to the other developers: this thing is meant to be independent and side-effect free.

Repeat the following mantras to aim for purity:

• My function shall be:
• short,
• side-effects free,
• realizing one and one function only.
• My function shall be strict on output. [1]
• My function shall be testable, and tested.
• My function shall be readable and read like a natural language expression.
• My function shall be documented. [2]
• My function shall be null-hostile.

[1] Whether it shall be strict or lenient on input depends on whether it's consumer code or library code.
[2] Self-documentation counts, comments for tricky parts count as well.

Of course, if you are in a generally non-FP-oriented code base, you won't manage to avoid shared mutable state for ever, but it's a very good, sensitive and no-BS guideline to follow. Even if you do get it wrong by over-modularizing and complexifying your class, it'll still be easier to pick up from there and refactor again than from a giant dump of code with high complexity and tight coupling.

Regarding the rules of the compositionality of your functions, these are your business rules. They are dictated by what you want to achieve, there's no automagical way of determining it for you.

-
We all know that short composable functions are a good goal but we still need to be able to deal with what to do when we have a long list of business logic to follow or a complex algorithm to implement. – hugomg Jun 20 '12 at 19:55
@missingno: you can address that in your class's documentation, and if you class really depends this much on a non-natural order of function invocations, then maybe having a builder to enforce it (or even getting close to a state machine) is what you want. You won't be able to avoid that anyways: you'll always need to know as the developer how things are supposed to be invoked, just as a mathematician knows in which order to invoke transformation functions for a desired result. There's no magic. – haylem Jun 20 '12 at 19:58
I think I was originally thinking of something simpler problem then the one you are worrying about. Say I give you a module with 10+ functions, how can you determine at a glance if those functions are only sub parts of a main function or if those functions actually implement a clever state machine with 10+ states? – hugomg Jun 20 '12 at 20:06
@missingno: in fact, you even say in your edit There will always be complicated algorithms or long lists of business rules that we still want to correctly be able to deal with. Exactly! These are business rules. Your code merely implements them. There has to be a higher authority on how things are to be ordered, there's not easy way around it, short of labelling them ordering prefixes or annotations (could be an interesting idea though, but as mentioned earlier, that goes more and more towards a state machine or expert system). – haylem Jun 20 '12 at 20:08
@missingno: re: your second comment, most probably the sub-parts would be private, whereas the states would be defined as publicly callable methods. Not necessarily though. Also, if what you want to describe are sequential states, then maybe an enum is what you want. It depends on your language and we're trying to be agnostic here, but for instance Java provides an extremely powerful enum construct since Java 5, making it very easy construct state machines, or simple classes with different values representing different stages in their lifecycle. – haylem Jun 20 '12 at 20:10

Just because your functions can call each other a variety of ways doesn't mean they do. Your program's complexity isn't suddenly quadratic because you've named the blocks of code you ran before...

All of the other advice is good, but missing a key issue: you have a function doing a lot of stuff. Breaking that function into a series of function calls doesn't make it do less stuff, it just pushes the stuff around. Having a function that does too much is a sign that you have poorly abstracted your problem. Fix your abstraction so that the functionality is a better mapping to what you need to do and the function will naturally be less complex.

-
+1 for saying "rethink your software" – Timothy Baldridge Jun 20 '12 at 20:57

More dynamic languages like javascript and python have the ability to declare functions inside of functions

function a(param1, param2) {
function helperFunc1() {
return param1 % param2 * param2
}
function helperFunc2(param3) {
return (param1 - param2) * param3
}
return helperFunc1() + helperFunc2(2) - helperFunc2(1.5)
}

These internal functions should have access to all vars inside the main function while hiding them from the outside, like a class would.

-
I don't think this is too different from the version using blocks to separate the subparts (as far as scoping is concerned at least). You gain the advantage I mentioned of passing arguments to the subparts and you avoid the rest of the code from seeing the subparts but there is still the complexity of the subparts being able to call each other that is not present in the undecomposed version. – hugomg Jun 20 '12 at 19:53
And you might lose a lot in testability. – haylem Jun 20 '12 at 19:59
discuessed in detail here: programmers.stackexchange.com/questions/137495/… – user281377 Jun 20 '12 at 20:09

I would suggest moving to a more functional programming style. The fact that you are talking about "calling in the wrong order" and showing functions that don't return data, this means that you are working with very impure functions.

The better approach is to work with pure functions:

1) Reject global state variables. They are bad. Enough said.

2) Every function should take arguments and return something.

3) Given the same arguments functions should output the same data.

When you follow these goals you will find that your code becomes much cleaner, functions can call other functions without fear of side-effects because functions are not allowed to mess with global state.

Now, in a web/gui app, you will have to modify global state at some point, in that case you can break the above rules in small fringe cases.

At this point, decomposing functions is extremely simple. You simply start taking logical parts in the code and breaking them into functions. When you find duplicated code, merge the functions into a single function. If you find yourself writing the same code over and over, then find a way to write that as a single function.

For instance, in a pseudo language, a for loop could be re-written as a map. So instead of this:

arr = [1, 2, 3, 4, 5]

for (x = 0; x < arr.length; x ++)
{
arr[x] += 1
}

You could write a map function once:

function map(func, arr)
{
out = []
for (x = 0; x < arr.length; x ++)
{
}
return out;
}

And in the future, instead of a for loop, you simply write:

function inc (x) { return x + 1; }

map(inc, [1 2 3 4])

So instead of a single-use-case imperative for-loop. Now you have 2 functions that you can reuse in thousands of places in your program.

Resources:

http://skilldrick.co.uk/2011/07/easy-functional-programming-in-javascript-with-underscore-js-part-1/

http://www.ibm.com/developerworks/library/wa-javascript/index.html

EDIT:

Update to address the OP's edits:

In your example, I suggest using maps in a situation like this. So in your example:

function foo(arg)
{
data = {"p1": func1(arg),
"p2": func2(arg),
"pN": funcN(arg)}
return assemble(data);
}

So in general, try to start with a empty hash map, and build out the structure by applying small functions to it until you build out your entire data set.

Or in a much cleaner way:

function reduce(fn, data, init)
{
for (x = 0; x < data.length; x ++ 1)
init = fn(init, data[x])
return init
}

Then simply:

function foo(arg)
{
fns = [p1, p2, pN]
data = reduce((l, f) -> f(l), fns, {})
reduce assembleData(data)
}

Using reduce you can also implement business rules with ease. Focus around data, and functional programming and you'll start to see how simple these things can really be.

Every day I spend close to 10 hours programming in both C# and Clojure, so I've seen both extremes and the functional approach simplifies things dramatically.

Perhaps a full real-world example would be in order?

-
Well, even though global state would make things more complicated this problem also applies to functional languages if you have to do a lot of stuff. I guess going with Haskell style "where" clauses is the way to go. – hugomg Jun 20 '12 at 19:43
Every function should take something and return something, but many languages do not have keywords to separate subroutines from functions, so we use the word function for subroutines too. The proper use of subroutines can make a function every clear so I wouldn't abolish subroutines all together. – Peter Smith Jun 20 '12 at 19:53
Wow! You've discovered modern procedural programming. At least use the built-in functions, makes a better point. – Paul Jun 20 '12 at 20:24
I would also say that in Clojure, once I hit about 10 lines of code in one function I think "I'm doing something wrong...I should break part of this into a separate function" – Timothy Baldridge Jun 20 '12 at 20:25

I would suggest that the OP look at refactoring the problem and solution before starting to code. If something is so complex that you start to worry about these issues, then you really need to jump outside of the box and look at the problem from a different angle.

Just for fun, Maybe you can create a DSL to solve your problem ^^

-