Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Comparing software engineering with civil engineering, I was surprised to observe a different way of thinking: any civil engineer knows that if you want to build a small hut in the garden you can just get the materials and go build it whereas if you want to build a 10-storey house (or, e.g., something like this) you need to do quite some maths to be sure that it won't fall apart.

In contrast, speaking with some programmers or reading blogs or forums I often find a wide-spread opinion that can be formulated more or less as follows: theory and formal methods are for mathematicians / scientists while programming is more about getting things done.

What is normally implied here is that programming is something very practical and that even though formal methods, mathematics, algorithm theory, clean / coherent programming languages, etc, may be interesting topics, they are often not needed if all one wants is to get things done.

According to my experience, I would say that while you do not need much theory to put together a 100-line script (the hut), in order to develop a complex application (the 10-storey building) you need a structured design, well-defined methods, a good programming language, good text books where you can look up algorithms, etc.

So IMO (the right amount of) theory is one of the tools for getting things done.

My question is why do some programmers think that there is a contrast between theory (formal methods) and practice (getting things done)?

Is software engineering (building software) perceived by many as easy compared to, say, civil engineering (building houses)?

Or are these two disciplines really different (apart from mission-critical software, software failure is much more acceptable than building failure)?


Thanks for all the answers and the interest in this topic. I would kindly ask you not to post any new answers if you do not have to add some observation that has not been covered by the existing answers yet. So please read all answers carefully before posting new ones.

I try to summarize, what I have understood from the answers so far.

  1. In contrast to software engineering, in civil engineering it is much clearer what amount of theory (modelling, design) is needed for a certain task.
  2. This is partly due to the fact that civil engineering is as old as mankind while software engineering has been around for a few decades only.
  3. Another reason is the fact that software is a more volatile kind of artefact, with more flexible requirements (it may be allowed to crash), different marketing strategies (good design can be sacrificed in order to get it on the market quickly), etc.

As a consequence, it is much more difficult to determine what the right amount of design / theory is appropriate in software engineering (too little -> messy code, too much -> I can never get finished) because there is no general rule and only (a lot of) experience can help.

So if I interpret your answers correctly, this uncertainty about how much theory is really needed contributes to the mixed love / hate feelings some programmers have towards theory.

share|improve this question

closed as not constructive by Steven A. Lowe, S.Robins, NickC, Walter, gnat Jun 2 '12 at 14:37

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

no, 90% of programmers are ;) – jk. Jun 1 '12 at 15:25
Well, in software you could start with building the roof and then work your way down to the foundation, while the finished parts are floating in the air. If something is not fitting, then you can use duct tape to make it fit. Try this when building a skyscraper. ;) – Secure Jun 1 '12 at 15:28
In theory there is no difference between theory and practice, but in practice there is. – Joris Timmermans Jun 1 '12 at 15:32
A good book to look up alogrithms? Most of software is just simple CRUD without anything close to what is included in any algorithm course or book. – Gilles Jun 1 '12 at 15:49
Theory is about modern languages and algorithms. Practice is arriving at work on your first day and being given the task to add a minor feature to Point of Sale software running on a cash register that uses software that was hand-converted from BASIC to K&R C by people who didn't know C, using a buggy compiler from a vendor who went bankrupt and being expected to have the feature working by Friday at the latest. – Steven Burnap Jun 1 '12 at 16:44

22 Answers 22

up vote 59 down vote accepted

I think the main difference is that with civil engineering, real world physics act as a constant, powerful reality check that keeps theory sane and also limits bad practices, whereas in software engineering there is no equally strong force to keep impractical ivory tower concepts as well as shoddy workmanship in check.

Many programmers have had bad experiences with runaway theory becoming an active impediment to getting things done (e.g. "executable UML", super-bureaucratic development processes). Conversely, dirty hacks and patches can get you pretty damn far, albeit slowly in the end. And as you observe in your last paragraph: failures are usually not as final and thus not as problematic.

share|improve this answer
I agree with you that in software engineering it is important to have the right amount of formalism. Too much means you can never get started and maybe when you find out you have made a mistake it is too late. Too little means you can make a mess. I think you have a very strong point saying that productivity and quality is much more difficult to measure in software engineering than in civil engineering. – Giorgio Jun 1 '12 at 15:39
"... whereas in software engineering there is no equally strong force to keep impractical ..." I think you mean there is no longer such a force. Back in the day, the limitations posed by weaker processors, less memory and little/no storage acted as such a force. – blesh Jun 1 '12 at 19:30
@blesh: I don't think so. Tight hardware limits constrain good and bad engineering equally. – Michael Borgwardt Jun 1 '12 at 20:03
Your examples are not the theory of programming. The constraints on software have to do with the technologies used and the mathematical capacity of the writers. – Paul Nathan Jun 1 '12 at 21:41
There is definitely something "theoretical" about UML... ...its utility! – ObscureRobot Jun 1 '12 at 23:43

Software engineering and civil engineering have little in common. Civil engineering efforts are limited by the physical properties of their materials and the environment. Civil engineers spend a lot of time learning about soil properties and material characteristics. Software development is physically limited only by the speed of the processors and the available storage. Both of these are easy to understand, and we don't spend much time studying them. The major limitation to software development is the human intellect. And no two developers are alike. Is this code maintainable? By whom? A three-line implementation of quicksort in Haskell may be obviously correct to some, but incomprehensible to others. A team of two may complete an application in a month, while another team of ten struggles to deliver in a year.

Software development is all design, the artifacts being designed are orders of magnitude more complex than any manufactured article, and each one is unique.

share|improve this answer
I agree with your observations that the human factor is much stronger in software, but still, I think that trying to analyse a problem or structure your solution is a general attitude / tool. My question is why some programmers think it is not an useful approach (or even a waste of time). You mentioned Haskell: I made the effort to learn some Haskell even though I did not use it in any project because I thought it would help me to write better code (even in C++ or Java). Even if I cannot measure this, my feeling is that I have become more productive: I get more things done. – Giorgio Jun 1 '12 at 15:54
A three-line Haskell quicksort? Hmm... is it even possible to implement Quicksort in a language where everything is immutable by design? – Mason Wheeler Jun 1 '12 at 17:08
@MasonWheeler: good point. quicksort and Haskell were pretty random choices. I don't know Haskell. – kevin cline Jun 1 '12 at 17:23
@MasonWheeler First result from Google: Quicksort in Haskell. – chrisaycock Jun 1 '12 at 18:47
@Mason: the runtime is still O(n log n). It also requires O(n log n) memory, unlike an in-place quicksort, which requires only O(log n) additional memory for the recursion. – kevin cline Jun 1 '12 at 22:11

As a more-or-less honest-to-gosh mechanical engineer (with some civil) turned programmer, then CS (AI) PhD, then teacher, then programmer again (excuse me, Software Engineer), I've got a rant on this general subject.

In traditional engineering

  • you gotta know your math and science because everything you do is based on it
  • the "heroes" in the field are people who invent new things, discover new ideas, solve problems considered unsolvable

There is a "physics" that applies to software - information theory, but software engineers get little exposure to it, and certainly nothing applied. The most theory they get is computability and big-O.

Also I'm continually amazed at people who think knowing programming is enough, and they don't need to understand the subject matter of what their programs are about.

What's more, inventiveness is not encouraged. It is discouraged, in favor of least-common-denominator group-think methods, disguised as "readability". (Imagine if aeronautical or nuclear engineers were encouraged not to do anything that might be hard to understand for their junior peers.)

The things they do learn, like how to program web apps, are of great value. So is the skill of a plumber or electrician, but it is not engineering.

share|improve this answer
Physics can tell you whether or not some structure will collapse under it's own load. CS tells you that you can't tell whether a given program will halt given a certain input. IMO formal methods scale much better in civil or mechanical engineering than in software mostly because the systems are less complex and less dynamic... – Guy Sirton Jun 1 '12 at 19:08
@GuySirton "CS tells you that you can't tell whether a given program will halt given a certain input." if that's all you think CS does, I think you might not know as much about CS as you think you do ... – gregghz Jun 1 '12 at 22:52
Guy, it is incredibly unlikely that you've ever used software materials that no one has used before. McCarthy did, and Turing did, but really, software engineering is not that freaking amazing. If it was okay that the building sank to the bottom of the ocean because you could just reboot it, that would be like software engineering. – philosodad Jun 2 '12 at 0:03
I'd give you a +1 except for the crack about readability. Maintainability is 80% of the cost of software so readability is not a small matter. In addition, when that aeronautical or nuclear engineer is making something that will be manufactured having other people understand it is important. The military, government or even large institutions aren't happy with a magic invention that can't be replicated or understood by anyone other than the inventor. – Thomas Jun 2 '12 at 6:23
@Thomas - The assertion that viable solutions are often discarded at the alter of "readability," by subpar minds, doesn't necessarily mean the solutions aren't as legible as they ought to be. I've seen it happen. Hell, I've caught myself doing it. – Erik Reppen Jun 2 '12 at 6:40

If I cut a corner on most software, and do something that's not the best design, but will get the job done, nobody is going to die. It's the same reason a hut in the garden doesn't need the same standards as a 10 story building. However, I can build a very large app like facebook, and if it screws up and loses some data, or whatever, it's not really that big of a deal. It's also simpler to fix the foundation of a large app after the fact, than it is to replace the foundation of a 10 story building. It all comes down to context, and calculating risk.

I can also, safely and simply keep adding to an app. You can't easily toss in a new third floor of a 10 story building (making it 11). I can toss in a new feature to a large app every day if I want to.

Now, good design makes all this easier in programming. But it's not impossible with poor design, and the risks, are...buggy software. Not usually death.

share|improve this answer
Well you hope they won't die...depends on your software ;) – Rig Jun 1 '12 at 15:35
@Rig, that's why I said 'most' and 'usually'. Some software is much more critical. – CaffGeek Jun 1 '12 at 15:44
I think this is increasingly becoming a very bad point of view, sure most software doesn't have any safety implications but there is money and privacy involved in a lot of software, getting these wrong could also land you in court – jk. Jun 2 '12 at 8:48

Bear with me on this one. I have a point.

I had a professor tell me once that procrastinating leads to more procrastinating, even though most people after a night of harried paper writing/cramming/programming say to themselves, "I'll never do that again. Next time, I'll start early and get done early." In my experience as the consummate procrastinator, I've found this to be true, and here's the professor's explanation why: no matter how unpleasant the experience of procrastinating is, in most cases, you get done having achieved relative success. This is high risk/high reward behavior. After a while, you forget about all of the unpleasantness, and only remember the reward. Thus, the next temptation to procrastinate is all the more enticing, because you succeeded the last time.

I think an analogy can be made here to the "get things done" programming technique we all too often see. A programmer or team of programmers, maybe out of ignorance, laziness, or perhaps a genuine time constraint, takes the "get things done" approach to programming, throwing all of your theory and maths and good practices out the window. And you know what? They get things done. It's not elegant, pretty, or maintainable, but it gets the job done. Maybe a non-technical superior who doesn't know a semicolon from a semaphore gives them some high praise for "getting things done". Thus, the next time the programmer is tempted to take this slack approach to programming, it's all the easier, because hey, it worked last time didn't it? It's the "easy" way out, unless your the poor, unfortunate soul who inherits such an application years later and has to maintain it.

I've been that poor, unfortunate soul, and so have many of you probably. I implore you all. Don't take the easy way out! :)

share|improve this answer
If you have to get it done once and forget about it it is fine. But if you have to extend and maintain it afterwards you are looking for trouble. You need to have a feeling for how much theory: too much means you'll never get it done, too little means you're going to do it 10 times before it is really done. My 2 cents. – Giorgio Jun 1 '12 at 15:29
But sometimes you need to get your software out the door NOW. You need to beat a competitor to market. Or you have a legal requirement to provide some information. Or you just need to get cash-flow so you will still exist when the mess you made in your "get it done" approach is a problem...which is sometimes a good problem. Because if you didn't have it, you didn't release on time, and your company is dead before it starts. – CaffGeek Jun 1 '12 at 15:48
@Chad - I agree with you. It is a balance. All of the things that you mention would fall under "a genuine time constraint" as reasons for get-it-done programming, and in the short term, it's okay and even advantageous as you point to do so. – FishBasketGordo Jun 1 '12 at 16:18
@FBG: Brilliantly said. – Kuba Ober Jun 1 '12 at 19:01
@Chad, good point. Martin Fowler makes a similar point at Sometimes it's a worthwhile tradeoff. – John M Gant Jun 1 '12 at 20:59

Your premise is flawed. The main reason civil engineers use engineering when designing large buildings, bridges, tunnels, etc. is to ensure that they are using minimum amount of material (concrete, structural steel, etc) which satisfies the required safety standards. It is entirely possible to build a tall building without much in the way of mathematics (e.g. the pyramids of the ancient Egyptian and Mayan civilizations) if costs of material and of labour is no object, but once built, there is usually no point in modifying them to make them use material more efficiently.

There is a somewhat different dynamic in designing large software systems. If anything, they are usually under-designed, but this is because the design can be changed dynamically as work proceeds, which simply cannot be done so easily with civil engineering projects.

The common factor is cost. Design on a traditional civil engineering project reduces costs (both actual, in terms of material, and potential in terms of liability), whereas there comes a point in software development where the cost of design increases beyond the value returned.

share|improve this answer
"there comes a point in software development where the cost of design increases beyond the value returned.": I have explicitly written "the right amount of theory". I know that over engineering does not increase productivity. – Giorgio Jun 1 '12 at 19:57
There are IMO almost zero projects that are designed up front that actually follow their design. Civil engineering is (generally?) building the same thing over and over (a road, a damn, a tunnel, a building, a bridge). The techniques are well known. This is not true in software. Because it can be changed easily and because people don't know what they want or what works until they try it serious up front design is a waste of time. We build, test and iterate. Something that's not possible with Civil Engineering as pointed out above. The 2 disciplines are not comparable. – gman Jun 1 '12 at 22:14
Sorry, I have to point out the typo: I do not think civil engineers build a damn. ;-) – Giorgio Jun 1 '12 at 22:31
I imagine in the future, when us software engineers, build cool civil engineering simulation software, civil engineers can do away with all this math stuff. Just build a 10km tall virtual skyscraper. If it doesn't collapse under its own weight in the first 100 virtual years and can withstand a cat 5 virtual hurricane, then use the special skyscraper 3D printer to build it. – emory Jun 1 '12 at 23:20
Civil engineers use engineering because otherwise they can't build the desired structure. Singling out "minimum amount of material" is silly. That is one of many concerns. (Will it withstand an earthquake? Will it fit on the lot? Etc. etc.) – Rex Kerr Jun 2 '12 at 5:07

I would also point out, in addition to several other excellent responses that mankind has been doing the equivalent of "civil engineering" since easily the time of the Egyptians so we've had a lot of time to perfect the general theory of how things should be done. We've been building software for somewhere around 70 years or so (depending on what you consider the first "software"); I mean that we've not had the same amount of time to develop the same sort of body of experience.

share|improve this answer

An architect's/civil engineer's blueprints are virtually never identical to the "as built" plans. Something ALWAYS changes. Why? Because there are, and will always be, "unknown unknowns". There are things that you know and so can plan for, things that you know are unknown and so you can research and estimate, and then there are things that you don't know you don't know; "surprises". You aim to eliminate these in most systems by learning all you can, but all it takes is one little building code violation (which may be based on a rule that didn't exist 2 years ago when your building was being conceptualized) and your all-encompassing master plan has to change, sometimes quite drastically.

Software is very much like this; there's always an unknown unknown. However, unlike civil or structural engineering, software development is inherently much more tolerant of change based on the problems the unknown unknowns create. If you're building a 10-story building and you overestimated the load-bearing capacity of the foundation you put in your design, you either can't build the building to 10 stories or you have to tear out a significant amount of work to get back down to the foundation and reinforce or rebuild it. However, in software, if you underestimated the demands on a particular tier of the overall solution structure, there are many options for fixing that tier that don't involve invalidating all the other work. You can replace a single DB server with a more powerful one, or a replication/failover cluster, or a load-balancing/distributed cluster. Same for the webserver. If you coded an algorithm that is inefficient but simple based on faulty assumptions of input size, you can almost always simply remove and rewrite the implementation in a relatively surgical manner, without affecting other code that has knowledge of the algorithm (calls and passes input to it, or expects an output from it).

This relative ease of change allows a software engineer to code based on what he knows without worrying unduly about what he doesn't know. This allows for laxer application of theory and up-front conceptual design; you dive in and get it done, and along the way you find the things you coded that need to change, and change them. You must still know the concepts and theory, because when a problem is uncovered it's those things that will help you identify the cause and create a solution. But, you're allowed to make a snap decision without succumbing to "analysis paralysis", because if it turns out you made the wrong decision based on something you didn't know or didn't factor in to your "calculations", the mistake is easier to correct.

share|improve this answer
There's also a lot more unknown unknowns in software development - you might start out building a skyscraper, but when the client looks at it they tell you "actually I wanted a ten-story high Rubix cube". – Tacroy Jun 1 '12 at 18:08
@Tacroy: Interestingly enough, a civil engineer would probably consider this a bad client who is wasting your time and resources, a software engineer will try to develop a new methodology to satisfy him. :-) – Giorgio Jun 1 '12 at 19:58
@Giorgio, or bill accordingly... – CaffGeek Jun 1 '12 at 21:25

The difference is primarily because of the known requirements:

  • On the theory side, everything is defined up-front, so you can know exactly what you need before you start.
  • In practice, they're often not all there, or you discover something midway through the implementation that causes you to have to redesign something. So it's much better to jump with at least rudimentary designs, so you can discover these problems early on.

Additionally, when talking about "theory", it usually means the theory side of computer science, rather than software engineering. This is the part of computer science that's largely about finding better and more efficient algorithms, proving whether something is or is not possible (P and NP, for example), and so on. While it's good to have these in the back of your mind, they don't come up in software development very often.

We use libraries for that kind of thing as much as possible.

share|improve this answer
+1 for "when talking about 'theory', it usually means the theory side of computer science". – Joshua Drake Jun 1 '12 at 18:47

There are actually quite a few levels of software engineering depending upon what the software you are building is doing.

NASA needs software to control manned shuttles in space so naturally the level of engineering process is much stricter than that of building a website to show pictures of rockets.

One of my co-workers that worked for NASA previously described their software engineering process as writing hundreds of pages of justification and hundreds of hours of meetings to justify writing a single line of code!

Don't misunderstand me because I'm not trying to sound disrespectful when I say this but even after all that cost of time, resources, and billions of dollars the space shuttle still blew up.

Even civil engineers know that no matter how much theory they put into a design something will eventually break it so they also need to develop contingency plans.

When building software the cost of it crashing rarely causes loss of life so it is much easier to quickly throw stuff out there and test drive it. Let's agree that getting things done quickly results in weak code. Even if this is always the case, seeing software in action is the best way for a developer to see where it is weak and needs to be made stronger versus where it is weak and still many times stronger than it needs to be to keep up with the load.

To sum up, Premature optimization is the root of all evil or as my boss would always say Shipping is a feature!

share|improve this answer
+1 for "Shipping is a feature"! Once I heard a similar sentence: "Perfection does not exist. This piece of software has the advantage that it exists." Of course it is a bit of a joke. Regarding mission-critical software: an uncaught exception can cause a rocket to crash. – Giorgio Jun 1 '12 at 21:28
this software has the advantage that it exists ... i hadn't heard that one yet but it is going into my list of great software quotes. i like it – Albert Lang Jun 1 '12 at 21:46
@Giorgio: JSF and MISRA C is written so that there are no exceptions. Exceptions and rockets don't mix. – Coder Jun 2 '12 at 4:06

So my question is why do some programmers think that there is a contrast between theory (formal methods) and practice (getting things done)?

Building software is unlike building a bridge. In software, there are many objects to be built which may or may not be defined at the onset. Standards exist to increase ease of maintenance and developer collaboration, not to adhere to arbitrary mathmatical formulas or other such ideals. For example, when selecting behavior based upon a variable sometimes it makes sense to use a switch, other times a factory pattern. It depends on the ease of maintenance and identified pain points such as performance problems.

Another example can be made with data manipulation. It often makes sense to use delegates in the context of .NET. It is not so easy in Java because it does not have the framework support for functional programming style that .NET has. In other words, in the general case it simply is not possible to do X in situation Y. This is due to the fact that X and Y depend on N number of variable factors.

Is software engineering (building software) perceived by many as easy compared to, say, civil engineering (building houses)?

I'm not sure "easy" is the correct term. A lack of tangible evidence can lead to the perception that no work is being done. Or, similarly, that existing work is easily changed.

Or are these two disciplines really different (apart from mission-critical software, software failure is much more acceptable than building failure)?

Traditional Engineering and Software Engineering are very different for the reasons I already stated.

share|improve this answer

Lots of good answers here, but I think the comparison between Computer Science and Civil Engineering is flawed.

Strictly speaking, what professional software developers do is more like Software Engineering than Computer Science. A better analogy is that Computer Science is the Physics for Software Engineering. Similarly, Civil Engieering is a collection of simplifications and approximations of Physics for practically building stuff.

I imagine that Civil Engineers rarely have to take into account general relativity when going about their job. Much of Civil Engineering can be safely built in Newtonian Mechanics. Similarly, Software Engineering can be accomplished very successfully with a roughly approximate understanding of theoretical computer science.

The big difference is that bridges, skyscrapers, and other products of Civil Engineering are reasonably well understood things. Software engineers are often building novel constructs, or using novel methods to build well understood things. Software Engineering is FAR less mature than Civil Engineering, and that will likely continue to be true for the foreseeable future.

TL;DR: Theory and practice are different in Software Engineering just like they are everywhere else. The proper analogy is Software Engineering : Civil Engineering :: Computer Science : Physics. But in practice, it is a little more complex than that :)

share|improve this answer
"I imagine that Civil Engineers rarely have to take into account general relativity when going about their job. Much of Civil Engineering can be safely built in Newtonian Mechanics.": As far as I know they have to use quite a lot of calculus (integrals and stuff like that). This is not quantum mechanics but IMO it is definitely non-trivial. – Giorgio Jun 1 '12 at 21:00
Sure, but you don't need to derive a wave equation for every component of your bridge and then explain how they interact. – ObscureRobot Jun 1 '12 at 21:13
You are right. However, my point is not how much theory is used in civil engineering wrt to software engineering. Rather, civil engineers know they have to use their formulas and do calculations on how to construct some building. In software engineering I have the impression there is more improvisation and sometimes if you want to sit back and analyse a problem (just to get it right, not to write a PhD thesis about it) you can be frowned upon: we want to get it finished, not to make it perfect. But IMO some theory (not too much) is exactly what can help to get it finished more quickly! – Giorgio Jun 1 '12 at 21:21
You need to find a balance point appropriate for your project. Junior developers are typically more gung-ho about throwing crap together to see what will stick. If they came from a very theoretical background, they may even have more crazy and excessively complex ideas. Managing junior developers effectively often involves helping them take a step back and analyzing their work. On the other hand, senior developers can be too focused on long term design issues to the point where they have trouble focusing on immediate needs. – ObscureRobot Jun 1 '12 at 21:43
Wow, sorry this is off topic but without reading your answer I ended it exactly the same—with a TL;DR and then literally the exact same analogy. SAT format. I edited it out of my answer so it doesn't look like I'm copying you, but it's still freaking me out. Maybe programmers think too much alike. – Jarsen Jun 1 '12 at 23:15

Your perception may be wrong here, or it includes many resources from people who haven't written sufficiently complex software.

Your experience is inline with what most people I know of (who have designed and written sufficiently complex software) would say.

That said, when it comes to most programmers, when the task of writing something gets to them the design ("the maths" as you put it) has already been done by the architect/lead/etc. before the task of writing it gets to them. So it may appear that way from the front-line level.

share|improve this answer
"the maths ... has already been done": not only, consider all the library functions, frameworks, DBMS's, protocols, and tons of other heavy stuff that we can just use in our code by calling a function with some parameters. As a programmer, I sometimes feel more like the worker who walks on the scaffold than as the engineer who has designed the building. – Giorgio Jun 1 '12 at 22:05

I think the reason for this contrast is that the life cycle of a software project and hardware or architecture project is different. Most software evolves gradually, it is not planned from the beginning to the end. Software developers can apply an iterative approach to development: plan, implement and listen to feedback. If the feedback is positive, carry on, it not - take a step back and reconsider your strategy. That's why software developers have things like agile development, minimum viable product and so on.

Civil engineers don't have such luxury. For them, once something is planned, you cannot change it as easily, as with software, because the cost of such changes can be dire. For software development, on the other hand, it's doesn't cost that much, and this can be used for their advantage.

But not every branch of software development can afford such approach. Making software, for instance, for aviation or medical services require very careful planning and a lot of prior calculations.

share|improve this answer

It seems the same to me. You build a large building out of standard blocks, standard strength concrete, standard steel. You build a big app out of standard libraries.

You don't try and mathematically formally prove a large app correct in the same way you don't try and write the wavefunction for a 100storey building

share|improve this answer
So what's the software equivalent of a finite element analysis of the 100 storey building? How many tall buildings have bugs in therm/crash? :-) – Guy Sirton Jun 1 '12 at 19:00
@GuySirton - you can only analyse a large building at a very coarse level, less detail than you would unit test a typical app. A lot of large buildings have faults, windows fall out, walkways collapse, they create wind tunnel effects. Or in the case of one curved highly reflective hotel in Vegas you create a death ray in the pool! – Martin Beckett Jun 1 '12 at 19:08
You can go pretty fine grained in FEA and predict behavior to a very high degree of accuracy. People still make mistakes. IMO it's simply impossible to make similar predictions on a complex piece of software. The faults you mention are a miniscule fraction of the total number of buildings. Defect rates in software must be two orders of magnitude higher. That said, it's obviously a continuum between where formal methods are useful and where they are useless. – Guy Sirton Jun 1 '12 at 19:16
@GuySirton - I think the difficulty is that you rely on other things. Nasa can test flight avionics to a very detailed level (although still not prove it correct) because they also create the OS and hardware. Writing on a general OS with toolkits and libs is like building a bridge where you aren't allowed to know the details of the steel or concrete. – Martin Beckett Jun 1 '12 at 19:35
That's certainly a factor. Limiting complexity and throwing more time/money at it also help... – Guy Sirton Jun 1 '12 at 19:42

I was a mechanical and manufacturing engineer before I discovered some 20 years ago that my aptitudes lay in software. I sympathise with many of the elements which you have laid out.

I suspect that the true nature of the problem is about how we get things done. We've now got ten or so years of agile development under our collective belts, and the message is clear. Don't progress by layers; progress by features. Sure - there will be projects when you need to progress by layers (e.g. build your network stack before your web server) but for the vast majority of real-world projects, we have learned the lesson that delivering working features, one or a few at a time, is much more effective building huge untested theories, and then trying to implement them.

So let's take your hut example (I usually talk of making a bridge by throwing a log across a stream vs. a kilometer long suspension bridge... whatever!), and bring it to the world of software engineering. The main difference I see is that in software, most of the work is of a scale that it doesn't need big up-front modelling to succeed. The beginner's mistake is often to assume that things need more of this than they actually do, and for most of us, having made that mistake a few times, we're chary of making it again too often.

No argument - there are projects that need to begin with a committee of 17 software architects. In truth they are about as rare as 20 carat diamonds.

share|improve this answer

I think the analogy is flawed. As far as I'm aware, civil engineering does not have the same sort of theoretical basis as computer science; computer science was born from theoretical mathematics—like Turing machines. Civil engineering is about creating structures that resist mother nature and maybe even look beautiful. Again, I really don't know much about civil engineering, but I don't think there are civil engineer equivalents of P vs NP, the traveling salesman, and other fun things to bash your brains against. And there is definitely a place for our computer science theory—if someone solves the traveling salesman or the halting problem we are in for a lot of awesome new advances. But for a software engineer, whose business is to architect software, such problems are really only fun and games.

Now, I also think it depends on what you mean by "theory." Are we talking design patterns, or pumping lemma? Because having a good solid understanding of design patterns is absolutely critical to being a good software engineer. However, when architecting a large software system, theorizing about P/NP problems isn't useful. In that sense, I believe there is a very stark contrast between software engineering and theoretical computer science.

Or does theory refer to algorithms? You don't spend a whole lot of timing writing algorithms you learned in your algorithms class. Why? Because you typically only need them in particular cases (and then you look it up and research it), or you use a library already written for you. No need to write another bayesian classifier. Abstraction is an important principle in computer science. I think software engineers tend to not learn how an algorithm works until they need to.

Another reason is there are currently several popular "get it done" software development methods that are effective. For example, in agile development, you do not architect out an entire system beforehand. The reason for this is because you don't know exactly what you're building yet—you want what you're making to be flexible and adapt to new information and requirements. Designing it all out from the get go and then building just that does not always produce the best software. However, it is not the solution for everything. For example, say you're designing some distributed-computing-cluster-crazy-new thing. You can't do some napkin sketches and start your SCRUM.

TL;DR. I think there is some equivocation around the word "theory." Traditionally, theory refers to the theoretical mathematical aspects of computer science. Unless you are researching newer ways of computing, for the most part theoretical computer science plays no part in the day to day life of a software engineer. Software engineers care about design patterns and system architecture. Specific implementation details of certain algorithms are not important. Often times with less complicated ideas it is appropriate to not design a lot and just start coding. And I think this is where the idea comes from that programmers don't like theory.

share|improve this answer
I see some similarities between our answers, but your ideas are obviously original and there are some differences. I don't agree that understanding P/NP isn't useful. You don't have to study Complexity Theory deeply, but a working software engineer should be able to estimate the O(n) of any given piece of code and say intelligent things about the cost of alternative solutions. A point that you almost made, but didn't, is that theory is often encapsulated in libraries. That is a good one to consider. – ObscureRobot Jun 1 '12 at 23:42
"If someone solves ... the halting problem we are in for a lot of awesome new advances.": Well, unfortunately theory has proven that this is unsolvable (there exists no program that can decide it) so I do not think that any research effort is being spent trying to solve the halting problem. – Giorgio Jun 2 '12 at 9:19
Turing Machines can't "However, not all machines conceivable to human imagination are subject to the Church–Turing thesis ... It is an open question whether there can be actual deterministic physical processes that, in the long run, elude simulation by a Turing machine, and in particular whether any such hypothetical process could usefully be harnessed in the form of a calculating machine (a hypercomputer) that could solve the halting problem ... It is also an open question whether any such unknown physical processes are involved in the working of the human brain..." -Halting Problem,Wikipedia – Jarsen Jun 6 '12 at 17:54
So, as far as I'm aware, and correct me if I'm wrong, I think we still have a lot of discovery to do about computation. As has been mentioned multiple times in this thread, computer science is still very young; there could be much beyond Turning Machines and the Von Neumann architecture. – Jarsen Jun 6 '12 at 18:09
@Jarsen: It is true that computer science is very young, but any computer that has been built up to now can only do Turing-computable stuff. As far as I know (very little indeed) even quantum computers cannot do more (they could solve certain problems more quickly, but they wouldn't be able to solve more problems). So, yes, who knows what can be invented, but any computing formalism that has been imagined during the last 70 years cannot do more than a Turing machine. – Giorgio Jun 11 '12 at 22:19

The gap between theory and practice is too large at the moment. When doing theory, you are given three axioms and it is subsequently shown that a one-line theorem has a thousand page proof, or no proof at all. In software engineering, you're given inconsistent APIs of thousands of functions which give you a myriad of (bad) manners of implementing an underspecified feature.

Real software engineering would drive most of those in the formal field mad, and real mathematical software development drives those in engineering mad. Both fields require people of different aptitudes, and I don't think the aptitudes often overlap.

share|improve this answer

Formal theory assumes that you can accurately plan everything in advance like a manufactured product, that software will exist indefinitely within the same environment, and that solving a general abstract problem is always the goal. It assumes a 4D software-as-a-product lifecycle: design, develop, deploy, done. Formal theory is about solving the problem of software design using analysis, abstraction, generalization, and predicting future change. This is good if you have a well-defined problem in a straightforward domain that's easily analyzable, predictable, and fairly static.

Practical programming is about solving the right problem (not that of software design) in the right way right now, so that your coworkers can do their jobs better/faster/at all, or so that revenues can flow in to the company. Much software is not like a product, never 'done', but more like a living thing, that begins highly-specialized for one ecological niche, and may have a widely-variable lifespan during which it needs to solve new, unforseen problems in a wide variety of ever-changing environments. In the business world, with politics and legalities and competition and ever-changing organizations, structures, and trends, the requirements are often ambiguous, convoluted with all kinds of special cases, poorly-defined, and subject to rapid unexpected change. They're not analyzable, predictable, or static, and often not logical or reasonable. The software is as likely to be irrelevant in 2 weeks as to be still in use in 20 years. It comes into the world not knowing much or able to do much, and needs to be nurtured, groomed, and trained throughout its lifetime to grow up strong, flexible, and able to adapt to its ever-changing environments and new problems. If you neglect it after birth, it will go feral if it survives long enough, and cause pain and suffering, solving problems with blunt force.

Formal theory doesn't address the needs of much real-world business software. It tricks us into believing that software can be designed and done. That it's a product that can be occassionally fixed, polished, or have things tacked on, but not a living thing that needs to be raised properly with constant care and attention throughout its lifetime. So we end up with really ugly feral legacy code, but formal theory probably wouldn't have helped with that.

That all sounds pretty negative, but in reality I love using formal theory. A beautiful design always brings a smile to my face. However, that's mostly in my hobbyist programming that isn't subject to the vicissitudes of business. At work, I mostly deal with organic code and just hope that I can give it enough attention that it will grow up right, make me proud, and not be obnoxious and rude to others who have to deal with it.

share|improve this answer

The stakes are lower, the job is easier, and management rarely sees the value in good design. System instability, maintainability, and integrity are an "IT" problem--not a "Business" problem. All executives have one thing in common. They are either 95% focused on money, or they report to someone who is.

The rest of the battle is with your fellow programmers. Many of them cannot or will not commit to thinking about a problem before coding begins. Due to the above, a good deal of these people are senior developers, making it even more difficult to get a good design into production.

I have watched project leads waste years adding ad-hoc features and fixes to projects that were rocky to begin with, and then shoot down every attempt to bring order to the chaos with phrases like "too complicated" or "wasting time." It is not pleasant to watch a major project spiral to its inevitable doom because management will not admit they are building their own prison on a daily basis; however, I fear it is an unfortunate reality that many developers have witnessed and--for better or worse--learned from.

I try to find a medium in my work. I write no more code in "tainted" projects than is absolutely necessary, and I take every opportunity to move functionality out of them. "Between projects," I spend time on design and cleanup in the projects I actually have control over.

In the end, it is a big mess of politics and personal integrity that 75% of the world's programmers don't have the stomach for. I can barely stand it, myself.

share|improve this answer

First of all, I love this question. I've written like three 1000 word answers and they were all horribly wrong by the time I got to the end of them.

The problem with attempting to compare the two as analogous, I think, is that programming is a modelling process that can be as abstract or tightly bound to concrete as you want.

Structural engineering theory, on the other hand, is tightly bound to very specific sets of reality-based laws that you have to conform to. You can't just alter the context or the laws. The problem itself is rooted in those laws. In programming, however, sometimes the solution is actually altering the nature of the question or simply placing it in a different context.

Whether the MVC pattern, for instance, is a perfect fit, has a lot to do with that context. A desktop application typically deals in one language and one language only, not counting config files.

The front end of a web app on the other hand, consists primarily of two declarative (non-programming) languages and JavaScript. The one physical thing you can't entirely abstract away is the fact that there is always this http wall to chuck things over between server and browser. Regardless of how you bury it in code, that takes time and asynchronous design.

Obviously you can't use a popular and well-regarded pattern like MVC to handle front end concerns on the web exclusively without altering the way you might handle it in a desktop app context. In fact I would argue, you should be aware of what makes MVC useful but not even try to implement it in a particularly exacting or wholesale manner. The web app paradigm is unique in that all the look-at-me stuff is handled by the user's browser and all the data/model-ish stuff is typically on the server somewhere. But where does that leave the controller? All on the server or all on the front end? Somebody has to own that. Or maybe MVC isn't 100% the best fit for the scenario. Not a bad fit for .NET back end stuff. Not terrible in the context of specific UI widgets. But attempting to apply it to everything for the sake of consistency could be a bad move, IMO.

Building a house solves a problem. Typical programming problems, however, often involve solving problems within problems and sometimes the solution is to redefine the outer problem. Reality isn't especially keen on that idea unfortunately.

share|improve this answer

Glenn Vanderburg presents a great view on the differences between software engineering and more traditional engineering disciplines:

If a civil engineer could test his designs without any costs before building the final thing he would make much less use of theory. If within seconds he could build a bridge a thousand times for free to test when it will break he would do so instead of spending months calculating when it might brake in theory...

In software development thats exactly what you do. Instead of calculating how fast your algorithm is in theory you can just test it and know the answer within seconds.

In fact most software today isn't limited anymore by physical constraints like computing power or memory. The limitation of software is the complexity that amounts in larger and larger systems. Its managing this complexity by keeping the system comprehensible by humans what makes the huge challenge in programming today.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.