Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

After watching National Geographic's MegaStructures series, I was surprised how fast large projects are completed. Once the preliminary work (design, specifications, etc.) is done on paper, the realization itself of huge projects take just a few years or sometimes a few months.

For example, Airbus A380 "formally launched on Dec. 19, 2000", and "in the Early March, 2005", the aircraft was already tested. The same goes for huge oil tankers, skyscrapers, etc.

Comparing this to the delays in software industry, I can't help wondering why most IT projects are so slow, or more precisely, why they cannot be as fast and faultless, at the same scale, given enough people?


Projects such as the Airbus A380 present both:

  • Major unforeseen risks: while this is not the first aircraft built, it still pushes the limits if the technology and things which worked well for smaller airliners may not work for the larger one due to physical constraints; in the same way, new technologies are used which were not used yet, because for example they were not available in 1969 when Boeing 747 was done.

  • Risks related to human resources and management in general: people quitting in the middle of the project, inability to reach a person because she's on vacation, ordinary human errors, etc.

With those risks, people still achieve projects like those large airliners in a very short period of time, and despite the delivery delays, those projects are still hugely successful and of a high quality.

When it comes to software development, the projects are hardly as large and complicated as an airliner (both technically and in terms of management), and have slightly less unforeseen risks from the real world.

Still, most IT projects are slow and late, and adding more developers to the project is not a solution (going from a team of ten developer to two thousand will sometimes allow to deliver the project faster, sometimes not, and sometimes will only harm the project and increase the risk of not finishing it at all).

Those which are still delivered may often contain a lot of bugs, requiring consecutive service packs and regular updates (imagine "installing updates" on every Airbus A380 twice per week to patch the bugs in the original product and prevent the aircraft from crashing).

How can such differences be explained? Is it due exclusively to the fact that software development industry is too young to be able to manage thousands of people on a single project in order to deliver large scale, nearly faultless products very fast?

share|improve this question
82  
Interesting question. I'm tempted to say the quality of the average worker in the software industry is less skilled and qualified than, say, civil engineering where every engineer has completed a rigorous and intensive degree and likely gained his charter too. Furthermore, the complexity space of large software (eg. an OS, MS Office) is probably much greater even than an aeroplane. Certainly many more places to fail! And a final important point: most software more or less works, even if was poorly written and highly buggy...certainly the cost of failure is normally much less than an aeroplane! –  Noldorin Jul 29 '12 at 13:58
83  
Find someone who actually works in any of those other industries (but not in PR) and ask them about "large faultless projects". I can virtually guarantee that you'll earn pained laughter. –  Michael Borgwardt Jul 29 '12 at 15:50
84  
The realisation of a software project takes seconds or minutes; it's what happens when you click "compile" in your IDE. On the other hand, programming is design. How long did it take to design the A380? –  Ant Jul 29 '12 at 16:36
29  
That TV program is a hype. They only telecast successful projects. Any channel will make programs for viewers attention. –  pandu Jul 29 '12 at 18:13
48  
'imagine "installing updates" on every Airbus A380 twice per week...' Imagine enemy robots constantly probing the plane for vulnerabilities while untrained pilots push buttons at random. I bet you'd need regular patches. –  Nathan Long Jul 30 '12 at 15:15
show 30 more comments

29 Answers

up vote 190 down vote accepted

Ed Yourdon's Death March touches upon a number of these meta type questions.

In general, the software industry lacks a lot of the following, which gets in the way of large projects.

  • Standardization and work item breakdown.

    • This has certainly gotten better, but the design constructs still aren't there to break out a big system. In some ways, Software can't even agree on what's needed for a given project, much less being able to break things down into components.
    • Aero, Building construction, Auto, etc.. all have very component driven architectures with reasonably tight interfaces to allow fully parallel development. Software still allows too much bleed through in the corresponding areas.
  • A large body of successful, similar projects. The A380 wasn't the first big airplane that Airbus built. There are a lot of large software applications out there, but many of them have suffered dramatically in some aspect or the other and wouldn't come close to being called "successful."

  • A large body of designers and builders who have worked on a number of similar and successful projects. Related to the successful project issue, not having the human talent who has been there, done that makes things very difficult from a repeatability point of view.

  • "Never" building the same thing twice. In many ways, an airplane is like any other airplane. It's got wings, engines, seats, etc.. Large software projects rarely repeat themselves. Each OS kernel is significantly different. Look at the disparity in file systems. And for that matter, how many truly unique OSs are there? The big ones become clones of a base item at some point. AIX, Solaris, HPUX, ... herald back to AT&T System V. Windows has had an incredible amount of drag forward through each iteration. Linux variants generally all go back to the same core that Linus started. I bring it up because the variants tend to propagate faster than the truly unique, proprietary OSs.

  • Really bad project estimation. Since the repeatability factor is so low, it's difficult to project how large it will end up and how long something will take to build. Given that project managers and Management can't put their hands on the code and actually see what is being done, unrealistic expectations regarding timelines get generated.

  • QA / QC is not emphasized as heavily as it could or should be for larger projects. This goes back to having looser interfaces between components, and not having rigid specifications for how components should work. That looseness allows for unintended consequences and for bugs to creep in.

  • Consistently measurable qualifications. Generally, people speak of the number of years they've worked in X language or in programming. Time in is being used as a substitute for caliber or quality of skill. As has been mentioned many times before, interviewing and finding good programming talent is hard. Part of the problem is that the definition of "good" remains very subjective.

I don't mean to be all negative, and I think the software industry has made significant strides from where we've been. Forums like this and others have really helped promote conversation and discussion of design principles. There are organizations working to standardize on "baseline" knowledge for software engineers. There is certainly room for improvement, but I think the industry has come a long way in a reasonably short period of time.

share|improve this answer
14  
It was difficult to pick an answer to accept among several very good answers, but I finally select this one despite the fact that it doesn't have the highest votes. In fact, both this answer and the one by m3th0dman precisely why there is such specificity in IT industry, helping to understand what to do in the future to close the gap. Compared to the answer by m3th0dman, this one seems much more detailed. –  MainMa Jul 29 '12 at 18:43
2  
+1 but might I just add that since software exists in the realm of the mind, it has almost infinite possibilities, whereas every plane every built must contend with the finite requirements of reality. –  Spencer Rathbun Jul 30 '12 at 12:27
3  
Very well answered. As an interesting example--imagine if a large plane was designed and implemented by a bunch of people without process or company history--people that just got together and formed a business to build a plane on the scale of a 747 from scratch. That's how 90% of the software projects I've seen are done. The other 10% with experienced architects and companies with history and process seem to be a lot more successfull. For a counter-example look at the development process behind software that causes people to die when it fails. –  Bill K Jul 31 '12 at 20:05
1  
@MainMa you should not choose the highest voted answer, but the one that helped you the most. –  user1249 Jul 31 '12 at 23:40
1  
@Thorbjørn Ravn Andersen: I know and I already did it before, but I still find it useful to explain why I disagree with the votes where there is a considerable gap between the highest voted answer and the one I accept. –  MainMa Aug 1 '12 at 7:35
show 8 more comments

The answer is surprisingly simple: those 'other industries' do have a high failure rate. We're just comparing the wrong things. Writing software is often called 'build', and so we compare it to the manufacturing or construction phases in other industries. But if you look at it, it's not construction at all: it's design. Software designs are written in code, and building is done by computers, whether by compiling software or directly interpreting it on the fly.

Many designs in other industries either take way longer than originally estimated, cost way more, or simply never see completion. Sound familiar?

So, what are we doing when we're planning software? Well, we're still designing, but at an earlier stage.

In software, there's no manufacturing line of note. Building the final component is (comparatively) cheap, and replication of that final product is both perfect and effectively free--you copy the build artifacts.

share|improve this answer
21  
Even in the industry the OP mentioned, Aerospace, the Boeing 787 Dreamliner and JSF F-35 have both had significant delays. Last week a carpark collapsed in one of the major shopping centres in Sydney. Sydney has very rigorous building standards but mistakes happen. –  teambob Jul 29 '12 at 22:10
10  
A thousand times this. Even the question's schedule link shows that the project was actually in development from 1988. The source code is the design: developerdotstar.com/mag/articles/reeves_design.html –  pkh Jul 30 '12 at 17:45
4  
+1 for seeing the code as the design! –  Martijn B Jul 31 '12 at 15:16
1  
@MrLane:In the real world it works like this. A schedule is set for when the hardware is supposed to be done and the software is supposed to be done. The hardware designers provide an ICD to the software team so the sw team can write their code without the hardware. The hardware slips its schedule, by a lot and changes its interfaces to work around the hw issues, frequently without notifying the sw team. Finally, the hw sort of works and is delivered, way late. Of course the sw doesn't work because of the myriad of unexpected hw "features". Because it is cheaper to fix hardware problems in... –  Dunk Oct 25 '13 at 17:06
1  
software, the sw team now has to incorporate the changes to the ICD and come up with workarounds for buggy hardware. So in addition to the hw being delivered way late and now the sw team is also fixing buggy hardware, who gets the blame for being late? Well, the software team isn't done yet. It is the software that is late. Everyone always forgets about the electrical, mechanical and systems engineering schedule slips that sw depended upon and then that forced sw to be rewritten and have extra requirements. All they see is that the sw team is still coding. Thus, the software is late. –  Dunk Oct 25 '13 at 17:09
show 2 more comments

To point out some figures:

  1. Change of requirements after implementation started; for example when the first Airbus A380 started to be created in the factory I cannot believe that if someone wanted 200 more seats, those would be put there; but in a large software project even after the programmers started development 5 more types of users can be added.
  2. Complexity - large software projects are extremely complex; probably the most complex projects human kind designed and developed;
  3. Not enough resources are spent in architecture and design phase;
  4. Field immaturity - software engineering is relatively a young discipline compared with other engineering sisters; this has two implications: a) Not so many heuristics and good practices; b) Not so many very-experienced specialists;
  5. Lack of mathematical proof - in most of the cases mathematical formal methods are not used to prove that a piece of software works as required; instead testing is used. This does hold true in other engineering fields which rely more heavily on mathematics; the reason of this is as complexity;
  6. Rush - many managers have unachievable deadlines; so quality of code is placed second, because of the deadline.

Answering strictly to the question - I tend to believe that others have very high expectations (especially in delivery time) from programmers and do not understand exactly how difficult programming large software is.

share|improve this answer
31  
Formal mathematical proof in software, besides the fact that it is often practically impossible to do right, is ultimately nothing more than writing the program twice (once in the actual programming language, and once in the formal-proof specification language) and verifying that both versions do exactly the same. –  tdammers Jul 29 '12 at 14:51
2  
tdammers, there are tools that can help you write both at once: Coq supports "program extraction" to extract a program in OCaml or Scheme from a certified Coq program. –  jkff Jul 29 '12 at 17:19
32  
"Change of requirements after implementation started". I call this the "moving the toilet problem". You are building a house, are putting finishing touches on the bathroom, and the owner wants the toilet in a different place. You give them the cost estimate. They balk, saying "why so much, I just want the toilet 8 feet away?". You then explain that you have to install new plumbing, rip open walls/floors, etc. to be able to move to toilet. Late changes are always expensive. –  The Lazy DBA Jul 29 '12 at 18:08
2  
I'd say testing an airplane is actually much more difficult than testing a software. With airplane, all the mathematic magic you conjured ends up useless when you figured that the software simulator or the wind turbines you created doesn't really reflect the way things works when you're up there. Formal proof in software is only a hard problem, compared to formal proof in engineering which is impossible. –  Lie Ryan Jul 29 '12 at 21:50
2  
#1 in your list is the most important IMO, i'd add two more things to it: -Lots of big changes can be made in a short timeframe (for example 'lets switch the communication protocol!'), that wont break stuff in the short term, but many of these make the project very hard to manage in the long term. - The changes in the environment where the software runs can change drastically over a short time. While the basic premises for an airplane will stay the same (must fly in storms, must land on solid runways,..), a software can totally break, if the new verison of the OS comes out for example. –  sydd Jul 30 '12 at 0:29
show 8 more comments

The premise of the question is a bit flawed. Both the A380 and the Boeing 787 were delivered years late.

In the case of the A380 much of the delay was caused by the French and German units of Airbus using different and slightly incompatible versions of CATIA design software. This incompatibly manifested itself as wiring harnesses that didn't quite fit the airplane.

There wasn't anything wrong with CATIA, which is the most widely used aircraft design software, but someone somewhere dropped the software configuration ball.

The Boeing 787 also suffered from software related delays, but most of its problems were more traditional new airplane problems like weight control and delivery of substandard parts by suppliers.

Both the A380 and the B787 had to modify their wing designs after the initial aircraft had structural issues.

Large complex projects are difficult for humans in all fields.

share|improve this answer
8  
Agreed. I think there's a false attitude that software is "produced more sloppily" than physical stuff because bad software ends up with fixes which are very visible, plus everyone has to deal with broken software. If a plane is a piece of crap and they have to do some work on it you don't send it back, it's just something the mechanics complain about amongst themselves in most cases, unless it's a huge defect. And those happen too. –  Ben Brocka Jul 29 '12 at 15:51
6  
I think the question still stands even though the examples are flawed. It is statistically proven, that software projects have much bigger final costs and take much longer that predicted. –  Euphoric Jul 29 '12 at 15:56
14  
It should be noted that the design and implementation of a commercial airliner system inherently includes the completion of a massive, and massively complicated, IT project, one that has to be fully and correctly functional. –  Pointy Jul 29 '12 at 17:47
4  
And given that the A380 had a major recall as recent as 2010, I wouldn't call it "flawless" even then. –  Muhammad Alkarouri Jul 30 '12 at 1:34
3  
Also, LOTS of concept airplanes have been funded and canceled over the years, particularly military contracts. Airplanes are not good examples at all, because we don't hear about many of the (classified) failures until many years or decades later. –  SilverbackNet Jul 30 '12 at 2:59
show 3 more comments

Skyscraper guy here. Not sure if I can answer your question but I can surely shed some light into various items in the thread. Buildings do indeed occur very fast. A major constraint is locale (regulations). But in general it takes 3 to 10 years for a tall building from start to finish.

I think comparing a new building with a new software project is not very accurate. A new building is closer to a new version of a kernel or OS. In this respect software development is much faster. We never build from zero as this will be a high risk task the client would never sign up for. Most design and development in buildings is derivative of proven and completed projects.

From personal experience only one in ten projects ever get built. The process is development-driven rather than design-driven, so the moment something like the price of steel goes up the whole project is out the window, or designed, as design is the cheap component of the process.

Design takes a month for concept to schematic, two to six months to design development, another six months to details and construction documents by a team of architects, planning consultants, structural engineers, wind engineers, services engineers, quantity and cost consultants, surveyors, accessibility engineers and the list goes on...

The argument of virtual versus physical is not very accurate. We also work mainly with virtual tools, and moreover we are both time- and scale-remote from our final product. In most cases we can not even test aspects of buildings in mockup scale and we use simulation to try predict what may come about.

Complexity-wise there are differences, but overall it is maybe about the same. We not only have interrelated units and multiple levels of tiered abstractions and interfaces but also people are very much specialized in small parts of the process that make communication very difficult.

As for the argument of design versus development, I think both processes are design-built. It sounds academically nice to keep these separated but it is not possible to design if you don't know how things work. You just increase the risk of failure.

Overall my (potentially wrong) estimation as per OP's question is that programming is more of an art than engineering. Why would people take pleasure and even do it for free, find expression and elegance in it? Computer science is also (as on the tin) more of a science than engineering. Why would you try to prove algorithms instead of just patching existing parts together and work to make things just work? Not sure if this makes any sense; I'm not a software guy.

One aspect that strikes me with software design and development is about the medium itself. Computers make human-centric work very unnatural. Everything is so very explicit and there are few tolerances. It's hard to mentally work your way around this, and some get away with it by dumping complexity within. If nothing else this may have something to do with it?

share|improve this answer
1  
+1 Thanks for the insight. "to design if you know know how things work" -> " to design if you don't know how things work" ? –  tokland Aug 1 '12 at 10:07
show 1 more comment

Then how long did design of those took? Year? Two? Ten years? The design is the most complex part of building something, the construction itself is easy.

Based on this article, it is slowly being understood, that software development is mostly design process where design document is the source code itself. And design process is totaly different from production process. It requires experienced people and is impossible to plan because even small requirement can result in huge change in overall architecture of the project. This understanding is basis for agile methodologies that focus on improving code quality as final design document and taking testing and debugging as parts of design process, just like they test airplane models in wind tunnels.

The construction itself is handled automatically by compilers. And thanks to that, we are able to build whole product in matter of minutes.

The reason why software projects are finished with huge delays and inflated costs is because managers still think they can estimate, predict and plan such design process. This backfires more often than it is actually valid. They still think that by tying people into a rigid construction process they can somehow increase quality even though end result is mostly opposite with increased costs and missed deadlines.

share|improve this answer
4  
@NathanLong: Especially if they came out with new forms of concrete every three years, and somebody figured out how you could run multiple elevators in a single shaft, and suddenly steel wasn't cool anymore and everyone decided to build their frameworks out of carbon fiber. Stuff like that goes on all the time in the software industry. –  TMN Aug 1 '12 at 21:15
show 1 more comment

As someone with a Mechanical Engineering background working in IT I've often wondered about the reasons of the low success rate in IT.

As others in this thread, I've also often attributed the failures to the immaturity of IT, the lack of detailed standards (yes I'm serious, have you ever checked the standard sheet of a simple bolt?) and the lack of standardized components and modules. Other industries, like building construction or ship building also have much more "beaten paths": knowledge and experience of a particular solution prototype, which - in customized form - is re-used again and again. Ever wondered about why buildings, ships or airplanes of different size and purpose somehow look so similar? (there are exceptions to the rule of course...) That is because those prototypes are well researched, well understood, generally used and have a proven track record. Not because it couldn't be done any other way. In IT standardization is rarely the case: (large) projects tend to re-invent components, doing research and delivery at the same time and with the same people! These inevitably lead to one-off products, which are expensive to develop and service, are error-prone and fail in unpredictable ways under uncertain conditions. And because of this, of course, these products are much quicker obsolete, written down and replaced at equally great costs with only slightly better ones. What IT needs is the equivalent of the industrial revolution, which turned middle-age artisans into efficient factories.

That said, there are factors that make IT truly unique however. As opposed to those other mentioned industries, IT is truly ubiquitous: it is used in every aspect of our modern life. So it's a small miracle IT achieved this much progress and is capable of delivering the results it does.

share|improve this answer
4  
+1. Good example for the standardization (standard sheet of a simple bolt). In IT, rare are the components which are normalized. Take registration forms: every website reinvent their own, and few are the developers who know how their registration form behaves with unicode, with empty strings, with strings too long, etc. –  MainMa Jul 29 '12 at 23:20
1  
@MainMa: poor example, do you create your buttons, textboxes, option boxes, option boxes from divs? No, you reuse the browser's form elements; and the browsers used the Operating System's form elements. –  Lie Ryan Jul 30 '12 at 15:31
3  
I were rather speaking about the internals, not the controls. Take some random website. Can you use Chinese characters for the password? Can you use 25-characters passwords? What will happen if you put a whitespace in password or user name? All this could be normalized, but it's not, and every person is reinventing the wheel for every project, often badly, i.e. no hashing and/or salting, or passwords limited to sixteen characters (example: MSN), etc. –  MainMa Jul 30 '12 at 15:49
1  
Changing IT from "artisans" to "factories" may not be possible. Factories are executing a process which has already been created. Workers in a factory execute their process with little or no human thought. Many factories have replaced humans with robots due to this fact. In software you are not executing a process, but creating one. Creating software would be more akin to designing the factory and it's processes rather than running the factory. Although software creation could benefit from standards, it cannot fundamentally become a factory. –  mike30 Aug 1 '12 at 19:19
add comment

I'm afraid that I disagree with your statement.

Airbus and Boeing are two examples of companies that build planes. How many companies that build planes are there? Very few, if you would compare it to how many companies build software.

It is equally easy to screw an airplane project as to screw a software project. If only the entry barrier was so low in the aircraft-building industry as it is in the software industry, you will certainly see many failed aircraft projects.

Look at cars; There are high-quality manufacturers that build very durable and highly advanced automobiles (think Land Rover, Mercedes) and there are ones that build cars that won't last a year without having to repair them (think Kia or Cherry). This is a perfect example of an industry with slightly lower entry barrier, were you start to have weaker players.

Software is no different. You have lots of buggy products, on the other hand, Windows, Office, Linux, Chrome, or Google Search are very high-quality projects that were delivered on time and had similar quality level as an aircraft.

The other point that many people miss is how much maintenance goes into maintaining a car, a tanker or an aircraft that we just take as a fact of life. Every plane has to undergo a technical check-up before every take off. You have to check-up your car every several k miles and do so regular stuff like change oil, change tires.

Software needs that too. If only people spent as much time on diagnostics, prevention or auditing software's state and quality as they do with mechanical/physical products, I would expect way less statements like these. Do you read your application's logs each time before you launch it? Well.. if it was an aircraft you would have to ;-)

share|improve this answer
4  
Windows has often not been delivered on time (Longhorn, aka Windows Vista, was originally supposed to ship in 2003). I don't know about Office, but most of the other software projects you mentioned explicitly don't have timelines, or their release schedule is so frequent that they include whatever features are ready in the release, and push everything else off until it's ready. –  Ken Bloom Jul 30 '12 at 2:15
1  
@PaulNathan decent is very subjective ;) –  James Khoury Jul 31 '12 at 4:36
2  
@KarimA.: Building a safe aircraft isn't cheap, but a big part of what makes an aircraft safe, is software... So an important part of the aircraft design is actually software design! –  awe Jul 31 '12 at 9:56
show 3 more comments

Imagine, in the middle of the design of the Airbus A380, someone piped up in a meeting and said, "Heh, could build it as a triplane?" Others joined in saying, "Yeah, yeah. A triplane. More wings is better." Next 3 years spent turning the A380 design into a triplane. At another meeting, someone says, "A triplane? That's old. We want a biplane. Just remove one of the wings."

Or imagine, in the middle of a bridge construction project, someone says, "Heh, we just made a deal with a shipping company. They need the bridge to be another 40 feet higher because their ships are much taller. Fix it. Thanks."

These are but some of the reasons why software projects, big and small, end in failure at an alarming rate.

share|improve this answer
2  
This is exactly what happens. The management types or the clients change their mind every 10 seconds which just frustrates the developers. I quit my last job because of crap like this –  Erin Drummond Dec 17 '12 at 9:45
add comment

Engineering standards and practices are very different in IT (as an independent industry) than in Aerospace. This is perhaps most easily understood by considering how IT professionals react when encountering standards documents for IT in aerospace. For example, the Joint Strike Fighter C++ Standards that have made their way around the internet in recent times.

Many express bemusement or wistful resignation (wish we could do that way); and many respond with ridicule, claiming there is no practical way to deliver consumer products in this way. This may even be right, given the expectations, not of consumers, but of management. There is a great deal of distrust for coders who just code and code for a few weeks, not demoing anything. God help the coder who merely designs something for two weeks. Not so with airplanes.

In software, people really expect to have something right now. Sure, the reasoning goes, it will take a little while to have it really solid; but can't we have >some complex thing described in simple terms< in a week? One learns, also, that complex things described in honest, complex terms rarely excite the imagination; and thus many engineers end up being complicit in an imagined world of really simple things being put together in creative ways (as opposed to a world of hard things being done really well).

share|improve this answer
add comment

I have often wondered the same thing. It certainly feels (occassionally) like we're a bunch of amateurs that don't have any idea what we're doing. I dislike explanations that put the blame on managers or other external factors -- we the developers should be responsible for what we create.

I think we are in a business where errors are cheap. Patching software is cheap, compared to rebuilding a skyscraper, or recalling every sold cellphone.

This has created a culture where bugs are a part of every day life. They are accepted with a shrug. While some bugs are probably unavoidable, should they dominate our day to day work? I completely understand managers who don't feel that QA is worth the trouble, precisely because they expect bugs anyway. I don't understand programmers who don't make every effort to produce error-free code, because correcting bugs is boring as hell.

In essence I believe it is a culture problem and I hope it will change.

share|improve this answer
5  
Do you understand programmers who don't make every effort to produce error-free code, because that would take twice as long and management is breathing down their necks to implement these new features yesterday? –  Beta Jul 29 '12 at 18:48
3  
Shouldn't that be true of other industries as well? I don't deny that external factors don't exist, but I believe that a change must come from the inside. If software engineers embraced their role as experts in their field, their recommendations and estimates would be respected and not second-guessed by management. Or am I being to naive? –  waxwing Jul 29 '12 at 20:01
2  
I am often surprised when I occasionally visit a customer, and watch them use the product i am programming. There are bugs and functionality that makes the way they work very difficult, and as a programmer, I see how easy that could be made much better for the user, but the user has never complained about it, because he thinks it's not worth the bother to report it. –  awe Jul 31 '12 at 10:01
add comment

Digital building blocks can be 1 or 0. There is no inbetween.

A mechanical design has a level of tollerance. I can put one less than perfect rivet into a bridge and it will most likely will not fall down, however, in code even just once instance of putting a 0 where a 1 should be can fail the entire program.

Due to the logical and interative nature of computing, any, even very small changes, can lead to drastic failure.

share|improve this answer
11  
I once heard someone say "Construction would be just like programming if when you put the final doorknob on the house backwards, the entire house exploded". –  Morgan Herlocker Jul 30 '12 at 19:34
add comment

Some stuff from me:

1- Standards and parts: They are plane manufacturers, not developers.I am not entirely sure but my guess is that alot of parts are outsourced. They don't build their own electronic/instruments, they get seats from some company, the engines are probably developed elsewhere etc. etc.

Software-Projects on the other hand almost always start from scratch with just some small frameworks/helpers in place.

2- Time to hit the market: Time is not a critical issue for planes. I bet the design of the Airbus was finalized years before it was finished and they did chose to neglect any major breakthroughs that might happen in that time. (Same for car manufacturers f.e., the cutting-edge technology they develop at the moment will hit the streets in 5-10 years.)

For software you need to be very agile, if I start a huge project now and take 3 years to develop it without any change the chances are pretty high that I am relying on technology that is not available anymore or my product is completely outdated. This in turn offers alot of problems.

3- Release-cycle and Versions. - A plane needs to be completely finished when it is "released". There are no stable beta versions, nightly builds or similar. Additionally once it's done it can only be modified in a small way. You can't add an additional level with 100 seats to an existing boeing, it's just not possible.

Software on the other has incremental changes that are often just "builds that work", but not necessarily finished products. Also, in IT it's not unusual to say "hey, lets add another luggage compartment to our plane which holds additional 50 tons".

share|improve this answer
add comment

I have a shorter version for you:

whatever is easy to do, or streamlined, we write a program to do it instead of us.

And then fight with the meta-process instead.

It's not that much true, per se, but every day thousands of blogs are set up, instead of writing blog engines. Every workday thousands of excel macros are written, instead of writing specially-designed database applications for these.

There are a lot of other factors - some of them mentined here - but I wanted to add this point to the discussion.

share|improve this answer
show 1 more comment

Software engineering and management is fundamentally different than a lot of other engineering areas. The deliverables aren't physical, and the production process is the design and development process. Creating another copy of a piece of software has essentially zero marginal cost; all the cost is found in developing the first copy.

Because of the relative youth of software engineering and management as a discipline, there is some misinformation and falsehoods out that are still taken as fact (see this reference) which hinders software development and engineering as a whole.

share|improve this answer
3  
+1 on the immaturity of Software development. Civil Engineering has had a couple of thousand years to develop it's practices. Aerospace has had a hundred, and is based on other engineering disciplines. Software is still young. It's also normally poorly understood. People can build bridges over streams or make paper planes as kids - the same isn't really true of software. –  Andy Burns Jul 30 '12 at 13:03
add comment

Not all developers are created equally. Some are good, others are, well, not.

Try reading other people's code all the time to get a feel of what I'm saying. Too many extra logic statements can add risk. These risks can lead to ill behavior or bugs. Not enough logic statements and now you have null references. The good programmer understands this and knows when to do what and where. But no one is perfect. Things are complex. Add the poorly thought out work of others and it is easy to see how projects run away.

share|improve this answer
add comment

I think the answer is quite simple:

1) Physical construction and implementation have been around for as long as people have - we've had thousands of years to develop our methods and techniques for implementing physical projects. Software 'construction', which requires an entirely new and different skill-set, is no more that 50 years old - we haven't had enough time figure it all out yet.

2) Virtual Construction is harder - you have to 'see' things in you mind that have no physical reality whatsoever. It requires you to analze and abstract a lot of information before you even know what your product is suppose to look like and the steps it will take to create it. Not so when building a bridge or a building.

3) It's often much more difficult to find the source of a software failure or bug than it is when doing physical engineering. If a girder buckles, you see where it's buckling and you see the supports that are holding it and failing, etc. Finding a software defect can entail examining a great deal of code and interacting processes - difficult, time consuming, and not bound to the laws of physics and math in the way that physical structures are.

share|improve this answer
add comment

Cathedrals used to take up to 100 years to build.

Some Airbus airplane needs 1 million lines of code to work.

The more time you have been improving something, the more improvement you get, so give the software industry a couple of centuries of trial-error to get better, and we'll see how much it takes a to develop a solid huge project without bugs or flaws.

share|improve this answer
add comment

Large projects often occur in large organizations. If you've never worked in a large organization, there is one thing that is guaranteed to kill performance and productivity: bureaucracy.

Surprisingly, many people do not know what bureaucracy is (it is often confused with politics), or even if/when they have a bureaucracy problem.

We recently concluded a project to implement smart card authentication. It was originally estimated at three months. It took 15 months. There were not any cost, budget, scope, or technical reasons for the delay. The scope was actually quite narrow - only for accounts with elevated privileges (administrator accounts), about 1,200 total accounts.

Another significant factor is your business partners. This would include vendors. If your partners have a problem that introduces a delay in your project, there aren't many options that will actually fix the delay problem. They don't work directly for you, and you may not be able to fire that one person at a partner that may be the cause. The partner can be fired, or can be subject to financial penalties or disincentives, but that does not change the fact that the project has incurred a delay. This is precisely what occurred with Boeing when they undertook a mammoth sourcing strategy with the 787.

share|improve this answer
add comment

Most large projects have a high degree of separability of sub-projects, where you can define a small number of design constraints; the whole project will work when those sub-projects each are completed. If something goes wrong in a sub-project, the whole effort is not thrown into question; you just look for alternate ways to complete the sub-project (e.g. use a different engine).

This is possible but difficult, both practically and as a matter of human nature, in software projects.

In part, other industries have learned the hard way that this sort of separability is a good thing. For example, if you're going to use Rolls Royce aircraft engines, you do not need to use special Rolls Royce bolts and attachment points that only work with wings with a particular design, and then if you try to switch to Pratt and Whitney, you have to redesign your entire wing from the ground up (which, in turn, requires a complete redesign of the fuselage, which in turn requires you to buy different seats, which in turn requires you to buy a different in-flight entertainment system, which...). There may be a few linkages--you can't just swap engines without a care--but big projects generally work better when such things are minimized.

I postulate that big software projects designed as a cluster of small software projects with clean interfaces between each other will not fail particularly often, as long as the big project is actually solved by the cluster of small projects. (It is possible to make a mistake in this regard.)

share|improve this answer
add comment

Building software systems is very different from building physical structures. That is, the implementation is very much different. While for example building a huge tanker, you do lots of relatively simple(not easy though!) tasks, such as welding parts together by a robot or by hand, tightening all the nuts and bolts, painting, do the decoration by carrying in all the equipment and furniture and such. All of this is very simple stuff to do, really.

However, when it comes to software, it gets much more complex. For example, how exactly do you implement the secure login and user credential storing part of the product? What libraries and tools can you use? With what libraries and tools are you familiar with? How exactly do you go about writing the hashing + salting function and how do you ensure it is secure? You can do this in so many ways that it's impossible to set any actual practical design patterns for these kind of things. Yes, the said design patterns do exist on a smaller scale as "best practices", but every single software system is very different from the others, and the field advances and changes at so rapid pace that it's essentially impossible to keep up.

When building a house, you don't really run into such problems where you realize that the main supporting walls seem to be inadequate and need to be replaced, requiring you to demolish the progress so far and start from the base by redoing the support walls. You tackle such issues at the design phase, because it's relatively simple to predict what kind of support walls your building needs. Thatis not the case with software though. You can't really design the whole product as a single entity and then implement it. Software design process is usually iterative, and the goals and requirements change as the product is being implemented and tested. Software development as a whole is an iterative process in which things usually change when least expected, and many times such changes impose challenges which require more work, more complexity and unfortunately and ultimately more money, time and hard work to get right.

So, in essence, the reason why it is hard to deliver big projects and estimate project timelines and roadmaps is that software development and especially working design are very complex fields. Complexity is the root problem.

share|improve this answer
show 1 more comment

Lack of accountability..., people get sued when an aircraft crashes. The software industry declines any responsibility in any software defect, therefore creating a lack of insentive to create robust and safe product

share|improve this answer
1  
I've been saying that for a long time. If you got sued when your app crashed, code would be a lot more robust... And there are a lot of companies I'd like sue - take MS for example: how many hours are lost because of all their updates and patches and bugs and revisions that break other stuff. Sue them for those lost hours and quality will increase QUICKLY. –  comeAndGo Jul 29 '12 at 22:45
3  
And the cost of a Boeing 737 is $50-80 million. You pay each time you get on one-- do you pay each time you open up Office? If I got paid every time somebody used my software damn straight i'd be dedicated to reliability. –  FlavorScape Jul 30 '12 at 3:00
show 7 more comments

Though it's hardly the only thing that could be mentioned, I think one basic thing is worth pointing out. Most products are intended to fit with existing behavior. Even a product that's a radical breakthrough (e.g., the car) is generally built to fit with existing behavior, and simply make it a bit simpler/easier/cheaper/whatever to do that. Yes, there's often some side effect on existing behavior as well (e.g., getting fuel for the car instead of food for the horses) but most of the latter tends to be a fairly minor side effect.

By contrast, almost any software that doesn't change the behavior of the users (e.g., let them do their job considerably more easily) is basically guaranteed to be a complete failure from day 1. Worse, large software projects don't just involve the behavior of users on an individual level, but the behavior of large groups -- often the entirety of the organization.

In short, designing the software itself is often the easiest part of the job. The hard part is redesigning peoples jobs for them. That's difficult to start with; doing it in a way that will not only work, but also be accepted is much more difficult still.

share|improve this answer
add comment

Airbus 380 was not a successful project as you have mentioned, I happen to work in a CAD/CAM company and I was told that it(we had airbus prioject too) was delayed by few years because they were using different version of software in different company i.e. different parts were being designed in different part of the world, And while integrating they came to know that all the design can'nt be integrted so they have to redesign it in one version. So looking at it I don't think it was successful, had it came 2-3 years before it would have been game changer for Airbus. Also regarding robust software, you look at any airplane, car(ABS,EPS, climate control etc) or space shuttle they have more than 50% software which are running them and belive me they are very robust. Its just that we are more close to software and there are many more software program so we see more error in them. Visit : http://www.globalprojectstrategy.com/lessons/case.php?id=23 and see how much successful was airbus 380

share|improve this answer
add comment

The definition of "large project" is skewed.

A large project, technically, can be delivered on time, and flawlessly, granted it is something that's been built many, many times over the years.

  • A Pac-Man clone.
  • A calculator
  • A text editor

I'm sure you're thinking..."but those are small projects! A text editor is simple." I would disagree with you. Computers are outrageously complicated. Just installing and setting up users on an operating system can be difficult at times, and you didn't even write the OS, or build the hardware.

The projects you're talking about are huge projects, akin to space exploration. How do you know how long it takes to develop inter-galactic travel? What model do we base it on? You have the known knowns, the known unknowns, the unknown knowns, and finally, the unknown unknowns, which happen to come up a lot in software development.

I think the problem is one of expectation. Just because the technology is there doesn't mean using it is going to be successful (or wise to use) for a while. If other industries behaved like the software industries did, we'd have black hole powered vacuum cleaners for sale by the end of the decade. Or some "visionary" would have the resources to build a moon base, and decide that a Starbucks would really "round out" the experience for visitors. I don't think the problem is the software industry, but the expectations placed on it.

share|improve this answer
add comment

I will try answering using a verbatim copy of an article from Jack Ganssle's embedded muse. While this says firmware everywhere, just mentally replace it by software.

Compared to What?

Firmware is the most expensive thing in the universe. In his wonderful book "Augustine's Laws," Norman Augustine, former Lockheed Martin CEO, tells a revealing story about a problem encountered by the defense community. A high performance fighter aircraft is a delicate balance of conflicting needs: fuel range vs. performance. Speed vs. weight. It seems that by the late 70s fighters were at about as heavy as they'd ever be. Contractors, always pursuing larger profits, looked in vain for something they could add that cost a lot, but which weighed nothing.

The answer: firmware. Infinite cost, zero mass. Avionics now accounts for more than half of a fighter's cost. That's a chunk of change when you consider the latest American fighter, the F-22, costs a cool third of a billion a pop. Augustine practically chortles with glee when he relates this story.

But why is software so expensive? Tom DeMarco once answered this question with these three words: compared to what? He went on to discuss relatively boring business cases, but that answer has resonated in my mind for years. Compared to what? With software we routinely create product behaviors of unprecedented complexity. Sure, the code's expensive. But never in the history of civilization has anyone built anything so intricate.

Consider the following bubble sort, lifted shamelessly from Wikipedia and not checked for accuracy:

void bubblesort(int * A, int n){

    for(int i(0); i < n; ++i)

        for(int j(0); j < n - i - 1; ++j)

            if(A[j] > A[j + 1])

                std::swap(A[j], A[j + 1]);

}

It's a mere 110 non-space characters, perhaps tossed off in an hour or two. Suppose we didn't have software and had to implement a sort using some other strategy. What would it cost?

A mechanical engineer might boast that his profession built sorters long before computers. Consider IBM's 1949-era model 82 card sorter (http://www.columbia.edu/acis/history/sorter.html) with a throughput of 650 cards per minute, rather less than our code snippet might manage even on a 4 MHz Z80. The model 82, of course, only sorted one column of a card at a time; to completely sort a deck could take dozens of passes.

How long did it take to design and build this beast? Years, no doubt. And its functionality pales compared to our code which is so much faster and which can handle gigantic datasets. But that was 1949. How long would it take to build a bubble sort from electronic components - without FPGAs and VHDL, or a CPU?

In an hour I managed a rough block diagram, one above the chip level (blocks have names like "adder," "16 bit latch" and the like). But the sequencing logic is clearly pretty messy so I've just tossed in a PLD, assuming at some point it wouldn't be too hard to write the appropriate equations. And, yes, perhaps that breaks the no-programmable-logic rule, but to design and debug all that logic using gates in any reasonable amount of time is as unlikely as buck-a-gallon gas.

Assuming 16 bit words and addresses, the circuit will need around a dozen 16 bit latches, adders, and the like. Plus memory. And I have no idea how the unsorted data arrives into the RAM or how the results get exported. Those are unspecified design requirements. The software-only solution naturally resolves these requirements just by the act of writing the function prototype.

Translating the rough block diagram to a schematic might take a day. Then there's the time to design and produce a PCB, order and load parts (and change the design to deal with the unexpected but inevitable end-of-life issues), and then of course make the circuit work. We could be talking weeks of effort and a lot of money for the board, parts and appropriate test equipment.

All this to replace 7 little lines of code. Few real embedded programs are less than 10,000; many exceed a million. How much hardware and how much engineering would be needed to replace a real, super-sized computer program?

Consider a real program like a spreadsheet. How much circuitry would it take to make one without a processor? It could be the size of a city.

Firmware is the most expensive thing in the universe, but only because of the unimaginable complexity of the problems it solves. But it's vastly cheaper than any alternative. So when your boss irritably asks why the software takes so long, you know what to say. Compared to what?

So there! Software/firmware has unparalleled complexity.

share|improve this answer
add comment

Software engineer here, with an engineering background, and a wife who works in construction. We've had long discussions (and arguments) on the differences of our jobs.

Software engineering is about designing new things. Almost everything basic has been done in an open source library somewhere. In almost any job where a software engineer is hired, she has to design something that doesn't exist.

In something like construction and most forms of engineering, things that would otherwise be in a 'library' in software are already fully designed. Want to build a tower? Just copy and paste the plans from an existing structure, with a few modifications.

In fact, one of the main reasons I decided not to become an engineer is that you spend most of your time designing a 10% improvement to an existing invention, when that same time could be used to program something more visible, like a social network.

There are not many new designs in engineering; an extremely skilled engineer is someone who can manipulate an existing design into something new or improve on it. But almost every programmer is expected to modify designs, hack them, or create something new.

Look back at how far the standards have changed completely, from Assembly to C to C++ to Java, Javascript, C#, PHP, and so on. There's not a lot of code that can be recycled from 10 or 20 years ago. This is very different to say... the automotive or aeronautics industry when you can keep improving on designs from decades back.

Project management is notoriously difficult in software. Time estimates are best done by people doing the work, but when they're busy making estimates, they're not writing code. This tempts people to avoid any project management at all.

Often, a lot of code depends on specific people, and if these people are late or unable to perform, the code does not move ahead. In contrast, if you wanted to build a car, you can simply hire different people to assemble the tires, the chassis, the battery, the engine, and so on. Object oriented and existing frameworks makes this possible, but it may not be practical when you're designing everything from scratch.

Failures may be allowed in software. Testing can be costly. In software, it's very tempting to skip all that testing, when you can just fix a crash. In most forms of engineering, a 'crash' can be fatal.

You do have programming methods that use extensive testing, like Extreme Programming (which was actually used on software megaprojects). But with tight deadlines (that can be made tighter on purpose), it's tempting to skip all that programming and launch with bugs. The Joel style of "always fixing all bugs" will save more time in the long run, but the undisciplined will skip this and fail.

Small projects are better. My wife once asked me to get a job in a big company. It ended up in an argument that big companies are bad companies... to her, a big company had a lot of resources, experience, functional project management, and the right procedures. To me, a big company is a dinosaur, where most of your time is spent on fixing code, testing it, and documentation.

I've seen million dollar IT projects worked on by less than 10 people. More people would have slowed down the project and added unnecessary bureaucracy. Whatsapp is an example of a 'small' project that's worth billions of dollars. It's not that big projects aren't possible, but you simply don't need thousands of people to produce billions of dollars worth in software.

share|improve this answer
add comment

There is active pushback and resistance from software developers to improving the process of software development. Requirements gathering is dismissed as unrealistic and always resulting in change which means that documentation is always out of date.

On the other hand, agile methods are resisted as well. Daily releases and constant refactoring are presumed to only be allowable for teams that have more time and budget.

There's no guarantee of a culture of professional. The culture and the processes of software engineers varies from company to company and that's one of the problems. You cannot rely on a minimum accepted standard other than "can you code something in language X using libraries A, B, C and get it done by Tuesday?"

This lack of professionalism and active resistance to professionalism is what causes bugs. We can hardly get people to do regular code reviews though there are numerous studies that prove how effective they are at reducing the number of bugs in a product. We can hardly get them to document their own code which can lead to new bugs and leads to wasted days months down the road when everyone's forgotten exactly what the code is supposed to do and why it was built that way.

share|improve this answer
2  
do you have any data to backup your argument? –  Hoàng Long Aug 20 '13 at 9:29
3  
I will check the book. However, I don't think these problems (lack of professionalism, active resistance) don't happen in other industries. –  Hoàng Long Aug 21 '13 at 2:20
show 1 more comment

For me the main problem that software engineering face is use cases, user and cross platforms.

use cases

How many use cases does an airplane has? Most of it is just to fly from one place to other. Maybe there are more such as radar, traffic control, etc, but the use case is clear and not much. In software engineering, we are faced with unclear requirements and user who do not know what they want. Different user need different configuration / flow, can an airplane be customized as user want (I want to have tv and refrigerator!)?

user

Who operates airplane? A pilot, a copilot, some stewards (if counted) and tower operators. They are all pros and has their job desc clear. Software are used by noobs and dummies, not to mention evil hackers and crackers, while still need to be integrateable with other modules (such as openId or google adsense). If an airplane can be operated by dummies while still survive from missiles or ninja robbers, then you can say that the airplane has same quality with software.

cross platforms

An airplane fly only on the earth's sky. I'm unsure about how they handle the foggy or windy or hot, cold and humid climate, but an airplane does not designed to fly at different gravitation level (I will be amazed if it can fly to mars). Sometimes, an application must survive different platforms, such as IE, chrome, firefox and safari for browser (sorry opera), or windows xp / 7 / 8, or linux for OS level. Not to mention mobile devices and different resolution and orientation.

share|improve this answer
add comment

protected by gnat Apr 3 at 10:45

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.