Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Often we as programmers see large organisations wasting huge sums of money on bloated and inefficient solutions to problems. This pains me greatly because I like organisations to benefit from best of breed solutions. However, my abilities as a programmer are limited when it comes to influencing the key decision makers and often my perspective on the matter is constrained to my own little technical world.

So, my question is this. After encountering an egregious waste of money on some software and/or hardware that really got your goat, what did you do about it to get it fixed or were you doomed to bite the bullet and mutter forever under your breath? I'm interested in hearing your overall experiences and especially what lessons you learned about how to tackle this sort of thing in the future. Let's not name names, the experience of how to tackle the problem is more important than the actual offending product.

share

locked by Yannis Sep 14 '13 at 21:43

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.

closed as not constructive by ChrisF Jun 7 '13 at 18:38

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

9  
+1 both for a good question and for using the word egregious. – Jon Hopkins Nov 12 '10 at 12:51
    
Thanks Jon, I do my best. I've edited the question to highlight what people have actually done to tackle the waste of money. I'd like to know more about what approaches people took when they encountered said problem. – Gary Rowe Nov 12 '10 at 13:31
    
I'd just like to say thanks for everyone taking the time to answer this question. Your efforts are appreciated! – Gary Rowe Nov 12 '10 at 17:24
3  
Well, how can I add this without naming the product :) Microsoft's lost eight years online: More than $6 billion down the tubes – Halil Özgür Nov 12 '10 at 18:55
    
+1 for tracking the answers with summarizing comments – Ilia K. Nov 12 '10 at 22:11

27 Answers 27

up vote 20 down vote accepted

Paying for big, bulky, buggy commercial products, in the range of:

  • Application servers;
  • Testing tools;
  • Development environments.

when the open-source or light-weight alternatives are obviously superior.

My steps usually are:

  1. establish an alternative as a reference - e.g. "I will experiment with the app server X instead of app server Y. I had good experience with it, because (...).";
  2. sell this proposition to my colleagues - "I am now developing faster since server X restarts much faster and I don't wast all that time";
  3. sell this to the immediate manager - "Our team now develops faster since we're using server X. It all began as a little experiment but everyone likes it.".
share
    
The question asked for product names not to be included. Could you elaborate on how what you did to deal with the issues encountered instead? – Thomas Langston Nov 12 '10 at 16:54
    
Agreed. Most of us know the culprits that are out there, it's more about what can be done to limit their damage. – Gary Rowe Nov 12 '10 at 16:59
    
@Thomas, @Gary : good points, and sorry for misreading. I've rephrased, hopefullly it's more useful now. – Robert Munteanu Nov 12 '10 at 20:09
    
+1 for both responding to critique and providing a way forward for tackling the problem. Good advice that others can take away and use. – Gary Rowe Nov 13 '10 at 1:33
    
+1 for qualifying when alternatives are superior. – Thomas Langston Nov 15 '10 at 14:12

I've seen too many examples to name a favourite, but I've noticed a few general trends in my main field, web-development:

  1. Vanity Websites. These are websites that serve no useful purpose to anyone outside the small organisation that commissions them and are built around an obsessive compulsion with logos, photos of themselves and self-indulgent waffle. The worst part is these are usually public-sector funded and commissioned by people who have no clue about the web. (For instance, once had a NHS hospital trust who wanted to develop a mini-version of Facebook for their own staff intranet).

  2. Paid for is Best. The mindset that insists that paid-for software must intrinsically be better than open-source. After all, it's paid for, right? I've seen so many clients insist on making stupid choices simply because they work in a culture that automatically discounts anything open-source as a matter of policy.

  3. Design by committee. This is where a huge group of people have a "brainstorm" and then try to incorporate every crack-pot idea there is into the design, inevitably resulting in a ill-thought through mess that compromises on everything in favour of trying to please everyone (and by everyone they mean the committee making the decisions, not the people having to use the application).

  4. Consultants. This is where you pay a middle-man (who knows neither business practices nor software development) to get in the way and cream-off money by protracting the development process with confusing techno-babble and business-speak.

share
5  
+1 for Vanity Websites. Going into a law firm as a development manager my biggest achievement was actually putting a price on the development of these up front which killed them stone dead (strangely no-one was willing to sign off £100k). – Jon Hopkins Nov 12 '10 at 10:07
7  
Re: (3) "A camel is a horse designed by a committee" – JBRWilkinson Nov 12 '10 at 14:00
2  
"Self-Indulgent Waffle". Beautiful. – khedron Nov 12 '10 at 15:43
3  
One thing: "Paid for is best" is a misnomer. People shy away from open-source because there is no support, but most importantly, NOBODY TO BLAME WHEN THINGS GO WRONG. – Stu Nov 12 '10 at 18:52
4  
+1 for consultants. They're the plague of the industry. – user8685 Jun 15 '11 at 13:32

I don't see that anyone has mentioned this one yet.

Building your own solution when you could buy it.

Variations of this pattern:

  • not even considering the buy-vs.-build tradeoff
  • significant scope creep of the in-house solution
  • limited scope, but also limited utility of the in-house solution
share
5  
Slightly different version: we can't use libraries because they all have bugs. – Stu Nov 12 '10 at 20:10
    
@Stu +1 because the worst bugs are in open source libraries which we can't, er, fix... – Gary Rowe Nov 13 '10 at 1:26
    
+1 because a fairly-priced, well-supported off-the-shelf solution can be much better than making the development team reinvent the wheel. – Gary Rowe Nov 13 '10 at 2:01
    
@Gary: well... then spend more of your personal time to fix it. – rwong Nov 13 '10 at 5:42
    
Bear in mind that the opposite is true as well... especially for large organisations. Many times it would make more sense to build from scratch rather than purchase a generic app that needs to be extensively modified by expensive consultants. An example would be Seibel, which is great for organisations that are going to use it OOB, but not so great once you start trying to get it to integrate with legacy apps. – Michael Rutherfurd Nov 13 '10 at 6:40

My two favorites:

  1. Hiring consultants (freelance) just to add more production capacity, while they should invest in their own employees instead, by hiring consultants to bring new knowledge and coaching their existing people.

  2. Hiring project managers that manage other project managers that manage other project managers that finally (think) they manage the development team. While they should let the team self manage and focus on business instead. I've seen software projects where they had more project managers than developers. Imagine the meetings.

share
16  
Sometimes companies need extra production capacity on a temporary basis to overcome a short term demand without requiring skills transfer. This is a key function of the freelancer. If this continues for longer than a few months without skill transfer then your point definitely stands. – Gary Rowe Nov 12 '10 at 9:32
6  
In 10 years of consultancy, I never seen that working properly. The Mythical Man-Month. – user2567 Nov 12 '10 at 9:40
    
Oh yes, TMMM often holds (I've been there myself), but I have seen it managed properly when contractors successfully deliver a well-defined component and then walk away. Sort of drifting off topic now though. Good points. – Gary Rowe Nov 12 '10 at 9:46
4  
@Gary Rowe, the law is "Assigning more programmers to a project running behind schedule will make it even later". But hiring consultants to start a new project because you can't find any permanent staff is 100% valid. I wanted to clarify that. So my statement is about "just add more capacity" to an existing team (of permanent staff). – user2567 Nov 12 '10 at 9:52
3  
My current project is me as only dev and 2 project managers. Yes, the meetings are far from the best I've ever been in. – Matt Lacey Nov 12 '10 at 17:21

Limiting Long-term Raises and Bonuses

I think its taught in Business 101 to not give employees raises. A secondary case is to limit salaries of star performers because they need to fit inside of some certain salary range.

Eventually employees will realise their pay-scale is not in-line with their industry (or output). The people that have the resume and skills will eventually leave, and take with them all their knowledge and probably a few of their friends. The people remaining (who are the bottom performers) will have to pick up the slack and then spend more time hiring a new person (at market rate). So the company just traded a star employee for JR level one, and just lost all the "savings" of keeping the salaries low.

As this continues, the development team will struggle to stay on par, and will probably get worse and worse until something drastic is done.

share
5  
+1 for pointing out that talent should be rewarded. Owners of businesses make a lot of money when they sell out, but it is often the talent employed that makes the business worth buying. Owners - pay your talent well. Everyone wins. – Gary Rowe Nov 13 '10 at 1:24
2  
Not paying properly for skills = the skills walk out the door. They do this every evening anyhow, just one day they do it for the last time. And the managers wonder why. – quickly_now Nov 13 '10 at 4:50
2  
I was at a company that cut a $40/head team-building expenses to save on the bottom line. I left shortly after that. That was probably the most expense $40 the company ever saved, cause I'm sure I wasn't the only one. – cmcginty Nov 13 '10 at 9:55
1  
Sadly too many people think that paying a pittance is good enough. If they know you're making X, they'll offer X+1 instead of Y, where Y is the average, and then wonder why you leave in under a year there. – Wayne M Aug 31 '11 at 17:57

This answer is somewhat different than most: not firing an employee soon enough, or stated differently, being overly tolerant of an employee's mistakes habits. These were things I've observed and couldn't do much about as a consultant.

  • The dev that poorly drove the design decisions of a project that led to its eventual rewrite (it was a complete mess).

  • The dev that sent sensitive unencrypted data to Google Charts because they thought it would be cool to show a pie chart (was a pie chart a requirement? Nope!).

  • The dev that consulted with a company in the past and accepted a position with them directly. He did an about face and turned into a prima donna that sought the Technical Lead position and went as far as talking to the lead's manager stating they thought it would be good for him to take over as the lead. Talk about audacity! A lot of devs no longer like the guy and he burnt a lot of bridges within his first 2 weeks as an employee. To top it all off he's a very green dev that only graduated 2 years ago but thinks he's awesome.

A few mistakes are understandable, but when there's a consensus between many devs about someone's attitude or skill level companies ought to get rid of them sooner rather than later.

share
    
+1 for a call out on a developer being worth his weight – Gary Rowe Nov 12 '10 at 17:23
    
+1 for the whole idea of not firing somebody soon enough – quickly_now Nov 13 '10 at 4:44
1  
It's almost like you have someone in mind, there... :) – Dan Diplo Nov 15 '10 at 10:43
    
rweowr! so catty! – Chris McCall May 21 '13 at 23:46

Several times I witnessed management bringing in consultants for the sole purpose of spending money. Most of the time this happened at the end of the year when they were under-budget frantically trying to spend the money. Usually these consultants would be paid hundreds of dollars an hour and they would spend weeks on a PowerPoint presentation that would never be used.

share
8  
+1 for the "must spend the budget or it'll be taken away" management anti-pattern. What can be done about that? – Gary Rowe Nov 12 '10 at 15:25
2  
"Frantically spending money because they are under-budget"---a friend and classmate of mine, who was working shifts in the physics computer lab, told me they had to spend the rest of their budget or it would get cut the next year. So they bought $5000 worth of new printers, paper, and a scanner, I believe. – Mark C Nov 12 '10 at 15:55
14  
@Mark C - See, that's the way to do it. If you're under budget, and absolutely must spend money now, splurge on gear for your team. Maybe get some new chairs, or dual 32" monitors for everyone, or maybe just a powerful new integration server. If your equipment doesn't fall under the same budget, you'd be surprised what most companies let you get away with as a "team building exercise". – Inaimathi Nov 12 '10 at 17:44
4  
@Inaimathi +1 for a good hack - others please take note – Gary Rowe Nov 13 '10 at 1:20

There is a much bigger issue at play here.

Many companies have one goal - to increase shareholder wealth. What they produce is irrelevant. How they produce it is irrelevant. How much waste they produce is irrelevant. The cost to society and the planet is irrelevant.

So - go work for or start a company that does something of benefit for society / the planet.

share
    
But surely, if the waste is affecting the bottom line the shareholders will start to sue? – Gary Rowe Nov 12 '10 at 13:25
    
+1 for pointing out the environmental costs. (would up vote if I could) – DevSolo Nov 12 '10 at 13:36
    
+1 for absolutely brilliant point! – Jas Nov 12 '10 at 20:54
    
Do you think that corporations should be forced to have a social conscience? – Gary Rowe Nov 13 '10 at 1:56
5  
"Many companies have one goal - to increase shareholder wealth. " Actually under corporations law the directors can get set to prison if they do anything else. – quickly_now Nov 13 '10 at 4:46

Paying large software companies not only for their product, but for their "support."

I was working at a Government agency for a team that was deep in bed with Oracle. Over the course of many years, they had been paid bajillions of dollars for their software. Coming from a startup background this made no sense to me - "why not use MySQL or Postgres?" I was told that it was mainly because of the support that Oracle provides, if something goes wrong they help you find the solution quickly.

The support was an absolute joke. There was a problem where one web app kept crashing the whole system. It appeared to be a result of a slow database query with a combination of horribly written code (which was written by a team of consultants, which should be a whole other answer). A "task force" (groan) was assembled to pinpoint the issue and fix it. Included on the task force was an Oracle support member. Every day at EOB there would be a conference call where the task force members would update the rest of the team with the findings. It was a long enough call that nobody wanted to be on b/c it started at 5, and the Oracle person just made it worse. Why? Well, saying "person" isn't even correct. It was a number of people. It seemed like every two or three conference calls, the Oracle rep would be somebody new, who explained their predecessor was now on another project or had gone on vacation. The new people were never briefed by anybody at Oracle, so every time somebody new came in we had to waste ten minutes of the conference call explaining the problem again. Their contribution would then be asking for J2EE log files, which not only can any monkey read, but were also useless because the horribly written code was doing things like throwing IndexOutOfBounds exceptions when the programmer found errors in parsing XML.

share
    
There's other reasons to use Oracle. It's known to be scalable and reasonably secure, and it's been around for a long time. It does things MySQL really won't. I'm not all that familiar with PostgreSQL, but your agency was probably committed to Oracle before it was known to be good enough. That said, you are precisely correct about paid support. Some of it I've found excellent, but much of it is much as you've described. – David Thornley Nov 12 '10 at 17:43
    
Many technical reasons to employ Oracle. I assume you've never worked at a large software/consultancy organization? It's not always the direct fault of those assigned to help, they get pulled in hundreds of different directions based upon $ and priority. My guess is someone else was paying more $ for support and got the attention, market values. – Xepoch Nov 12 '10 at 19:16
2  
@David Thornley, for curiosity: what's something that Oracle will do that MySql won't? I mean, if you can explain an example easily. – Dan Rosenstark Nov 13 '10 at 4:40
    
MySQL ain't a real database system. MS SQL server is friendlier and just as capable. – Job Nov 18 '10 at 3:59

Having programmers to 1st line phone support.

Having programmers do testing.

share
1  
What kind of testing? A certain level of developer testing is necessary, but having programmers do full QA is not. – Adam Lear Nov 12 '10 at 18:46
4  
It can perhaps best be rephrased as "having no testers". Besides, developers are the worst testers. In. The. World. – Stu Nov 12 '10 at 18:50
    
this is so common, I don't think its really worth mentioning – cmcginty Nov 12 '10 at 19:49
5  
Wait, what? Are you serious? It is an egregious waste of money, but we shouldn't mention it because it is common? – Stu Nov 12 '10 at 20:08
    
+1 because programmers should really be 3rd line support (unless the company is a startup and call volume is low). I think programmers should do testing as part of development (TDD anyone?) and should be in close contact with the testing team to point out areas where they think the product is weak so that the testers are better targeted. They hate repetition as much as programmers do. – Gary Rowe Nov 13 '10 at 1:36

I know that this is an old question & I'll be lucky if 3 people read this answer, but it's a fun story to tell, so what the hell.

I came into a project (embedded systems, safety-critical firmware, very high stakes) and I was appalled by what I found. People using C (especially pointers) incorrectly, no static analysis, no code reviews, no testing other than "integrate it together, run it, beat on it, see what breaks."

I wrote a very long email my first week there (as a consultant). It was dicey because I was basically saying it was mis-managed, the developers were in over their heads, no process was being followed, etc. It should have gone to the corporate VP, but instead I sent it to the development manager who hired me. He was not entirely defensive about it, in fact he acknowledged many of the shortcomings & told me I wasn't the first to point them out (no kidding, right?)

To answer the crux of the original question: I offered to spend AT MOST 1 man-week getting Gimpel's Lint (PC-Lint / Flexelint) static analysis tool configured & running on their platform, and to run a full report of everything that was found. I told them I was absolutely sure that we'd find several lurking "timebombs" as a result.

They calculated my hourly rate, multiplied it by 40, and determined it was "too expensive to do that." Long story short, I left there within 60 days. About 3 years later, I learned of a product recall, the cost approached 9 figures ($100M), not to mention damage to the company's reputation.

I won't mention the company, the product, or the industry, but I still keep in touch with one of the engineers there, and when he explained to me what caused the recall, my eyes rolled - it was a problem that would have been caught by even a basic static analysis tool (accessing an array out of bounds). In fairness, I cannot say with certainty that the problem was in the code when I was there, but I'm sure if they'd spent the money on some kind of static analysis tool, that bug would not have escaped.

So they saved $295 by not buying PC-Lint (OK, they also saved a week of paying me, at most) - but I'm nowhere good enough to charge $100M for a week.

That's what I call a pretty damn big waste of money.


Reminds me of a joke that many of you may have already heard:

Ever heard the story of the giant ship engine that failed? The ship’s owners tried one expert after another, but none of them could figure but how to fix the engine. Then they brought in an old man who had been fixing ships since he was a youngster. He carried a large bag of tools with him, and when he arrived, he immediately went to work. He inspected the engine very carefully, top to bottom.

Two of the ship’s owners were there, watching this man, hoping he would know what to do. After looking things over, the old man reached into his bag and pulled out a small hammer. He gently tapped something. Instantly, the engine lurched into life. He carefully put his hammer away. The engine was fixed! A week later, the owners received a bill from the old man for $10,000.

"What?!" the owners exclaimed. "He hardly did anything!"

So they wrote the old man a note saying, "Please send us an itemized bill."

The man sent a bill that read:

  Tapping with a hammer ........ $ 2.00

  Knowing where to tap ......... $ 9998.00

Effort is important, but knowing what you're doing makes all the difference.

share

Bloated development teams and terrible productivity in software companies.

This is a consequence of the common pattern in the business world: the importance of a manager is measured by the number of subordinates, therefore a manager's number one concern is not productivity but quite the opposite: poorer productivity is the best justification to hire more people.

share
2  
+1 for mentioning that waste is deliberately introduced as a result of self-interest – Gary Rowe Nov 12 '10 at 13:24
2  
I've never seen too many total developers. Quite the opposite too many managers, too few workers. – Xepoch Nov 12 '10 at 19:15
    
The problem of empire building in organisations is not restricted to software development... – Richard Nov 13 '10 at 11:16

In a company that sold software...giving salespeople full commission on all custom mods sold, so that selling something that already existed and we could just profit on was not nearly as profitable for them as selling one-offs. This was combined with moving the sales staff halfway across the country from the technical staff.

This also meant that we in Development couldn't possibly meet the sales deadlines, making customers unhappy, and had a great deal of difficulty in getting any core work done that would make the product better for everybody. The increased pressure caused code quality to decrease and hurt morale, particularly when we heard stories about the sales office (which I never confirmed).

Lots of us resented Sales, but in fact it wasn't their fault. They were going out and selling as much as they could, doing what they were rewarded for in accordance with the limits that had been placed on them. It was bad management that caused all these problems.

share
    
+1 for recognising that sales are not the enemy, they have their own motivations and that these must be harmonised with those of the developers (and other aspects of the organisation). – Gary Rowe Nov 12 '10 at 17:44
2  
+1. Commissioned sales is a very bad idea in general. (Think about this: How much of the housing bubble would never have happened if it weren't in both the realtors' and the bank loan agents' self-interest to sell people houses at prices they can't afford?) – Mason Wheeler Nov 12 '10 at 18:03
1  
@David: That's just the point. Commissioned sales creates an inherent conflict of interest, especially in a product that's sold on debt and not for cash up front. The people who were making that decision were the loan officers, the very ones benefiting from commissions on bad sales. "It is difficult to get a man to understand something when his salary depends upon his not understanding it." -- Upton Sinclair – Mason Wheeler Nov 12 '10 at 18:35
1  
Yes, but no one at any level did, because it was making short-term money for everyone. Now, if we had a law that made it a crime to receive a commission payment for any transaction before it had been paid in full, the entire problem would go away almost instantly. Suddenly it would be in the loan agents' and realtors' best interests to give people loans they could afford to pay off, and pay off quickly. Absurdities like 30-year mortgages would disappear overnight, and everyone would be happy except the parasites who cause problems like this in the first place. – Mason Wheeler Nov 12 '10 at 18:44
1  
@Xepoch: I am thinking of the effect; I'm just looking to cause different effects than the status quo. Long-term financing is not a good thing. A worker is generally expected to enter the workforce around age 20, give or take a few years, and leave around age 65. If he wants something as fundamental as a home to call his own, he's supposed to be in bondage to a bank for two-thirds of his productive life?!? I don't know how anyone ever got the idea that that was supposed to be a good thing, but I call it a crime against humanity. – Mason Wheeler Nov 12 '10 at 22:07

There are two that I have experienced.

  1. Cancelling a project that had a huge ROI for the business that was about 80% completed and then handing out 100 engraved and gold plated iPods to senior executives.

  2. Laying-off several hundred people and then the following day announcing substantial pay raises and bonuses for the senior executives.

These are not totally programming related, but most certainly wasted a lot of money, plus provided a slap to the face for all those involved.

I didn't get laid off, but I also didn't get a raise or an iPod either...

share
    
+1 Some days I get the impression we all work for the same company. Not my current company, mind you, but an ex-Fortune 500 company I've worked for in the past was much like this. – Jesse C. Slicer Nov 12 '10 at 14:51
    
@Jesse: Should we draw any conclusions from your "ex-"? – David Thornley Nov 12 '10 at 17:29
    
@Jesse - these events were from a Fortune 500 company. Doesn't really matter which one, like you said, they're all the same... – Walter Nov 12 '10 at 17:40
    
@David: yes, there are some pretty easily-come-to conclusions you may draw. – Jesse C. Slicer Nov 12 '10 at 19:27

I've seen a couple of horrible outsourcing projects that succeeded in significantly increasing costs while either failing to increase or actually reducing efficiency.

In the worst case the new outsource team were put in place and up skilled but the existing on-shore team remained in place because the outsource team weren't trusted to actually do any of the critical work.

At this point the logical thing to do would obviously have been to accept failure and shut down the outsource team but because management weren't willing to publicly admit that it hadn't worked both teams were left in place (at a significant increase in cost with no increase in efficiency or usable capacity) until the whole thing could be buried.

In another instance development was outsourced and the original team laid off. Two years later they realised that it hadn't worked and paid to bring the whole lot in-house again only to find that in addition to the very significant costs of another handover, the impact of lost knowledge, recruitment fees, contract terminations and so on, the outsource organisation had lost a significant chunk of the source code.

(Note: I'm not saying outsourcing can't work, just that too many times people are seduced by potential savings and don't consider the realities of their new world, the change to process and working practices and so on which leads to majorly screwed projects)

share
1  
+1 for outsourcing badly. I've seen one outsourcing team pushed on us by a major investor and everything they did had to be rewritten, another who just didn't do what they were asked (when contractors are asked to produce an MSBuild-based build script, MSBuild wrapping NAnt isn't good enough), and one team who did a great job but they've started on v2 and have been working for us so long they're effectively just really expensive employees – JohnL Nov 12 '10 at 13:10
1  
+1 for the "can't admit failure" anti-pattern (a very sweeping pattern that crosses management and enters international politics but let's not go there...) – Gary Rowe Nov 12 '10 at 17:38

Technical Debt

I've seen is the chronic "beating the dead horse" of legacy code. Or more to the point, from the viewpoint of the trenches, countless hours spent in maintenance mode when the whole team knows we should be in replacement mode.

What we've done.... is still on going. Trying to invoke positive change from within

Performance Testing

Simply, not doing it. Again, still working on the positive change from within.

share
1  
+1 for working to pay off technical debt: martinfowler.com/bliki/TechnicalDebt.html – Gary Rowe Nov 12 '10 at 14:44
    
How are you getting management buy in for refactoring and general code hygiene issues? – Gary Rowe Nov 12 '10 at 14:45
    
As I said, "it's on going" and not easy. I'm also not sure if it's any one thing. Adopting agile practices and bringing transparency is a start. Example employing a daily stand-up helps. The CTO attends this nearly every day and pays attention to the "face planting" that is reported with the legacy code. The helps to identify some things. – DevSolo Nov 12 '10 at 14:55
    
Good job in getting the CTO (major stakeholder) to take a direct interest in the daily stand up. Sadly I doubt that approach could scale though. – Gary Rowe Nov 12 '10 at 16:25
    
Ironically, it was their call to attend. Which is good. The need for change is seen on the macro scale. Once we're at the code level, it's more challenging. It's not perfect, but any way to bring transparency is good. – DevSolo Nov 12 '10 at 16:27

I've been working with a few state institutions and they are amazing at wasting money on IT. From buying bloated middleware to solve extremely simple problems to paying thousands and thousands of dollars to a vendor to have them create a CSV. Without in-house people with sufficient experience it seems they either get fleeced on the upfront cost or on the maintenance.

share
    
+1 for the bloated middleware to solve extremely simple problems observation. What could state institutions do in the absence of good internal people? – Gary Rowe Nov 12 '10 at 17:47
    
The problem is that state institutions have bad management. – syrion Sep 27 '11 at 1:23

In non-software companies (banks, insurance) with in-house IT the money comes from various business groups. The business groups directly gets sales pitch from vendors and will push it on to the IT. They are paying for the software/hardware and your salary so your protests will go no where.

  • Paying for bloated applications and middleware that costs mid-high five figures and doesn't even fit within the existing system architecture
  • Using expensive software like HP QualityCenter, BMC Remedy, HP LoadRunner etc where better and cheaper options are available
  • With multi city teams a lot of travel costs, sometimes for only a few hours of meeting
  • Paying for windows 7 license that somes with new machines and then paying again to downgrade to windows XP as the new SOE (designed in 2010) is still XP
  • Over capacity in hardware
share
    
Have you been able to give them advice for better, cheaper solutions with any success? How did you manage to convince them? – Gary Rowe Nov 13 '10 at 17:24

I work in the performance testing profession and I witness (literally) millions of dollars a year being flushed down the drain by organizations for four reasons

  1. Hiring an outsourcer based on price alone, not qualifying skills and not regularly auditing the skills of the performance testers. Hiring an amateur performance tester is a lot like hiring an amateur plumber or an amateur electrician, it will take them a lot longer to work through basic tasks, a lot of checks and balances in process are lost, and when you do find out just how bad they were its horribly expensive to fix (in production). As a moderator for half a dozen forums in this field I observe regularly people showing up who lack fundamental skills in testing, communication, project management, development, systems analysis, etc... and they have simply been thrown at a tool. To the person that noted LoadRunner as a waste of money earlier, if you throw a fool at a tool there is only one result you should expect. The irony is that open source tools require an even more mature skill set to be successful with them.

  2. Not collecting performance requirements. This impacts the whole organization because you will have a different perspective on performance in Architecture, Platform engineering, Application Engineering, Functional QA and performance QA, none of which may actually match the business stakeholders (and frequently doesn't). This is a process problem for in many organizations the performance test team is asked both to collect the performance requirements and to test against them. For proper checks and balances you should do one and not the other. Related to 1 above with immature staff you will have people who cannot even recognize a proper performance requirement, do not have a measurement point to validate against with a load profile, and yet they are still building "scripts to run." This is a collosal waste of time and effort and does little to improve quality. Performance needs a common perspective across the organization and is not something that can just be tacked on at the end if it wasn't engineered in to begin with.

  3. Performance Test Environment Management. I cannot tell you how many organizations are delayed to to test environments not being ready to run at the time that the test organization is ready to proceed. Just in one client I can see this as a multi-million dollar problem in terms of hours lost while waiting

  4. Project managers who have no understanding of what performance testing is, what tasks are involved or the level of effort in place, but who are dictating how long the activities should take place. This leads to variances in the project schedule which are entirely related to how items were scheduled (and cost overruns as a result). This is directly related to 1 above as well for immature testers are not able to accurately project either the number and types of tasks nor how long the tasks should take. It is an axiom that if you allow someone who does not understand what you do and why you do it to dictate how you work and how long you will take, then this path will lead to failure. It happens all too often in performance testing.

share

Proprietary version control systems. Given the state of Git and Mercurial, I don't see why people would go for something with a gate keeper.

Not only do you have to pay for the VCS, you also have to pay per user. Additionally, your flexibility gets shot in the foot. You might as well wear a T shirt that says "I ♥ Vendor Lock In!!!"

I feel that it's just nuts these days to not use a free (D)VCS. If you want lots of add on perks to go with it, stuff like Kiln is available.

I don't think I'd go to work for someone that insisted on BitKeeper or similar.

I almost said the same thing about emulators, but products like Simics continue to offer significant advantages over free alternatives.

share
5  
Sort of have to disagree: after using a proprietary and expensive Version Control System and then moving on... by comparison, CVS and SVN are horribly horribly sucky. The system I used was expensive, and by heck we used it hard. And it stood up for year after year, simple to understand, use. No experience of Git or Mercurial (which seem to be todays big things), but some of the other free stuff is just hideous. Once you have used quality, going to something lesser is HARD. – quickly_now Nov 13 '10 at 10:55
1  
@quickly_now - Which VCS was that? I've used a lot over the years & haven't found anything free or paid that I'd use in preference to Hg – mcottle Sep 1 '11 at 1:12
    
Sitting down... IBM/Rational ClearCase. On really big multi-projecct / multi-branch developments, its expensive, takes a bit of learning, and I'd go back to using it in a shot because its so damn good. – quickly_now Sep 1 '11 at 5:34
    
Borland StarTeam was pretty awesome, and was light years ahead of the free stuff when I used it wayy back in 1999. – Neil N Sep 27 '11 at 3:41
    
@Neil N Did you stick with it? – Gary Rowe Sep 27 '11 at 9:29

Status meetings & weekly reports

An organization I worked in was all about weekly status reports - rolled up at 3 different levels. The dev leads and test leads for each of the 4-6 projects in flight report their progress in a lengthy email, which then get rolled up by the next manager up, which in turn gets arbitrarily summarized by the next one.

The following business day, all project leads gather in a 1 hour meeting to go over the report.

Effectively one day each week is spent on reporting that week's progress. Keep in mind that this is all separate from the daily standups and weekly demo / retrospective meetings.

share
    
The other thing I wonder is how accurate the reports are, after going through two layers of management. – David Thornley Nov 29 '10 at 16:00

I work for a public body. There's really no way to adequately explain the level of waste that can go on when the workplace is so heavily legislated and unionised that sacking someone is practically almost impossible.

Managers play pass the parcel with bad staff, and hope to remove them all at once under cover of restructuring. Some bad staff are promoted, just to move them out of an area that needs improvement. Any good staff end up struggling constantly just to make up for the bad staff's work. Staff you wouldn't keep for 3 months forge 40 year careers. The amount of money they waste over such careers is astronomical.

I worked in the private sector previously, and saw a lot of waste, but public sector waste is a whole different sport, let alone ballgame.

It was suggested in a comment that establishing sinecures for underperforming staff would help. It would help in that it would limit the damage they could do, but wouldn't impact the root causes of the problem. I think the best thing would be the adoption of some private sector hiring and management procedures, and changes in legislation to make it easier for public bodies to let staff who underperform go. Unions should also change their policies in consultation with the government - their role of protecting their members is important, but they should recognise that sometimes their members are truly out of their depth, and should be moved on

share
    
Can you suggest any strategies that may help to reduce the waste? Perhaps creating sinecures for bad staff - would that help? – Gary Rowe Nov 29 '10 at 23:16
1  
It would help in that it would limit the damage they could do, but wouldn't impact the root causes of the problem. I think the best thing would be the adoption of some private sector hiring and management procedures, and changes in legislation to make it easier for public bodies to let staff who underperform go. Unions should also change their policies in consultation with the government - their role of protecting their members is important, but they should recognise that sometimes their members are truly out of their depth, and should be moved on. – Dan O Nov 30 '10 at 9:52
    
+1 for the strategies - you should edit your answer to include the comment otherwise it may get lost in the froth – Gary Rowe Nov 30 '10 at 11:32
    
"pass the parcel" - we used to refer to that game as pass the trash. – HLGEM Aug 31 '11 at 19:50

One project I worked on with a large financial institution. There were huge amounts of conference calls daily, and I estimated that they burned about $100k per day just on conference calls. The project lasted about 2 years. They had tons of legacy systems and when the daylight savings changes were made a couple years ago, they paid Microsoft about half a million dollars to come up with a DST patch for NT 3.51.

share
    
Just to clarify - this system was still running NT3.51 in 2008? Sheesh kebab. – Gary Rowe Nov 12 '10 at 17:40
    
@Gary, they had some NT 3.51 based computers in their data centers. The story was that the software running on those servers was not certified for running on anything newer. The system I was part of was a huge unix/windows project, and they finally got off of SQL Server 2000 in 2008 (and replaced them with SQL Server 2005). Meh. Finance industry. For all the talk of modern technology, it would not surprise me to find abaci, clay tablets and punch cards tucked in back behind all the fancy blinkenlights. – Tangurena Nov 12 '10 at 19:10
1  
+1 just for use of abaci ;-) , but overall it seems that the "too expensive to rewrite" anti-pattern is at work here. What do you think? – Gary Rowe Nov 13 '10 at 1:46
    
@Gary, they had money to burn, after all it was the fees from people who invested in mutual funds paying for it. Sometimes they just didn't have time to rewrite applications, and had delayed XP SP2 for a couple years as the changes to the networking stack broke too many custom internal applications (they've since moved to Win7, bypassing Vista). For the systems that I was involved in, there were 4 parallel stovepipes of integration/system testing (for quarterly releases) even though it would take 6-8 months to squeeze through one of the stovepipes. – Tangurena Nov 13 '10 at 3:10

We were having a smallish amount of work and barely making bills and payroll in a small shop in which I worked. The solution: hire an efficiency consultant and a personal secretary for the boss so he could go perform more "meat and potatoes" work.

Solve a budget shortfall by increasing spending...fail.

On the plus side - the efficiency expert supplied a dry erase board where we tracked our billable hours and paid hours...guess who had the least amount of billable hours.

share
    
This is standard operating practice. I'm surprised the boss did any work at all. Half the owners/execs I've worked for seemed to not actually DO anything all day except look important, if they were even in the office; half the time they were out, presumably sleeping or playing golf or on the yacht. – Wayne M Aug 31 '11 at 18:44

Let's see, we once spent well over half a million dollars doing the work to win a million dollar contract. So much for the profit on that one. Some of us on the project proposal development team tried to point this out but it had become a thing of pride for our little company to win over the Fortune 500 companies we were competing with. We did win and lost money hand over fist onteh contract for that and other reasons but we had bragging rights.

As a government contractor once, I was forced to work unpaid overtime because the contract allowed it and the contractor got paid for my overtime. Not only that I was caught up on my work and spent 4 hours every Sunday surfing the Internet with no work to do. Needless to say I moved on very quickly after they started that nonsense.

Buying Clarity as our project management system, a commercial app that is so bad, 100% of the people who use it have begged to go back to our old home grown system (the one guy who liked and chose it has moved on to some other company), people even volunteered to work on their own time to add the reporting they wanted to our old system. But we have invested the money so we are stuck with it. In other words, refusing to ditch something that doesn't work just because it was expensive.

share
    
+1 for a clear case of "it's too expensive to ditch" even though it causes more problems than it solves – Gary Rowe Nov 13 '10 at 1:50
    
Ah yes I love the "Our tiny 5-person company can compete with 1000+ employee corps, even if we don't make a dime!" type if idiots that always seem to think they're amazing entrepreneurs. – Wayne M Aug 31 '11 at 18:37

Sheer waste. An IT spend that had to be cut by many millions. So the way to do this was to fly the IT people in from all over the world. Put them up in a flash hotel for a week. Then in the building where the meetings were held, lay a new floor. Marble of course. And overnight, between the meetings each day, the building was redecorated. Thats every evening for a week.

Err... priorities anybody?

Fantasyland.

share
    
A waste of money, but imagine the cool factor if you redecorated the office each day with different "themes". One day it could be a jungle, one day a medieval castle, one day a cave. That would be awesome. – Wayne M Aug 31 '11 at 18:45
    
And doing that whilst planning all the jobs you were going to cut and the people you are about to sack. Hmmm. – quickly_now Sep 1 '11 at 5:38
    
Well yeah, that's still a crappy thing to do. But the idea of having a different themed office every day or every few days is really cool for a startup premise. Get a reputation like Google for being a cool place to work :D – Wayne M Sep 1 '11 at 12:23

The company I work for paid $800 for a CHART FX License - It's not even my money yet I feel robbed.

http://www.softwarefx.com/sfxNetProducts/ChartFX/

Just for kicks, their software will place files all over the place including the registry and program files.... yep all that for some naff looking charts.

share

Not the answer you're looking for? Browse other questions tagged or ask your own question.