Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In theory, customers should be able to feel the software performance improvements from first-hand experience.

In practice, sometimes the improvements are not noticible enough, such that in order to monetize from the improvements, it is necessary to use quotable performance figures in marketing in order to attract customers.

We already know the difference between perceived performance (GUI latency, etc) and server-side performance (machines, networks, infrastructure, etc).

How often is it that programmers need to go the extra length to "write up" performance analyses for which the audience is not fellow programmers, but managers and customers?

share|improve this question

10 Answers 10

up vote 19 down vote accepted

Although @jwenting makes some good points, I have to disagree with the general assessment.

A user often does not notice minor performance improvements.

With that, I can agree.

Where I disagree revolves around this statement:

most end-user facing applications spend most of their time waiting for user input.

Now, before you jump up and down, I agree with that statement too! However, this statement highlights a fact often overlooked by those who do not adequately understand how a user really perceives a system.

A user will notice that an application is slow when they have to wait for it to load. A user will notice it when they have to pause for the program in between entering their data.

Software performance is evident for a user when it breaks up a natural and fluid interaction with the system.

A user will only not notice system performance when it is functioning perfectly and not holding up the user.

share|improve this answer
5  
Unfortunately some programmers feel the need to play to their users' expectation of waiting. I saw this in production code the other day: Thread.Sleep(1000); //pretend this does more than change a 0 to a 1 in the database. – Ben L Mar 8 '11 at 12:01
    
That's were code review by reasonable developers should step in. That, or the people further up the food change should have their decision making licence revoked. – Dan McGrath Mar 8 '11 at 12:08
2  
@BenL on the other hand I experienced the opposite, some operation that was slow before we made fast, so fast that users thought the action either hadn't executed or failed. – Pieter B Apr 1 '15 at 6:28
2  
@BenL: Thankfully, modern UX explicitly recommends using animations over inserting arbitrary delays. – rwong Apr 1 '15 at 6:44

Some performance enhancements aren't noticed as performance. The customer will just notice that the system "feels" nicer.

The subconcious works at speeds way faster than the concious. Our brains are programmed for instant feedback and when faced with a sequence of tasks we need to churn through them one after the other. A slight pause in feedback causes this process to become disjointed, which increases stress levels. People will automatically double click a button without thinking about it if there is a pause in the feedback.

share|improve this answer
    
Yes, but there's limits. Humans won't notice things much faster than a tenth of a second, so any response of 100ms or less is golden. Getting the response down from, say, 250ms to 100ms is going to make things feel a lot smoother. Going from 100ms to 40ms won't have nearly the same effect. – David Thornley Mar 7 '11 at 18:53

Quite often performance improvements are so minor the customer never notices directly. At best, they may have a slightly more fluent application flow over their use of it, but not enough to be consciously noticed.

Remember that most enduser facing applications spend most of their time waiting for user input, not processing that input. So even if you shave 10% off the 100ms needed to process that button click and update the screen, the user will barely notice as she'll not do anything with that updated screen for another 10000ms after.

Who will notice is the sysadmin who sees a batch job that used to take 2 hours to complete now complete in 90 minutes, but he'll only notice that if he has to wait for the result and migh get angry the faster return interrupts him halfway through his movie :)

share|improve this answer
    
+1 for interrupting sysadmin's movie. :) – Michael Kjörling Mar 7 '11 at 9:48
    
From the sysadmin's point of view, that may also mean having to have three rather than four servers up to handle the load, and that's significant. There was also that place I worked at where the daily run took 18-20 hours, assuming nothing went wrong. The manager would have loved to cut that down. – David Thornley Mar 7 '11 at 18:56
    
yes, that's the extreme cases. Worked on one such where a job that needed to run once a day required 36 hours to complete. Was able to refactor it down to needing "only" 20 hours. Customer was happy :) – jwenting Mar 8 '11 at 6:19

As other's say today, it's more about perceived performance and "fluiditiy" than actual raw speed.

That means that you can get away with having a slow(er) system just by having a natural feel and rhythm in your software's UI, instead of having some things too speedy and others very slow. (As humans, we notice irregularities very well, since it might be a tiger sneaking upoin us...)

This is essential in hiding latencies that you can do nothing about, so it's a good skill to practice.

share|improve this answer

I just wanted to jump in here and offer an unusual case where....

*THE CUSTOMERS FREAKING CARE ABOUT PERFORMANCE AND NOTICE EVERY LITTLE CHANGE!.

It's in my field where we cover production rendering which tends to be analyzed to death in terms of performance by the customers themselves. A 2% slowdown in performance over a minor version can equate to the slowdowns being reported in the form of "bug reports" en masse.

Forum threads are often started with customers benchmarking their scenes against various versions of the software, where the customers are actually benchmarking more than the developers themselves. "This scene took 1 hour and 40 minutes to render in version X. It now takes 32 minutes in version Y."

"This scene took 18 minutes to load in version X, now it takes 4 minutes to load in version Y."

They're extremely appreciative when optimizations are applied, and that alone may suffice to warrant a purchase of a new, very expensive upgrade of the software, and sometimes with only modest improvements like a 10% reduction in times.

In some larger contexts, it may also save the customer enormous amounts of money when the product is sped up, since some bigger studios use render farms where they have to pay for hundreds of machines rendering all day long, and any improvement in times here can speed up their whole production process (and possibly even yield better results when artists are more productive creating art rather than waiting for it to render).

So there exist fields like this where the customers really, really, really notice -- sometimes even more than the developers themselves, and this is outside of UI interaction concepts which are more about latency than throughput.

How often is it that programmers need to go the extra length to "write up" performance analyses for which the audience is not fellow programmers, but managers and customers?

In our case, all the time, with just about every minor release. Speed is one of the top selling points, and even the most technical benchmarks and performance analyses are actually appreciated and understood by the customers and managers. The perception of customers is often like rabid wolves, hungry for more optimizations, and trying to make suggestions to the developers about how to potentially make things go faster. In this case, it actually takes discipline to resist some of the customer urges to optimize further and focus on other metrics like maintainability and feature enhancements.

share|improve this answer

The only times I come across are:

  1. The software improves by doing more work in the same time-frame.
  2. Offline rendering or processing is markedly speedier, but unseen.
  3. Speed boost are somewhat nominal but improvements prevent future bottlenecking
  4. Parallel processing which accomplishes the same work at the same speed, for many.
  5. Any time unnoticeable speed increases strongly affect productivity.
share|improve this answer

If the customer won't notice speed improvements, why did the developer work on them? There is probably a good reason. Why monetize that work if it is transparent for the user?

An example: Apple charges about $130 for each Mac OS X upgrade. Except on Snow Leopard which is sold $30. Developers have worked hard on that version but there is very few improvements that are visible from the user viewpoint. So Apple decided to charge a minimum.

share|improve this answer

How often is it that programmers need to go the extra length to "write up" performance analyses for which the audience is not fellow programmers, but managers and customers?

I believe it depends upon the industry. In the wacky world of defense contracting, we do this fairly frequently. We have specific requirements to make the products perform in certain ways - and these performance metrics are not always directly related to something an end user experienced. And we generally do our own testing to see where the product bottoms out. Both types of tests are written up in reports with some serious thought on what it means.

That said, I'm not sure it holds true in situations where the customer and deployment base is less specialized (ie, the commercial world). Given that we do purchase COTS which needs to meet performance specs, I can say that some purchasers do ask for such performance specs, but in my experience, the COTS companies I've gone to don't always have so many performance analysis white papers available. It seems to depend on the industry, the size of the company and the nature of the competition. Ah... capitalism.

share|improve this answer
1  
Having supported a lot of COTS products, I can assure you that performance is not anything they care about. The users do care when they are on a call to a customer and it take ten minutes to move from one screen to the next (Actual case I dealt with of a poorly designed COTS product that costs over 100K). But COTS manufactures don't directly deal with the irate users and thus it isn't important to them. – HLGEM Mar 7 '11 at 19:41

My opinion is if the performance gains are not noticeable, then they are not marketable. In other words, why would someone pay more for software that is not noticeably improved?

I think marketing claims of unnoticeable performance improvements have led users in general to give little weight to such claims. For example, when I wanted to start using distributed version control, I ignored claims of git performance because I believed the differences would be negligible. It was only through trying it out for myself that I found they were credible, especially when combined with inotify support.

I will make an exception for performance gains that are not directly related to the end user experience. For example, server throughput would matter to people who purchase and maintain servers, even if the end user doesn't notice a difference. In that case, a simple "percent improvement over X" is sufficient.

share|improve this answer

It depends who you are selling your software product to.

More often then not, your customer is not the end /day to day user. So often you end up with making more nice and shiny reports instead of fixing performance issues. Because really you're selling to management, not end user.

Which means in that case, you will be hard pressed to mark up for some performance issues but will make top dollar on automating that report.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.