Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

We build some software platform, and now we need to make hardware sizing sheet for the server-side piece of this software. So, I should prepare some mathematical model that, for example, given the number of concurrent users, can predict the number of physical servers or storage capacity or network bandwidth. How to make such model?

share|improve this question
1  
Whatever model you use, it will be defeated by the reality. The best is to make a scalable system. –  mouviciel Mar 20 '12 at 9:08
1  
I agree. But when I sell my software, I should tell the customer how much will be the cost of hardware. Microsoft, IBM, SAP, Adobe offer their customers some calculations of hardware based on business parameters (kind of user activity, number of concurrent users, etc). I want to make the same model for my software. –  Pavel Surmenok Mar 20 '12 at 9:32
1  
One obvious solution is prototyping. Run your software on some hardware and add concurrent users until performance requirements are no longer met. Then upgrade hardware and add new concurrent users. With few iterations you may estimate both initial cost and additional cost per user. –  mouviciel Mar 20 '12 at 10:15

3 Answers 3

up vote 2 down vote accepted
+50

You may want to start with a mathematical model, but I'd advise instead to do a simulation - make a "user": a program that acts like a user and generate randomized user-like activity. Then, load the system with a real-life like dataset (sized accordingly) and deploy your users against the structure you created. Rinse and repeat until you get as many measures for every data point so that you have a reasonable confidence the number represents the performance at a given load and that noise has been reduced to a level that doesn't interfere with your measurements. Keep an eye on maximums, because they may appear in real-world usage and derail the application. Also, remember to test it with absurd numbers of users and absurd volumes of data, as it often offers you some insight on problems that may not be evident on more normal scenarios.

If possible, publish alongside your chart the methodology you used (how your users behave, what hardware did you use - if real metal, if EC2 instances - and how many measurements you made).

This is a highly worthy exercise, BTW. You may discover some bottlenecks you never imagined you'd have, as well as new ways to tune your application on different hardware configurations that never occurred to you.

I come from a web background and I try to make every system I build capable of generating one or more JMeter configuration files for load testing purposes. Nobody knows how to better simulate load than the system itself - it has access to previous usage patterns, as well as to its own working database.

And then when you have a ton of experimental data, you can refine your mathematical models in order to predict conditions you may not easily simulate.

share|improve this answer

I think the advice to prototype is right on. Run it, see how it runs with X users. Then, add in a "fudge factor" on top of your results so that you have some breathing room with your customers. So if your test results find that you need 512 MB of RAM for 100 users, write 768 MB or 1 GB in your recommended specs.

The big guys (HP, Microsoft, Oracle) all have recommendations. Routinely those recommendations are either loosely adhered to in development and test environments, or ignored completely because the hardware used is "whatever is available". Just a thought about that.

share|improve this answer

A "purely mathematical" approach is unlikely to be very accurate. One of the basic questions to ask when optimising software for performance is "have you measured it"?

My advice would be to perform some stress testing of your software. Do this in a carefully controlled way, and observe the behaviour of the system while it's under different levels of stress. This information, combined with your inside knowledge of the system itself, should allow you to come up with some educated guesses about the way your system will perform with different hardware and different loads. I would advise against trying to extrapolate to hardware that varies wildly from your test setup - it's one thing to predict what will happen if you go from two cores to four, but will that scale to 64 cores, or make efficient use of hundreds of GB or RAM? I would stay away from wild guesses like that.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.