What skills do you think a performance engineer should have?

Question

I believe performance tuning (or even testing) is one the most challenging for an engineer. Well, in lots of company, this is the lowest priority than others "important" thing.

My purpose of opening this post is to know what do you think*good* performance engineer should have.

I can list some things like:

1. Solid database,programming knowledge.
2. Do single thread performance testing.
3. Good knowledge of using the load generator tools to simulate the concurrent loads.
4. Use different tools to monitor/measure the app/db server performance status
5. Understand and can debug the codes. Even tune the codes.

Any more ideas are always appreciated!

Answer 1

I run a consulting firm (80 people) uniquely focusing on Performance Engineering. My opinion is that performance engineering is extremely complex because it cannot be seen as a Sw engineering matter only but it is a multidisciplinary science that is extremely difficult to master.

Performance best practices (and so optimization initiatives, when needed) span across the whole IT "stack":

• application design and algorithms
• application code
• framework code
• platform (e.g. Apache, IIS, ...)
• data access
• operating system
• hardware infrastrutture and processing distribution
• network

The difficulty of the role resides in the fact that there are typically multiple ways to get to a result (e.g. Code optimization vs. Capacity plannning) with competing technical time and economical contraints.

So the best performance engineers are smart, flexible, trained in multiple disciplines and need strong communication and problem solving skills to succeed in their job.

Answer 2

Add to your list a solid understanding of algorithms. That is a must.

Answer 3

I think good performance has a lot to do with

• software architecture and desingn decissions (i.e. support for scalability in a Serverfarm)
• database optimisations (i.e. indexing)
• load balancing on high volume websides
• product knowledge with their pros and contras

Answer 4

• Knowing that profiling comes first. Too many developers start optimizing what they think is the problem- and they are often wrong.
• Being able to understand profiling output and generate "real load"
• General knowledge of parallelization techniques, RDBMS and operating system behavior
• Good eye for algorithms and caching

More often than not, I've seen that big optimization gains are easy; just find the hot spot, and try different approaches. In most shops, it's rare to need thorough optimization- thoughtful coding from the start normally means that performance problems are far and between (esp. in this age where fast IO is relatively cheap and most working sets fit into RAM).

Answer 5

A general understanding of how to access the configuration of the software being tuned,e.g. IIS, along with what parameters are there to improve performance such as what hardware changes may improve performance or what configuration changes in software may have a large impact. This could mean getting to know something as general as an operating system or as specialized as a customization on top of off-the-shelf software.

Knowing how to analyze data for patterns would also be a standard requirement to handle things like extrapolation that someone may ask.

I'm used to thinking of this as a "Systems Engineer" who has some system administrator responsibilities as it is production that generally is getting tuned and not some lower environment. There could also be configuring network appliances like load balancers or firewalls for this person in addition to server configurations.

Nice to have:

ITIL's Service Asset & Configuration Management process experience and knowledge

Answer 6

Math, modeling, and statistics are very important. You gotta be able to figure out trends, and understand the difference between errors and residuals.

In addition to this they've got to understand how the whole system works, all the way from how many spindles in a RAID array, to how are the worker threads in the application working out shared resource contention.