Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My boss is planning to use the metrics from our continuous build (builds and runs tests on every commit) as part of our performance reviews (in our 'quality' metric). This seems like a REALLY bad idea to me, but I'd like to know if anyone has studied this or seen this tried before.

My thought is that it's going to have our developers not put in as many tests as they otherwise would, for fear that the tests are going to fail. I feel that he's turning a valuable developer tool into a stick to beat the developers with.

The obvious counter argument is that it will promote people being more careful before they commit, and therefore leading to higher quality.

Am I off base here? Please leave aside the question of whether we should be doing performance reviews at all or not - that's been answered elsewhere.

share|improve this question

closed as off-topic by Snowman, durron597, GlenH7, Dan Pichelman, MichaelT Apr 16 '15 at 2:22

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions seeking career or education advice are off topic on Programmers. They are only meaningful to the asker and do not generate lasting value for the broader programming community. Furthermore, in most cases, any answer is going to be a subjective opinion that may not take into account all the nuances of a (your) particular circumstance." – Snowman, durron597, GlenH7, Dan Pichelman, Community
If this question can be reworded to fit the rules in the help center, please edit the question.

Any system that can be gamed is a horrible input for a performance eval. – Steve Jackson Aug 10 '11 at 18:20
Everyone has the option not to test? – JeffO Aug 10 '11 at 18:56
@Steve, And "systems" that can't be gamed give you a tiny narrow view of the bigger picture. To truly accurately track performance would require leg work. – maple_shaft Aug 10 '11 at 19:22
Note that some things work well on developers machines, but fail on the build server (an accidental dependency on an external jar, wrong way to use / and \ on Linux boxes etc). The primary reason for the build server is to catch these things, not to harass anybody for not testing for them first. In other words, this is a bad idea. – user1249 Aug 10 '11 at 19:27
Follow up: After we started doing this I found that the biggest problem had nothing to do with the other engineers and willingness to write proper tests, but rather with the fact that our existing tests were REALLY unstable, so every commit had a pretty large chance of breaking through no fault of the person doing the commit. This factor deadened everyone's enthusiasm for testing far more than any effects of the performance reviewing. – Michael Kohne Dec 11 '11 at 3:07

Performance reviews are fine but how about useful metrics like:

  • Percentage of Unit Test coverage on features
  • Ability to meet deadlines
  • Clear and concise documentation
  • Follow proper coding conventions
  • Communicates well with others
  • Ability to turn requirements and user stories into tasks

These are all good ways to measure performance but the problems that management seem to have with these are that they actually require... ummm.. well ya know... ACTUAL WORK on their part.

Unfortunately most management has the attitude, "To hell with that, I want to judge my employees on metrics that don't actually require me to keep up with what they are doing."

share|improve this answer
+1 for providing some good choices of what metrics are useful. – David Ruttka Aug 10 '11 at 19:27

Gaming the system here is quite likely, in my opinion, and your boss has to find ways to prevent that from being the reality. The other case you didn't mention is where developers commit tons of times so that there is this flood of check-ins where the number of modifications is relatively low as if there is some part of the review where the count of builds is used this is where this becomes a new tool that could be misused rather easily. I'm thinking of check-ins where something is renamed or white space is changed is a check-in and counts as some form of productivity would be the pedantic view.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.