Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I understand that to measure a project or code, we can use The Joel Test, but is there any simple standard test (like The Joel Test) that is able to measure and filter how good a programmer is?

My plan is to have this test as a quick filter first before going to a more detail test.

share|improve this question

closed as primarily opinion-based by gnat, Giorgio, MichaelT, Bart van Ingen Schenau, GlenH7 Jan 25 at 0:32

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

If there are tests like this that are being used, I think programmers should know about it. They may or may not be valid like: Hire the one with the most body modifications. –  JeffO Feb 15 '12 at 14:39
Interesting, when I asked this question, it was downvoted to hell (now positive again, yay, and quickly closed). Is it really so different than this one? programmers.stackexchange.com/questions/133691/… –  ripper234 Feb 15 '12 at 15:43
@ripper234, reasons for a question being closed on SE are a bit like those unexplainable, intermittent software bugs - a mystery really. A bit like human nature really. –  tehnyit Feb 15 '12 at 17:41
Joel himself provides a simple-yet-rigorous test: Smart and Gets Things Done. :P –  Dan J Feb 15 '12 at 20:27
FizzBuzz! Surely! –  CraigTP Feb 16 '12 at 10:56

7 Answers 7

There is the programmer competency matrix.

As with the Joel test it's just a vague guide. The only way to properly assess a programmer is to ask good programmers who have worked with them.

share|improve this answer
So the only problem left is to assess whether the guy's past/current teammates are good... Oops. –  Péter Török Feb 15 '12 at 11:42
Yes, its is recursive :) –  Tom Squires Feb 15 '12 at 11:50
That's nothing like the Joel test. Your answer points to a huge detailed matrix, Joel's test is a series of 12 very simple questions to answer. –  Bryan Oakley Feb 15 '12 at 12:14
@BryanOakley - that's true, but the PCM is also the very first thing I thought of when reading the question. The upshot: there are no simple questions you can answer to measure a programmer! –  Joris Timmermans Feb 15 '12 at 13:39
@BryanOakley point of matrix being more complex than series is well taken; to me closer analogy to Joel's test would be a series made of Level 1 column elements in PCM - "explain and use Arrays..., Basic sorting... etc" –  gnat Feb 15 '12 at 14:51

I would turn the Joel test around:

Have they used source control?

Do they know how to automate a one step build?


The only one that doesn't seem particularly applicable is the have testers question. The others that seem kind of off turn in to a this is how we handle it how have you handled it in the past sort of questions(This is how we handle keeping our schedule up to date how have you handled scheduling in the past?).


Basically you don't get the stuff in the Joel test for free, you have to hire people who can make it happen. You want to establish their ability to make that happen.

share|improve this answer
+1 this is a smart answer. :) –  Rudy Feb 20 '12 at 10:12
All of Joel's questions are more about the environment than the programmer. If my team doesn't use source code control, then my not integrating with them by using my own source code control is hardly an improvement. Getting the team to use source code control is an improvement. –  Edwin Buck May 1 '12 at 22:37

The Joel Test is just an informal baseline check to quickly judge whether a place has good working conditions for programmers. Even if it scores a perfect 10, it can still be a hell-hole that is going to go bankrupt six months down the road. A low score is an indication of something not quite right, and makes for excellent interview questions ("You're not currently using source control; are there any plans to do so in the future?"), and the answers might be such that you'd accept the job despite a low Joel score.

The Joel Test is not a 'standard' test either; it's just a checklist Joel Spolsky posted on his blog.

As far as 'measuring' the quality of a programmer goes; unfortunately, the really important skills and qualities of a good programmer are hard or impossible to quantify, so there is no replacement for thorough human evaluation. You can weed out the completely clueless candidates quite easily though, using a very simple programming task - ideally, something that involves recursion, tree structures or pointers (a programmer who doesn't 'get' these is unlikely to be of much use). For those who pass this test, you'll have to evaluate skills manually: read code they wrote, test drive applications they wrote, give them more programming tasks (both design and implementation), watch them work, talk to them, see if you can spark a professional discussion. If you're looking for a specialist / language guru, you might also want to ask a few knowledge questions, but for a general programming candidate I wouldn't bother.

share|improve this answer
+1 Judging the skills of a good programmers is one task is hard to quantify. –  Karthik Sreenivasan Feb 15 '12 at 11:41
("You're not currently using source control; are there any plans to do so in the future?"), and the answers might be such that you'd accept the job despite a low Joel score. You would be making mistake to accept the job by the way. Eventually every developer learns that Plans to do so in the future is just something that interviewers say to deceive you but that they never act on because of terrible management. How many times have we heard something to the effect of Oh, we are moving towards Agile... and it turns out to be yet another micromanaged waterfall shop? –  maple_shaft Feb 15 '12 at 12:00
@maple_shaft: yeah, probably not a good example... –  tdammers Feb 15 '12 at 14:46
A perfect 10 on the Joel test would actually be 12... just saying :) –  MattDavey Feb 16 '12 at 9:36
@MattDavey: That critically depends on your competencies to drive change. I had one of those experiences when I'd been in the business for two years (yeah we'll move to C++) and got the expected result. Nowadays, it would be a different matter. I can figure out if it's a sincere wish but inability to change, and then make it possible. –  MSalters Feb 16 '12 at 12:46


Do you program in your spare time?

In all my experience, this single question is most indicative of how good a programmer is. If they enjoy it; if they have passion for doing the task, then they're going to be good at it.

And frankly, a lot of 9 to 5 jobs don't involve a lot of coding. They don't involve a lot of iterating through the life-cycle of designing new programs and seeing how that design works/fails. Without that iteration, there's simply not the practice needed for programmers to gain core program design skills.

And they don't involve a lot of learning. Programmers who even simply hack on things at home are going to be exploring new and interesting solutions without the constraints of big business.

share|improve this answer
Why does this have only one vote? IMO this is the real distinction between mediocre teams and those that are truly innovative. –  Repo Man Mar 5 at 16:21

Eh, I have qualm with the wording at the start. It's not "have you used X" or "Do you know of Y", it's a matter of actually using and doing. Any programmer that hasn't touched or heard of the items on the Joel test is simply disconnected and needs to get a clue. But you're right, code-shops fail the Joel test because people in the shops let it fail. The only defense I can see runs along the lines of "I tried, but did not have the authority. And now I'm applying here".

share|improve this answer

It's not as detailed as the Joel Test, but asking them to write a fizz buzz program will be a good indication to see if they can code at all.

http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html and http://imranontech.com/2007/01/24/using-fizzbuzz-to-find-developers-who-grok-coding/

That will not tell you about the software engineering maturity of the individual, but it will screen out the worse.

share|improve this answer

Do you use source control?

Yes, but

  • It doesn't really tell you anything.
  • How do you know whether I forward merge?
  • How do you know whether I pull changes before pushing?
  • How do you know whether I build before committing to repository.

Can you make a build in one step?

  • Yes, our CI lead writes scripts and I just run them in powershell.

Do you make daily builds?

  • Our CI server does

Do you have a bug database?

Yes, but I haven't configured it and I don't administer it, I simply use it.

Do you fix bugs before writing new code?

  • In a perfect world where I have unlimited resources - yes I do. In real world I'm sometimes forced to log them and work on something else.

Do you have an up-to-date schedule?

Nope, that's not my job.

Do you have a spec?

I get given a spec, I then analyse it and produce relevant documents.

Do programmers have quiet working conditions?

  • Will you not hire me if I listen to music, talk to my colleague and make a joke? Software development supposed to be creative - working conditions will vary from one organisation to another.

Do you use the best tools money can buy?

You don't know what the best tools are and if you think that you do, there will always be someone to argue your point.

Do you have testers?

Yes. Actually, yes and they are not very good, but that wasn't in the question.

Do new candidates write code during their interview?

Yes and they fail. Yes and they pass. What does this tell you?

Do you do hallway usability testing?

No, but what if we do something better?

To conclude:

  • This test might work well in a perfect academic world where everything just works, everybody gets along, shares knowledge and there are unlimited resources.
  • What you want to know is what kind of engineer I am. Simple bitwise answers won't tell you anything useful and I think that somebody has to be very naive to fall for them.
  • Answers above, both negative and positive don't tell you anything about my ability to produce clean code.

This is not a rant, but I would be very interested to hear what kind of developer you think I am based on the answers that I have provided. This will hopefully prove my point.

share|improve this answer
I don't really get the point of this. These questions are about the team/company, not the individual programmer. And the OP didn't suggest using the same questions to asses the programmer, he just wants a set of simple questions. –  CodesInChaos Apr 24 '12 at 21:18
I think that I am as good as my team is, or the company that we work for. In my opinion tests like this make a lot of fuss, but they don't really tell you anything useful about the company or the developer. –  CodeART Apr 24 '12 at 21:20
How do you know whether I pull changes before pushing? Well, I don't know what source control you're using, but in SVN at least, if you try to commit to a folder with changes that you don't have yet, the commit will fail until you run Update. –  Mason Wheeler May 1 '12 at 21:31
We are using TFS :) I really need to get my hands on other version control systems. –  CodeART May 1 '12 at 21:36
Soooo people actually do use TFS. I learnt something. –  Fabinout Jan 20 at 16:21

Not the answer you're looking for? Browse other questions tagged or ask your own question.