Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

From the Joel Test, of the 12, which do you think are the absolute must-haves to at least have a decently running software department/company?

I realise there is no absolutely right answer. I'm just trying to get opinions of others out there. My own organization only manages a measly 5 of 12. If you check listings on Careers 2.0, most companies don't score a full 12 either but I'm sure they're doing fine.

Does SO publish the stats for those anywhere? Or has anyone tried scrapping the results? Would be interesting to know which are practised the most. And whether because they are easier to implement or whether they actually have the most impact.


EDIT: These are the ones practised by my company and for the given reasons

  1. Having source control. - Have to agree with one of the answers that that is the one I would not negotiate about.
  2. Fix bugs before writing new code. - Matter of principle but not a must-have.
  3. Have a spec. - In a small software house with tight margins, specs save life. But I could see it done away with in other companies without affect.
  4. Candidates write code during an interview. - Saved us hiring the wrong candidate many many times over. This is the other non-negotiable must-have for me.
  5. Hallway usability testing. - Easy for us. Our whole office is a hallway. It's really easy to grab people anytime. But that also means we don't have a quiet working environment for developers.
share|improve this question

closed as not constructive by gnat, Walter, Glenn Nelson, GlenH7, MichaelT Feb 19 '13 at 15:16

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. If this question can be reworded to fit the rules in the help center, please edit the question.

this question IMO has some serious problems. For one thing, it can grow without bound: everyone can have their own list of which items are "bare minimum". Second, it is asking 3 questions in one: which are absolute must-haves / soliciting opinions / any stats? You should refocus this on scraping the statistics which is at least answerable and would provide data. Otherwise this would be best expressed as a poll, which is not what we do here. –  Jeff Atwood Mar 14 '11 at 7:51
“My own organization only manages a measly 5 of 12.” — which ones? –  Paul D. Waite Mar 14 '11 at 9:06
As I recall this was Joel's list of "bare minimums". –  Joel Etherton Mar 14 '11 at 15:12
I want to know where these companies that fix bugs before writing new code are. –  Tom Hawtin - tackline Mar 14 '11 at 20:12

10 Answers 10

up vote 72 down vote accepted

Number 1: Using source control.

This is the only one that I absolutely wouldn't, under any circumstances imaginable, ever, negotiate about.

share|improve this answer
These days I would give a company using a non-distributed-SCM a 0.5 instead of a 1. It's hard to go back... :) –  BinaryMuse Mar 14 '11 at 3:52
@binarymuse Amen to that! –  Mike Brown Mar 14 '11 at 5:02
@TokenMacGuy, I don't care whether it's centralised or not. What I care about is that DSCMs are designed for merges, and do them much better than SVN. –  Peter Taylor Mar 14 '11 at 7:14
I went over the list, got ready to post an answer.. and then realized it was the same as yours. Using source control is the only one that I absolutely wouldn't, under any circumstances imaginable, ever, negotiate about. –  Carson63000 Mar 14 '11 at 12:21
It almost sickens me that universities are keen to stuff design patterns and good coding practices down the throats of students, but they will not even mention source control to the vast majority of them... –  Mike B Mar 14 '11 at 14:08
  • Do you use source control? Required
  • Can you make a build in one step? Nice
  • Do you make daily builds? Nice
  • Do you have a bug database? Required
  • Do you fix bugs before writing new code? Nice
  • Do you have an up-to-date schedule? Required
  • Do you have a spec? Required
  • Do programmers have quiet working conditions? Required
  • Do you use the best tools money can buy? Required
  • Do you have testers? Required
  • Do new candidates write code during their interview? Nice
  • Do you do hallway usability testing? Nice

So I would say 7. YMMV.

share|improve this answer
I agree with most of that. I disagree on requiring "the best tools money can buy". I prefer, "use open source when feasible". I also disagree with having new candidates write code during their interview. I've met enough BS artists to want a filter for that. –  btilly Mar 14 '11 at 6:26
The "build in one step" is required to me. The reason is that this is the "is the build automated" question, and automation is the key to reproduceability. –  user1249 Mar 14 '11 at 7:05
Besides dedicated testers being nice, I would agree completely. –  JustinC Mar 14 '11 at 7:48
I feel that daily builds are more important than having testers or a spec... But I'm not in the industry. –  jokoon Mar 14 '11 at 9:39
•Do new candidates write code during their interview? Nice ???? –  Geek Mar 14 '11 at 10:29

That's an interesting question. I have always felt that the Joel Test was a lot like code smell; scoring a 12 doesn't mean that the company is doing everything right, and scoring an 8 can indicate that there are some bad practices but doesn't necessarily mean that the company is run poorly (but like code smell, you should tread warily).

Instead of focusing on the score, I would figure out which parts of the test were important to me and then base my decision on that. If you really wanted to get "scientific" you could give each part of the test a weight or percentage and base it off that.

share|improve this answer
+1. And anything lacking is an opportunity for you to improve the way your organisation works. That's a good way to get the respect of your peers and management both - and surely a good way to be rewarded for your work? –  Carl Norum Mar 14 '11 at 4:48
I agree, although it may not be the best idea to show up on your first day with your guns slinging complaining about how they should fix their problems. :) Make sure to earn your reputation and their trust! –  BinaryMuse Mar 14 '11 at 15:01
complaining is probably the wrong way to go about it, yes. But if you implement any of the missing pieces, you'll be a hero. –  Carl Norum Mar 14 '11 at 18:12

Depending on the company, not having 12 on the Joel's test can be seen as an opportunity for you to provide them with high value that will be probably rewarded accordingly.

Therefore, I personally prefer to work for companies with a low rate as it will imply that I will able to come up with solutions to their problems.

If they refuse those improvements, then it's probably time to move on.

It's easier to do what I describe here if you are freelance which signs a new contract every 3 months. But even as an employee, that could be challenging for some of us.

share|improve this answer

I will make the caveat that it depends what they're bringing you in for. If you are a coder, rather than an architect or high-level developer, you should probably look for a high number since you likely won't have any control over your destiny. They might be looking for some help in getting closer to a 12 though, and that shouldn't be discounted if the company has other great things going for it.

Environment, culture, and willingness to improve from a 0 to a 12 is far more important to me than a company that claims 10+ and employs a bunch of idiots.

With all that said, my beliefs are as such:

  • Do you use source control? YES If this is a software shop with more than one person, this is a must have to organize yourselves effectively. Even one person should probably at least use a DVCS simply for history.
  • Can you make a build in one step? NO As long as it's trivial to do the build, # of steps is not terribly important to me.
  • Do you make daily builds? NO This is not important (IMO) for small teams and certain kinds of software development, and for certain stages of business.
  • Do you have a bug database? YES Although I think it should really be more of a to-do list
  • Do you fix bugs before writing new code? NO This is important in large groups but not a deal-breaker for me
  • Do you have an up-to-date schedule? NO Not important for effective work as long as programmers and project stakeholders meet often enough to be in sync
  • Do you have a spec? YES Preferably written and drawn.
  • Do programmers have quiet working conditions? YES This should really be rephrased to say "option for quiet working conditions", such as working from home, or a spare room for working on difficult tasks.
  • Do you use the best tools money can buy? YES Dual monitors, fast machines (laptops if that's what they want), operating system and software of the programmer's choosing, if possible
  • Do you have testers? MAYBE This depends on the maturity of the product and the size of the company
  • Do new candidates write code during their interview? YES They should write code, solve problems in their head, and be able to discuss theory during an interview
  • Do you do hallway usability testing? YES I like the idea of this a lot, but YMMV greatly on this one, depending on the quality of people in the rest of the organization
share|improve this answer
-1 "Yes. If this is a software shop with more than one person" . . . Sorry, even a shop with one person needs source control. Also disagree with several others, including your opening statement sorry. –  Binary Worrier Mar 14 '11 at 11:31
When you have a wife and kids, you would not want to work from home for quiet. Oh, how little Gracie interferes with my side projects... :-) –  corsiKa Mar 14 '11 at 14:03
@glowcoder: as long as you can have 30+ minute quite periods now and then at home, it's more productive than an office environment. –  Codism Mar 14 '11 at 21:36
As for control of destiny, I think there are some companies that operate like what you described, but if there are jobs for both an architect and a junior at a company, guess who's going to be making those decisions? I just don't want someone to be getting their hopes up about turning a Joel Test score around, only to realize they simply don't have the attention of the people that may be in charge of making those decisions. –  Jordan Mar 15 '11 at 15:04
@Matt: The music I listen to at work is so familiar to me that I'm only subliminally aware of it. It gives my ears something to listen to that will not distract my brain, as it distracts my ears from local noise. I agree ones own office is superior, but in 20 odd years I've never had one, nor do I know anyone who has. –  Binary Worrier Mar 24 '11 at 18:22

I would be perfectly happy to work someplace that scores a big fat ZERO on the test, so long as the work was interesting and pay was right. If anything, I'd be hesitant to work for a company that even knew its Joel score, or worse, bragged about it. That would be an indicator to me of a certain slavish adherence to group-think, which is not the kind of environment I thrive in.

The only items on the test that I would really be concerned about would be quiet coding conditions and a decent spec. But even then, it would depend on what the actual project was.

share|improve this answer

All Joel Test questions are created equally, but some are more equal than others (apols to Orwell).

I do agree that these questions represent a good measure, but not all of these questions should have the same weight. I don't see that something as fundamental as using source control is weighted the same as usability hall testing (especially for small companies that may simply not be able to grab 5 people to test their software). In addition some questions are pre-requisites for other questions; for example no company without source control will be doing daily builds.

For me the most important aspects of the Joel test (which is 10 years old) are:

  • Do you use source control?
  • Can you make a build in one step?
  • Do you have a bug database?
  • Do you fix bugs before writing new code?
  • Do you have an up-to-date schedule?
  • Do you have a spec?
  • Do you have testers?
  • Do new candidates write code during their interview?

As to the importance of the questions, I largely agree with @SnoopDougieDoug analysis.

  • Do you use source control? Required
  • Can you make a build in one step? Required
  • Do you make daily builds? Nice
  • Do you have a bug database? Required
  • Do you fix bugs before writing new code? Required
  • Do you have an up-to-date schedule? Required
  • Do you have a spec? Required - but how much detail is the important issue
  • Do programmers have quiet working conditions? Not required - a chatty office is more productive than a deathly quiet one IMHO
  • Do you use the best tools money can buy? Nice - open source has moved on since 2000
  • Do you have testers? Required - not necessarily in house
  • Do new candidates write code during their interview? Required
  • Do you do hallway usability testing? Nice - if not always practical
share|improve this answer

The real answer depends on your market. If you are doing internal LOB development, consulting, or research, all of which focus on "mostly works right now" vs "works right mostly" then you may be able to get away with lower scores. But if you are in the comercial software business you need a high score.

The bare minimum is 11. And that's only if you are really small (< 5 people) and literally can't afford testers. 10 is a stretch. Anything less and you are bordering on the edge of suckyness. I'm going to focus on the items marked as "nice" in this answer.

  • Can you make a build in one step?

    You can't make daily builds if you can't build in one step.

  • Do you make daily builds?

    If you don't have automated builds, your builds are going to be broken. If your builds are broken, your programmers can't integrate with each other. If your programmers can't integrate with each other, you can't ship on time.

  • Do you fix bugs before writing new code?

    My personal opinion is that the order you fix bugs isn't as important as them actually getting fixed. However, it's much easier to ensure bugs are fixed if you fix them up front.

  • Do new candidates write code during their interview?

    You would be surprised how effective this is at separating horrible candidates from good candidates and good candidates from great candidates. Even simple questions like "write code to convert a hex string into an integer" are really effective. When it comes to judging technical competence there is really no better filter. If you don't employ such a filter than you are relying entirely on luck. That's not to say that you don't want to apply other filters (behavioral and culutral matches), and that there aren't other things that are as equally important as technical skill.

    However, software is completely intangible. It's 100% abstract. When you sell software, you are selling repackaged thoughts. That is, someone took their thoughts and converted them into a format that a computer can understand. Running a software business is about harvesting brain waves and converting them into cash. Someone's ability to translate their thoughts into code is crucial to your ability to convert them into cash.

    Why would you not employ such an effective means of evaluating such an essential skill? More importantly, I would challenge any assertion that there is any other way to evaluate such skills.

  • Do you do hallway usability testing?

    If you don't do hallway usability testing, how do you know that your stuff doesn't suck? If you don't know, as in you have ethnographic evidence (you've watched people using it), that your stuff doesn't suck, then you have 2 problems: your software sucks and you don't know it. Fixing the first problem (having software that doesn't suck) is hard. Fixing the second one (knowing that your software sucks, and why it sucks) is easy. It also happens that fixing the second one will give you a road map to fixing the first. Even if you aren't selling software on the open market (where too much suckyness will damn you) there is a huge business case for investing in usability. Check out this book for more info.

share|improve this answer
I've never had faith in hallway usability testing. I've felt your users will let you know if your interface is wonky (if it makes it that far.) The usability is all in the interface. You'll have many more people working on the backend than on the interface. You'll get just as good (if not better) feedback from a set of alpha or beta users than you will from the other engineers. So why take another developer away from what he was doing when there are a lot cheaper and better ways to get the information he'd give you? –  corsiKa Mar 14 '11 at 21:48
I'm not a huge fan of HUT either BUT one useful thing it does do is ensure your product can sell itself. And we want it to sell itself. If the sales guys have no chance of selling it to people, we might as well stop now. –  Tim Williscroft Mar 30 '11 at 23:28

The hallway usability testing question only makes sense if you are building end user facing software. If you are working on say device drivers or embedded code it makes no sense at all.

share|improve this answer
Sure it does. All software has users, even though the user for a device driver might just be another programmer. –  Karl Bielefeldt Mar 14 '11 at 15:51
I disagree too. There are a lot of ways you can ruin interfaces (talking abstractly) for the person next to you. Your obvious can be different to someone elses. –  burnt_hand Mar 14 '11 at 15:59
For sure, but how you test that interface will be very different, and grabbing random people out of a hall to test your UI does not make sense in some contexts –  Zachary K Mar 14 '11 at 16:50
Hallway usability is a degenerate case of if the target market find it unpalatable, sales will be an issue. If developers don't like the look of the library, who would buy it? If your device driver is not faster & visibly better to an end user than the MS basic functions device driver who would bother installing it ? The device driver also sells the device. If the drivers are bad, the news gets around and the device is avoided. There is always a customer. The embedded code in a device is the user experience (and can break it) Again, sells devices. Sales get us paid. –  Tim Williscroft Mar 30 '11 at 23:32

My current contract

  • Fixes bugs before writing new code (usually)
  • Has quiet working conditions (telecommute)
  • Has the best tools money can buy (Because we buy our own)

The benefits of working from home -- at least for some of us -- far outweigh some of the missing points. Some examples:

  • Folders named "Project X 03302011" kinda work as long as only one person is working on Project X at a given time.
  • Having no testers is really more of a loss to the company than the Developers, IMHO. If you want me to waste everyone's time going back and testing/rewriting, that's your business decision to make. I will do whatever the company is willing to pay my hourly rate for - within reason.

Obviously, there are some people whose homes are actually very noisy. It pays, quite literally, to live alone. Also consider you can listen to music without headphones, you can pair program or do other group work with online meeting tools, and the company saves a crapton of money by not having to pay for an office. Of course they do anyway, but they really don't need to.

Also, since we buy our own tools, they're much better. I've never seen a work environment where the company always gave their people the best of the best; and my home system has always been a few years newer. It's almost embarrassing, really.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.