Bare minimum on the Joel Test [closed]

Question

From the Joel Test, of the 12, which do you think are the absolute must-haves to at least have a decently running software department/company?

I realise there is no absolutely right answer. I'm just trying to get opinions of others out there. My own organization only manages a measly 5 of 12. If you check listings on Careers 2.0, most companies don't score a full 12 either but I'm sure they're doing fine.

Does SO publish the stats for those anywhere? Or has anyone tried scrapping the results? Would be interesting to know which are practised the most. And whether because they are easier to implement or whether they actually have the most impact.

Thanks.

EDIT: These are the ones practised by my company and for the given reasons

1. Having source control. - Have to agree with one of the answers that that is the one I would not negotiate about.
2. Fix bugs before writing new code. - Matter of principle but not a must-have.
3. Have a spec. - In a small software house with tight margins, specs save life. But I could see it done away with in other companies without affect.
4. Candidates write code during an interview. - Saved us hiring the wrong candidate many many times over. This is the other non-negotiable must-have for me.
5. Hallway usability testing. - Easy for us. Our whole office is a hallway. It's really easy to grab people anytime. But that also means we don't have a quiet working environment for developers.

Answer 1

Number 1: Using source control.

This is the only one that I absolutely wouldn't, under any circumstances imaginable, ever, negotiate about.

Answer 2

• Do you use source control? Required
• Can you make a build in one step? Nice
• Do you make daily builds? Nice
• Do you have a bug database? Required
• Do you fix bugs before writing new code? Nice
• Do you have an up-to-date schedule? Required
• Do you have a spec? Required
• Do programmers have quiet working conditions? Required
• Do you use the best tools money can buy? Required
• Do you have testers? Required
• Do new candidates write code during their interview? Nice
• Do you do hallway usability testing? Nice

So I would say 7. YMMV.

Answer 3

That's an interesting question. I have always felt that the Joel Test was a lot like code smell; scoring a 12 doesn't mean that the company is doing everything right, and scoring an 8 can indicate that there are some bad practices but doesn't necessarily mean that the company is run poorly (but like code smell, you should tread warily).

Instead of focusing on the score, I would figure out which parts of the test were important to me and then base my decision on that. If you really wanted to get "scientific" you could give each part of the test a weight or percentage and base it off that.

Answer 4

Depending on the company, not having 12 on the Joel's test can be seen as an opportunity for you to provide them with high value that will be probably rewarded accordingly.

Therefore, I personally prefer to work for companies with a low rate as it will imply that I will able to come up with solutions to their problems.

If they refuse those improvements, then it's probably time to move on.

It's easier to do what I describe here if you are freelance which signs a new contract every 3 months. But even as an employee, that could be challenging for some of us.

Answer 5

I will make the caveat that it depends what they're bringing you in for. If you are a coder, rather than an architect or high-level developer, you should probably look for a high number since you likely won't have any control over your destiny. They might be looking for some help in getting closer to a 12 though, and that shouldn't be discounted if the company has other great things going for it.

Environment, culture, and willingness to improve from a 0 to a 12 is far more important to me than a company that claims 10+ and employs a bunch of idiots.

With all that said, my beliefs are as such:

• Do you use source control? YES If this is a software shop with more than one person, this is a must have to organize yourselves effectively. Even one person should probably at least use a DVCS simply for history.
• Can you make a build in one step? NO As long as it's trivial to do the build, # of steps is not terribly important to me.
• Do you make daily builds? NO This is not important (IMO) for small teams and certain kinds of software development, and for certain stages of business.
• Do you have a bug database? YES Although I think it should really be more of a to-do list
• Do you fix bugs before writing new code? NO This is important in large groups but not a deal-breaker for me
• Do you have an up-to-date schedule? NO Not important for effective work as long as programmers and project stakeholders meet often enough to be in sync
• Do you have a spec? YES Preferably written and drawn.
• Do programmers have quiet working conditions? YES This should really be rephrased to say "option for quiet working conditions", such as working from home, or a spare room for working on difficult tasks.
• Do you use the best tools money can buy? YES Dual monitors, fast machines (laptops if that's what they want), operating system and software of the programmer's choosing, if possible
• Do you have testers? MAYBE This depends on the maturity of the product and the size of the company
• Do new candidates write code during their interview? YES They should write code, solve problems in their head, and be able to discuss theory during an interview
• Do you do hallway usability testing? YES I like the idea of this a lot, but YMMV greatly on this one, depending on the quality of people in the rest of the organization

Answer 6

I would be perfectly happy to work someplace that scores a big fat ZERO on the test, so long as the work was interesting and pay was right. If anything, I'd be hesitant to work for a company that even knew its Joel score, or worse, bragged about it. That would be an indicator to me of a certain slavish adherence to group-think, which is not the kind of environment I thrive in.

The only items on the test that I would really be concerned about would be quiet coding conditions and a decent spec. But even then, it would depend on what the actual project was.

Answer 7

All Joel Test questions are created equally, but some are more equal than others (apols to Orwell).

I do agree that these questions represent a good measure, but not all of these questions should have the same weight. I don't see that something as fundamental as using source control is weighted the same as usability hall testing (especially for small companies that may simply not be able to grab 5 people to test their software). In addition some questions are pre-requisites for other questions; for example no company without source control will be doing daily builds.

For me the most important aspects of the Joel test (which is 10 years old) are:

• Do you use source control?
• Can you make a build in one step?
• Do you have a bug database?
• Do you fix bugs before writing new code?
• Do you have an up-to-date schedule?
• Do you have a spec?
• Do you have testers?
• Do new candidates write code during their interview?

As to the importance of the questions, I largely agree with @SnoopDougieDoug analysis.

• Do you use source control? Required
• Can you make a build in one step? Required
• Do you make daily builds? Nice
• Do you have a bug database? Required
• Do you fix bugs before writing new code? Required
• Do you have an up-to-date schedule? Required
• Do you have a spec? Required - but how much detail is the important issue
• Do programmers have quiet working conditions? Not required - a chatty office is more productive than a deathly quiet one IMHO
• Do you use the best tools money can buy? Nice - open source has moved on since 2000
• Do you have testers? Required - not necessarily in house
• Do new candidates write code during their interview? Required
• Do you do hallway usability testing? Nice - if not always practical

Answer 8

The real answer depends on your market. If you are doing internal LOB development, consulting, or research, all of which focus on "mostly works right now" vs "works right mostly" then you may be able to get away with lower scores. But if you are in the comercial software business you need a high score.

The bare minimum is 11. And that's only if you are really small (< 5 people) and literally can't afford testers. 10 is a stretch. Anything less and you are bordering on the edge of suckyness. I'm going to focus on the items marked as "nice" in this answer.

• Can you make a build in one step?

  You can't make daily builds if you can't build in one step.

• Do you make daily builds?

  If you don't have automated builds, your builds are going to be broken. If your builds are broken, your programmers can't integrate with each other. If your programmers can't integrate with each other, you can't ship on time.

• Do you fix bugs before writing new code?

  My personal opinion is that the order you fix bugs isn't as important as them actually getting fixed. However, it's much easier to ensure bugs are fixed if you fix them up front.

• Do new candidates write code during their interview?

  You would be surprised how effective this is at separating horrible candidates from good candidates and good candidates from great candidates. Even simple questions like "write code to convert a hex string into an integer" are really effective. When it comes to judging technical competence there is really no better filter. If you don't employ such a filter than you are relying entirely on luck. That's not to say that you don't want to apply other filters (behavioral and culutral matches), and that there aren't other things that are as equally important as technical skill.

  However, software is completely intangible. It's 100% abstract. When you sell software, you are selling repackaged thoughts. That is, someone took their thoughts and converted them into a format that a computer can understand. Running a software business is about harvesting brain waves and converting them into cash. Someone's ability to translate their thoughts into code is crucial to your ability to convert them into cash.

  Why would you not employ such an effective means of evaluating such an essential skill? More importantly, I would challenge any assertion that there is any other way to evaluate such skills.

• Do you do hallway usability testing?

  If you don't do hallway usability testing, how do you know that your stuff doesn't suck? If you don't know, as in you have ethnographic evidence (you've watched people using it), that your stuff doesn't suck, then you have 2 problems: your software sucks and you don't know it. Fixing the first problem (having software that doesn't suck) is hard. Fixing the second one (knowing that your software sucks, and why it sucks) is easy. It also happens that fixing the second one will give you a road map to fixing the first. Even if you aren't selling software on the open market (where too much suckyness will damn you) there is a huge business case for investing in usability. Check out this book for more info.

Answer 9

The hallway usability testing question only makes sense if you are building end user facing software. If you are working on say device drivers or embedded code it makes no sense at all.

Answer 10

My current contract

• Fixes bugs before writing new code (usually)
• Has quiet working conditions (telecommute)
• Has the best tools money can buy (Because we buy our own)

The benefits of working from home -- at least for some of us -- far outweigh some of the missing points. Some examples:

• Folders named "Project X 03302011" kinda work as long as only one person is working on Project X at a given time.
• Having no testers is really more of a loss to the company than the Developers, IMHO. If you want me to waste everyone's time going back and testing/rewriting, that's your business decision to make. I will do whatever the company is willing to pay my hourly rate for - within reason.

Obviously, there are some people whose homes are actually very noisy. It pays, quite literally, to live alone. Also consider you can listen to music without headphones, you can pair program or do other group work with online meeting tools, and the company saves a crapton of money by not having to pay for an office. Of course they do anyway, but they really don't need to.

Also, since we buy our own tools, they're much better. I've never seen a work environment where the company always gave their people the best of the best; and my home system has always been a few years newer. It's almost embarrassing, really.