Can anyone explain Developer Twin Syndrome?

Question

I realise this is a fairly odd question to ask but in recent rounds of hiring we have noticed a rather odd trend in new developers that spectacularly fall short of our expectations. So much so that they to all intents and purposes appear to be an entirely different person to the one interviewed and tested.

We pre-screen applicants during their initial interview with a fairly nasty PHP/MySQL written test which tests both their knowledge of the PHP/MySQL as well as some fundamental aspects of application design. You cannot score highly on this test by simply memorizing the PHP and MySQL manuals (We've made that mistake in the past). I'm sure It is by no means fool-proof, but is intended to be sufficient to weed out those who have clearly lied on their resumés.

Applicants that pass this stage are offered a second interview which is followed by a 2 hour practical test where they are required to develop a small application to a given specification. They are not necessarily expected to complete the task but are judged on their approach to the given task.

Three times now he have hired the "best" applicant based on interview and test results, and each time the person that has showed up for work on the first Monday morning has been completely devoid of the skills and experience they have previously demonstrated. Each of these applicants scored over 90% on the written test and exceeded our expectations when given a practical task to complete.

Hence my coining of the phrase "Developer Twin Syndrome"

Has anyone else experienced a similar trend in hiring developers?

Answer 1

Maybe your tests have been leaked. A candidate might get a friend to interview before them and prep them on the problems they encountered.

If so, the best solution is to develop a suite of basic programming challenges that can be solved quickly on a whiteboard by a candidate with genuine skills. Randomly selecting a subset of the tests will make it very difficult to game the interview process. If you have ten tests, and give each candidate two of them, you can dish out 45 different combinations before repeating yourself. Even a determined cheat with two friends would have less than a one-in-three chance of knowing both questions.

Also, do you put candidates in front of real programmers? It isn't clear from your question, but I get the impression that the only technical engagement they have is via written tests. A technical interview with a dev team member usually weeds out idiots.

Answer 2

I've seen smart, smart people get dropped into a new situation and suddenly forget how to tie their shoes. Their first Monday morning is no time to be assessing their credentials.

They're madly trying to figure out how to check their new email account and get to the code repository and understand Jimmy-in-the-next-cubicle's code. Doesn't seem like a real fair time to judge them, especially after putting them through such a wringer in their interview process.

And then there's this: when you hire someone, you're committing to them just as much as they're committing to you. You're saying: Look! Here's this person. I found them and they found me, and I'm going to support them with whatever they need to be the developer I need them to be. If that's NOT the attitude you have with new hires, let me know your company name so I can avoid even bothering with your interview gauntlet.

Answer 3

Does your work environment have anything like the level of structure you have for your hiring process? It could be that you're not testing for the skills that employees will need in order to find a useful role in the company.

If your company's culture gives employees a good deal of autonomy, you might want to try to test more for candidates with the soft skills to deal with that rather than testing on pure technical merit.

Answer 4

A common problem. I've seen a strong correlation between those that do really well on very specialiazed questions/tasks and a lack of productivity. While those that tend to answer in generalities (often to the point of seeming like they don't know much about it) tend to do quite well in the real world. Perhaps the specialist group only know those exact items and the generalist answerers can't answer specifically as they know too much in general. No matter how well we try to simulate the work conditions for applicant testing, it just doesn't happen. I don't believe there is a way to build a suitably scientific method at this time. Subjective interviews/measures still seem to hold the upper hand on getting the best employees, despite all the attempts to objectify the process.

Answer 5

Maybe you're testing the wrong things? Instead of testing for "mysql", it should be about core database design concepts. You don't want to miss out on someone good just to avoid the week or two it takes to learn a DBMS. If it was a DBA, then I would be more specific about the platform. The same for PHP. Anyone experienced with web applications in any language won't have an issue picking up PHP quickly.

Answer 6

I think you need to take a long hard look at your hiring methods and revise them heavily, you have some major problems if your "best" has been a dud 3 times. You should test your interviewers and see if they are improperly ranking interviewees. You also need to check your tests to make sure they haven't leaked, are up to date and truly do test what you think they test, it would be worth it to involve your current developers in this process, get a senior devs comments/suggestions and have a couple junior devs that have been there a while take the test. Also you need to decide if your tests are even worth it, if you are getting unqualified candidates passing the test you are wasting a ton of time. Also I'm willing to bet that your company has a bad reputation with your local pool of developers and more qualified candidates aren't applying. good programming jobs are plentiful enough that good developers can be picky which means employers can't be dicks about hiring, or offer lower pay/less benefits and expect to attract desirable candidates. Personally I would feel insulted to be asked to take a written exam and then come back another day and take a second test, I would really recommend combining these into a single day and keeping the total test time at ~2 hours so with an initial interview that's still 3-4 hours per candidate for a first interview which is a pretty long time for what seems to be an entry level position.

Answer 7

It's likely that your tests have leaked or they don't test the knowledge of the programmer. You can teach people how to use a calculator, doesn't mean that they can do math.

You might find this a usefull read:

http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html

Answer 8

The only explanation I can think of is that you are assuming that your tests are complex and throughout, but in reality they're not.

You could also ask the candidate to work with the team for a few days, doing regular work, before signing a contract. So you put the "first Monday" problem as part of the hiring process, not the firing process.