Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

So we present a straightforward coding exercise to new candidates with some well defined requirements. Occasionally we receive solutions which don't really solve the problem at hand, but are over-engineered to solve a perceived problem - often outside the bounds of the exercise.

Now my question is, is this a warning sign?

EDIT: Quite a lot of the discussion is based on the test being flawed - which is a fair point. As I described in a comment, the basic premise of the test is to show how you can read the data from the file in a sensible way (and you'd be amazed at the variety of approaches we see), and how to match the items before calculating the latency between the updates. Now for this to work, certain assumptions have to be made about the data, and we look for these assumptions, and we also state explicitly that we want to see the approach you take (including OO approach etc.) All this in a two hour time frame.

IMHO, when I was interviewing it was the most complete exercise I came across.

The particular scenario which I'm pondering about is where a candidate, rather than reading from the file, accepted "network" input in a multi-threaded application, which clearly is not in scope.

share|improve this question

closed as primarily opinion-based by gnat, GlenH7, dietbuddha, MichaelT, mattnz Sep 16 '13 at 23:17

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

42  
can you please include an example of what the exercise is. The problem could be in the challenge and not the candidate. –  Mathew Foscarini Sep 13 '13 at 12:44
13  
Are you sure the candidates understood the problem presented in the exercise? Straightforward to the person presenting the exercise doesn't always equal straightforward to the candidate under stress to perform. –  cdkMoose Sep 13 '13 at 12:47
21  
Doesn't the fact that you called it "overengineering" sort of dictate the answer? It's like asking "Is an overconfident candidate a warning sign"? Sure, unless he's just confident, but you've already put your conclusion into the premise of the question. –  psr Sep 13 '13 at 17:16
3  
@MathewFoscarini Isn't overengineering always negative? It implies the person focused on the wrong things, and implemented a solution which is needlessly complex, time-consuming or hard to understand & maintain. How could it not be perceived as a flaw? –  Andres F. Sep 13 '13 at 18:25
2  
@AndresF. it's an interview. Over Engineering the answer to a question in an interview given that most interviews last only an hour. Could be an achievement. Yes, writing a 1,000 lines of code to sort something is over engineering but he wrote a 1,000 lines of code in less than an hour! If over engineering is an issue that needs to be filtered out in the interview process. There should be a more specific test related to design scope and complexity. I would rather give the person a software architectural challenge to solve. For example; "create a UML diagram for a self-driving car system". –  Mathew Foscarini Sep 13 '13 at 19:07

10 Answers 10

The problem is the test is skewed. You're asking someone to demonstrate their ability to write complex, enterprise-level software using a simple exercise taking only a few minutes. There are other interviewers at other companies who complain that candidates don't show enough skill in object oriented design with these exercises, so people tend to overcompensate. It doesn't necessarily mean your candidate is incapable of using simpler code when the situation warrants it.

If you want to know if that's the case with your candidate, just ask them to redo it, giving them some specific guidelines. Say, "I can see you were showcasing your object oriented design skills, but it seems overkill for such a simple problem. Can you rewrite it using only two small functions?"

share|improve this answer
5  
where in the question does it say "complex enterprise-level software"? –  Mathew Foscarini Sep 13 '13 at 13:37
12  
@Nim - I think Karl's point is that what you consider overengineering, other interviewers may consider a good representation of the interviewee's grasp of OOD. The reference to pseudo-code may not be as explicit as you think in describing the type of approach that you expect. –  Mike Partridge Sep 13 '13 at 14:44
7  
I'm not saying anything about your intentions, @Nim. Candidates read things into questions that aren't explicitly stated, and a lot of interviewers actually expect that. Candidates have no way of knowing if you're that kind of interviewer or not, so they err on the safe side. –  Karl Bielefeldt Sep 13 '13 at 14:45
5  
@KarlBielefeldt: yes, sometimes people read things into questions that aren't explicitly stated - for example, into questions asked here on PSE ;-) –  Doc Brown Sep 13 '13 at 14:46
3  
What about a simple solution of just adding a sentence at the end of the exercise saying "describe in as little code as possible" or something in those terms –  user60812 Sep 13 '13 at 17:38

I would say that this is a clear warning sign, but not necessarily disqualifying for a candidate.

There are two separate problems that candidates seem to be having:

  1. Missing the point of the exercise - This is fairly alarming. All of the programming skill in the world won't create a good result if someone is unable to cast the problem that they are solving properly. I have found that the most productive engineers are the ones that are able to identify the true problem to be solved, even if it isn't exactly the problem posed. Having the ability to think critically about the question being asked, and perhaps reformulate it into something that is simpler to solve is a critical skill. Missing the point when the problem is being clearly stated should be a big red flag.
  2. Overengineering the solution - This is a secondary issue and is often the result of the first issue. There a couple of different pathologies that can have this result. First, not properly understanding the problem, or casting it too broadly can result in a solution that is just too complex. This is clearly related to the first point above. Second, trying to "show off" by thinking through future scenarios that might not really be relevant. This can be problematic because it indicates that the candidate hasn't understood the value of simplicity in solutions and deferring complexity until it is truly necessary. This is something that can be corrected with good guidance, but is a watch out when bringing someone into the organization.

I would suggest following up with the candidate about their answer during the course of the process. Rather than simply looking at their result, try to ascertain what led them to the result, and given some guidance, how they would respond and change their answer. This will probably be more revealing about their capabilities than just their direct response to the problem. The "whys" of their approach can give you a sense of whether they are just not getting it, or if their understanding was just slightly off in how they chose to respond. This kind of follow up will also reveal more about their overall approach to problem solving.

Also, reexamine the problem itself and see if maybe it is unclear and perhaps causing folks to get on the wrong track as they formulate their answers.

share|improve this answer

No, not during a job interview. Too many interviewers do things like intentionally underspecify the requirements and expect the interviewee to ask further questions, or look at attention to real-world issues not explicitly mentioned.

Here are some things, off the top of my head, that your requirements probably didn't mention:

  • Coding standards

  • Comments

  • Exception handling

  • Descriptive names of variables, classes, methods

  • Following idiomatic usage of the language

  • Proper object oriented design

  • Attention to possible concurrency issues

  • Writing test cases for the code

Did you pay attention to a single one of these things without explicitly stating it? How would the candidate know which ones you care about, and which you don't?

An interview is a highly artificial environment. The interviewer is often trying to "trick" the candidate a little bit in order to make it difficult for the interviewee to tell him whatever he wants to hear, and the interviewee is trying to guess what the interviewer really wants.

Generally, making a mistake on that guess is pretty different than making a mistake on real world design decisions. If you want to know if someone over-engineers things you probably have to talk about design rather than look at a very artificial coding exercise.

share|improve this answer
    
I've never seen this done. In actuality most companies want the simplest, most succinct solution. I would be wary about working for any company that can't form a proper request, and for lack of an applicant being able to understand "clear requirements" I would not hire him/her. –  Shaun Wilson Sep 14 '13 at 4:40
1  
@ShaunWilson: That very much depends. I imagine big companies might be interested in seeing what you can do with clear specs. In smaller teams, people depend on each others abilities to empathize, extrapolate, read between the lines and explore the problem space yourself. –  back2dos Sep 14 '13 at 9:27
    
@ShaunWilson I've seen it done many times. Give an assignment, even tell the candidate to ignore things like error checking and provide only the basics, then fail them because they didn't include every single corner case and contingency. It's sadly very, very common. –  jwenting Sep 16 '13 at 9:39
    
For a coding exercise, it's a bit much to expect candidates to stick to coding standards and style - but consistency is a fair expectation. We expect that candidates will use the language idiomatically, but we're not after test cases - the time frame is only two hours (I think it's unrealistic.) I don't believe in tricks in interviews, there's no point - I've been in those situations before, and frankly I find that they are an ego trip for the interviewer and so best avoided. We do explicitly mention OOD (and yet it's amazing to see solutions that use no OO..) –  Nim Sep 16 '13 at 9:50
    
@jwenting, let me assure you, we do no such thing, that's just underhanded. If we do proceed however, we will in the f2f interviews, talk through how they could expand to add corner cases, but only if the candidates bring it up. –  Nim Sep 16 '13 at 9:52

IMHO the answer is clearly yes, it is a warning sign, if

  • the coding exercise had a clear task
  • has well defined (ans also well-written) requirements,
  • the candidates had a chance to ask questions to make sure they solve the correct problem.
  • you want people who are smart and get things done in your team, not architecture astronauts.
share|improve this answer
1  
+1 for the interactive element. In many cases specifications are vague, incomplete, or contain terminology that is domain specific. Without an opportunity to clarify any issues it may be inappropriate to fault the candidate. –  HABO Sep 13 '13 at 13:20
1  
+1 for the fact that your answer models the process perfectly. You clearly answered exactly the question the OP asked. –  dcaswell Sep 13 '13 at 13:39
1  
+1, this is my current thought process, I guess it's good to see it's not naive or plain dumb... Thanks for the two Joel links... –  Nim Sep 13 '13 at 14:08
1  
Don't be so quick to scorn architecture astronauts. Being an architecture astronaut is a phase a developer needs to go through before becoming a truly proficient duct tape programme. See this answer by Aaronaught to the Frankly, do you prefer Cowboy coding? question. –  Marjan Venema Sep 13 '13 at 15:42
1  
@MarjanVenema: I have doubts that Aaronaught meant the word in the same sense as Joel Spolsky, who created that term. And honestly, I don't think I "scorned" anybody - if you want devs in your team creating, well, lets say visionary solutions, feel free to hire them. –  Doc Brown Sep 13 '13 at 18:41

Not nearly as big of a warning sign as not solving the problem at hand. The fact that he failed the quiz and didn't provide the correct solution is a warning sign. It's not necessarily a no-go scenario because it depends on how and why he didn't provide the correct solution.

A lot of times the question is complete crap and simply unanswerable. Don't pretend that interviewers never make mistakes. It's still a failure on his part to suss out why the question is crap. If you're hiring an engineer who is expected to help make requirements, this is a problem. If you're hiring a coder, you just don't expect him to do that.

Presuming that the coding exercise isn't horribly messed up, it appears that the way he failed it was misinterpreting the problem and going off into the weeds trying to solve a problem that wasn't there. Yes, that's a warning sign.

share|improve this answer

Maybe.

Not solving the problem is of course a clear cut warning sign that something is wrong. What went wrong? Either they misunderstood the problem, or they made a bad solution. A bad solution for something simple is a clear sign that the candidate is poor.

If they misunderstood the problem, then take a long look at your requirements. Even things that seem crystal clear to you may be unclear to others unfamiliar with a domain, or from a different background (locale, age, upbringing). If the candidate was there in the room with you, or offered an opportunity to ask questions and still failed, I would consider that a failure on their part. If the requirements were thrown over the wall, I would likely give them the benefit of the doubt (and think about fixing the interview process).

If the solution was correct, then it becomes less clear. Personally, I think that a number of people take YAGNI too far. If you can take a specific problem and create a general solution without increasing complexity or harming maintainability then why not make the general solution? (Think reversing a string vs. reversing any collection) That sort of "over-engineering" is clearly good in my opinion.

Everything else is that grey middle ground. Does the solution address likely axes of change? Does the solution add complexity or harm maintainability? Adding a little complexity to solve future problems that are nearly guaranteed to a win. Harming maintainability greatly to account for something that is totally unlikely is not.

Being a good software engineer means striking the right balance here. Being a good interviewer means striking the right judgement/inferrence about how and why the candidate chose that balance, often using other parts of the interview to evaluate.

share|improve this answer
2  
If the problem is hard to understand, or hasn't been communicated well, this is the time for them to demonstrate the critical skill good to moderate programmers MUST have - defining the problem. –  Adam Davis Sep 13 '13 at 14:33
    
The question doesn't say that the candidate didn't take advantage of a chance to ask questions. –  dcaswell Sep 14 '13 at 17:56

Maybe but consider the following:

  • Interviews are hard: People are under stress. This should be factored heavily into any problem you give someone

  • Requirement Length: How long and straight forward are the requirements? Did you make them extra wordy to make sure you included everything. As a result, they may be clear to you but the requirements maybe over engineered! I took a job interview once for an hour problem with about 3 pages of text for a problem that was relatively simple. Sometimes all that text can be tough to read and interpret on an interview and can be misinterpreted as well.

  • Sometimes Less is More: I would much rather have a few bullet points, sentences, and or examples showing the main requirements and then a discussion with someone to ask questions to and maybe reach out to along the way if needed. I guess what I am trying to say is you should first check your test requirements are not overly complicated for a simple problem.

  • Communication Skill: you should be testing the candidates ability to communicate and ask intelligent questions about the problem first and once you know they have demonstrated that they understand the problem then you can set them forth on their implementation.

  • Bottom Line: If you haven't checked that the problem is understood than you really don't know what to make of the over engineering. As others have said it could be a good or bad thing but if you didn't check the understanding or communicate with the candidate about the problem up front, it is harder to know their general understanding of the problem and what to make of it.

share|improve this answer
1  
Solid answer, but it's hard to wade through the wall of text. Consider editing your answer and breaking out the major sections. –  GlenH7 Sep 14 '13 at 1:16
    
thanks for the tip, I broke it up –  bjackfly Sep 14 '13 at 16:58

A lot of this could be attributed to how you word the question and how you've put the entire interview into perspective.

It's like, "Let's see how creative you are. What is 2 + 2?" Four? All you could come up with is the most simple, obvious and accurate answer? Young / entry level developers who are so eager to impress during an interview will take the "We want to test your coding skills or see how good of a programmer you are." to mean "do something very sophisticated." We all like to think simple is best except when it makes things more difficult.

There are ways to see if someone is prone to always be over-engineering things. Give a code example of something that was too complex and ask for a simpler solution. When someone provides a solution that you don't think will work, this is a great opportunity to see how they react to criticism. Personally, I'd like to see someone be open to new ideas and recongnize a better solution than someone who is going to make the same mistakes over and over again.

And in reality, don't we always have an opportunity to change our code when it doesn't work? You can under design or over design initially. Once you've recognized the simple solution, shouldn't it be easier to implement?

share|improve this answer
    
"What is 2+2?" 4 versus "Let's see how creative you are. What is 2+2?" The limit of the sequence 3.9, 3.99, 3.999, 3.9999, ... –  emory Sep 13 '13 at 15:07
    
"Let's see how creative you are. What is 2+2?" 5, for sufficiently large values of 2. –  Michael Sep 13 '13 at 15:36
    
and define "overengineering". Depending on the environment, something that might appear overengineered to someone unfamiliar with it may be considered as taking way too many liberties and shortcuts for someone in that environment. Think missile control software when looked at by someone whose main field is writing games for mobile phones... –  jwenting Sep 16 '13 at 9:42

It depends, but generally no.

Design in general is a skill learned with experience. Aaronaught's description of that progression linked by Marjan is generally a good one.

Communication in any form also depends a lot on context. What may seem perfectly clear to mean one thing in one context, may just as clearly mean something else within a different context. Knowing which questions to ask is also something that comes with experience.

Their thought process and how they reason about their decisions is far more important than their solution. Without reviewing their solution and their decisions with them you cannot fully assess the context it was developed under.

Given the progression above, a candidate with the over-engineered solution may well be further along than the candidate with the simple one.

Also: What level position are you hiring for? Over-engineering out of an entry or intermediate level candidate is less of an issue than over-engineering out of a senior level candidate.

share|improve this answer

If the programmer didn't solve the problem, then all of the extra code is the coder's attempt to obfuscate the non-answer. It's the same technique used on an essay test by a student who doesn't know much about the subject. He or she will ramble on about a compelling, but unrelated issue in hopes his ignorance is masked by word count.

If the programmer did solve the issue but added a great deal more code, consider the programmer's background, as some areas of programming require greater tolerances than others.

For example, code running the auto-pilot in a commercial passenger jet has much less tolerances for failure than a free Android game. A developer used to programming embedded devices that are hard to reach and almost impossible to update will tend to code more what-if's then a developer that can push out 15 updates in a day.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.