Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

The position is an entry-level position that consists of reading C++ code and identifying lines of code that are vulnerable to buffer overflows, out-of-bounds reads, uncontrolled format strings, and a bunch of other CWE's.

We don't expect the average candidate to be knowledgeable in the area of software security nor do we expect him or her to be an expert computer programmer; we just expect them to be able to read the code and correctly identify vulnerabilities.

I guess I could ask them the typical interview questions: reverse a string, print a list of prime numbers, etc, but I'm not sure that their ability to write code under pressure (or lack thereof) tells me anything about their ability to read code.

Should I instead focus on testing their knowledge of C++? Ask them if they understand what a pointer is and how bitwise operators work? My only concern about asking that kind of question is that I might unfairly weed out people who don't happen to have the knowledge but have the ability to acquire it. After all, it's not like they will be writing a single line of code, and it's not like we are looking only for people who already know C++, since we are willing to train the right candidate. (It is true that I could ask those questions only to those candidates who claim to know C++, but I'd like to give the same "test" to everyone.)

Should I just focus on trying to get an idea of their level of intelligence? In other words, should I get them to talk and pay attention to the way they articulate their thoughts, and so on?

share|improve this question

4 Answers 4

we just expect them to be able to read the code and correctly identify vulnerabilities.

So have them read some code and identify the vulnerabilities.

Failing that, ask them to demonstrate something like a buffer overrun, using either C++ or pseudocode.

share|improve this answer

Don't ask questions, give them some code and ask them to find the vulnerabilities. Then ask them to prove to you each bug or vulnerability. Make sure you have an answer sheet with the correct results and reasons so that you can show them it was a fair test.

I did this kind of work for a while and when people had to stand behind their work in front of the paying customer, you would not believe how fast some of them would not stand behind their work and pass the buck. It is one thing to say you found a bug in a report; it is another to convince someone in person who probably has much more knowledge about the code and probably more years of experience than you and explain to them where they made a mistake.

More importantly, why are you not using tools to do this? Granted the tools are not 100% full proof but are faster and if you get the same result from two different tools you can be pretty sure it is a true problem. Also since you are paying a person to do all of the code manually, the tools can build a working list of possible bug candidates that a person can check.

share|improve this answer

One approach is to be completely open ended. Ask them to start a list on the white board of secure coding practices and see how many they have from the following list.

https://www.securecoding.cert.org/confluence/display/seccode/Top+10+Secure+Coding+Practices

This site has an amusing picture too.

When I interview candidates, I will sometimes run my own version of the word association test that has been used by psychologists since Karl Jung.

In this test, I have a sheet of paper that lists twelve to twenty common terms and concepts from a topic area like embedded systems, testing, or for your example, software security / writing secure code. Typically, I have two copies of this list, one for myself which I mark, and one for the candidate. I request the candidate spend no more that 10-20 seconds on each because I am looking for breadth rather than depth. Once we complete the list, I can ask for more depth. I also have the candidate return the list.

My list for software security might include:

Basic

  • buffer overflow
  • code injection
  • SQL injection
  • strcpy();
  • format string vulnerability
  • authentication
  • denial of service
  • password strength
  • cryptographic hash
  • certificate
  • SSL
  • sand boxing
  • firewall
  • security risk model
  • security by obscurity
  • salt
  • protected execution

Medium

  • AES
  • non-repudiability
  • PKI
  • web of trust
  • digital signing
  • elevation of privilege
  • acl
  • group policy
  • DMZ
  • man in the middle
  • WEP
  • default permissions
  • TPM
  • smart card
  • NFC
  • CAPTCHA
  • challenge / response protocol
  • replay attack

Expert?

  • Schnierer
  • Alice and Bob
  • Malory
  • Tempest
  • digital cash
  • touch screen voting machine
  • Enigma
  • cipher

Certainly these lists need some tuning. Interviewers should take precautions to insure that the questions they ask are relevant, so you might want to evaluate each item against your business, but if you do web sites, I think you need SSL, SQL injection, certificate. If you ever use passwords, you probably need to understand what security by obscurity means and what it implies you should never do. If your team administers servers, firewalls, salt, authentication, acl, and probably group policy are important.

Security threats are growing more common and more nuanced. If you can get someone who knows a lot about software security, grab them. If you can't, try to talk someone in your team into becoming an expert.

share|improve this answer

I feel a disturbance in The Force when I read "entry-level position" and "reviewing code for security vulnerabilities" in close proximity to each other. Seriously. I've been grinding out code for 40 years and I still can screw up on stuff like this. The people doing code review for security problems need to be more experienced than the people who wrote the code in the first place.

You're far more likely to find actual buffer overflows and sanity-check failures by doing things like Fuzz testing than you are by hiring "entry-level position" programmers. This is the type of tool that actual Bad Guys® use in trying to crack an app. (BTW, a good round of Fuzz testing can prove very embarrassing to the original coders.)

I think good interview questions would be:

  • Strictly off the record: how many sites have you broken into?
  • How old were you when you first hacked into someone else's site?
  • Have you served any jail time for illegal access to a computer system?
share|improve this answer
    
+1 for having a few more years of experience than me. –  Guy Coder Oct 7 '12 at 15:36
2  
+1 for some sane logic. I cringe every time I hear "but those experienced guys cost a lot - we can get two juniors for the same price!" :) –  Daniel B Oct 11 '12 at 8:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.