Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I have heard of security firms who consult on the security of a clients systems. All of the people I know in this field are network engineers, but I know programmers get involved in security as well. What do security programmers who perform audits/consulting actually do? Do they literally go through the codebase looking for every vulnerability in people’s legacy systems? I have always assumed that is what they did, but it seems like this would be highly unreliable and not do much more than providing a false sense of security. Note: I’m not talking about programmers who write encryption algorithms or anything like that, only those concerned with software security audits/consulting.

share|improve this question
1  
Feel free to browse security.stackexchange.com for an audience of many more security folks! –  Rory Alsop Apr 12 '11 at 13:28
1  
^ what he said, we are both pro tem moderators over there. –  user4051 Apr 12 '11 at 22:50

3 Answers 3

up vote 12 down vote accepted

Frankly we try not to go through your codebase, we try to write tools to do that for us.

First, the theory. Security is a requirement of a software system, so like other requirements (functionality, usability, accessibility, performance etc) it should be considered at every stage of the software engineering workflow from requirements gathering to deployment and maintenance. Indeed, this is possible, and guidance exists to help software project teams do that. Even though I mainly work with iOS developers, my favourite description of "the secure development lifecycle" is from Microsoft Press.

In this model, application security begins when we're trying to elicit requirements from our users. We need to discover their security and privacy concerns, which is not easy because we are the experts, not the users, and where they do understand their security requirements they might find it hard to express them. We also need to discover what risks the software will be exposed to in deployment, and what level of risk is acceptable.

We design our application with fulfilling those requirements in mind. We write the code with an eye to satisfying those requirements, and avoiding additional risks associated with code-level security mistakes. We test the software to ensure that our model of its security is consistent with what we really built, then we deploy the software in a fashion that matches the assumptions we made about the environment when we designed the thing. Finally, we provide support and maintenance that helps the user operate the software in a way consistent with their security requirements, and which allows them (and us) to react to novel changes in the risks presented.

Ok, so much for theory. In practice, for reasons that are explained very well (albeit in a nontechnical fashion) in Geekonomics and which are mainly due to they way software companies are motivated, most of the above stuff doesn't happen. Instead, we get this. Developers will:

  • hire a security guy or gal to be present when they're bidding for a contract, to show that they "get" security.
  • write software.
  • hire a security guy or gal to validate the software before release, fixing many problems that arose in step 2.
  • patch everything else after deployment.

So what most app security people are really doing is, as you say, finding bugs. This is really a glorified code review but it's a highly focussed code review carried out by people who are experts at the sorts of bugs this review is looking for, so there still is value in getting external help in doing that. That's a general rule of teting, of course: always get someone else to test who wasn't involved with making the thing.

If we accept the above as true then it follows that people making purchasing decisions are likely to equate "capable security guy" with "finds lots of bugs". Those who get computers to do the work for them will find more bugs than those who don't, so of course they rely heavily on static analysis tools and will aim to spend more time extending the tools than coding for specific issues for specific clients. Then we conclude that app security people are more likely to write tools to read code than to read code.

** Warning: what remains is personal opinion and speculation **

Reality is broken. You will notice that the theory of software security was all about identifying and responding to the risk of relying on a software system, while the practice was about finding as many bugs as possible. Sure, that will still reduce risk, but only as a side-effect. The point of the game has become less important than "winning" the game, so the rules are changed to make it easier to win.

What can you as a software developer do about that? Play the game by its original rules. Find someone on your team (preferably actually on your team, rather than a contractor, so that they are motivated to provide long-term results rather than a quick win) who understands the importance of security and train the bejeezus out of them. Give that person the responsibility to direct the team in providing the end-to-end security described at the beginning of my answer.

Also, give that person the authority to follow through. If a design doesn't express the security requirements, it must be revised. If the implementation does not meet the security requirements, it must not be released. Your security person can make the judgement call, but must be permitted to act on that judgement. I realise this might sound like the security guy saying "OMFG security is the most important thing", but that's not what I mean. If your product doesn't meet the functionality, usability or performance requirements either, you also shouldn't release that thing.

Why would you want to do that? It ought to be cheaper: we've all seen (and probably quoted for a cheap +10 rep) the Code Complete table where defects get more expensive the later you fix them, right? Well security defects are defects too. I the real-world rules of the game, most of them are problems in the requirements that are fixed in maintenance. Is that cheap?

Ok, now what can I as a security gun for hire do about that? Well it turns out that I can refuse to play by the amended rules, too. I can tell developers that it's all about reducing risk, that this can be done at every stage, and then I can help them do that.

share|improve this answer
    
For person in your position, I'm surprised that you can't offer more to the discussion. I'd be very interested in hearing what you have to say. –  Steve Evers Apr 12 '11 at 1:58
    
You're right, I was tired (jet lag) when I wrote the answer. I'll try to fill it in a bit. –  user4051 Apr 12 '11 at 10:09
    
@snorfus I should apologise for not giving a good answer. I'm truly sorry. –  user4051 Apr 12 '11 at 10:18
    
Great answer - thanks! –  nealmcb Apr 12 '11 at 14:24
    
@Graham Lee: I suspected that you had a great answer hidden from us :) Your edits have proven just that; thank you! –  Steve Evers Apr 12 '11 at 15:31

From 15 years of running security assurance programs against small and extremely large apps, environments, systems etc I would say there is a bit of everything. In my teams I have always had a few who are hardcore coders.

At the detailed level, some of it comes down to very in depth code review - as an example I am currently working on a multi-million line codebase, using tools to narrow down the possible issues, and then looking at each and every one to categorise.

(Admittedly I then hand over to developers to remediate or to explain to me why the issue does not pose a risk)

But this is a specific engagement for which the risk profile makes sense to carry out this sort of resource intensive work.

Far more standard, and much more cost effective is trying to understand the risk profile of the organisation and focus on the risks top-down, eg:

  • Risk Appetite - impact to the business, threat modeling
  • Governance - regulatory compliance, reporting etc
  • Policies - definitions to ensure governance framework is effective
  • Processes - technical and human
  • Standards - definitions for each security control
  • Implementation - the How To

The programming side really only comes into the last two, with code review and custom penetration testing. For some organisations it is of very low importance, for example if you have layered security controls which have already been peer reviewed extensively (various encryption types, say) then while you may check your implementation, you aren't usually going to recheck all the code as that has been done before.

share|improve this answer
2  
I +1d, but beware "or to explain to me why the issue does not pose a risk". Developers will often find reason to avoid changing the thing they made (speaking as a developer), and besides may not understand the customers' risks. After all, it was developers who wrote Windows 98 ;-) –  user4051 Apr 12 '11 at 13:13
    
@Graham - you said that which was not to be named :-) And I do like your new longer version answer! +1 –  Rory Alsop Apr 12 '11 at 13:22
    
oh, right. I deliberately said that because I didn't want to name windows 98-but-three-years-earlier. –  user4051 Apr 12 '11 at 13:36

I have never encountered any that go much further than discussing architecture/best practices in a vague manner during design and/or running attack/fritzing/exception testing suites against finished projects.

In almost all cases I can even tell what tools they use by the attack vectors they try and the manner in which the attack is carried out after one of the audit passes on an existing system.

I imagine that there are a few that actually take the time to examine code and do some review/whitebox testing, but I have yet to encounter them in real life.

share|improve this answer
    
it sounds like the company you work for consistently cheaps out and takes the hacks, who talk a good game but don't really get the point. Besides myself, and the other answerers here, I have worked with, and trained, many that do it right. Admittedly, I've probably met more of the kind that you've had, too... –  AviD Apr 12 '11 at 13:49
    
@avid I didn't mean it as a negative. I am certain if you paid top-dollar you could find enough competing firms, but even when you do you tend to get a lot of more suggestions to buy something than you do advice on improving / implementing something. THAT IS NOT A BAD THING using a known product with a good security record is better when it is a proper fit for the problem space. The OP mentioned AUDIT specifically, and at the range you pay for your annual 3rd party audit you get through the 2nd, 3rd and 1/2 of the 4th of Rory's points. –  Bill Apr 13 '11 at 22:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.