When in the project life-cycle would you fit in external penetration/security testing of the software?

Question

Assuming that you're working on a piece of software which is required to pass a third party security/penetration test before being released to the client, at which point in the project life-cycle would you perform the tests?

Passing the test means that no major flaws are detected, or, more likely, that any flaws detected have been properly corrected before release.

Answer 1

I guess it depends on how expensive (in cash, time and resources) is to conduct the test, how often it can be repeated, and how much in advance you need to negotiate your test dates. And also about how limited its validity is, i.e. if you do the simplest change in the code after passing the test, does it invalidate the whole test, forcing you to repeat it fully, or partly?

If it is expensive and hard to arrange, I would schedule it to the UAT phase and try to run as much similar testing as possible in-house before. Obviously, the easier and less expensive to do it, the earlier it is worth starting the tests (once I have a reasonably stable version including all critical features, that is).

Answer 2

The obvious simple answer would be: at the end. Doing it half way through just leaves room for flaws to creep in during the second half of the development, etc.

More effective would be doing it every so often. That way you'll be able to find and fix the flaws closer to when you actually made them, which is much easier than remembering what you did months ago. If you can put some of the basics of penetration testing into a unit test, then first up a continuous integration server which can alert you if you make any real stinkers (though can't really replace real testing).

Answer 3

When in the project life-cycle would you fit in external penetration/security testing of the software?

It happens at every step in the life cycle.

First, penetration testing is only one small piece of a more complex security puzzle.

Second, penetration testing includes a run-time configuration and proper administration as well as a software implementation issue. Penetration testing requires proper patching and configuration of OS, web server, database, firewalls and application software.

You do penetration testing of the architecture before you start doing detailed designs. Is the architecture itself secure?

You do penetration testing of the design. How secure is the design? What penetration opportunities will be present?

You do penetration testing as part of coding and unit testing. Does the code meet the design? Does it properly use architectural features to prevent security problems?

You can then subject the final software product to all kinds of security scrutiny including -- but not limited to -- penetration testing.

Security is pervasive. Or it has holes. It's not a "step" in the work plan. It's an underlying principle.

Answer 4

Obviously it is a risk to do at the end, fixing things might just cost to much time.

Obviously it is a risk to do it to early, added features might add security vulnerabilities.

My advice, do it as late as possible but reduce the risk.

• Get insight form the testing company what they are going to test so you can design and build for that
• Build the potential attack surface (Website/web service anything that allows interaction with the outside world) first and when is ready do the test.
• during building use static analysis tools like fortify to test basic security
• create your own security integration tests.
• keep enough time after the test to fix and retest.

Hope this helps

Answer 5

I would make it part of the system integration test. Unless your system is highly modular, it probably wouldn't be worthwhile to do it as part of your unit testing. I'd also schedule it early in the integration test phase, as security issues are notorious for causing significant system redesign.