The Fallacy of Secure Software
This weeks guest post is by Justin Clarke, an unusual blend of Web Application Security freak and Kiwi. Justin is the lead author of "SQL Injection Attacks and Defense" - a book that Richard Bejtlich recently voted as his 2009 "Best Book Bejtlich Read" (high praise indeed). Justin is the man you call when your webapp security program needs to "keep it real" ;-). Thanks Justin!
So, you've got yourself a security solution for software that your organization develops. Maybe you've just invested in some automated tools, or developer training. Maybe you did hear a bit of FUD in the sales presentation, but it all seemed to make sense? Great! You're going to be producing secure code now aren't you? Security problem over?
Well, no.
There are a number of problems with solving the problem of insecure software. One of the biggest is that there is no one-size-fits-all solution to the problem, regardless of what some less-than-honest vendors will tell you. All approaches that are parts of solving the insecure software problem have their upsides and downsides. To give a few examples, lets look at some of the popular approaches to assuring software security:
Penetration testing/blackbox testing - probably the most popular and widespread approach to securing software such as web/network applications. For enterprise situations, this would normally be performed near to completion, generally in a user acceptance or pre-production environment. As an approach, it is intended to simulate the risk of an external party (such as an attacker or malicious user of the system) attempting to compromise the system.
The generally accepted upsides of this approach are that it should give you a good picture of where you are with regards to someone breaking in. Downsides include that it is resource intensive (and therefore expensive), and very dependent on the skill of the people performing the assessment. It is also very late in the development cycle, so in many cases major rework can result in cost and schedule overruns and delays. Very commonly used in environments where the possible loss due to a security issue is high, such as banks and other financial institutions.
A variation is the use of automation for some or all of the testing. These are generally tools that implement some from of "fuzzing" approach, with the most mature products being those in the web application space from HP and IBM. This provides the ability to cover many more applications in the same amount of time, but seldom with the same level of depth or flexibility that a person performing the same assessment could. Very commonly these are used in conjunction with other assessment approaches in order to provide greater coverage or to find easier to detect issues that would then warrant additional manual attention.
Code Review - becoming increasingly popular in the United States, and to some extent in Europe, is the review of the source code of an application for security vulnerabilities, either on a manual, automated, or combination of both basis. Manual analysis of source code is extremely resource intensive and as such can be very expensive, however recently a number of static analysis tools (from vendors such as Fortify and IBM) have become effective in allowing the analysis of larger volumes of source code for security issues.
The upside to this approach is that it should provide far greater coverage than black box testing, as all of the code included in the solution and all possible execution paths can be evaluated, potentially allowing a lot more issues to be discovered - including those occurring from edge cases that may not be picked up with other approaches. The downside to this approach is that it can still be resource intensive, even if using automation. Also, the pace of technology change restricts the ability for automated tools to support newer technologies - the wide variety of technologies and frameworks in use means that it is extremely difficult for automated tool vendors to support a wide selection of the technologies that are currently in use. Code review is commonly used in areas where security failure would be especially critical or result in major loss.
Platform Defenses - another area that is seeing increasing use if to have platform (or in some cases, framework) level defenses in place, normally so that some level of protection is afforded to applications running in that environment regardless of whether the application itself is secure. Some widely spaced examples are buffer overflow protections (such as stack canaries and non-executable stacks) and web application firewalls.
The upside to this approach is that, in many cases, the application developer will need to make minimal (if any) changes in order to take advantage of these security features. These can also be a useful approach for providing additional security over applications purchased into an organization. The downside to this approach is that they can only provide a band-aid level of protection if an underlying security issue does exist. Also, for each of the approaches there is generally widely known issues, limitations, and bypasses for getting around that particular control.
Training - always popular is security training for developers. For enterprise situations this can be customized to be most relevant for the types of issues that are most commonly present in that organization's applications. Upsides to this approach are that a good training program can increase awareness of issues, and change development practices in the short term. Downsides are that training is a point in time activity, and as such must be reinforced or followed up to ensure that new starters are trained and that trained security knowledge is not lost over time.
Architectural/Design Analysis - areas that are increasingly seeing attention are approaches that seek to tackle security issues at the requirements, design and implementation stages. These could take the form of security reviews of the architecture and/or design and threat/risk analysis activities (such as Threat Modeling).
Upsides to analysis at this level include the ability to detect and correct architectural or design issues that would be extremely difficult (or in some cases, impractical) to address during or after development, potentially leading to significant cost savings over addressing these later in the development process. The main downside of this approach is the difficulty of developing and maintaining these types of analysis with popular development approaches (e.g. agile development approaches), largely due to the velocity of changes to the design during development.
OK, so those all work then? And other people are using them. OK, I'll pick the best one of these approaches for me and I'll be fine!
Well, no. Anyone who tells you that, or uses that approach while selling you something is selling you a load of FUD... there is no silver bullet. First of all, there isn't any real amount of scientific research available which would give you any guidance as to what approach would be the best, or the best for you. Also, generally held belief if you have a look at what is available, especially if you look at what information is available on what other organizations (such as Microsoft) are doing would seem to imply that you need to be doing all of these approaches and more. Which would be both far too expensive, and far too much change for most organizations to even consider implementing.
So, is there anything out there that would tell me what I should be doing if I want secure software? Something useful, and FUD-free?
Well, yes and no.
Recently a couple of closely related frameworks have been released that at provide some structure to this question though. Although a number of approaches to the Secure SDLC have been publicly available in the past (notably Microsoft's SDL), these take a different approach of looking at the maturity of security considerations in the development process. These are the Open Software Assurance Maturity Model (OpenSAMM) and Building Security In Maturity Model (BSIMM) models. These two models both look at the processes you would normally expect to see within a "Secure SDLC" at various levels of maturity in a similar way to a Capability Maturity Model. The main difference between the models is that the research that has gone into BSIMM captures actual secure development practice at large organizations with initiatives in place (such as Microsoft, Google, and Wells Fargo) whereas OpenSAMM presents a more academic model of leading practice.
These models are very useful for answering questions about what you could do - especially BSIMM. They are even pretty good at giving you information and materials that would help justify why you would want to introduce certain activities. What they're not good at is providing you with the vision of secure development you'll need for your organization. They are good at providing a framework for tying together disparate efforts into a consistent picture, but you'll have to provide the driving force behind how you will get from where you are now to where you want to be. They are also pretty good for having in your back pocket for after you've got a couple of secure development activities in place (through guerilla means or otherwise) to be able to relate gaps in the organizational processes and skillsets to activities that should be looked at next.
Does this answer the question of what your organization should do? Well, no - what it may do is help you figure out for yourself what would work for the organization. After all, its you who has the knowledge of how the business works that is critical in the success of getting any business change in place.
And after all that, what is the activity that both OpenSAMM and BSIMM both consider to be the most important things with developing secure software?
Pentesting?
Code review?
Nope - its having someone who is championing and driving software security within the organization. Having a group of folks who are ready and willing to shepherd and drive through all of the various changes to how the organization works over time. These are sometimes (in BSIMM in particular) referred to as the Software Security Group (SSG), and in many cases can be make or break in getting adoption and use of security initiatives within the organization.
After all of that, it turns out the best thing for software security in your organization may well be you...
