We’re breaking rank and posting a day early this week.  Why?  To give this post some time to breath before a small gathering in San Francisco of security wonks.  My thanks to Jeremiah for this post, and I fully agree with his call to action!  You?

By Jeremiah Grossman

I’m told fudsec is a place to float, among other things, half-baked and incomplete security ideas. I’ve no shortage of those I assure you. Fortunately the infosec community is not shy about telling you so. For today’s thought let’s provide some background…

A few weeks ago a consultant by the name of Larry Suto published, “Analyzing the Accuracy and Time Costs of Web Application Security Scanners,” [1] which reviewed desktop black box website vulnerability scanners: Acunetix, IBM AppScan, BurpSuitePro, Cenzic Hailstorm, HP WebInspect, NTOSpider, and Qualys WAS (Software-as-a-Service). Larry faced off these products using the vendors’ very own public-demonstration, vulnerability-laden “test websites” as the scan targets. For those curious, WhiteHat Security politely declined to participate because Sentinel is delivered as SaaS solution and not a product like the others tested. [2]

You may read the report yourself, but I’ll save you the suspense. The results for nearly all scanners were basically horrible. Large percentages of vulnerabilities were missed, there were false-positives galore, and significant human configuration time was required. Perhaps these are benefits if you are looking for tools to help fill the gaps in your day and provide job security. Several vendors wasted little time in defending themselves, attacking the report’s methodology and Larry himself, which is presumably to be expected anytime you call someone’s baby ugly.

The conclusion from the vendors: Don’t take these results seriously. For best results, scan real-live production websites, like your own environment, and not test websites.

You know, I can agree with that! I’ve been recommending the same for quite some time. First though, try something a little different. Turn the tables around. Instead of running your websites through the gauntlet, risking downtime from intrusive scans, only to discover you have vulnerabilities just like everyone else — how about making the vendor eat their own dog food.

Ask the sales rep for a trial license and permission to scan THEIR production commerce website(s). That’s right, scan the vendor! Imagine their FUD-induced response. If they really believe in their product’s capability, safety, and marketing hype this shouldn’t be an unreasonable request. A “right to test” is no more than any reasonable cloud computing client would ask for. Right? Plus, doing so will provide a good reference point for when you scan your own websites, if, in fact, Larry’s results were atypical. The sales rep might say they don’t have authority to grant such authorization. Fair enough, but go ahead and press a little. It’s not like the bad guys are asking permission to scan these sites everyday anyway. Just ask [3] xssed.com [4].

[1] http://ha.ckers.org/blog/20100203/accuracy-and-time-costs-of-web-application-security-scanner-report/
[2] http://jeremiahgrossman.blogspot.com/2010/02/wheres-whitehat-re-scanner-comparisons.html
[3] http://www.xssed.com/search?key=hp.com
[4] http://www.xssed.com/search?key=ibm.com