Since the founding of Fudsec we've looked to expose FUD, but until today it's been a little like Justice Stewart's definition of obscenity - I can't define it, but "I know it when I see it." In this week's invited post, information security and risk management consultant Gal Shpantzer blows the lid of that problem with the Shpantzer Coma Scale. We at the Fudsec Institute For FUD Studies are delighted that he could bring clarity and metrics to such an important topic - because if you can't measure it, you can't ... well, you know.

By Gal Shpantzer (@shpantzer)
When considering the veritable cornucopia of vendor offerings in the information security niche, you'll find a spectrum of quality in the products and services themselves, from the ridiculous to the incredibly useful and well-designed. You'll also find a wide variety of approaches to sales and marketing these very same products and services.

Some vendors are consistent and have good products as well as sales/marketing teams. This is a rare vendor indeed. Treasure them if you find them. The majority within the vendor space have either good products or good marketing. Then there are those with neither. Inconsistency breeds hilarity.

Please consider this friendly scoring system, inspired by a combination of the Glasgow Coma Scale, APGAR and some other medical scoring schema for survivability of trauma and disease. Note: We're carefully calibrating the rating system with an old Cray supercomputer in @rybolov's basement. YMMV)

Let's add up some points!

Vendor inappropriately uses absolute terms like "always" and "never" in order to delude the sucker, er, I mean prospect into thinking there's any certainty to be had in the security niche. Take one point off for every absolute term, starting at 5.

• Bottom score of -5 for FUD lameness.

Number of minutes from start of presentation until vendor uses the term "APT".

• +1 point for every minute past start. Max -5 points 
• Bonus 3 points for not mentioning it at all, unless prompted to.

If, when prompted to address APT, vendor says "oh yeah, we've been doing APT since before 9/11".

• -5 points

If, when asked, "How do you approach the APT issue, exactly?" they respond "That's on our roadmap".

• -5 points

Vendor claims to fully detect malware on your endpoint. The more certain the claim sounds, the more points you can take off, starting from 5.

• Bottom score of -5

Vendor has something that goes beyond a default OS build for its products: Starting at 0, add points for each aspect of security hardening credibly claimed.

• +1 point per feature, Max 5 points.

Vendor has credible claims to integrate with relevant third party applications and services.

• +1 point per feature, Max 5 points.

Vendor offers some level of choice in pricing model.

• +1 point per choice, Max 5 points

Vendor has recent history of catastrophic encryption implementation failures.

• -5 points

Vendor offers a 99% discount off retail pricing for year one software licensing. When pressed for total cost of ownership over 3 years, they reveal their plan to stick you with maintenance based on MSRP for years two and three.

• -5 points

Vendor offers some level of ability to update and upgrade the software they're selling.

• Max 5 points

Vendor actually responds to vulnerability reports in a way that remotely resembles something a reasonably responsible business would.

• +5 points

Vendors offers some level of centralized management of distributed product.

• Max 5 points

Vendor's central management of said distributed product causes DoS on your network.

• -5 points

Vendor has some sort of third party certifications for their crypto library and/or device as a system (FIPS 140-2, Common Criteria, UK gov't, German gov't, etc).

• Max 5 points

Vendor doesn't use proprietary encryption algorithms (yes, this is still being done, see Onix International and EncryptStick polymorphic…).

• +5 free points for using AES or other accepted algorithm.

Vendor has technical capability to deploy in a flexible manner, to suit your virtualization strategy, if relevant.

• +5 points

Vendor has real scalability in technical and pricing terms. Ask for references, don't just buy the canned demo.

• +5 points

Vendor has reasonable licensing terms that allow for configurations that serve different use cases.

• + 5 points

Vendor can integrate with two-factor authentication tokens/cards, at least for administrative interface.

• +5 points

Vendor is very negative and constantly disparages other competitors in their space.

• 0 points

Vendor is negative when disparaging obvious lamers like those who use polymorphic encryption.

• Max +3 points.

When asked about reference customers, vendor claims that the entire DoD and civilian government uses their products. When pressed for a confidential phone call, under NDA? "That's classified, but just between you and me, we're all over Langley and Ft. Meade."

• -25 points and a call to the FBI Counterintelligence office.

Vendor is an otherwise credible up-and-coming security player that has been around for more than a year and can legitimately support an enterprise customer, in theory.

• Max 5 points.

Vendor product does in the testing lab something close to what it says in the slideware.

• Max 5 points Bonus 3 points for having a reasonably responsive pre-sales engineer available via webex to help with a qualified bake-off.

Vendor is an otherwise credible security player that's been around for a while and has actual, reference-able enterprise customers.

• Max 5 points.

SCORE:

Negative Score: Bring back the pillory and the scarlet letter.

Under 30: Run, don't walk. Then keep running. Write a blog post to lower your blood pressure.

31-50: Ask for a webinar and have them explain polymorphic encryption to you.

51-70: Possible long-list candidate with value play

71-90: Probably gonna make it to shorlist for tech eval

91+: Might be able to deliver on the promise and not the peril