I'm delighted to welcome back Nick Selby, now Managing Director of Trident Risk Management, for a special fudsec Thanksgiving edition.  Thanks Nick!

Update 30th November: modified text at request of Nick re: mitigation to avoid distracting from the main point of the post - Ed

By Nick Selby

The critical infrastructure security debate has reached, well, a critical juncture. However in the United States, the debate has been limited to either more government regulation or proactive mitigation on the part of private utilities. Since I write from America on the day we Americans give thanks for that which founded our country and made it great, let's attack this issue from a third front.

Let's get the customers pissed off, so that they vote with their wallets.

Because the US' infrastructure is mainly privately owned, the only way utilities will upgrade or properly configure their systems is under pressure of market demand for it. If the US business community, armed with the understanding of the risk of utility interruption to their enterprises, demands better service - that is, they demand that their businesses are better protected by those they pay to provide them with power - then the utility markets that are the most competitive will become the safest.

There's a strong business case here: many exploits of the vulnerabilities in our electrical power grid cost little to mount and cost a lot to remediate. As security researchers, practitioners and thought leaders, we can articulate a business case to American business leaders:

• You're being forced to accept risk. Utilities are offloading risk of an outage to their customers, by charging for power and reliability and not mitigating even obvious, well-known risks;
• The risks are to your people, your property and your profits - that is, they are to your brand. If your business relies on power for mission-critical or safety processes, the failure on the part of utilities to remediate means your customers and your brand are at risk - on the terms of the utilities, not your business' risk managers or your shareholders;
• Cleaning up after the risk becomes reality is a hidden tax on your business and on you. The consequence management aspect of a loss of the power grid of even 20 minutes are massively high in terms of life, safety, profits and our national sense of well-being and safety. By not remediating these risks, utilities are offloading to us taxpayers the cost of clean-up and restoration after a catastrophic failure.

With respect to the last point, I seem to recall us fighting a war over taxation without representation. I submit that this is another one. I know that some utilities will be mad at me for saying this, but as far as I can tell, they've had their chance to take action. Now it's our turn.

Some high-level context
This may be stating the obvious, but what's obvious to people who look at this problem a lot is not obvious to people who don't.

For years, public and private security researchers have been pointing out that the networks at electric utilities were reliant on the thinnest veneer of security - if that. This was not because utilities didn't care, it was because utilities built themselves for the functionality of production of electricity in an age when their networks were truly air-gapped - that is, they were physically separated from the Internet.

To further state the obvious, one big problem is that these networks haven't been truly air-gapped for years and years, but the utilities continue to behave as if they are. And there's a great deal of reliance on plain old security-through-obscurity.

The government can make recommendations and even some regulations, but at the end of the day, and here's another obvious statement, the reason the majority of electric utilities in this country haven't upgraded their security is because doing so is expensive and there's not been any publicly released information about a compelling reason to spend the money.

Hacks or DOH! - Cause Is Less Important Than The Impact
Whether a successful attack on a US utility has happened already, it will happen (not for nothing, but there are active investigations of such attacks underway now). Regardless of the cause, bringing down power networks has life-and-death consequences. Security professionals sometimes forget the 'A' part of the CIA triad (of confidentiality, integrity and availability).

I wrote recently that in 2008 an ice storm blacked out much of my county for eight days - my family spent eight days with sub-zero temperatures and no water, heat (except my woodstove) or even telephone. Life changed dramatically for us, very fast. It is, being obvious again, very important that we safeguard against attack or misconfiguration or any other event that brings down the power grid.

In a recent post on Errata Security, Robert Graham rightly pointed out two important things:

As a pen-tester, I know that our power grid is insecure...I know I can hack in from the Internet and cause power outages. However, government regulation isn't the answer.

Not only has government regulation not been the answer, but private industry has ignored, largely, government initiatives of exactly the kind I would expect would resonate with the security community and the public at large. In many cases, the guidance is specific, limited in scope to what is necessary, driven by expert analysis and input from leaders in security research, vendors, private and public employees and regulators; in short, it's the findings that come after Mr Smith went to Washington.

And still, it's pulling teeth.

A Good Example: Aurora
A perfect example is the Aurora vulnerability (See the Power Point here, page 8, for more), because it has been public knowledge for about two years, the cause is understood and the mitigation is straightforward and well-understood. There's so much great published research and congressional testimony on the problem and its solution that I cannot believe that there has been such low takeup in doing that.

In just two days of scouring open source, unclassified documents I was able to put together a basic mitigation strategy sheet (and to scare the crap out of myself about how easy and inexpensive it would be to mount an Aurora attack). Yet, anecdotally, it seems that only a really small percentage of substations have been protected against this well-known vulnerability. By the way, I don't charge customers to see this remediation sheet.

What Is To Be Done?
After consulting with a number of people in and out of governments, I've decided that the best way to use this information is, at no charge to them, telling businesses which depend for mission critical processes on the public power grid. The at-no-cost part is important to me, because I believe that this is an issue too important not to share.

It's my hope that in sharing this information, outlining the issues and explaining to business leaders how they can and should raise them with their utilities, the utilities will see that there is in fact customer demand for mitigation, and come at this from the market side.

I had asked for a debate and a discussion, so here's my contribution: I'm suggesting all pen-testers and consultants who've looked at this to get vocal - find something within the field that raises your level of concern, something that can be mitigated rather easily.

Then, as opposed to trying to monetize that knowledge directly, help your customers articulate concern in a way that matters to the private utilities: "We, your paying customers, find this to be a risk that you should mitigate. Please do so." We should also help the utilities find federal money to contribute to their effort to help mitigate these risks. Hell, if they're going to throw all that money around on "infrastructure" projects let's at least get some in this area - the government has made it clear that it would like to.

If many of us who have the ear of the customer and the knowledge of the issues do this in a constructive way, we can go a long way to raising the bar. In the end, the real questions remain,

• How hard is it to exploit vulnerabilities in our system?
• How can we make it harder?
• What help is there for private industry to raise its bar?

Many have said that action is not that important, because "no attacks have happened yet on American soil." Arguments about whether attacks have happened are for another forum, but if your main argument against mitigation is justifying the cost with evidence of an attack, I'll ask you this question:

What is the cost of wrong?

I hereby pronounce today "Cyber-FUD-Friday".  I don't know about you, but I tend to whince anytime someone uses the word "Cyber".  Combine that with an emotive word like "war" and suddenly everyone has an opinion and is touted as an "expert".  Huh, kinda reminds me of Cloud Security ;-).  This weeks guest post delivers a much needed dose of perspective.  Thanks Jayson!

By Jayson E. Street

456 BC: Aeschylus, a Greek playwright, was killed when an eagle dropped a live tortoise on him, mistaking his bald head for a stone. The tortoise survived.

Dying by a falling turtle has been documented and therefore is a proven threat. However it still remains unlikely for you to die that way. Cyber-War (what the cool kids are calling it) has in fact happened.  This proven threat does not necessarily mean a country’s smart grid is going down anytime soon.

I started doing research for a book I am writing which includes cyber-warfare. During that process I was startled by a few things I observed.
1.People who know what is going on don’t talk about it to either confirm or deny it. Conversely, people who don’t really know what is going on have no problem speaking about it at great length with much authority.
2.In a realm where anonymous attacks are the norm not the exception, people are really quick to lay blame on who is doing what.
3.Everyone is INVOLVED!

Observation One: I am not an expert on cyber-warfare. This is just something I started researching for supporting material in a book. Like a lot of people I had been reading about on this subject, I had not been to any of the countries commonly named as participants in cyber-warfare.  I knew I would not get good answers without “boots on the ground” experience.  I applied for my passport and took my first trip outside of the USA.  I wanted to see what was really going on.

The best place to begin seemed like China.  After all, the people where were doing the talking were dropping that name with great frequency. I attended Xcon where I had dinner with GoodWell, the founder of the Green Army.  He is commonly known as the godfather of the Chinese hacker movement in with activity going back to 1997. He has gone the way of his Western counterparts.  He has left his past to apply the knowledge gained from underground hacking and illegal breaches for a more legitimate profession that pays better and comes with cool business cards.  He now consults with billion-dollar clients.

I was amazed to sit there and listen to his concerns of how hacking has become more a tool of crime rather than exploration and political action. Here was one of the major figures of the Chinese hacking culture expounding on the problems with criminal hackers and worried about so many attackers assailing Chinese networks. In fact, the typical Chinese home computer user is under constant attack from bots, Trojans and also a virus here and there (sound familiar?). 

So my first trip abroad was a real eye opener.  I learned to not be so quick to judge or take everything I here about “Cyber-Warfare” as gospel.  It was after I returned home that I started listening more to what “experts” were saying about cyber-war.  I realized most have been using data from certain 2003 incidents.  Their opinions were not based from data gained first-hand.

Since then I have traveled to other countries and gained a more open perspective of what is going on in this realm. The most important thing I have learned still remains what I knew from the beginning. I am not an expert, but I can form opinions based on what I know first hand.  I am limited to information in the public domain, but that is not all there is to the story.  Most of the sources offering opinions have the same limitation.

Observation Two: I believe this to be the biggest problem facing those who are on the front lines – the battlefield is virtual. A physical attack is much easier to detect and trace back to the source. You can see the path the attackers take.  You can see the bullets they fire. The person attacking you with a DDOS is harder to trace.

The recent attack on South Korean and United States websites showcases the perils of being quick to judge and even quicker to accuse. For example, within a week of the attacks Congressman Peter Hoekstra of Michigan (1) insisted we needed “to send a strong message.”  Yet to this day there has been no positive proof who was actually responsible.

With $50,000 USD anyone can hire a botnet to replicate these attacks. It is that easy because most criminals are not motivated by politics but by money. This also poses another problem. When anyone can hire or create their own army of compromised computers does it make the impact less because it was a guy in Paraguay who was curious and wanted to see if he really could take down the White House website? In a way it would be more comforting if such activity were limited to the high tech branch of a rouge nation launching an opening salvo in a cyber-attack. That can be an easier target for a response.  But the same damage is felt regardless of who dealt the blow.

As time goes on expect to hear about more cyber attacks that are “thought” to be either this country or that country but with no publicly available proof of who was responsible. This is a problem that will not be going away. So how can you protect and more importantly trace the attacks when the bullets appear from everywhere including from your own side?

This brings us to Observation Three: who is now involved in cyber-war activity? The answer is EVERYONE! I would say (just my opinion based on my research) that most every industrialized nation is working on a military hacking division (or whatever a government wants to call it). The Chinese were probably the first with the Indonesian cyber-skirmish in 1998(2). 1998 was also a notable year for the ramping up of cyber-warfare capabilities in the USA.  Attacks on Serbian air command were used to help facilitate USA airstrikes as well as targeting enemy bank accounts (3). Also in the late 1990s, a computer specialist from Israel's Shin Bet was able to compromise the mainframe of the Pi Glilot fuel depot north of Tel Aviv (4).

So here we are over 10 years later still wondering what “Cyber-Warfare” is, who is doing what, and what can we do to defend ourselves?  It is also a safe assumption that everyone is also getting much better at attacking.

We are not learning from the past and the old adage bears true that we will likely repeat it. The 1980’s were the decade to fear the nukes. This decade we fear the digital arsenal. The good news is we did not die in atomic fire (though that was a proven threat). The bad news is we found something else to fear (and we always will).

We need to understand the threat of a digital holocaust is a possibility. And so could a nuclear war break out, Swine flu become an epic pandemic, a meteor wipe out all life on the planet or a falling turtle kill one of us.  The threats are real.  But should we panic?  No, probably not.

1. http://www.scmagazineus.com/cyber-retaliation-debate-is-north-korea-guilty-of-ddos/article/139968/
2. http://www.disasterpreparednessblog.com/disaster-preparedness-blog/2009/10/22/chinas-cyber-warfare-capabilities-highlighted-in-report-to-c.html
3. http://findarticles.com/p/articles/mi_qa5332/is_1_48/ai_n28827258/?tag=content;col1
4. http://www.alertnet.org/thenews/newsdesk/LV83872.htm

TGIF!  A recent flashmob poll of CISOs discovered that the flagrant abuse of statistics, graphs and number theory misleads at least 5*9+(sqrt 10)^3 of decision makers "most of the time".  Returning guest Lori Mac Vittie came across a recent "study" that caused her to reach out for a key tool of the professional defudder -  the humble calculator.  Ah yes, ladies and gentleman - every number tells a story - which shelf in the bookstore that story belongs is a different matter.  Read on as Lori takes aim at the numbers from a recent "study".  Thanks Lori!

Lori Mac Vittie

Technical Marketing Manager for F5 Networks.

The latest study “State of Internet Security” from WebSense indicates that 95% of all user-generated content is, well, to put it simply: crap. Even more frightening is the conclusion that “61 percent of the top 100 sites either hosted malicious content or contained a masked redirect” and “77 percent of Web sites with malicious code are legitimate sites that have been compromised.”

OMGWTFWEB2.0?

It’s enough to keep you away from social networking sites, surely! After all, the “top 100 most visited Web properties…tend to be classified as ‘Social Networking’ or ‘Search’ sites.” Facebook? Twitter? MySpace? My god, they’re probably all infected. Grab a face mask and pull that cable from the wall lest you catch some social (networking) disease from visiting your BFF Jill’s Facebook page.

Now that we’re done (I hope) having hysterics and fear-induced panic attacks, let’s consider the math for a minute, shall we?

Facebook now has 300 million users. Let’s assume that each user has at least one piece of user-generated content on their Facebook page cause, well, it’s a very user-content driven site. That means that of the 300 million home pages on Facebook that 95% (285 million) has either a malicious link or other insecure content. Conversely that means that 5% (15 million) are clean, uninfected, safe pages.

The average Facebook user has 120 friends or 281 friends, depending on which news article you might be reading. Let’s just assume for mathematical purposes that the number is somewhere in the middle, at about 200 friends per user. Let’s pretend, too, that you visit every friend’s page in a single day. Because it’s your day off, of course, you wouldn’t actually do that at work.

The mathematical likelihood that one of your 200 friends is one of the 95% that is infected is infinitesimal. Visiting a second-order friend (a friend of your friend) makes it more likely, but in mathematical terms one could still categorize the risk as statistically insignificant. In other words, all this hubub about how much content is malicious and insecure is a blown a bit out of proportion; considering the magnitude of the numbers we’re dealing with we could say 99% of all content is crap and still not raise your security risk much higher than it is today.

That is, of course, purely a mathematical view of the security risks associated with social networking. Generalizing statistics can be useful, as can statistical sampling. But we  - both as pushers of that data and as consumers of the same – need to be more aware of how the magnitude of the data behind those statistics affects the actual risk involved. It’s always more fun to say 95% than to give a real number, especially when those numbers are so large that they essentially lose meaning to human beings. And we know that people will interpret 95% to mean 95% of the content they visit because that’s the way it’s presented. But is that reality? Likely not, unless their behavior on-line is such that it puts them more at risk because they’re visiting and connecting with a higher percentage of the content out there.

The reality is that there’s only so much providers and vendors can do to protect individuals online. Web application firewalls. Firewalls. IDS. IPS. Vulnerability scans. Anti-virus. SPAM filtering. These technologies are necessary to reducing risk in general and they do, but the best and primary protection mechanism in every user’s arsenal should be themselves. Users need to educate themselves on the risk inherent in today’s increasingly connected web of content and proactively examine content presented to them with a more educated eye. And they need to be aware that at least part of the risk incurred from user-generated content is self-inflicted: the more content, the more friends, the more connected they are, the higher the risk of stumbling into malicious content.

The danger in generating such a false sense of insecurity is that users will begin to fear content and links to content, which means they’ll fear the Web in general because the whole premise of the Internet, of the World Wide Web, of Web 2.0, is links and content and the intricate relationships between them. The web is useful because of links and content and user-generated content and yes, much of it contains malicious code and other nasty tricks. But rather than scare users with statistics that don’t accurately portray the risk to them we ought to do a better job educating them on how to recognize malicious content and provide simple ways for them to report or tag or otherwise mark malicious content when they do find it so we, as protectors of data and users and content, can continue to innovate new ways to automatically handle removing such content from our applications and sites.

Instead of scaring users let’s engage users and make them part of the solution rather than just another part of the problem.

By Paul Asadoorian

Microsoft patch Tuesday, or "Black Tuesday" as it is referred to as, has been around for some time.  At first it seemed like a good thing, with all these patches coming out it gave people time to plan accordingly and get them installed in their environment.  I firmly believe it was a knee-jerk reaction that has plagued us since 2001-2003 when "worms" and network-based vulnerabilities were being exploited on a regular basis.  The time has come to react more quickly to the wide variety of threats posing organizations infrastructure, and have the flexibility to apply patches as they come out, not just once a month.

I remember it fondly as I worked for a University at the time patch Tuesday was being dreamed up.  It was a time in a land where XP firewalls did not exist, nobody blocked the NetBIOS ports from the Internet, and the term "automatic updates" was something that did not exist.  Machines were getting compromised faster than they could be rebuilt, in fact many became compromised WHILE they were being re-built.  At some point, someone hit Microsoft with a clue bat and they began coming out with patches, lots and lots of patches.  Before you know it systems administrators were overwhelmed with patch installations, and were not armed with the appropriate tools to handle the patches.  So they do what some systems administrators do best, they bitch.  I have to admit, when I was a systems administrator, I did my share of bitching.  Its that kind of job, somewhat thankless, so bitching came with the territory.  But in this situation, it caused Microsoft to take a step back and think about how they release patches.  Then, Patch Tuesday was born (along with exploit Wednesday).  This meant Microsoft was now holding back patches, saving them up for one release!  While I started to grow hatred for this method, systems administrators did less bitching, which makes everyone's lives easier.  Another factor that played heavily into the decision to release regular patches was desktop management.  Very large corporations with 10s of thousands of desktops incurred costs when applying patches.  Each of these desktops, and servers, needed to be rebooted in order for the patches to apply!  This adds an economic factor to the equation, meaning is costs money for organizations to apply patches.

Times have changed and its time to evolve.  The regular patch release schedule got us over a hump and got systems administrators in a grove of applying patches. However, Imagine this situation:  

There are terrible viruses running around in the real world (like ones that infect you as a person, not your computer).  Your doctor is getting the vaccinations as soon as they are available.  However, its left to your doctor to determine when they are applied.  So to make things easy for everyone, vaccinations are done on the second Tuesday of every month.  In the mean time, you and your family (are users like children?) are vulnerable to the virus, some of which can be fatal, and others that are not so bad.

You should be equally as outraged about patch Tuesday, and here are some things you can do about it:

Set your own patch schedule

This is something I have been preaching for some time.  As an organization you need to evaluate the threats against your business, and prioritize the defensive measures you implement (if any).  You should not have to wait for an third party to release patches, they should be applied according to your own priorities.  Security is about evaluating risk and I can assure you that you can do a better job than Microsoft of evaluating risk for your organization.  

Define your own threat level

The other aspect of security that Microsoft believes they have covered for you is determining the criticality of each bulletin (which can contain multiple vulnerabilities, which doesn't help!).  What is deemed "critical" by Microsoft may not be as critical to your organization.  Evaluate each vulnerability, not just the bulletin, and think about the impact it has on YOUR organization, not the rest of Microsoft's customers.  Home users, small businesses, governments, health care, universities, they all use Microsoft products in some capacity, do they all treat threats the same way?  Hell no, and neither should you.  Define what "critical" means to your business, and that means reading each of the security bulletins when they are released, in detail.  So, on the second Tuesday of every month, get a bigger cup of coffee, sit down, relax, and enjoy the ride.

Conclusion

Dumb Questions: Is it "wormable"? Is there "public exploit" code available?

The real question you should be asking: "How does this impact our business?".  

Lets get one thing clear, evil bad guys have exploits.  They have exploits for stuff we don't know about yet.  When a patch is released, its too late. Shortly after a patch is released, its really too late.  So patching needs to be built right into your operations and balanced with your business plan.  Don't get caught up in the hype around "remote exploits", "wormable", and all that crap, take matters into your own hands, at least as much as Microsoft lets us. The game is risk mitigation, not patching.  But we can't rely on MS to provide reasonable, workable, mitigants to many of their bugs based on track record.