How to neutralize a single point of failure

Imagine for a moment that all your business relationships, sales, and profits depended on one small behind-the-scenes interaction — a quick contact with a provider or an automatic operation in your system. If that one action were to fail, your entire operation could immediately fall apart.  

Sounds a bit dramatic, doesn’t it? Unfortunately, this is the reality for many small businesses: since resources are thin and simplifying is a top priority, it can be difficult to put all the necessary backup plans in place, leaving operations to rest on a single point of failure (SPoF).

What is a single point of failure in business?

A single point of failure can take different forms. A person can be a SPoF if they have restricted access or if they’re the only one who can handle certain business tasks. In terms of technology, a SPoF is often a piece of software, a network, or a point in a supply chain that will stop the entire system from working if it fails.

As cyber attacks grow bigger, stealthier, and more frequent, smart cyber risk management becomes more and more important for small businesses. One key element in your strategy should be to identify and remove any single point of failure — and where that’s not possible, learning what protections to adopt to avoid a crippling cyber incident.

Am I exposed to a single point of failure?

Considering a near universal dependency on the supply chain, you can safely assume your business is exposed to a SPoF somewhere down the line. But you might also have one or more SPoFs closer to home, like a database or a managed service provider (MSP) that oversees your IT infrastructure.

Determining your risk level

Not all companies can afford extensive IT departments, equipment, and redundancies. In the real world, many (if not most) businesses will need to outsource some tasks, count on third-party tools, or depend on one or two internal experts — and this is where the SPoF threat is greatest.    

You can answer a few key questions to determine your SPoF baseline risk:

What’s the state of my hardware?

If a piece of hardware on the server side or the user side fails and you don’t have a backup in place, you could find yourself in a troubling situation. Protecting against this SPoF can involve elements like a redundant power supply or automatic data backup to another drive.

Do my people share knowledge?

While it’s important to employ the principle of least privilege (PoLP) for your experts in charge of crucial processes, the requisite knowledge and understanding to handle those tasks shouldn’t fall on any one person. When you’ve got one or more people who you truly couldn’t operate without, you’ve got SPoFs.

What does my vendor list look like?

A managed service provider (MSP), offsite data storage service, contracted maintenance operator, or other third-party vendor may have software vulnerabilities. If there’s a problem on their end (like a system failure or a ransomware attack) and you have no redundancy in place, you could be sidelined for as long as they are — or worse, you could be breached.

The bottom line: you must understand how your systems and processes rely on each other, but you also need to think beyond your own security to truly be protected.

Supply chain attacks and SPoF

Managing supply chain risk is important, given that there can be hundreds, if not thousands, of SPoFs in an IT environment. Third-party software can be corrupted in a number of ways: from malicious code injection to vulnerabilities programmed into open-source software, businesses along the chain could be at major risk simply by using their standard operating tools.

The downstream effect

In a supply chain attack, a particular asset is targeted and compromised (like a provider’s software) with the intention to impact those downstream (the software provider’s clients). In a case like this, a SPoF not only endangers the initial target, but all the connected entities, too.

Without a separation between client environments, they’re all vulnerable if an attacker targets the shared service.

What’s on the line?

There have been a few notable supply chain attacks that have rippled across the commercial landscape in the past few years. The NotPetya attack in 2017 caused $10 billion of damage across the globe — $300 million of losses was reported by Maersk alone. The SolarWinds attack in spring of 2020 transferred malware to about 18,000 organizations, both public and private; experts suspect the aim was to infiltrate major technology supply chains.

Large tech companies aren’t the only targets. When a ransomware gang known as REvil exploited zero-day vulnerabilities in Kaseya VSA, an IT management product used by MSPs, they were able to infect any and every company that used the compromised MSP server with ransomware. The Kaseya attack of 2021 has come to exemplify the SPoF danger to small businesses: without a separation between client environments, they’re all vulnerable if an attacker targets the shared service.

Attacks like Kaseya, SolarWinds, and NotPetya showcase how indiscriminate supply chain attack damage is (even if certain targets were prioritized) and how difficult it is to control the consequences. Larger software vendors have so far sidestepped debilitating attacks, but a major disruption to platform vendors like Microsoft, Apple, or Google could wreak havoc on all types of business.

Protecting against SPoF

Mitigating a single point of failure means adding another “point” to the mix to strengthen your defense with a safety net. You need to put backup plans, aka redundancies, in place. Here’s how to start.

Audit your software, systems, and services

The first step is to identify where the problems are, which can be easier said than done. So much third-party software is often bundled with other software that is bundled with other software. In turn, experts are predicting that a Software Bill of Materials (SBOM) to vet your digital supply chain may become a requirement.

With a focus on the three main sources of SPoFs outlined above — your systems and hardware, service providers, and people — have your IT team search for issues. Namely, they should be on the lookout for things like data that isn’t backed up, any hardware or software systems that have no redundancy, and any unmonitored devices on the network.

Weigh the risk

You may not have the time, budget, or help to deal with all the SPoFs you find, and that’s alright. Consider the nature of each and decide if it’s worth the effort to resolve.

For every part of your network, identify what you would stand to lose if this particular “link” were to go down. Would everything grind to a halt, or would it temporarily make for some extra work? Would the future savings and peace of mind justify the cost to fix it?

Some SPoFs may be quick to resolve, so they’re well worth your attention. Data backups would fall into this category. However, others may be too expensive to deal with — in these cases, you might want to consider other business continuity options instead of putting redundancies in place.

Call on experts

To expedite this audit process and your redundancy plan, you might consider hiring outside help to identify potential SPoF vulnerabilities.

Breach Counsel
While people generally turn to a breach counsel during or after an incident, this legal service can also prove valuable before a cyber incident. With a pre-incident risk assessment, they can help you identify potential technical exposures. A tabletop exercise can surface questions to get you thinking about the consequences of specific system failures. Both are useful ways to spot SPoFs.

Incident Response Firms
Likewise, an incident response firm can provide technical guidance and expert advice on how and where to look for SPoF exposures before they catch you by surprise. They will often work with your breach counsel to cover all angles.

Wondering what sort of support you can get with Elpha Secure? We’ve brought together leading incident response professionals who can help you before, during, and after a cyber incident. Check out our panel of cyber incident response providers!