The Pyramid of Pain and Sandfly

Linux Security Linux Forensics

June 20, 2018
The Sandfly Security Team

There is a great post called The Pyramid of Pain by David Blanco that details the six ways to cause adversaries the most trouble when attacking your network. The Pyramid of Pain is below:

Pyramid of Pain

We’ve been in the security industry for years and the Pyramid of Pain is the best way we’ve seen to illustrate the ultimate goal of intrusion detection systems (IDS) and other security products. That goal is to cause adversaries so much hassle that they move onto easier targets. In terms of persistent attackers, we want them to expend tremendous resources to accomplish their mission which lowers their overall effectiveness.

The Layers of the Pyramid of Pain

We’ll summarize the layers of the Pyramid from the perspective of Sandfly. This can give you some insight into how Sandfly’s agentless intrusion detection operates and why we do and don’t do certain things to find attackers.

Hash Values – Your basic file signature matching scanner. It can detect static attacks, but falls apart when the slightest change is made to the code. Sandfly does not use any kind of hash databases to find attacks because it is largely ineffective and resource intensive on the endpoints being protected.

Sandfly does collect cryptographic hashes of suspicious files we find as part of an attack for forensic purposes though. You can use these hashes with online malware tools to see if it is something known or unknown as part of your investigation. Or, you can use these hashes to check your other systems as part of an incident response.

IP Addresses – You can deny network connectivity to attackers by blocking and blacklisting specific IP address ranges or blocks. This works temporarily, but is not a long-term solution. Sandfly does not operate at the network level, but monitoring and blocking IP addresses used by attackers as a stop-gap solution during an incident can be helpful.

Domain Names – Malicious domain names are simple to block but also simple to evade with a brand new domain. Again, Sandfly does not operate at the network level, but using domain name monitoring/blacklisting to spot malicious activity is useful again as a stop-gap.

Network/Host Artifacts – Network traces of suspicious activity could be unique such as command and control protocols, covert channels, lateral movement methods, and so on. On hosts, you can look for signs of intrusion that are left behind by attackers.

Sandfly has mechanisms to detect host artifacts of successful attacks and this can reveal that a system has been compromised.

Tools – If you can spot the tools the attackers are using consistently, you are now affecting their ability to operate. Detecting attack tool usage takes away an adversary’s main ways to gain and retain access. Further, it causes them to create new tools or abandon the approach altogether.

Sandfly does not look for specific tools (again we don’t use signatures which go out of date). Rather, we look for indications that some kind of tool was used to do something suspicious. For instance we can spot a missing entry in a system audit trail for user logins. We don’t know which of a dozen+ tools did it (nor is it even relevant), but we do know that something erased a user login entry and you should be alerted about it. Here we are hurting the ability of attackers to use common tools indiscriminately as they always have.

Tactics, Techniques, and Procedures (TTPs) – This is where Sandfly operates mostly. We are specifically targeting how attackers operate and infiltrate directly. We want to take away their ability to hide, their ability to spread, and their ability to maintain access.

Sandfly has hundreds of ways built in (and growing) to spot common and uncommon ways attackers may try to hide, operate, and alter a host to ensure they can get back onto it. For instance, we search for suspicious directories where attackers hide files. We search for processes doing things that are suspicious such as deleting themselves from the disk. We also target other techniques to maintain persistence on the host or maintain access.

By targeting the tactics, techniques, and procedures you make entire classes of attacks much harder. Even if the attacker re-writes their tools entirely, if they follow any of the previously seen TTPs then they will be spotted again just the same. This is forcing the attacker to move onto new approaches or try riskier tactics that exposes them to detection in other ways.

Sandfly Philosophy: Cause Forced and Unforced Errors

The most important point of the Pyramid of Pain is this:

As you move up the Pyramid you are causing your adversary to expend vast resources and time to avoid being caught.

The corollary to the above is this:

The more complicated you make the job of an adversary, the more likely you’ll cause an error that will increase their chances of being seen.

Basically, we want to cause forced and unforced errors. Sandfly is designed to cause adversary errors which leads to detection. We look for so many ways an attacker may hide or compromise a system that the slightest misstep by an attacker and they will be spotted. When you take attackers out of their comfort zone with the tools and tactics they’ve used for years, you are increasing their risk of being detected.

Target Tactics and Procedures

Sandfly focuses on the . We are targeting the tactics and procedures used by intruders to enter and remain on Linux systems undetected.

For instance, there are dozens of ways to start a bindshell backdoor on a Linux system. You can use shell scripts, Python, Ruby, PHP, Perl, etc. Now to detect this kind of thing you could create signatures for all the variants and search the entire file system constantly, but this has some big problems:

  1. It’s very intense and you will hammer the system performance.

  2. It’s trivial to change or obfuscate code so you won’t see anything.

Sandfly takes a different approach. If we want to detect a bindshell, we do it in a way that is directly targeting the outcome of a backdoor attack. Namely, we look for system shells that are connected to network ports in a way that we know are suspicious or dangerous. The best part about this is that an attacker can create many variants of a bindshell and there’s a good chance Sandfly will see them even with these changes. We target how these tactics work. The outcome is pain for the attacker because those methods no longer operate without being seen.

The same goes for other tactics attackers may employ. For example, they may want to alter an audit log. Great. We check for the ways an audit log will show tampering and we can spot that activity even if they use one of a dozen tools that you can find online. Even if they manage to tamper with the logs unseen, we check for inconsistencies that they missed. Same for hiding malicious processes, files, directories, or whatever. We don’t focus on the tools, we focus on the foundations needed for these tools to operate which are very difficult to conceal.

Sandfly is not just an agentless intrusion detection system for Linux. Sandfly is designed to remove common and uncommon tools from an attacker’s arsenal so they have to go back and re-invent new ways to avoid being found. That’s pain. When you cause attackers pain, you are being effective.

Let Sandfly keep your Linux systems secure.

Learn More