How To Decloak Stealth Linux Cryptocurrency Mining Malware

Increasingly on Linux we are seeing malware deploying anti-detection and evasion tactics. In this post we’re going to go over a recent piece of malware that used a commonly available tool for hiding itself. We’ll show you how to de-cloak this kind of malware if you are investigating a host suspected of being compromised with this tactic.

Preload Stealth Rootkit Basics

The malware we’re investigating is doing cryptocurrency mining and used a ld_preload style rootkit to hide itself once active. With this style of rootkit standard system commands like ps, ls, netstat and so on will not show anything from the command line.

This rootkit abuses the ld_preload feature of Linux that allows someone to insert a library that can intercept normal system calls. The main area this attack focuses on is inserting a library path into the file /etc/ld.so.preload. Paths in this file are read first before the core system libc library is used. Any library can intercept calls at this point and have them return whatever the author wants.

Preload Libraries Are Almost Always Bad News

On most Linux systems it is rare that ld_preload needs to be used. There are three main reasons for ld_preload usage:

Debugging tools looking to profile or watch system processes.

Some security tools looking to monitor system calls.

Other performance enhancing libraries for specialized applications like databases or memory management.

There are some other small exceptions, but mostly ld_preload libraries are probably up to no good. If you see a file listed under /etc/ld.so.preload, and nobody knows why it is there, you should just assume it’s malicious until proven otherwise.

Sandfly Finds Something is Wrong

Sandfly is our agentless security platform for Linux. We are not affected by this kind of rootkit and saw several things wrong with the victim host when we scanned it. We’ll only go over the process cloaking we saw, but as you can see this malware had a lot of tricks up its sleeve.

Let’s Cheat With Sandfly’s Agentless Forensics

We’re going to cheat a bit and use Sandfly’s detection to get us onto the trail faster. If you aren’t running Sandfly you can still use what we are about to show you, so don’t despair.

There are many attributes with this malware, but for this article we’ll just use the suspect Process ID (PID) of 6345. The other arrows show further areas of interest such as working directory, open files, cryptographic hashes, network ports open, etc.

Malware Is Cloaked

First, if you logged into this host and wanted to look for a problem, what would you see? Well, maybe you’d see from external tools high CPU usage, but if you used command line tools it wouldn’t show anything. Below shows a ps listing. Our PID we want to see, 6345, is not there.

ps -auxw

We are pretty sure we should see something there so we try to look directly at the /proc directory and again the PID is not there:

ls -l | grep "^d"

Check /etc/ld.so.preload

We now suspect games are being played with us because we are certain something is using a lot of CPU on this host but standard commands are not allowing us to see anything. Let’s look for basic process hiding by running this command:

ls -al /etc/ld.so.preload

This shows us that something is present in the system /etc/ld.so.preload file. Again, you must assume guilt until proven innocent when you see anything in this file.

Now let’s see what’s in this file:

cat /etc/ld.so.preload

We see a file referenced as: /usr/local/lib/libjdk.so

Assume Guilty Until Proven Innocent

Again, we’ll assume anything listed under /etc/ld.so.preload is guilty until proven innocent. We’ll look at our suspect file with these commands at first:

ls -al /usr/local/lib/libjdk.so
file /usr/local/lib/libjdk.so

Running a basic ls command we see the file and date it was made. Note that the date made matches the time Sandfly said the process was started above (21:01 on November 20th). This again may not be available if you don’t run Sandfly, but it gives you a good time bracket to use to investigate the host for other artifacts.

Now we run the file command and see it’s a library file that isn’t stripped. This is good fortune as it can help leak more details about what is going on.

Investigating Suspicious Library Binary

Now that we have our suspect /usr/local/lib/libjdk.so, we’ll take a look at it with a simple strings command.

strings /usr/local/lib/libjdk.so

There are really interesting things here. For one, this library has references to /proc which is where process information is registered by the kernel. If you wanted to hide something, /proc is the prime target and this library seems to have a lot of interest in this area.

Next we see a name of processhider.c in the text which doesn’t look like it’s baking cookies. Then we see process_to_filter which is a weird name like it has processes it is trying to hide.

Finally, we see a very peculiar interest in directory commands. As if this library is intercepting directory calls and doing something with them. It has references to original directory as if maybe it’s swapping things around.

Put Down Your Debugger

People get intimidated by Linux malware and think the first thing they need to reach for is a debugger/disassembler when they encounter it. The truth though is that these tools are frequently not needed and can sometimes make things worse by destroying evidence on a live process.

The library here is clearly malicious just on its face. We don’t need to do anything more tricky at this point to figure out what is going on. If you wanted to take this file offline and analyze it that’s fine. For now though, we have enough information to suspect the worst.

De-cloak the Malware

Now if you wanted to push things after you have your backups, etc. done and the system has been isolated, you can try to de-cloak the malware further. De-cloaking an ld_preload style rootkit is very easy. Simply rename the referenced library, or move the file /etc/ld.so.preload to another name to see what happens like below:

mv /etc/ld.so.preload /etc/ld.so.preload.bak

By simply renaming this file the library will not be called and now the system commands will work again.

De-cloaked Process Forensics

Now let’s run our standard commands again. Our ps command shows something new:

ps -auxw

You can now see the PID 6345 is using 99.2% of the CPU which is typical for a cryptominer.

Our listing of /proc now shows the hidden PID and we can begin .

Stealth Malware Is Becoming More Common

We are seeing more Linux malware using stealth techniques to evade detection, but it doesn’t mean they can’t be found. With a solid understanding of the basics you can de-cloak any stealth malware on Linux in our experience. And yes, you can do it without debuggers or disassembly. Good attention to detail can go a long way.

This malware did many other things to this host, but once you de-cloak it you can then work standard forensics and pull out the artifacts. Although it can be done by hand, having our product Sandfly hunting for this activity for you 24 hours a day is the biggest advantage you can have. We are Linux intrusion detection and forensic specialists. Let agentless Sandfly find malware for you without loading anything on your endpoints. Give it a try.