How To Recover A Deleted Binary From Active Linux Malware

Often, Linux malware will delete itself after it starts so that file scanners and integrity checks won’t see the binary present. It also can make forensic analysis harder if you can’t get to the binary easily. However, it is easy to recover a deleted process binary on Linux as long as the process is still in memory. Let’s take a look.

Recovering a Deleted Linux Process Binary Command

The basic command to recover a deleted process binary is simple. Just do the following:

cp /proc/<PID>/exe /tmp/recovered_bin

Where <PID> is the Process ID of the deleted process you want to recover. The above command will copy the binary to /tmp/recovered_bin so you can do with it what you will.

Practicing Recovery of a Deleted Process

We’re going to use the sleep command to simulate a process that has deleted itself from the disk. You can run this series of command safely on your own Linux system so you can practice recovery of a deleted binary.

cd /tmp
cp /bin/sleep x
./x 3600 &
rm x

Now check that the binary is in fact gone:

ls -al /tmp/x
ls: cannot access /tmp/x: No such file or directory

The above copies the sleep command as “x” under /tmp and runs for 3600 seconds and then exits. We then delete “x” so they system binary appears removed.

You can see the process still running in the ps listing below.

Forensics Examination of Linux /proc for a Deleted Process

Now let’s look at the /proc directory for this process. The /proc filesystem is a special filesystem on Linux that tracks almost all kernel and process information you would ever want.

Under /proc the system flagged the binary as deleted. The command format is:

ls -al /proc/<PID>/exe

Where again <PID> is the process ID we’re interested in like so:

ls -al /proc/12725/exe
lrwxrwxrwx 1 root root 0 May 23 19:39 /proc/12725/exe -> /tmp/x (deleted)

Recovery of Deleted Linux Process

But, the Linux file system will not actually delete that file as long as a running process still has it open. The file is there, but it simply is not being shown to you any more.

This /proc/12725/exe link will easily get you the binary that started the process. Simply copy it to where you want and the binary that was deleted is now yours:

cp /proc/12725/exe /tmp/recovered_bin

The hashes of the recovered binary and the sleep command that we used to emulate the attack are identical which proves the file recovered correctly.

sha1sum /tmp/recovered_bin
d76373f39b93bbc5191e5f8ca0f9c2f692e5aa71 /tmp/recovered_bin
sha1sum /bin/sleep
d76373f39b93bbc5191e5f8ca0f9c2f692e5aa71 /bin/sleep

You can now use this binary for further analysis as you wish such as a quick look with strings or disassembler of your choice.

Never Kill a Suspicious Process Blindly

The above is reason 4,387 to never simply kill a suspicious process you find until you have thoroughly investigated what is going on. If you were to kill the process first, you’d lose the binary as it would be released by the filesystem and likely destroyed.

If a system is infected with something, isolate it on the network and then take your time and look around. Do not go in with guns blazing because you will destroy critical data.

As we advise in the Linux command line forensics cheat sheet: Don’t Panic. Take your time, collect your evidence, and only then rebuild the system after you’ve figured out what is going on and how the attack happened (root cause analysis). What’s the point of rebuilding a box if you don’t know how the person entered and whether they’ll just do the same thing again immediately when you bring it back online?

Finally, did you know that Sandfly’s agentless Linux security platform automatically hunts for deleted processes and grabs forensic data about the situation for you? Why not give it a try?