Sandfly Filescan Open Source File Entropy Scanner for Linux

We’re releasing an open source tool today called sandfly-filescan. Sandfly-filescan allows Linux admins and incident responders to quickly scan for compressed or encrypted executable files often linked to malware.

sandfly-filescan is a utility to quickly scan files and report on their entropy (measure of randomness) and if they are a Linux/Unix ELF type executable. Some malware for Linux is packed or encrypted and shows very high entropy. This tool can quickly find high entropy executable files which often are malicious. We will also generate the MD5, SHA1, SHA256 and SHA512 hashes of any files found.

For example we’ll look inside a system /tmp directory for packed binaries with high entropy:

sandflysecurity # ./sandfly-filescan -dir /tmp -entropy 7.7 -elf
filename: upxtest
path: /tmp/upxtest
entropy: 7.78
elf: true
md5: cd7d187644f2fd085ed85240fbdd1369
sha1: cdc729c5d49c8776d9011953a20e8db97c63ee80
sha256: 2f50b82cc29050cb580f2565d1d755484424f4f26d996f6ef96cd968b148ce18
sha512: 9051972e161d1b229ce3cfa1e1b84fa16e38a4c935ffae48c4c83e5c58c35e7ddff05a70a4a1c7f2cc80fe799c49367756114ad300da
18c6b09fc5af78f1131b

You can also use it to scan individual files that are suspicious:

sandflysecurity # ./sandfly-filescan -file ../../../rootkits/malware/x86_64.worm
filename: x86_64.worm
path: ../../../rootkits/malware/x86_64.worm
entropy: 7.97
elf: true
md5: 16fce1497314474c7520fd5b3774ada2
sha1: e7f6b58801b0d5331a34410efa159f9accb59b70
sha256: cd6d320fef843d1dc7e9759a8fdfa7096be83c7666e50395bdc7ddb8116b860f
sha512: fafd5e3eb66ca8b8bb1d2bc91e0ac96ae14b026767426b1b75206dfb7321220f46c3bae2f50f75c2317bff1e1049c247db40d423a7eafcf479d4927380e2446d

Why Scan for Entropy?

Entropy is a measure of randomness. For binary data 0.0 is not-random and 8.0 is perfectly random. Good crypto looks like random white noise and will be near 8.0. Good compression removes redundant data making it appear more random than if it was uncompressed and usually will be 7.7 or above.

A lot of malware executables are packed or encrypted to avoid detection and make reverse engineering harder. Most standard Linux binaries are not packed because they aren’t trying to hide what they are. Searching for files that are high entropy and executable can make finding malware easier that is hiding on your hosts. Even finding just an executable file in an odd location can help find malware (e.g. executable files under /tmp).

Here’s a scan through a malware directory to see how quickly it can spot packed malicious binaries:

sandfly-filescan -dir /path/to/dir -elf -entropy 7.7

Examples

Search for any file that is executable under /tmp:

sandfly-filescan -dir /tmp -elf

Search for high entropy (7.7 and higher) executables (often packed or encrypted) under /var/www:

sandfly-filescan -dir /var/www -elf -entropy 7.7

Generate entropy and cryptographic hash values of all files under /bin and output to CSV format (for instance to save and compare hashes):

sandfly-filescan -dir /bin -csv

Scan a directory for all files (ELF or not) with entropy greater than 7.7: (potentially large list of files that are compressed, png, jpg, object files, etc.)

sandfly-filescan -dir /path/to/dir -entropy 7.7

Quickly check a file and generate entropy, cryptographic hashes and show if it is an ELF executable:

sandfly-filescan -file /dev/shm/suspicious_file

Use Cases

Do spot checks on systems you think have a malware issue. Or you can automate the scan so you will get an output if we find something show up that is high entropy in a place you didn’t expect. Or simply flag any executable ELF type file that is somewhere strange (e.g. hanging out in /tmp or under a user’s HTML directory). For instance:

Did a high entropy binary show up under the system /var/www directory? Could be someone put a malware dropper on your website:

sandfly-filescan -dir /var/www -elf -entropy 7.7

Setup a cron task to scan your /tmp, /var/tmp, and /dev/shm directories for any kind of executable file whether it’s high entropy or not. Executable files under tmp directories can frequently be a malware dropper.

sandfly-filescan -dir /tmp -elf
sandfly-filescan -dir /var/tmp -elf
sandfly-filescan -dir /dev/shm -elf 

Where to Download

sandfly-filescan is released under the MIT License and is available on Github:

sandfly-filescan repository

Thanks to the Community

We’re releasing this tool as a thank you to the information security community in hopes it helps keep your systems safe.

Sandfly Security produces an agentless intrusion detection and incident response platform for Linux. Automated entropy checks are just one of hundreds of things we search for to find intruders without loading any software on your Linux endpoints. Learn more here.