Advanced Malware Detection – Signatures vs. Behavior Analysis

By John Cloonan, Director of Products, Lastline, Inc.

Malware has threatened our computers, networks, and infrastructures since the eighties. It is constantly evolving, and deploying products that effectively detect it is crucial to preventing costly data breaches. There are two major technologies to accomplish this, but surprisingly, most organizations rely almost exclusively on just one approach, the decade’s old signature-based methodology. The more advanced method of detecting malware via behavior analysis is gaining rapid traction but is still unfamiliar to many.

Signature-based malware detection is a proven method for identifying “known” malware. Unfortunately, new versions of malicious code appear daily that are not recognized by signature-based technologies. These newly released forms of malware can only be distinguished from benign files and activity by analyzing its behavior.

Signature-Based Technologies Track Known Threats

In computing, all objects (including operating system components, executable programs, documents, images, and others) have attributes that can be used to create a unique digital fingerprint or signature. Algorithms can quickly and efficiently scan an object to determine its digital signature.

When an anti-malware solution provider identifies an object as malicious, its signature is added to a database of known malware. These repositories may contain hundreds of millions of signatures that identify malicious objects. This method has been the primary technique used by most malware detection products and remains the fundamental approach used by the latest firewalls, email and network gateways, and other intrusion detection systems.

Signature-based malware detection technology has a number of strengths, including:

• Signature-based malware detection is well known and well understood. The very first anti-virus programs used this approach.
• It’s fast. Signature-based technologies can rapidly identify known malware.
• Signature-based malware detection is relatively simple and will run in minimal endpoint environments.
• It’s readily available within a number of leading network security tools such as next-generation firewalls, email gateways, and IPS.
• It provides good protection from the many millions of older, but still active threats.

But there are over a million new versions of malware released daily. Many of these have very specific targets—often just one. As a result, on top of the much greater number of overall threats, fewer organizations are successfully discovering and then reporting these highly targeted attacks, making information about new attacks less available for informing signature-bases solutions.

Don’t Wait for Signatures

Verifying that a new file is malicious, and adding its signature to a database of known malware usually takes several days and is complicated. And often the malware has already evolved by then. The Cisco 2017 Annual Cybersecurity Report found that up to 95% of malware files they analyzed were less than 24 hours old, indicating a very fast “time to evolve.” The delay in identifying new forms of malware makes corporations vulnerable to serious damages.

Modern malware will often strike immediately, inflicting incredible damage in a short period of time. Jigsaw, a particularly nasty form of ransomware, starts deleting files within 24 hours. HDDCryptor, another ransomware monster infected 2000 systems at the San Francisco Municipal Transport Agency before it was detected. Being vulnerable to infection while waiting for a signature is very risky.

Another major problem with signature-based malware detection is that today’s advanced malware can alter its signature to avoid detection. Signatures are created by examining the internal components of an object. Malware authors simply modify these components while preserving the object’s functionality and behavior. There are multiple transformation techniques, including code permutation, register renaming, expanding and shrinking code, and the insertion of garbage code or other constructs.

Another example that has seen a significant increase over the past few years is Metamorphic malware, which automatically changes itself with each new instance or infection.

Behavior-based Malware Detection

Behavior-based malware detection evaluates an object by its intended actions before it can actually execute that behavior. This is typically accomplished by activating it within an isolated environment such as a sandbox.

An object’s behavior, or in some cases its potential behavior, is analyzed for suspicious activities. Any attempt to perform actions that are clearly abnormal or unauthorized would indicate the object is malicious, or at least suspicious.

There’s a multitude of behaviors that point to potential danger. Here are some examples:

• Any attempt to discover a sandbox environment
• Disabling anti-virus or other security controls
• Modifying the boot record or other initialization files to alter boot-up
• Installing rootkits
• Registering for autostart
• Shutting down or disabling system services
• Downloading and installing unknown software
• Deleting, altering, or adding system files
• Modifying other executable programs
• Connecting with known malicious sites
• Encrypting files that are unrelated to the program
• Adding or modifying user accounts
• Dynamic code building to enhance evasion capabilities
• Executing a dropped file
• Spawning Powershells
• Performing any actions that are highly abnormal

Evaluating an object for malicious behavior as it executes is known as dynamic analysis. Threat potential or malicious intent can also be assessed by static analysis, which looks for dangerous capabilities within the object’s code and structure.

Static analysis is extremely efficient and is often performed prior to dynamic analysis. It’s also useful for detecting malicious activities within code that may not execute during dynamic analysis. Dynamic analysis monitors actual behavior and detects malicious actions that are missed by static analysis. Both approaches have their advantages and are important for behavior-based malware detection.

While no solution is one hundred percent foolproof, behavior-based detection is the leading technology today to uncover new and unknown threats in near real-time. Some examples of where behavior-based technology succeeds when signature-based systems fail are:

• Protecting against new and unimagined types of malware attacks
• Detecting an individual or one-time instance of malware targeted at one organization or one person
• Identifying what the malware will do in a specific environment when files are opened
• Obtaining comprehensive information about the malware, helping analysts classify the object and respond appropriately to potential threats

There are, however, a few important limitations to be aware of.

• If malware determines it’s running in a sandbox, it will attempt to avoid detection by curtailing malicious activities. It’s critical that a sandbox remains undetectable—and most fail to do this.
• It takes time to analyze the behavior of an object. While the static analysis can be performed in real-time, the dynamic analysis may introduce latency while the object is exercised. The ability to detect internal stalling is an important feature to maintain high throughput.
• Some behavior-based malware detection requires more hardware resources than signature-based detection.
• Many behavior-based solutions are exclusively cloud-based. Transmitting sensitive files to an outside service may be an issue for some organizations.

Not All Behavior-based Technology Is Created Equal

Conventional sandbox technologies have limited visibility and can only evaluate the interaction between an object and the operating system. By observing 100 percent of the actions that a malicious object might take, even when it delegates those actions to the operating system or other programs, CSOs can evaluate not only the malware’s communication with the operating system but each instruction processed by the CPU.

How Behavior-based Solutions Work

Advanced malware detection solutions observe and evaluate in context every line of code executed by the malware in context. Furthermore, they analyze all requests to access specific files, processes, connections, or services. This includes each instruction executed at the operating system level or other programs that have been invoked, including low-level code hidden by rootkits.

The technology identifies all malicious, or at least suspicious activity, which, when taken together, makes it very clear that a file is malicious before it is released onto the network to actually execute any potentially damaging behavior.

Both signature and behavior-based malware detection is important and have distinct advantages. The best security will come from utilizing both technologies simultaneously. Too many security officers are misled by vendors promoting “next-generation” firewalls and other “state-of-the-art” security tools.

They don’t realize that these “latest” products are relying exclusively on the decade’s old signature-based approach to malware detection that will miss evasive malware and zero-day attacks. No organization with sensitive data or critical operations to protect should be without behavior-based malware detection to augment the capabilities of existing security tools.

About the Author
John Cloonan is Director of Products for Lastline with a passion for creating innovative information security solutions. Of his nearly 25 years of professional experience, he has spent more than 15 years in Information Security software development and service delivery. John Cloonan is Director of Products for Lastline with a passion for creating innovative information security solutions. Of his nearly 25 years of professional experience, he has spent more than 15 years in Information Security software development and service delivery.