Malware Hides in Installer to Avoid Detection

At McAfee Labs we recently observed various threat families using the Nullsoft Scriptable Install System (NSIS). This practice is not new, but our analysis shows that several malware families are employing the same technique to hide their packed executable code. Usually every malware family uses its own polymorphic packers to obfuscate its payload. In this case four families have the same executable format to hide the malicious code.

The malicious NSIS package contains a DLL (acting as a decryptor and injector) and an encrypted executable payload. Once onboard an infected machine, the NSIS package drops a DLL and two data files and loads the DLL. The DLL decrypts the two data files and executes the final payload using process hollowing, a technique used by malware in which the original code is replaced with malicious code. If we were to analyze the DLL alone, we would not conclude that it was malicious because it relies on encrypted data in the two data files.

We found four malware families using this technique:

  • Cerber
  • Gamarue
  • Kovter
  • ZCrypt

Evading security products

Because the malicious payload and APIs are in encrypted and do not fall under any specific file formats, antimalware scanners will usually omit scanning these files. They also act as efficient hash busters and easily bypass emulation techniques. When these files are copied into other directories, the malware keep the NSIS file format to strengthen their defense. We also noticed that the decryption logic varies slightly among the malware.


The malware are distributed via spam campaigns:


A ZIP archive contains the executable:


NSIS file identification

The start of the overlay+8 offset contains the “NullsoftInst” string:


Malicious NSIS package

The sample we analyzed has the following components inside the NSIS package.

  • e: Data file contains encrypted APIs used for process hollowing.
  • fsv: Data file contains the final encrypted payload.
  • dll: Malicious DLL decrypts data files and executes the process hollowing.

The encrypted data file geanticline.e:


The decrypted geanticline.e:


The encrypted payload (tache.fsv):


The decrypted payload:


Decryption code for process hollowing APIs

Code in OpenCandy.dll decrypts both data files. The following code accesses the files:


The decryption key that unlocks the data file lies in the data filename itself. The decryption logic appears in the following screen:


An XOR operation decrypts the data file.

Decryption code for payload

We found the decryption key resides inside the DLL and varies among the malware families.

Decryption key location:


Decryption code:


Decryption logic for process hollowing

We employed python to write the decryption logic used by the malware. The encrypted data file path should be passed as an argument.

For each malware family, the value of MAXKEYINDEX can be changed or be equal to KEYLEN.


Decryption logic for payload



MD5 hash: 5AF3BED65AEF6F0113F96FD3E8B67F7A

I would like to thank my colleagues Sivagnanam G N and Manjunatha Shankaranarayana for their help with this analysis.

Introducing McAfee+

Identity theft protection and privacy for your digital life

FacebookLinkedInTwitterEmailCopy Link

Stay Updated

Follow us to stay updated on all things McAfee and on top of the latest consumer and mobile security threats.


More from McAfee Labs

Back to top