Understanding Malicious RATs: Threats and Detection Techniques

RAT-based APT Detection for Provenance Graph Analytics Northwestern University Jan. 15, 2016

Motivation ~300 APT White Papers [1] Most APTs are on Windows and involves malicious RATs Malicious RAT APT lifecycle [2] Establish foothold Escalate privileges Internal reconnaissance Move laterally Maintain presence Complete mission Initial compromise [1] https://github.com/kbandla/APTnotes. [2] APT1: Exposing One of China s Cyber Espionage Units, Mandiant, 2013. 2

What is a RAT? RAT stands for Remote Administration Tool, a piece of software that allows a remote "operator" to control a system. Interpreted as remote administration Trojan in some white papers. A malicious RAT is a backdoored spying horse zombie, in essence. Trojan - Disguises its identity as legitimate program. Backdoor Enables remote control and receives commands. Spyware Gathers information stealthily and sends back to master. Zombie Behaves like a zombie (allows its master to perform operation through it). 3

First malicious RAT and Today 1stmalicious RAT o Back Orifice is one of the first RATs, released in 1998. o It includes a GUI control panel. o Spawn/kill processes, play audio files and capture videos. Malicious RAT Today o An abundance of different RATs: NetBus, iControl, Sub Seven, Beast Trojan, Bifrost, Blackshades, DarkComet, LANfiltrator, Win32.HsIdir, Optix Pro, o RAT has become a service available to people without much computer knowledge. 4

Architecture of a malicious RAT Client/Server architecture o Server program installed on victim s machine o Client program with GUI installed on C&C server o Server program is remotely controlled by Client program RAT Server (unobtrusive) RAT Client (GUI) Control Response C&C Server Victim s Machine 5

Typical Features of a malicious RAT VM detection Antivirus tool detection/disabling File/registry/process manipulating Shell control Credential stealing Webcam/microphone access Keystroke logging Screen capture Network monitoring ... Features of DarkComet www.darkcomet-rat.com 6

Malicious RAT vs Bot Targeted attack vs. random attack o APT are targeted attacks with reconnaissance while most botnets are kind of random attacks, relying on port scan for victim discovery. Human manual operation (w/ vs w/o) o RAT is much more complex and powerful, and must require human operation, while bots are usually automatic programs without requiring human-driven activities. Attack Intention o RATs are mainly used for spying on victims while bots are typically harvested for orchestrated attacks. 7

Malicious RAT vs. Legitimate Program Malicious RAT vs Enterprise/Commercial RAT o They resemble each other, except a few features such as VM detection and antivirus tool detection/disabling. Malicious RAT vs other Legitimate Program o Malicious RATs have many distinct features that legitimate programs do not have: Keystroke logging Network connection monitoring File/registry/process manipulating Etc. o RATs always stay hidden while legitimate programs typically interact with users and show itself by displaying windows, buttons, dialog boxes and so on. 8

Related Work Very few academic works related to RAT detection o Z. Chen, P. Wei, and A. Delis. Catching Remote Administration Trojans (RATs). In Software Practice and Experience, 2008. o K. T. Gard sen. Detecting Remote Administration Trojans through Dynamic Analysis using Finite-State Machines. Master thesis, 2014. Industrial efforts on APT detection o Redline [1], a memory forensics tool designed by FireEye, provides host investigative capabilities to identify malicious activities through memory and file analysis. Volatility [2] - an open source memory forensics framework for incident response and malware analysis [1] Redline: https://www.fireeye.com/services/freeware/redline.html [2] Volatility: https://github.com/volatilityfoundation/volatility o 9

Our Detection Observations: API calls reveal the behavior of a RAT no matter how it disguises itself and how its code is obfuscated. API calls are stable for a specific behavior most of the time. Similar behavior triggers similar API calls. We perform RAT-based APT detection by o modeling the behavior of RATs by the API calls o creating the provenance graph based on the API call traces o developing signature-based and statistics-based methods for spotting suspicious behaviors 10

Trace Collection Experiment Setup Windows VM Windows VM RAT victim RAT controller (C&C server) Control Response API capture tool process RAT RAT Code kernel32.dll, ntdll.dll, ApiCapture dll

Trace analysis: Screen Capture DarkComet WSARecv(0x00000000000048BC, ) CreateDC( ) GetDeviceCaps( ) // Get Instruction from C&C server (attacker). CreateCompatibleDC( ) CreateCompatibleBitmap( ) GetDC() SelectObject() Share the same socket descriptor ReleaseDC() DeleteDC() // send all the collected information back to the attacker Send(0x00000000000048BC, ) WinAPI tracing: Attacking flow of DarkComet (RAT) 12

Trace analysis: Screen Capture Pandora WSARecv(0x0000000000005B14, ) // Get Instruction from C&C server (attacker). GetDC( ) Different RATs share similar API call patterns in similar behavior. GetDeviceCaps( ) CreateCompatibleDC( ) SelectObject() ReleaseDC() Share the same socket descriptor DeleteDC() // send all the collected information back to the attacker Send(0x0000000000005B14, ) WinAPI tracing: Attacking flow of Pandora (RAT) 13

RAT Behavior Detection Generate signature for RAT s screen capture behavior: Signature-based detection: o We generate signature based on our trace analysis. WSARecv(fd, ) WSARecv(fd, ) GetDC() / CreateDC() Identify critical path Graph signature based detection GetDeviceCaps() Critical WinAPI Calls CreateCompatibleD C Send() Send(fd, ) 14

RAT Behavior Detection (contd) Statistics-based detection: o Single feature/behavior cannot alone differentiate malicious RATs from enterprise RATs. o However, they typically differ in terms of characteristics of API calls such as the frequency and the time interval between API calls. o With traces of API calls collected from real-world malicious RATs, enterprise RATs, and other legitimate programs, we develop statistics-based detection. 15

Conclusion We identify that malicious RATs are used in most APT attacks on Windows systems. We propose signature-based and statistics-based method to detect malicious RATs for APT detection for provenance graph analytics. 16

Thanks! Questions? 17

Trace analysis: Get OS Information DarkComet WSARecv(0x00000000000048BC, ) // Get Instruction from C&C server (attacker). GlobalMemoryStatus( ) // Get global memory status. NtQuerySystemInformation(0x0, ) // Get detailed system information by classes, the 1st parameter indicate a unique class of system information NtQuerySystemInformation(0x3, ) Share the same socket descriptor NtQuerySystemInformation(0x2, ) // send all the collected information back to the attacker Send(0x00000000000048BC, ) WinAPI tracing: Attacking flow of DarkComet (RAT) 18

Trace analysis: Get OS Information DarkComet WSARecv(0x00000000000048BC, ) WSARecv(fd, ) GlobalMemoryStatus( ) Graph signature based detection NtQuerySystemInformation(0x0, ) Critical WinAPI Calls NtQuerySystemInformation(0x3, ) NtQuerySystemInformation(0x2, ) Send(0x00000000000048BC, ) Send(fd, ) 19