![Helping Non-Security Stakeholders Understand ATT&CK in 10 Minutes or Less [VIDEO]](https://assets.crowdstrike.com/is/image/crowdstrikeinc/video-ATTCK2-1)
![Qatar’s Commercial Bank Chooses CrowdStrike Falcon®: A Partnership Based on Trust [VIDEO]](https://assets.crowdstrike.com/is/image/crowdstrikeinc/Edward-Gonam-Qatar-Blog2-1)
![Endpoint Protection and Threat Intelligence: The Way Forward [VIDEO]](https://assets.crowdstrike.com/is/image/crowdstrikeinc/GK-Blog_Images-1)
"Continuous integration (CI) is the process of automating the build and testing of code every time a team member commits a change." — Sam Guckenheimer, Microsoft Azure
While not enabled by default, anonymous read access can be leveraged to access build history and the credentials plugin. In some cases, anonymous script console access was also enabled, which would enable full access to the Java runtime allowing command execution. Locking down access to Jenkins, in particular the web console, is highly recommended because improperly configured authentication plugins are common ways attackers gain access to Jenkins and further their mission.
Authentication plugins allow development teams to customize logins to their environments. These plugins vary across organizations, for example, organizations without Active Directory may select to use the Google Login Plugin. It is important to note that regardless of implementation, these authentication methods should be properly secured. Adversaries have been observed leveraging authentication methods to obtain web console access and therefore, these methods should be thoroughly tested for edge cases. For example, if using the Active Directory Plugin, are all active directory users allowed authentication to the web console? If so, an attacker that has obtained domain credentials will be able to authenticate and attempt exploitation of the Jenkins server.
From here, an attacker could spawn a beacon, list files, decrypt stored passwords, etc. It is important to note that using the execute method, all commands are run as child processes of the Java process (Java.exe on Windows and /usr/bin/java on Ubuntu).
Identifying suspicious process trees can be a helpful indicator when detecting malicious Jenkins server activity. For example, when spawning PowerShell commands via the script console, the following is observed:
In some instances, the attackers may elect to avoid spawning a method of command and control (C2) by using the built-in Java methods, rather than relying on PowerShell to conduct post-exploitation. In many Jenkins compromises, attackers will attempt to gain access to the files 
Using the above Groovy Script, the attacker was able to retrieve each of the files without spawning a potentially malicious child process. The attacker also used the Base64 class method to retrieve the binary-formatted hudson.util.Secret file. Usage of this script can be seen using a Jenkins test instance.
Passphrases stored within the 
Access to the Jenkins script console gives attackers a variety of methods for obtaining critical and sensitive files on Jenkins servers, and therefore should be disabled or restricted from access.
With job creation access, attackers can create a local job on the Jenkins server and use it to execute commands, then view the results within the console output. Allowing users access to build history and console output can also potentially leak secrets, source code, keys, etc., to anyone with web console access. Console output and build history should be checked for sensitive information that could potentially be leveraged by attackers.
To gain command execution on a Jenkins server with job creation access, create a Freestyle project with a given item name.
Once created, a variety of options can be configured within the Freestyle project. For simplicity, ignore all options and click “Add build step.”
For this test instance, we will configure it to “Execute Windows batch command” and run some basic commands, including adding a local administrator account, however, this could be anything that would run within a windows batch file (.bat).
After clicking “save,” the newly created Freestyle project can be built by selecting the “Build Now” option from the web console.
Once built, the output can be viewed on the Console Output as shown below.
It is important to note that because the Jenkins server is configured to allow anonymous creation, the user associated with the Freestyle project creation is unknown.
With job creation, possibilities are nearly the same as with script console access, but what about situations where attackers can only reconfigure jobs? These situations are nearly identical, however, attackers must edit an existing job and schedule a build. In the following example, we are going to reconfigure the “BackupProject” Freestyle project to print out secrets stored within the credential plugin. To begin, select the “Configure” option for a modifiable project.
Once selected, attackers can reconfigure the Build Environment to store secrets and credentials within environment variables. These environment variables can then be used within Build Steps and output to files. In this scenario, the attacker can output the results to a globally accessible userContent folder at (C:/Program Files (x86)/Jenkins/userContent/).
In Windows systems environment, variables are accessed using the % character and the $ character on Unix systems.
After building the modified project, the results can be viewed at:
The userContent folder is a special folder where contents are not subject to any access controls beyond Overall/Read access. This folder can be a viable location to store credential/secret console output in instances where attackers may backdoor existing build projects. Console output results, including credentials/secrets, can be redirected to this folder after each build.
Introduction
Jenkins is the leading open source automation server and is popular among development teams. Recently, adversaries have been observed targeting wide-scale Jenkins servers to deploy cryptominers. They have also launched targeted breaches using Jenkins to maintain access to developer environments. There are several well documented blog posts that discuss exploiting and gaining access to Jenkins via exploits, web console, and post-exploitation. This blog focuses on techniques commonly used by adversaries to gain access, maintain it, and exfiltrate data. CrowdStrike's red team leverages these techniques to perform adversary emulation exercises where the development environment is compromised.Targeting Jenkins
Targeting and identifying Jenkins servers varies depending on use case. For most red teams, Jenkins will be accessible somewhere on the internal network. Gaining access to these servers can be achieved several ways. The most common methods observed are adversaries using recently disclosed vulnerabilities and exploiting them, misconfigurations in authentication plugins, and previously obtained credentials. Exploits are not always used in targeted breaches. However, recently, a Jenkins vulnerability was observed being used on several occasions.Java Deserialization
The Java deserialization vulnerability (CVE-2017-1000353) can be leveraged to gain remote code execution on unpatched Jenkins servers. Exploit-db (https://www.exploit-db.com/exploits/41965/) contains a modifiable proof-of-concept (POC) that can be used to test this. In cases where exploits were not used, adversaries commonly leveraged previously compromised credentials or misconfigured Jenkins servers to obtain access. By default, Jenkins requires authentication, but this is commonly changed by development teams and can leave the server vulnerable, depending on how it is configured. Jenkins supports a variety of authentication plugins, including LDAP, Kerberos single-sign-on (SSO), SAML, etc. One of the most common misconfigurations is the anonymous read access delegation within the Global Security Configuration shown below.While not enabled by default, anonymous read access can be leveraged to access build history and the credentials plugin. In some cases, anonymous script console access was also enabled, which would enable full access to the Java runtime allowing command execution. Locking down access to Jenkins, in particular the web console, is highly recommended because improperly configured authentication plugins are common ways attackers gain access to Jenkins and further their mission.
Authentication plugins allow development teams to customize logins to their environments. These plugins vary across organizations, for example, organizations without Active Directory may select to use the Google Login Plugin. It is important to note that regardless of implementation, these authentication methods should be properly secured. Adversaries have been observed leveraging authentication methods to obtain web console access and therefore, these methods should be thoroughly tested for edge cases. For example, if using the Active Directory Plugin, are all active directory users allowed authentication to the web console? If so, an attacker that has obtained domain credentials will be able to authenticate and attempt exploitation of the Jenkins server.
Post-Exploitation Jenkins
Jenkins is a java application that supports a variety of operating systems, most common are Windows, Ubuntu/Debian, and Red Hat/CentOS. While the Jenkins web application functions nearly identically, there are some notable differences between Windows and Linux installations, as described in the following:Windows
By default, when installed on windows, Jenkins will utilize the NT AUTHORITY\SYSTEM account. It is highly recommended that this user account be changed as the SYSTEM authority account has full permissions on the Windows system. If the script console were to be accessed, the attacker would have full control of the system with relative ease. In general, it is recommended that you use a service account with limited permissions on the local system.Linux
By default, when installed on Linux, Jenkins will create a service account. This user account is not given sudo or root access by default, however, this is always worth checking. If the script console were to be accessed, the attacker would have the same permissions as the Jenkins service account.Script Console
The Jenkins script console is an application viewed in the web console that allows users to execute Jenkins Groovy scripts. When accessed, the script console allows full access to Java and can be leveraged to do anything within the Java runtime process. Most notable is the ability to execute commands, as shown below for both Linux and Windows installations.
From here, an attacker could spawn a beacon, list files, decrypt stored passwords, etc. It is important to note that using the execute method, all commands are run as child processes of the Java process (Java.exe on Windows and /usr/bin/java on Ubuntu).
Identifying suspicious process trees can be a helpful indicator when detecting malicious Jenkins server activity. For example, when spawning PowerShell commands via the script console, the following is observed:
In some instances, the attackers may elect to avoid spawning a method of command and control (C2) by using the built-in Java methods, rather than relying on PowerShell to conduct post-exploitation. In many Jenkins compromises, attackers will attempt to gain access to the files credentials.xml, master.key and hudson.util.Secret. These files are responsible for encrypting secrets, and in some cases, storing credentials. The master.key file is used to encrypt the hudson.util.Secret file, which is used to encrypt secrets within the credential plugin. The credentials.xml file contains encrypted passphrases and keys for Jenkins users.
Obtaining these files can be achieved in a variety of ways. If SSH access or C2 is established to the server, these files can be directly copied from the server and exfiltrated. In this example, the attacker leveraged the built-in Java methods to obtain these files by leveraging the following Groovy Script:
Using the above Groovy Script, the attacker was able to retrieve each of the files without spawning a potentially malicious child process. The attacker also used the Base64 class method to retrieve the binary-formatted hudson.util.Secret file. Usage of this script can be seen using a Jenkins test instance.
Passphrases stored within the credentials.xml file can also be directly decrypted from the script console using the following script:
Access to the Jenkins script console gives attackers a variety of methods for obtaining critical and sensitive files on Jenkins servers, and therefore should be disabled or restricted from access.
Job Configuration/Creation
In situations where the script console is not accessible, users that can view the web console and potentially schedule jobs or view build history can still obtain valuable information, depending on configurations. On several assessments, the CrowdStrike red team identified situations where jobs could be reconfigured, but not created, and vice-versa. Identifying which permissions are allowed for the authenticated user can be done via the web console by viewing the default page, as shown in the example. In this scenario, users are not required to authenticate to configure/create jobs.
With job creation access, attackers can create a local job on the Jenkins server and use it to execute commands, then view the results within the console output. Allowing users access to build history and console output can also potentially leak secrets, source code, keys, etc., to anyone with web console access. Console output and build history should be checked for sensitive information that could potentially be leveraged by attackers.
To gain command execution on a Jenkins server with job creation access, create a Freestyle project with a given item name.
Once created, a variety of options can be configured within the Freestyle project. For simplicity, ignore all options and click “Add build step.”
For this test instance, we will configure it to “Execute Windows batch command” and run some basic commands, including adding a local administrator account, however, this could be anything that would run within a windows batch file (.bat).
After clicking “save,” the newly created Freestyle project can be built by selecting the “Build Now” option from the web console.
Once built, the output can be viewed on the Console Output as shown below.
It is important to note that because the Jenkins server is configured to allow anonymous creation, the user associated with the Freestyle project creation is unknown.
With job creation, possibilities are nearly the same as with script console access, but what about situations where attackers can only reconfigure jobs? These situations are nearly identical, however, attackers must edit an existing job and schedule a build. In the following example, we are going to reconfigure the “BackupProject” Freestyle project to print out secrets stored within the credential plugin. To begin, select the “Configure” option for a modifiable project.
Once selected, attackers can reconfigure the Build Environment to store secrets and credentials within environment variables. These environment variables can then be used within Build Steps and output to files. In this scenario, the attacker can output the results to a globally accessible userContent folder at (C:/Program Files (x86)/Jenkins/userContent/).
In Windows systems environment, variables are accessed using the % character and the $ character on Unix systems.
After building the modified project, the results can be viewed at:http://jenkins/userContent/out.txt
The userContent folder is a special folder where contents are not subject to any access controls beyond Overall/Read access. This folder can be a viable location to store credential/secret console output in instances where attackers may backdoor existing build projects. Console output results, including credentials/secrets, can be redirected to this folder after each build.
Conclusion
Automation systems like Jenkins are valuable targets for adversaries. It is crucial that administrators spend time securing and auditing Jenkins installations, as these systems are likely going to be targeted by adversaries within the network. To help address this issue, CrowdStrike recommends Jenkins administrators note the following, based on observations of recent adversary activity:- Can anyone access the Jenkins Web Console without authenticating?
- Does this include Script console access?
- Can they view the credentials or build history?
- Can they create builds or schedule jobs?
- What permissions do authenticated users have?
- Does this include Script console access?
- Can they view the credentials or build history?
- Can they create builds or schedule jobs?
- Is there any sensitive information stored in build history or console output?
- Is Jenkins internet-accessible? Does your organization need it to be?
- Is the Jenkins service account running with the least amount of privileges necessary to perform its functions?
- How are credentials stored?
- Who can access
credentials.xml, master.key, and hudson.util.Secret?
- Who can access
Learn more:
- For more information on CrowdStrike’s Incident Response, Compromise Assessment or Threat Hunting offerings, visit the CrowdStrike Services page or please reach out to us at Services@crowdstrike.com.
- Visit the CrowdStrike Falcon® product page for more information on next-gen endpoint protection.
- Test CrowdStrike next-gen AV for yourself. Start your free trial of Falcon Prevent™ today.
