Quanta Servers (Still) Vulnerable to Pantsdown

Eclypsium research has identified that Quanta Cloud Technology (QCT) server models are vulnerable to the well-known ‘Pantsdown’ BMC vulnerability (CVE-2019-6260). We have developed a proof-of-concept exploit demonstrating how even an unsophisticated attacker with remote access to the operating system could leverage this vulnerability to gain code execution within the BMC of QCT servers. 

(NOTE: This post covers technical aspects of the vulnerability, details of the proof-of-concept exploits, and technical mitigations steps. For a report on the overarching business and market impact of this vulnerability, please read QCT & Pantsdown: An Executive Summary.) 

It is important to note that the vulnerability in question is rated ‘Critical’ with a CVSS score of 9.8 and exploits already exist in the wild. This vulnerability can provide an attacker with full control over the server including the ability to propagate ransomware, stealthily steal data, or disable the BMC or the server itself. Additionally, by gaining code execution in the BMC, attackers could steal the BMC credentials, which could allow the attack to spread to other servers in the same IPMI group. This highlights how a single bug can potentially allow attackers to target many devices in the same network as seen in the recent attacks on ViaSat. These trends combined with the recent discovery of threat actors using BMC implants in destructive attacks in the wild (iLOBleed), it is particularly important that QCT customers verify the integrity of their devices.

We initially disclosed these vulnerabilities to Quanta on October 7th, 2021. At the time of writing, QCT has informed us that they have addressed the vulnerability and new firmware is available privately to their customers, but will not be made publicly available. 

Background Information

Baseboard Management Controllers (BMCs) are incredibly powerful and privileged pieces of technology that provide out-of-band management for modern servers. A BMC is effectively a separate, fully functional independent computer within a server (or sometimes even a workstation), equipped with its own power, firmware, memory, and networking stack. It is designed to allow a remote administrator to control virtually everything on the device from managing low-level hardware settings to updating the host operating system or managing the virtual hosts, applications, or data supported by the OS.

Sponsorships Available

Sponsorships Available

Sponsorships Available

BMC Attacks

One manner in which servers are managed through BMCs is through the use of IPMI managed groups, which share the same password.  As a result, attackers leverage arbitrary code execution within one BMC and then backdoor the login page to steal credentials. From there they can use those credentials for any other system within the same IPMI group.

A recent CISA alert (https://www.cisa.gov/uscert/ncas/alerts/TA13-207A) point this issue out along with a number of other IPMI risks: “Knowledge of one IPMI password gives you the password for all computers in the IPMI managed group.”

Pantsdown Vulnerability

The ‘Pantsdown’ vulnerability (CVE-2019-6260) was discovered in early 2019 and affected multiple firmware BMC stacks including AMI, SuperMicro, as well as OpenBMC versions up to version 2.6. The vulnerability allows arbitrary read and write access to the BMC’s physical address space from the host, which would allow an attacker to overwrite the existing BMC firmware, execute malicious code, and disable the device. The vulnerability was rated by NIST NVD as 9.8 Critical based on the CVSS v3 scoring system.

Analysis and Exploitation

As part of our ongoing research, we obtained a QuantaGrid D52B rackmount server for testing and updated it with the latest firmware provided on the Quanta website (qct.io). To document the state of Quanta’s site at the time of writing, we have included a web archive of the Quanta update page as well as the firmware package itself. This included update package version: 1.12 with a release date of 2019.04.23, and included BIOS version 3B13, and BMC version 4.55.00. This same firmware package names support for D52BQ-2U, D52BQ-2U 3UPI, D52BV-2U models of the server. On inspection, we found that the server contained an Aspeed 2500 BMC (AST2500(A2)) and was running a version of AMI-based BMC software vulnerable to Pantsdown. 

Using CVE-2019-6260’s ability to read and write to memory, we were able to patch the web server code while it was running in memory on the BMC, and replace it with our own malicious code to trigger a reverse shell whenever a user refreshes the webpage or connects to the web server.

This is only one example of many possible techniques that would allow an attacker to move from the host operating system and gain code execution within the BMC. It is important to note that this PoC required us to have root access on the physical server. These permissions are routinely provided by default when renting a bare-metal instance of a server. Additionally, an attacker could gain root access by exploiting a web-facing application and escalating privileges or simply taking advantage of any services already running with root privileges. 

In the context of a real-world attack, the attack chain would look something like this:

1. Attacker gains a foothold on the target server. Confirm there is a vulnerable BMC inside.
2. Dump the BMC RAM via CVE-2019-6260, then use memory forensics techniques to find running processes and binaries such as Lighttpd. Lighttpd is an open source web server the BMC uses.
3. Reverse Lighttpd in order to find the location of the connection_accept() function, which is called whenever a new connection is established to the web server.
4. Modify the BMC RAM via CVE-2019-6260 to replace the connection_accept() code with our own code that includes a reverse shell.

5. Trigger the reverse shell by connecting to the webpage, or simply wait for a sysadmin to access the webpage.
6. At this point, we have full control of the BMC, our code is running with root privileges, and we can transition to post-exploitation activities, which can include, but are not limited to:
   1. Modify BMC flash to maintain persistence within BMC
   2. Modify BIOS/UEFI flash to implant persistent malware into Host
   3. Read and modify Host main memory to modify the kernel and/or inject malware into an already-running Host
   4. Boot from arbitrary disk images presented to the Host as virtual cd/usb drives
   5. Inject arbitrary keyboard/mouse events to the Host
   6. Present arbitrary virtual USB devices with any vendor id, device id, or device class to the Host, which can cause the Host to route traffic through the fake device or any of a number of other interesting scenarios
   7. Lateral movement to attack/infect other devices on Management Network
   8. Brick the device such that it can’t be recovered without sending back to manufacturer at a specific time across all infected systems

Attack steps and methodology

Following the initial disclosure of the Pantsdown vulnerability in 2019, the researchers published a useful test and debug tool to allow researchers and security staff to further explore the vulnerability. Using this tool, we can dump the memory of both volatile and non-volatile memory.

Next, we can analyze the memory and see what processes are running. Lighttpd is a binary that handles the web server running on the BMC, and we can see it is loaded into memory every time the BMC resets or boots up.

Next, we can dump out the Lighttpd binary and find where the connection_accept() function lives, and verify that connection_accept() is in the memory dump. Then we can replace that code with our malicious payload.

The video below shows how we created our malicious payload and how we inserted it into BMC memory.

Payload generation using msfvenom.

Key Takeaways and Recommendations

Organizations that rely on QCT products or services should verify the integrity of their BMC firmware, and update their BMC when a fix is made available. Additionally, it should be noted that newer versions of the Aspeed BMC SoC (AST2600) and Open BMC have introduced support for Secure Boot. These improvements can help prevent an attacker from updating the firmware, but should not be considered a complete fix as numerous firmware attacks have been demonstrated in the presence of Secure Boot.

Additionally, organizations should be regularly verifying the integrity of their BMC firmware. The recent iLOBleed attacks demonstrated that attackers are already taking steps to prevent BMC updates and then falsely reporting that the update was successfully applied. Regular integrity verification would ensure that known-good firmware is always running on a server and reveal any modifications introduced by malware. 

Lastly, it should be noted that these types of issues are common in the OEM community. While the Pantsdown vulnerability was the most egregious, we have identified a number of other known and critical vulnerabilities on the same model including insecure protection of UEFI firmware, an outdated list of vulnerable bootloaders (DBX) which shouldn’t be allowed to run by firmware, and over 90 CVEs in Linux-based BMC firmware. This makes it critical that organizations have the ability to independently assess the devices that they rely on for vulnerabilities that could put the device at risk.

If you have any questions or comments regarding the research in this article, please reach out to the team at [email protected].