U.S. government agencies, critical infrastructure entities, and other private sector organizations are being affected by a cyber threat actor—or actors—beginning in June 2020 or earlier related to vulnerabilities in certain Ivanti Pulse Connect Secure products. These entities confirmed the malicious activity after running the Pulse Secure Connect Integrity Tool released by Ivanti to detect the integrity of Pulse Connect Secure appliances.

To gain initial access, the threat actor is leveraging multiple vulnerabilities, including CVE-2019-11510, CVE-2020-8260, CVE-2020-8243, and the newly disclosed CVE-2021-22893. The threat actor is using this access to place webshells on the Pulse Connect Secure appliance for further access and persistence. The known webshells allow for a variety of functions, including authentication bypass, multi-factor authentication bypass, password logging, and persistence through patching.

Technical Details:

CISA has observed the cyber threat actor performing cleanup as demonstrated by the following:

1. Threat actor was observed timestomping trojanized umount binary to match timestamps of legitimate binaries attempting to disguise the modifications; the touch command was used to modify the time stamp https://attack.mitre.org/techniques/T1070/006/:

          /bin/touch /tmp/data/root/bin/umount -r /tmp/data/root/bin/cp

    2. The threat actor deleted files from temp directories using “rm -f”: 

          /bin/rm -f tmp1
          /bin/rm -f tmp2

    3. Timestamps:

Note: for context, loop 6 is the active partition and loop 8 is the rollback partition of the device.

Date	Time (GMT)	Partition	Artifact	Activity
4/13/21	5:15:33	pulse-loop6	/bin/umount	Content Modification Time
4/20/21	19:09:14	pulse-loop8	/bin/umount	Metadata Modification Time
4/20/21	19:09:14	pulse-loop8	/bin/umount	Content Modification Time
4/20/21	19:18:49	pulse-loop6	/bin/umount	Metadata Modification Time
4/23/21	16:14:48	pulse-loop6	/bin/umount	Last Access Time
5/6/21	14:27:20	pulse-loop8	/bin/umount	Last Access Time
4/20/21	19:08:01	pulse-loop6	/bin/touch	Last Access Time
4/20/21	19:09:14	pulse-loop8	/bin/touch	Last Access Time

The suspected cyber threat actor modified several legitimate Pulse Secure files on the impacted Pulse Connect Secure appliances. The modifications implemented a variety of webshell functionality:

• DSUpgrade.pm MD5: 4d5b410e1756072a701dfd3722951907
  • Runs arbitrary commands passed to it
  • Copies malicious code into Licenseserverproto.cgi
• Licenseserverproto.cgi MD5: 9b526db005ee8075912ca6572d69a5d6
  • Copies malicious logic to the new files during the patching process, allowing for persistence
• Secid_canceltoken.cgi MD5: f2beca612db26d771fe6ed7a87f48a5a
  • Runs arbitrary commands passed via HTTP requests
• compcheckresult.cgi MD5: ca0175d86049fa7c796ea06b413857a3
  • Publicly-facing page to send arbitrary commands with ID argument
• Login.cgi MD5: 56e2a1566c7989612320f4ef1669e7d5
  • Allows for credential harvesting of authenticated users
• Healthcheck.cgi MD5: 8c291ad2d50f3845788bc11b2f603b4a
  • Runs arbitrary commands passed via HTTP requests

Many of the threat actor’s early actions are logged in the Unauthenticated Requests Log as seen in the following format, URIs have been redacted to minimize access to webshells that may still be active:

Unauthenticated request url /dana-na/[redacted URI]?id=cat%20/home/webserver/htdocs/dana-na/[redacted URI] came from IP XX.XX.XX.XX.

The threat actor then ran the commands listed in the below table via the webshell.

Time	Command
2021-01-19T07:46:05.000+0000	pwd
2021-01-19T07:46:24.000+0000	cat%20/home/webserver/htdocs/dana-na/[redacted]
2021-01-19T08:10:13.000+0000	cat%20/home/webserver/htdocs/dana-na/l[redacted]
2021-01-19T08:14:18.000+0000	See Appendix.
2021-01-19T08:15:11.000+0000	cat%20/home/webserver/htdocs/dana-na/[redacted]
2021-01-19T08:15:49.000+0000	cat%20/home/webserver/htdocs/dana-na/[redacted]
2021-01-19T09:03:05.000+0000	cat%20/home/webserver/htdocs/dana-na/[redacted]
2021-01-19T09:04:47.000+0000	$mount
2021-01-19T09:05:13.000+0000	/bin/mount%20-o%20remount,rw%20/dev/root%20/
2021-01-19T09:07:10.000+0000	$mount

The cyber threat actor is using exploited devices located on residential IP space—including publicly facing Network Attached Storage (NAS) devices and small home business routers from multiple vendors—to proxy their connection to interact with the webshells they placed on these devices. These devices, which the threat actor is using to proxy the connection, correlate with the country of the victim and allow the actor activity to blend in with normal telework user activity. 

Note: these devices are not related to the Pulse vulnerabilities, but rather, where the malicious internet traffic passes through.

Details about lateral movement and post-exploitation are still unknown at this time.

Mitigations

It is strongly recommended organizations using Pulse Secure devices to immediately:

• Review the Pulse Secure Connect Integrity Tool Quick Start Guide and Customer FAQs
• Run the Pulse Secure Connect Integrity Tool.
  • The tool requires a reboot.
  • If virtualized, take a snapshot before running.
  • If the appliance is physical, consider the consequences of rebooting and running the tool and contact Ivanti for assistance or questions.
  • the Pulse Secure team released Security Advisory SA44784 that addresses CVE-2021-22893, CVE-2021-22984, CVE-2021-22899, and CVE-2021-22900 with patches.
• Update to the latest software version.
• Using the Pulse Secure Integrity Checker. The Integrity Checker Tool (ICT) helps system owners understand if their Pulse Secure Connect device has been compromised. While the tool is accurate, there are several nuances to its effective use.
  • The ICT detects evidence of adversary cleanup only on the current, running version of PCS.
  • It may be necessary to roll back the current PCS version to have a valid run of the ICT.
  • During the upgrade process, the active version becomes a rollback partition.
  • Only one rollback partition exists on a device, as the rollback partition is replaced on each update.
  • Therefore, if an entity has updated their PCS device without running the correct version of the ICT, anomalous activity will not be detected.
     

In addition to the recommendations above, organizations that find evidence of malicious, suspicious, or anomalous activity or files, should consider the guidance in KB44764 – Customer FAQ: PCS Security Integrity Tool Enhancements, which includes:

After preservation, you can remediate your Pulse Connect Secure appliance by: 

1. Disabling the external-facing interface.  
2. Saving the system and user config.
3. Performing a factory reset via the Serial Console. Note: For more information refer to KB22964 (How to reset a PCS device to the factory default setting via the serial console)
4. Updating the appliance to the newest version.
5. Re-importing the saved config.   
6. Re-enabling the external interface. 

CISA recommends performing checks to ensure any infection is remediated, even if the workstation or host has been reimaged. These checks should include running the Pulse Secure Connect Integrity Tool again after remediation has been taken place.