uberAgent 7 Preview: EDR & Antivirus Performance Monitoring
While we’re finalizing version 7.0 of our user experience monitoring & endpoint security analytics products uberAgent UXM and uberAgent ESA, let’s take a look at another cool new feature: protected process performance monitoring (applies to EDR, antivirus, and other types of security products).
Endpoint security products need to protect themselves from tampering and deactivation by malware. One of the techniques to help with such protection is built into the Windows OS: protected processes light (PPL).
The PPL model represents a security boundary. When a vendor chooses to run its security agent as a protected process light, other user-mode processes cannot modify or terminate the agent any more – this includes elevated processes and processes running as
SYSTEM (access right details).
- Security hardening.
When a security product makes use of the PPL security model, some operations become more difficult, e.g.:
- Administrators cannot stop or restart the product’s system services.
- Monitoring software cannot query the OS for CPU or RAM performance metrics of the product’s processes.
A process’ protection level is stored in the
Protection member of the
EPROCESS kernel structure. The
Protection member is a
PS_PROTECTION structure containing the following three components (docs):
- Audit (unused)
Type can be one of the following:
- PsProtectedTypeNone: the process is not protected.
- PsProtectedTypeProtectedLight: the process is a protected process light (PPL).
- PsProtectedTypeProtected: the process is a protected process.
The actual protection level is a combination of
Signer. The following table lists the valid protection levels from high to low power:
The Windows security model does not allow regular user-mode monitoring products to query the CPU or RAM usage of a protected process. Only processes with equal or higher protection level can access protected processes, even if it’s only to collect performance data. There are two ways to solve this:
- Configure the monitoring product to run as a protected process itself, at a protection level that is higher than the processes it needs to monitor. This is impractical given that some OS processes like
lsass.exerun at very high protection levels (
- Collect process performance data from a kernel-mode driver. This is exactly what uberAgent 7.0 does and which enables it to collect performance metrics for all processes independent of their protection level.
The following screenshots demonstrate uberAgent’s new capability to monitor the performance of security agents and antivirus products. We’re using Windows Defender as an example. Typical for antivirus products, Defender’s main process
MsMpEng.exe has the following protection properties:
- Type: Protected light
- Signer: Antimalware
- Protection level: PsProtectedSignerAntimalware-Light
Please note that the screenshots were taken in a lab environment and show average values of multiple endpoints.
As you can see in the screenshot below, the baseline RAM usage is approximately 200 MB. Interestingly, the RAM usage increases significantly to almost 600 MB over a period of several hours once per day. These increases do not coincide with CPU spikes (blue line).
The average CPU usage of Windows Defender is very low. The most active process is
MsMpEng.exe. All Defender processes combined generate an average CPU load of less than 0.2%.
The average RAM usage of Windows Defender is a little more than 300 MB. Again, the most active process is
As with CPU and RAM usage,
MsMpEng.exe generates most of the disk IO. The value of 36,712 in the screenshot below was measured over a period of three full days. Disk operations per second (IOPS) are, therefore, low.
Defender talks to multiple hosts in the Azure Cloud. Actual hostnames and IP addresses will vary between geographic regions.
- Windows Internals, 7th edition, part 1
- itm4n: Do You Really Know About LSA Protection (RunAsPPL)?
- Microsoft docs: Protecting anti-malware services
uberAgent is an innovative Windows and macOS user experience monitoring (UXM) and endpoint security analytics (ESA) product.
uberAgent UXM highlights include detailed information about boot and logon duration, application unresponsiveness detection, network reliability drill-downs, process startup duration, application usage metering, browser performance, web app metrics, and Citrix insights. All these varied aspects of system performance and reliability are smartly brought together in the Experience Score dashboard.
uberAgent ESA excels with a sophisticated activity monitoring engine, the uAQL query language, detection of risky activity, DNS query monitoring, hash calculation, registry monitoring, and Authenticode signature verification. uberAgent ESA comes with Sysmon and Sigma rule converters, a graphical rule editor, and uses a simple yet powerful query language instead of XML.
About vast limits
vast limits GmbH is the company behind uberAgent, the innovative user experience monitoring and endpoint security analytics product. vast limits’ customer list includes organizations from industries like finance, healthcare, professional services, and education, ranging from medium-sized businesses to global enterprises. vast limits’ network of qualified solution partners ensures best-in-class service and support anywhere in the world.