Event Data Filtering
What is Event Data Filtering?
Event data filtering allows defining rules with conditions that are evaluated for every event before it is sent to the backend. This feature applies to all built-in metrics. With each matching rule, a pre-defined action is executed that controls whether the event is sent to the backend or not. Additionally, it allows clearing the contents of certain fields before the event is sent to the backend.
Use Cases for Event Data Filtering
Data Volume Reduction
Depending on the requirements, one might only need a subset of the events generated by uberAgent for certain sourcetypes. Filtering out unnecessary data at the endpoint may reduce the data volume significantly (see the documentation for other ways to reduce the data volume).
Example: every time a cmd.exe process is started on Windows, an accompanying console host process is started too. These conhost.exe processes are rarely of interest and can be safely excluded.
[EventDataFilter]
# Exclude "conhost.exe" (typically started from the path: \??\C:\WINDOWS\system32\conhost.exe)
Action = deny
Sourcetype = Process:ProcessStartup
Query = regex_match_path(ProcPath, r"^(\\\?\?\\)?%SystemRoot%\\System32\\conhost\.exe$")
Sensitive Data Removal
Some sourcetypes have fields with data that may be considered sensitive in nature, such as window titles. Event data filtering allows clearing such fields, on the endpoint, before the data is sent to the backend for indexing and searching.
Example: clear the contents of session detail window title metric.
[EventDataFilter]
# Clear the contents of window titles.
Action = clear
Sourcetype = Session:SessionDetail
Field = SessionFgWindowTitle
Query = trueThe query above always evaluates to true and therefore the action clear is executed for this particular event and the field SessionFgWindowTitle is cleared to an empty value.
Configuring Event Data Filtering
An event data filter is configured using the configuration file. The stanza [EventDataFilter] starts a new filter configuration, followed by multiple settings.
| Setting | Description | Valid Values |
|---|---|---|
| Action | Specifies the action to be taken if a query rule is matched. | allow, deny or clear |
| Sourcetype | The category and name of the sourcetype. | Please refer to the metrics documentation for a list of available sourcetypes. |
| Query | The query rule to filter using uAQL. | Please refer to the uAQL documentation. |
| Field | Optional setting to specify the field to be cleared if the current Action is clear. This setting can be configured multiple times. |
This can be any field of the given metric. |
Example: Exclude DNS Monitoring Events Caused by Browsers
A browser can generate a significant number of DNS monitoring events that are rarely of interest. Such events can be easily ignored using the following event data filter.
[EventDataFilter]
# Deny any DNS event caused by browsers.
Action = deny
Sourcetype = Process:DnsEvent
Query = ProcName in ["chrome.exe", "iexplore.exe", "firefox.exe", "msedge.exe", "opera.exe"]Example: Exclude Process Detail Metrics For Certain Processes
[EventDataFilter]
# Exclude processes whose name is exactly one of the given names.
Action = deny
Sourcetype = Process:ProcessDetail
Query = ProcName in ["cmd.exe", "conhost.exe", "csrss.exe", "lsm.exe", "smss.exe", "wininit.exe", "winlogon.exe"]Example: Exclude Everything Except…
Consider a use case where one wants to exclude an entire metric, except for specific data in events. This is achievable by explicitly allowing certain processes but denying all others. The stanza ordering is important because filter processing stops after the first match.
[EventDataFilter]
# Allow network events caused by John Doe and Jane Doe.
Action = allow
Sourcetype = Process:NetworkTargetPerformance
Query = ProcUser in ["John Doe", "Jane Doe"]
[EventDataFilter]
# Deny any network event that was not allowed in a previous filter.
Action = deny
Sourcetype = Process:NetworkTargetPerformance
Query = true
Per-Receiver Configuration
In addition to the fields of a sourcetype, certain built-in receiver fields may also be used. This allows creating event data filters that are active for certain receivers, only. Check the documentation on routing to different backends to learn more on the use case.
The available fields are Receiver.Name and Receiver.Index.
Example: Clear Fields Per Receiver
For this example, we take a look at the Session Detail metric. Consider there are two receivers configured, one for most metrics and an extra receiver for sensitive content with restricted access.
[EventDataFilter]
# Clear the contents of window titles for any receiver except "uberagent_sensitive"
Action = clear
Sourcetype = Session:SessionDetail
Field = SessionFgWindowTitle
Query = Receiver.Name != "uberagent_sensitive"This configuration will clear the contents of SessionFgWindowTitle in all receivers, except the receiver uberagent_sensitive.
Example: Filter Events Per Receiver
We can also use almost the same rule to achieve a different use case: exclude events from all receivers, except uberagent_sensitive.
[EventDataFilter]
# Exclude this event from all receivers except "uberagent_sensitive"
Action = deny
Sourcetype = Session:SessionDetail
Query = Receiver.Name != "uberagent_sensitive"Instead of clearing the field content for the non-matching receiver, we simply deny sending the event to all receivers, except uberagent_sensitive.
More Examples
More examples including rules created by our support for our customers can be found in the knowledge base article here.