Event Data Filtering
Event data filtering allows defining rules with conditions that are evaluated for every event before it is sent to the backend. This feature applies to all built-in metrics. With each matching rule, a pre-defined action is executed that controls whether the event is sent to the backend or not. Additionally, it allows clearing the contents of certain fields before the event is sent to the backend.
Depending on the requirements, one might only need a subset of the events generated by uberAgent for certain sourcetypes. Filtering out unnecessary data at the endpoint may reduce the data volume significantly (see the documentation for other ways to reduce the data volume).
Example: every time a
cmd.exe process is started on Windows, an accompanying console host process is started too. These
conhost.exe processes are rarely of interest and can be safely excluded.
[EventDataFilter] # Exclude "conhost.exe" (typically started from the path: \??\C:\WINDOWS\system32\conhost.exe) Action = deny Sourcetype = Process:ProcessStartup Query = regex_match_path(ProcPath, r"^(\\\?\?\\)?%SystemRoot%\\System32\\conhost\.exe$")
Example: compared to Windows, macOS runs significantly more concurrent processes. which can also lead to an increased data volume. One way to adjust this is to exclude the processes that belong to the operating system.
[EventDataFilter] # Exclude processes belonging to the operating system Action = deny Sourcetype = Process:ProcessDetail Sourcetype = Process:ProcessStartup Sourcetype = Process:ProcessStop Sourcetype = Process:ProcessStatistics Sourcetype = Process:NetworkTargetPerformance Query = AppId == "mcOS"
Some sourcetypes have fields with data that may be considered sensitive in nature, such as window titles. Event data filtering allows clearing such fields, on the endpoint, before the data is sent to the backend for indexing and searching.
Example: clear the contents of session detail window title metric.
[EventDataFilter] # Clear the contents of window titles. Action = clear Sourcetype = Session:SessionDetail Field = SessionFgWindowTitle Query = true
The query above always evaluates to
true and therefore the action
clear is executed for this particular event and the field
SessionFgWindowTitle is cleared to an empty value.
An event data filter is configured using the configuration file. The stanza
[EventDataFilter] starts a new filter configuration, followed by multiple settings.
It’s essential to note that filter rules are processed sequentially. Once a rule matches, the filtering process for that particular event stops. Therefore, the order of filters can be crucial for the desired outcome.
|Action||Specifies the action to be taken if a query rule is matched.||
|Sourcetype||The category and name of the sourcetype.||Please refer to the metrics documentation for a list of available sourcetypes.|
|Query||The query rule to filter using uAQL.||Please refer to the uAQL documentation.|
|Field||Optional setting to specify the field to be cleared if the current Action is
||This can be any field of the given metric.|
A browser can generate a significant number of DNS monitoring events that are rarely of interest. Such events can be easily ignored using the following event data filter.
[EventDataFilter] # Deny any DNS event caused by browsers. Action = deny Sourcetype = Process:DnsEvent Query = ProcName in ["chrome.exe", "iexplore.exe", "firefox.exe", "msedge.exe", "opera.exe"]
[EventDataFilter] # Exclude processes whose name is exactly one of the given names. Action = deny Sourcetype = Process:ProcessDetail Query = ProcName in ["cmd.exe", "conhost.exe", "csrss.exe", "lsm.exe", "smss.exe", "wininit.exe", "winlogon.exe"]
Consider a use case where one wants to exclude an entire metric, except for specific data in events. This is achievable by explicitly allowing certain processes but denying all others. The stanza ordering is important because filter processing stops after the first match.
[EventDataFilter] # Allow network events caused by John Doe and Jane Doe. Action = allow Sourcetype = Process:NetworkTargetPerformance Query = ProcUser in ["John Doe", "Jane Doe"] [EventDataFilter] # Deny any network event that was not allowed in a previous filter. Action = deny Sourcetype = Process:NetworkTargetPerformance Query = true
In addition to the fields of a sourcetype, certain built-in receiver fields may also be used. This allows creating event data filters that are active for certain receivers, only. Check the documentation on routing to different backends to learn more on the use case.
The available fields are
For this example, we take a look at the Session Detail metric. Consider there are two receivers configured, one for most metrics and an extra receiver for sensitive content with restricted access.
[EventDataFilter] # Clear the contents of window titles for any receiver except "uberagent_sensitive" Action = clear Sourcetype = Session:SessionDetail Field = SessionFgWindowTitle Query = Receiver.Name != "uberagent_sensitive"
This configuration will clear the contents of
SessionFgWindowTitle in all receivers, except the receiver
We can also use almost the same rule to achieve a different use case: exclude events from all receivers, except
[EventDataFilter] # Exclude this event from all receivers except "uberagent_sensitive" Action = deny Sourcetype = Session:SessionDetail Query = Receiver.Name != "uberagent_sensitive"
Instead of clearing the field content for the non-matching receiver, we simply deny sending the event to all receivers, except
More examples including rules created by our support for our customers can be found in the knowledge base article here.