The following walkthroughs illustrate how to build a Splunk search from scratch.
Scenario: we want to identify users who launch a specific executable more often than n times in a given time range.
Splunk SPL Search
Let’s build the Splunk SPL search step by step.
Step 1
We start with all events from uberAgent’s index.
Note: to facilitate changing the index name used by uberAgent, all our dashboards make use of the uberAgent_index macro which contains the actual index name. The macro is defined in macros.conf of the uberAgent searchhead app.
index=`uberAgent_index`
Step 2
We filter for the process startup sourcetype which contains one event per started process.
Note: The documentation of uberAgent’s sourcetypes and fields can be found here.
We ignore processes started by SYSTEM, LOCAL SERVICE and NETWORK SERVICE.
Note: The pseudo-users sys, lvc and nvc are defined in the lookup table systemusers.csv of the uberAgent searchhead app. They are auto-expanded to the proper user names SYSTEM, LOCAL SERVICE and NETWORK SERVICE in uberAgent’s data model.
Note: The only purpose of adding the field ProcName to the stats command is to make it part of the results table, too.
index=`uberAgent_index` sourcetype=uberAgent:Process:ProcessStartup ProcUser!=sys ProcUser!=lvc ProcUser!=nvc ProcName=Winword.exe
| stats count as Starts by ProcName ProcUser
Step 6
We only keep users with more than five starts in the results list.
index=`uberAgent_index` sourcetype=uberAgent:Process:ProcessStartup ProcUser!=sys ProcUser!=lvc ProcUser!=nvc ProcName=Winword.exe
| stats count as Starts by ProcName ProcUser
| where Starts > 5
Step 7
We rename fields to make them look nicer.
index=`uberAgent_index` sourcetype=uberAgent:Process:ProcessStartup ProcUser!=sys ProcUser!=lvc ProcUser!=nvc ProcName=Winword.exe
| stats count as Starts by ProcName ProcUser
| where Starts > 5
| rename ProcUser as User ProcName as Process
Step 8
We sort the results so that the user with the highest number of starts is listed first. The 0 in the sort command ensures that the output is not truncated after the 10,000th result.
index=`uberAgent_index` sourcetype=uberAgent:Process:ProcessStartup ProcUser!=sys ProcUser!=lvc ProcUser!=nvc ProcName=Winword.exe
| stats count as Starts by ProcName ProcUser
| where Starts > 5
| rename ProcUser as User ProcName as Process
| sort 0 -Starts
The Result
This is what the final Splunk SPL search for users with more than five starts of Winword.exe looks like. The screenshot below shows the search being run over the past 30 days. In practice, you would adjust the time range to any relevant time interval.
Accelerated Data Model Search
uberAgent comes with an accelerated data model. Searching an accelerated data model is a lot faster than searching the underlying index (by “a lot” we mean at least 50x), but requires a different search syntax based on the pivot or tstats commands. We are using pivot because of the easier syntax compared to tstats.
In this second example, we demonstrate how to search for starts of a “modern” UWP app, specifically the weather app that is part of Windows. Most UWP apps cannot be identified by process name – which is simply backgroundTaskHost.exe. Luckily uberAgent determines the real app name automatically.
Step 1
We start with a count of all process starts.
Note: The documentation for the Splunk pivot command can be found here.
Note: the macro uA_DM_Process_ProcessStartup resolves to the name of the data model containing the Process_ProcessStartup dataset. We use this technique to facilitate moving datasets between data models.
| pivot `uA_DM_Process_ProcessStartup` Process_ProcessStartup count(Process_ProcessStartup) as Starts
Step 2
We filter for the weather app.
Note: An easy way to identify the name of the weather app is to dig around with a search like the following: index=`uberAgent_index` sourcetype=uberAgent:Process:ProcessStartup AppName=*Weather*
| pivot `uA_DM_Process_ProcessStartup` Process_ProcessStartup count(Process_ProcessStartup) as Starts
filter AppName is "Microsoft.BingWeather"
Step 3
We split by user so that we get a count of process starts per user (renaming the ProcUser field to User in the process).
| pivot `uA_DM_Process_ProcessStartup` Process_ProcessStartup count(Process_ProcessStartup) as Starts
filter AppName is "Microsoft.BingWeather"
splitrow ProcUser as User
Step 4
We only keep users with more than five starts in the results list. We also sort the results so that the user with the highest number of starts is listed first. The 0 in the sort command ensures that the output is not truncated after the 10,000th result.
| pivot `uA_DM_Process_ProcessStartup` Process_ProcessStartup count(Process_ProcessStartup) as Starts
filter AppName is "Microsoft.BingWeather"
splitrow ProcUser as User
| where Starts > 5
| sort 0 -Starts
Step 5
We add the application name as a row to the results table.
| pivot `uA_DM_Process_ProcessStartup` Process_ProcessStartup count(Process_ProcessStartup) as Starts latest(AppName) as Application
filter AppName is "Microsoft.BingWeather"
splitrow ProcUser as User
| where Starts > 5
| sort 0 -Starts
| table Application User Starts
The Result
The resulting output is very similar to the first example above: a table with the application, the users and the number of starts that can easily be exported to CSV or otherwise be processed further.
Walkthrough: Building a Splunk Search
In this article
The following walkthroughs illustrate how to build a Splunk search from scratch.
Scenario: we want to identify users who launch a specific executable more often than
n
times in a given time range.Splunk SPL Search
Let’s build the Splunk SPL search step by step.
Step 1
We start with all events from uberAgent’s index.
Note: to facilitate changing the index name used by uberAgent, all our dashboards make use of the
uberAgent_index
macro which contains the actual index name. The macro is defined inmacros.conf
of the uberAgent searchhead app.Step 2
We filter for the process startup sourcetype which contains one event per started process.
Note: The documentation of uberAgent’s sourcetypes and fields can be found here.
Step 3
We ignore processes started by SYSTEM, LOCAL SERVICE and NETWORK SERVICE.
Note: The pseudo-users
sys
,lvc
andnvc
are defined in the lookup tablesystemusers.csv
of the uberAgent searchhead app. They are auto-expanded to the proper user names SYSTEM, LOCAL SERVICE and NETWORK SERVICE in uberAgent’s data model.Step 4
We add a filter for the name of the process we are interested in,
Winword.exe
in this example.Step 5
We count the number of (start) events per user.
Note: The only purpose of adding the field
ProcName
to thestats
command is to make it part of the results table, too.Step 6
We only keep users with more than five starts in the results list.
Step 7
We rename fields to make them look nicer.
Step 8
We sort the results so that the user with the highest number of starts is listed first. The
0
in thesort
command ensures that the output is not truncated after the 10,000th result.The Result
This is what the final Splunk SPL search for users with more than five starts of
Winword.exe
looks like. The screenshot below shows the search being run over the past 30 days. In practice, you would adjust the time range to any relevant time interval.Accelerated Data Model Search
uberAgent comes with an accelerated data model. Searching an accelerated data model is a lot faster than searching the underlying index (by “a lot” we mean at least 50x), but requires a different search syntax based on the
pivot
ortstats
commands. We are usingpivot
because of the easier syntax compared totstats
.In this second example, we demonstrate how to search for starts of a “modern” UWP app, specifically the weather app that is part of Windows. Most UWP apps cannot be identified by process name – which is simply
backgroundTaskHost.exe
. Luckily uberAgent determines the real app name automatically.Step 1
We start with a count of all process starts.
Note: The documentation for the Splunk pivot command can be found here.
Note: the macro
uA_DM_Process_ProcessStartup
resolves to the name of the data model containing theProcess_ProcessStartup
dataset. We use this technique to facilitate moving datasets between data models.Step 2
We filter for the weather app.
Note: An easy way to identify the name of the weather app is to dig around with a search like the following:
index=`uberAgent_index` sourcetype=uberAgent:Process:ProcessStartup AppName=*Weather*
Step 3
We split by user so that we get a count of process starts per user (renaming the
ProcUser
field toUser
in the process).Step 4
We only keep users with more than five starts in the results list. We also sort the results so that the user with the highest number of starts is listed first. The
0
in thesort
command ensures that the output is not truncated after the 10,000th result.Step 5
We add the application name as a row to the results table.
The Result
The resulting output is very similar to the first example above: a table with the application, the users and the number of starts that can easily be exported to CSV or otherwise be processed further.