How We Achieved Effectively Unlimited Scalability

Our largest customers have deployed uberAgent to hundreds of thousands of endpoints – each. This article explains which aspects of the product’s architecture make that kind of (effectively unlimited) scalability possible.

Splunk Backend

uberAgent’s preferred backend, Splunk, is a big data platform that has been designed to handle huge amounts of data. If architected well, Splunk has no built-in limitations and scales practically infinitely.

Search Across the Entire Estate

Splunk’s data index is typically spread across multiple servers, each of which stores only a subset of all data. Search queries are executed in parallel by all indexers, whose partial results are combined into a complete result set. Increasing the capacity of a Splunk installation is as easy as adding more servers. See this article for an overview of how Splunk works.

Even very large numbers of endpoints do not need to be segmented in any way, as is necessary with some other products. Splunk searches always cover the entire fleet of endpoints. It is, of course, possible to search only a subset of machines or users, e.g., a specific site.

Historical Data

uberAgent and Splunk have no limit on how much historical data can be stored or for how long data can be kept. Neither are historical metrics compressed in any way: all data is always available in full resolution, in the same way as it was collected by the endpoint agent. With Splunk, you’re only limited by the available disk space.

Endpoint Agent Data Volume

If you only have a few hundred endpoints, the amount of data that each endpoint generates does not really matter. Things are very different, though, when hundreds of thousands of endpoints collect and send detailed information on a regular basis. At scale, any optimization that reduces the data volume or the number of API calls over the network proves to be invaluable.

uberAgent’s Data Volume Optimizations

uberAgent’s data volume optimizations include, but are not limited to:

• Most data types are transmitted in the most compact form possible, as comma-separated text. Even headers are omitted; Splunk knows about the field structure.
• Common text constants, such as the name of a browser (e.g., Chrome), are not transmitted as text strings, but represented by single-digit numbers, e.g. 1. The human-readable names are added via automatic lookups in Splunk.
• Numeric fields very often have the value zero. E.g., most processes’ CPU usage effectively amounts to zero most of the time. Instead of transmitting zero values as 0.0, as most applications would do, uberAgent shortens them to 0, which reduces the data’s length from 3 bytes to 1, saving 66%.

Dashboards & Searches

Dashboards visualizing uberAgent’s data and the searches that are powering them cannot just query and attempt to display data about individual machines. Tables with more than a few hundred (or rather: dozen) rows are unusable. Inefficient searches that have not been optimized for huge result sets take too long to execute.

Overview Dashboard With Drilldown Capabilities

uberAgent’s Experience Score dashboard is the perfect entry point. It gives an overview of the state of the environment and offers drill-downs to any relevant KPI or metric one might want to inspect.

uberAgent’s dashboards never attempt to list all entities individually. They always start with overview metrics or charts that provide grouped information per machine type, for example. Drilldown capabilities offer quick navigation down to individual endpoints, users, applications, and the like.

Configuration Flexibility

Organizations with hundreds of thousands of endpoints tend to have heterogeneous networks and complex requirements. A product that thrives in such an environment needs to be inherently flexible in the way it can be configured and deployed.

Endpoint Agent Deployment

uberAgent’s installation packages can be rolled out with any software deployment tool a customer might have in place. License files need not be deployed: they can be centrally hosted and updated. Agents occasionally connect to the license repository and update their cached copies.

Endpoint Agent Configuration

The agent can be configured via config files or via Active Directory Group Policy (Windows only). uberAgent ships with two configuration sets: the default configuration and a second configuration optimized for data volume. The data collection frequency of any of uberAgent’s metrics can be configured freely, from very high resolution (mere seconds) to minute or even hour or day intervals (reducing the data volume).

Allowlists and denylists make it possible to only collect what is needed. Event data filtering allows for powerful manipulation of the collected data before it leaves the endpoint, e.g., to remove sensitive data or further reduce the data volume.