Event Data Buffering When Network Connectivity to Splunk is Lost
Many organizations want to be sure that the data collected by uberAgent reaches their Splunk infrastructure reliably even when there is only intermittent network connectivity. This article discusses how that can be achieved.
RAM Buffer
Before uberAgent tries to send new events to Splunk, it writes them to an in-memory queue (RAM buffer). The size of this buffer is 10 MB by default and can be set to any suitable value in the configuration.
Once an event’s data has been sent to Splunk, it is removed from uberAgent’s buffer. This simple algorithm ensures that network glitches and connectivity issues do not affect uberAgent’s reliability and visibility.
Long-term Buffering
While uberAgent’s in-memory buffer efficiently helps with short to medium phases without connectivity, it is not designed for use cases where computers are offline for days or weeks at a time. It also does not help when the machine is rebooted, in which case buffered data is lost.
Sizing the RAM Buffer
On typical client machines, uberAgent generates about 550 Bytes per second on average. With this number, we can easily size the RAM buffer. For example, if we wanted it to be just large enough to protect against a maximum of 60 minutes of offline time, we would set the buffer size to 2 MB.
Minimum Buffer Size
The minimum buffer size is 1 MB. Setting the buffer size to 0 is equivalent to an unlimited buffer size.
Offline/Laptop Usage
If you plan to use uberAgent on laptops or other devices that are offline for longer periods of time please read how to persist collected data on disk until it could be sent to Splunk successfully.