Christoffer Hjortlund

Software Engineer

TimedDict – an easy rolling window

When dealing with real-time* data (or events), I often find the need to ether; monitor the amount of data points, within a given time frame, or use the data for performing some rolling statistical calculating.

Okay, but it’s not that difficult!

Indeed, it’s relatively easily done, using a normal dict structure with a timestamp as key, then iterating over all the keys, in order to discard the ones too old, every time your want to perform your count or calculation.

But.. Hey! That adds a loop to on your real-time path, a loop that easily can be very large. If the precision needed, for old elements to be removed, is >1 ms (0.001 sec), using a thread to do the purging, this can be handled beforehand, allowing fast access to the data and preventing congestion and slowdowns.

This all of a sudden turns more complicated, and this relative simple feature clocks up the readability of the code, the solution: Introducing the TimedDict data structure.

Read more about the TimedDict on PyPI.

Example

The following code snippet illustrates the functionality of the TimedDict, from instantiation to stopping of the purge thread.

Output

Raw data:
{1534608053.6948583: 'value_1', 1534608054.6948583: 'value_2', 1534608055.6948583: {'values': {'value_4', 'value_3'}}}

- protect()
1534608053.6948583
1534608054.6948583
1534608055.6948583

- pause() followed by resume()
1534608053.6948583
1534608054.6948583
1534608055.6948583

Length of the TimedDict: 3
{1534608053.6948583: 'value_1', 1534608054.6948583: 'value_2', 1534608055.6948583: {'values': {'value_4', 'value_3'}}}
{1534608054.6948583: 'value_2', 1534608055.6948583: {'values': {'value_4', 'value_3'}}}
{1534608055.6948583: {'values': {'value_4', 'value_3'}}}
{}

* Soft real-time that is, as a deterministic execution time is not guaranteed, when using a garbage collected language.

Write a Comment

Your email address will not be published. Required fields are marked *