UltraTimeSeries: Data model and Concepts
UltraTimeSeries is a time series platform built to collect, process, and visualize time-related information. It’s built to be flexible, fast, and scalable.
At the core of UltraTimeSeries, there is a novel way to think about data. Instead of using columns and tables from the relational model, at UltraTimeSeries, we have created our own model. The Actor/Message Model with the goal of being as simple and flexible as possible.
At the core of our way to model data and solving problems lie three constructs:
Actor - any entity capable of producing data (i.e capable of sending messages)
Message - time-based events to be stored. Messages contain unit, action, and time a triplet capable of describing any event in the observable universe.
Roles - Roles are how we store actors’ metadata (data that is not time-related).
An actor is any entity that can generate data. It can be a physical entity such as a sensor or a mobile phone or a virtual or logical entity such as logging software. Actors are to whom the time series relates, they are the owners of the time series. Without actors there cannot be time series and yet a single actor can be the owner of multiple time series.
Actors are very simple structures and to create as they have only one field:
- Key a string that uniquely identifies the actor in the whole system.
When designing a system with this model one of the first things that needs to be done is to identify all the actors that take part in it. The fact that we only need to uniquely identify them means that we can start interacting with them after a bare minimum.
Creating a Time Series
Once we have registered actors in the database these actors can now receive messages. A message is an event with a timestamp. When an actor receives a message for the first time, a new time series is created with a single entry. As more messages arrive, more entries are added to that same time series. Time series can be sparse (very few entries - once a year) or dense (lots of entries - once every second).
A message has the following structure:
Subject - the owner(actor) of that time series.
Unit - unit is a 3 part construct
Dimension - a top-level unit of measurement (e.g. mass, time, length, etc)
Unit - the actual unit of measurement (gram or kilogram, hour or minutes, etc)
Value - numeric representation of the quantity, which is a separated argument when creating a message.
Action - a string (with a maximum length of 4096) describing what is the time series monitoring example:
(heights of) Ocean Tides
(temperature of) Sensor reading
(closing value of) Dow Jones. In parentheses, we have the unit of measure for the time series.
Time - the date of when the event took (or will take) place.
When modelling data in UltraTimeSeries, 2 things must be kept in mind:
Source - who is generating this data. The source is the actor.
What - what we are storing/tracking, what the action is, it must have the time component. Examples of actions are reading the temperature of a room over time. The cash withdrawals from an account over time.
Unlike relational databases where there can be direct relationships between tables here, there are no direct relationships between time series. These time series can be analyzed in conjunction as part of larger algorithms but each one exists on its own. This separation allows for later multivariate analysis without the constraints of a fixed data model.
A time series is uniquely identified by the combination of actor + action and an actor can be the owner of multiple time series (actions).
actor: john + action: Running - stores the distance that John spent running.
actor: john + action: Going to the Gym - stores the time that John has spent in the gym.
actor: mary + action: Running - stores the distance that Mary spent running.
actor: mary + action: Going to the Gym - stores the time that Mary has spent in the gym.
In the above examples, we have 4 different time series although both actors and actions do repeat.
In UltraTimeSeries we can generate and store as many time series as required to solve the problem at hand.
When working with a utility company helping then monitor domestic solar panels. Each set of solar panels was considered a single actor and for each actor, we had three different actions:
- Energy Produced - how much was produced by the installation, in watts.
- Energy Consumed - how much was used by the household, in watts.
- Energy Injected - how much has been put back into the grid, in watts. All of this data was being collected in a 15min interval which allowed the creation of these 3-time series. These time series made helpdesk life a lot easier because now they could simply compare Energy Produced with the neighbouring houses to detect anomalies.
Another example of modelling data comes from the banking industry where the job was to monitor the ATM network health. We achieved this by monitoring cash withdrawals. In this scenario, each ATM was an actor and each actor only had one-time series:
- Cash Withdrawal - the amount of money disbursed at a given moment. This data was used to create a forecasting model that would tell how much money was expected to be withdrawn the next day, at a 5 min interval. Less money being withdrawn usually means a problem in the network (i.e machine ran out of money).
Next, we will see how to store actors’ data such as an address, serial number, or installation date.
UltraTimeSeries has the concept of Roles, and actors can inhabit (multiple) roles. Roles are useful to store metadata because actors themselves only have 1 property (key - unique identifier). Using the example above of John and Mary let's imagine we want to store more information related to their sports life, such as:
- Full name
- Shoe size
We can create a new role named SportPerson with the following properties:
- FullName of type Text
- Birthdate of type Date
- Height of type Number
- Weight of type Number
- Gender of type Text
- ShoeSize of type Number
A role can have as many properties as necessary but properties can only be of 3 types:
- Text - a string with a maximum length of 4096
- Number - a double-precision floating-point number
- Date - a date value type represents dates and times with values ranging from 00:00:00 (midnight), January 1, 1601, through 11:59:59 P.M., December 31, 9999 A.D. (C.E.) in the Gregorian calendar.
Properties are also immutable, both in definition and in instance value.
Once a property has been defined, regardless of the role, a property with the same name cannot be defined again even if the type is different. In other words, once the property Gender is defined as Text it cannot be redefined or declared somewhere else with the type Number.
Properties also maintain the same exact value across roles. Assuming we have the following roles: SportsPerson and Employee both with property FullName. We also have an actor, mary, who’s both a SportsPerson and an Employee. Once an actor is instantiated in the role SportsPerson with the FullName Mary Doe, that value carries to all the other roles the actor has. A change in the value of that property will be reflected across all roles.
This convention allows an actor to have multiple roles without the worry of information clash or constant updating and ensures information consistency across the whole system.
UltraTimeSeries focuses on simplicity and that is why we created such a simple, yet powerful model. Time series are stored always in the same fashion, using the same 2 constructs:
Actor - source/generator of the information
Message - information that is being stored
Actors' metadata is stored in roles and properties. While there is no direct relationship between actors there is the inferred relationship of actors being in the same role and sharing the same set of properties.
This strictness brings predictability when interacting with UltraTimeSeries and lowers the cognitive load when rationalizing the information or the algorithms to build on top of it.
You can find this guide as a Postman collection including examples here.