types of data

agenda

types of storages

challenges

types of data

types of data

user data

• vendor (silvair Open)

• web language

• notifications

• brand

• email

• name, company

• sign up date

• login history (2 days)

• legal documents

• verification nonces

auth0

internal

types of data

project data

• name, timezone, location

• enabled features

• areas, zones (names, images, placement)

• schedules (gateway, ins, ELT)

• profiles (control, scenes, energy)

• energy saving settings

• ELT RESULTS

types of data

commissioning data ("topology")

• mesh network (keys, addresses, iv index)

• applications (mobile apps, gateways)

• nodes

  • uuids and names

  • mesh configuration

  • commissioning status

  • OTAU STATUS

• provisioning events

types of data

manufacturing data (licensing)

• uuids

• otau keys (crypto sets)

• firmware releases (binaries, manifests)

• firmware versions (per uuid)

• ...?

types of data

monitoring data

• raw readings

• aggregates

• reports

types of data

branding data

• filters

• renamings

• legal documents

• enabled features

types of storage

storages

document store

• external

• easy to modify

• inconsistent by design

storages

relational database

• internal

• hard to modify

• enforced consitency

storages

file storage

• internal

• permanent

• inconsistent by design

• inexpensive

• public access

storages

analytical database

• external

• ephemeral

• expensive

• inconsistent by design

storages

auth0

• external

• multiple instances (per branding)

• identity provider

• ... but we treat it as user database

storages

challenges

challenges

it's not obvious what goes where

• user profiles are in auth0

  • ... but some parts are internal

• commissioning data is in topology database

  • ... apart from commissioning history, which is in project database

• ...

challenges

querying

• need to join data from multiple databases

• data schema is undefined

• services do not provide complete rest api

challenges

performance

• everyone needs to talk to everyone else

  • ... all the time

• caching is harder than usual

challenges

migrations

• zero downtime deployments

  • ...except we don't really have them

• migrations take a long time

  • ...so we rarely do them

  • ...so we have outdated entries

2023

commissioning platform 101:
data flow

questions?