2023
silvair platform 101: data
types of data
agenda
types of storages
challenges
types of data
types of data
user data
-
vendor (silvair Open)
-
web language
-
notifications
-
brand
-
email
-
name, company
-
sign up date
-
login history (2 days)
-
legal documents
-
verification nonces
auth0
internal
types of data
project data
-
name, timezone, location
-
enabled features
-
areas, zones (names, images, placement)
-
schedules (gateway, ins, ELT)
-
profiles (control, scenes, energy)
-
energy saving settings
-
ELT RESULTS
types of data
commissioning data ("topology")
-
mesh network (keys, addresses, iv index)
-
applications (mobile apps, gateways)
-
nodes
-
uuids and names
-
mesh configuration
-
commissioning status
-
OTAU STATUS
-
-
provisioning events
types of data
manufacturing data (licensing)
-
uuids
-
otau keys (crypto sets)
-
firmware releases (binaries, manifests)
-
firmware versions (per uuid)
-
...?
types of data
monitoring data
-
raw readings
-
aggregates
-
reports
types of data
branding data
-
filters
-
renamings
-
legal documents
-
enabled features
types of storage
storages
document store
-
external
-
easy to modify
-
inconsistent by design
storages
relational database
-
internal
-
hard to modify
-
enforced consitency
storages
file storage
-
internal
-
permanent
-
inconsistent by design
-
inexpensive
-
public access
storages
analytical database
-
external
-
ephemeral
-
expensive
-
inconsistent by design
storages
auth0
-
external
-
multiple instances (per branding)
-
identity provider
-
... but we treat it as user database
storages
| DOCUMENT DB | RELATIONAL DB | ANALYTICAL DB | FILE STORAGE | AUTH0 API | |
|---|---|---|---|---|---|
| speed | ✔️ | ✕ | ✔️ | ✕ | ✕ |
| durability | ✔️ | ✔️ | ✕ | ✔️ | ✔️ |
| consistency | ✕ | ✔️ | ✕ | ✕ | ✔️ |
| cost | $$ | $ | $$$ | $ | $$$ |
| maintenance | low | high | low | low | low |
challenges
challenges
it's not obvious what goes where
-
user profiles are in auth0
-
... but some parts are internal
-
-
commissioning data is in topology database
-
... apart from commissioning history, which is in project database
-
-
...
challenges
querying
-
need to join data from multiple databases
-
data schema is undefined
-
services do not provide complete rest api
challenges
performance
-
everyone needs to talk to everyone else
-
... all the time
-
-
caching is harder than usual
challenges
migrations
-
zero downtime deployments
-
...except we don't really have them
-
-
migrations take a long time
-
...so we rarely do them
-
...so we have outdated entries
-