team discussion April 2020
(the plague year)
components we have...
>./sifget StudentPersonals
>./sifget SchoolInfos
retrieves SIF data from
data-hub or provider
system
configured once for environment
access
SIF xml output as
files,
stream,
web-response
generic xml / SIF in
generic JSON / SIF-PESC JSON out
converts between encodings, full round-trip, schema-aware for SIF.
schema built into
distributed binaries
based on specgen
output only
available via
NSIP
runs as application or web-service
inbound
full data
payload
outbound
redacted
payload
Privacy service is a web-service that redacts messages according to policies and rules.
Can suppress read and write operations.
Can remove, replace, empty any value.
Enforcement is data-path based.
policy service
maintains contextual
privacy control defintions
requested dynamically
by enforcer.
(PODS/own-format)
definitions of
redaction rules
Privacy service is itself
a client of policy-service
which issues redaction rules
for a given context.
n3w
publish()
query()
contexts
crdt
streaming / syncing
data-store
access to web services
protected by distributed PKI auth. & JWT claims
stored data
encrypted at rest
data encrypted at rest & in
transit over nats streams
benthos
links services, uses TLS and
PKI auth where needed.
input:
mqtt:
urls: [ tcp://TODO:1883 ]
topics: [ foo ]
pipeline:
processors:
- process_map:
premap:
id: doc.id
contents: doc.body
processors:
- lambda:
function: sentiment_analysis
postmap:
results.sentiment: .
output:
redis_streams:
url: tcp://TODO:6379
stream: baz
max_in_flight: 20
all actions
declared in external
configuration
not code.
publish()
query()
fetch xml from hub
convert to json
enforce inbound privacy controls
enforce outbound privacy controls
convert to xml
post xml to hub
store, link, analyse
inbound & outbound service orchestration
managed by workflows with error-handling, monitoring & tracing built in.
n3