privacy on the wire
team discussion April 2020
(the plague year)
components we have...
- SIF Client
- JSON-XML Converter
- Privacy Service
- nias3 Data Store
- workflow engine
SIF Client
>./sifget StudentPersonals
>./sifget SchoolInfos
retrieves SIF data from
data-hub or provider
system
configured once for environment
access
SIF xml output as
files,
stream,
web-response
JSON-XML
Converter
generic xml / SIF in
generic JSON / SIF-PESC JSON out
converts between encodings, full round-trip, schema-aware for SIF.
schema built into
distributed binaries
based on specgen
output only
available via
NSIP
runs as application or web-service
privacy
service
inbound
full data
payload
outbound
redacted
payload
Privacy service is a web-service that redacts messages according to policies and rules.
Can suppress read and write operations.
Can remove, replace, empty any value.
Enforcement is data-path based.
policy service
maintains contextual
privacy control defintions
requested dynamically
by enforcer.
(PODS/own-format)
definitions of
redaction rules
Privacy service is itself
a client of policy-service
which issues redaction rules
for a given context.
n3 components
n3w
publish()
query()
contexts
crdt
streaming / syncing
data-store
access to web services
protected by distributed PKI auth. & JWT claims
stored data
encrypted at rest
data encrypted at rest & in
transit over nats streams
workflow engine
benthos
links services, uses TLS and
PKI auth where needed.
input:
mqtt:
urls: [ tcp://TODO:1883 ]
topics: [ foo ]
pipeline:
processors:
- process_map:
premap:
id: doc.id
contents: doc.body
processors:
- lambda:
function: sentiment_analysis
postmap:
results.sentiment: .
output:
redis_streams:
url: tcp://TODO:6379
stream: baz
max_in_flight: 20all actions
declared in external
configuration
not code.
put it all together...
publish()
query()
fetch xml from hub
convert to json
enforce inbound privacy controls
enforce outbound privacy controls
convert to xml
post xml to hub
store, link, analyse
inbound & outbound service orchestration
managed by workflows with error-handling, monitoring & tracing built in.
n3
questions
privacy on the wire
By matt_farmer
