OpenSearchSymposium 2021 / 12.10.2021
Stephan Schwichtenberg
only protection of bilateral IP connections
not protecting different data objects, but apis
unsuited for rapid change of data owners / new data channels
trust perimeter has changed
fragmented information (flows) need protection
authn/authz must be possible everywhere
data objects governed by external/internal access policies (AP)
data object interactions main driver for future IT architecture
devices produce and consume data at the same time
respect different data owners per device
if one fails, all suffer!
www.neuropil.org
OpenSource CyberSecurity Mesh
development started in 2014
2016: first exhibition @FROSCON
2019: NGI Zero / EU funded
2021: beta-release HMI 2021
looking for pilots & partners
Random Identities
User Identities
Intent Identities
H256(X)
0...
8...
4...
c...
why we joined:
rules to enforce data ownership / sovereignty
increase data quality and transparency
building european-wide ecosystems
neuropil@IDSA
decentralized (meta-data) broker
each application/device is a connector
decentralized MQTT
Neuropil is a project that wants to turn the tables on online search and discovery: instead of search solutions calling the shots, data owners decide what content is publicly searchable in the first place.
They can do this through a new messaging layer that is private and secure by design. Data owners can send cryptographic and unique so-called intent messages that state what specific information can be found where.
The access to the actual information or content is also controlled by data owners, for instance to provide either paid or public free content.
Broker
a4
93
82
2c
3d
4e
central broker structures
_
b5
0a
1b
71
60
5f
central broker structures
_
Broker
central broker structures ?
_
Broker
a4
93
82
2c
3d
4e
b5
0a
1b
71
60
5f
_
de-central algorithms
NGI Zero / part 1
_
a4
93
82
2c
3d
4e
b5
0a
1b
71
60
5f
subject="urn:neuropil:photo:library:v1"
Id
N
I'
Identity Token
Node Token
Intent Token
{
"iss": FP(Id),
"sub": "mail:pseudonym@example.com",
„pub“: <binary data>,
...
} + sig
{
"partner_dhkey": FP(N),
"attribute_1": "super_secret_sauce",
"attribute_1": bin(x),
} + sig
=> H(sub)
NGI Zero / part 1
_
_
step 1 / obfuscate subject:
step 2 / send intent token:
step 3 / messages exchange:
NGI Zero / part 1 / the past
_
step 1 / obfuscate subject:
0fa6 472b a981 3c56 (32 bytes)
01010000 00100100 10001000 00010100 (3*4 bytes)
NGI Zero / part 1 / the future
_
step 2 / obfuscate attributes of intent token:
NGI Zero / part 1 / the future
_
step 2.5 / increase storage capabilities of ABF:
NGI Zero / part 1 / the future
NEW
_
step 2 / obfuscate attributes of intent token:
NGI Zero / part 1 / the future
{
"iss": FP(Id),
"sub": "urn:neuropil:photo:library:v1",
„pub“: <binary data>,
...
} + sig
{
"partner_dhkey": FP(N),
"attribute_1": "super_secret_sauce",
"attribute_1": bin(x),
...
} + sig
Object
=> BF(obj)
Object Fields
=> BF(attributes)
_
step 3 / discovery of best path and exchange security token
step 4 / messages exchange:
NGI Zero / part 1 / the future
_
now is the time for questions or a short coffee break
NGI Zero / part 1 / questions
_
initial idea of the NGI Zero project:
approach works for single words / URL's / etc.:
not every node wants to be part of a specific search index
Neuropil zero search
_
NGI Zero / part 2
_
Neuropil zero search
minhash signatures:
_
Neuropil zero search
compare mmh-signatures / push mmh signatures to bloom filters:
_
Neuropil zero search
CLKHash - Cryptographic Longterm Keys:
_
Neuropil zero search
CLKHash - Cryptographic Longterm Keys:
_
Neuropil zero search
NEW
LSH - Locality Sensitive Hashing (based on minhash):
_
Neuropil zero search
LPH - Locality Preserving Hashing:
_
Neuropil zero search
More options for text analysis based LSH / LPH
_
Neuropil zero search
NEW
can LSH and LPH work together?
_
Neuropil zero search
let's use a counting bloom filter to compare LSH table distribution!
revisit mmh signature / LSH: (b=8/r=1; t=0,125)
_
Neuropil zero search
minhash(8): 15 - 54 - 9 - 23 - 823 - 547 - 3948 - 336
assume we have a set of eight hash tables
revisit mmh signature / LSH: (b=4/r=2; t=0,5)
_
NGI Zero / part 2
minhash(8): 15 - 54 - 9 - 23 - 823 - 547 - 3948 - 336
revisit mmh signature / LSH: (b=2/r=4; t=0,84)
_
NGI Zero / part 2
minhash(8): 15 - 54 - 9 - 23 - 823 - 547 - 3948 - 336
revisit mmh signature / LSH: (b=1/r=8; t = 1,0)
_
NGI Zero / part 2
minhash(8): 15 - 54 - 9 - 23 - 823 - 547 - 3948 - 336
revisit mmh signature / LSH: (b=2/r=4)
_
NGI Zero / part 2
minhash(8): 15 - 54 - 9 - 23 - 823 - 547 - 3948 - 336
revisit mmh signature / LSH:
_
NGI Zero / part 2
L-Quartile
U-Quartile
Median
use the median to calculate relative importance of the eight tables
00
01
10
11
10
01
00
01
00
01
11
11
revisit mmh signature / LSH:
_
NGI Zero / part 2
L-Quartile
U-Quartile
Median
use the median ?? experiments with different approaches
00
01
10
11
10
01
00
01
00
01
11
11
NEW
_
Neuropil zero search
using LSH and LPH together - 256bit hash value
_
Neuropil zero search
can LSH and LPH work together - 256bit hash value
_
Neuropil zero search
_
Neuropil zero search
_
Neuropil zero search
_
Neuropil zero search
NEW
_
Neuropil zero search
NEW
_
Neuropil zero search
NEW
_
Neuropil zero search
H("urn:np:search:node:v1")
SearchNode "Server"
SearchNode "Client"
H("urn:np:search:entry:v1")+
H("urn:np:search:peer:id")
H("urn:np:search:query:v1")+
H("urn:np:search:peer:id")
H("urn:np:search:result:v1")+
H("urn:np:search:peer:id")
SearchNode "Private"
urn:np:search:peer:id=H("/dev/random")
urn:np:search:peer:id=H("/dev/random")
urn:np:search:peer:id=H("/dev/random")
H("urn:np:search:node:v1") +
H("my:private:secret")
NEW
_
Neuropil zero search
bm25 - 5kmer - space
org-science-AI space
H256(X)
0...
8...
4...
c...
your-private space
NEW
_
Neuropil zero search
_
Neuropil zero search
_
Neuropil Zero Search
add curator
select ranking
exec bm25
select curator
_
NGI Zero / part 2
now is the time for questions or a longer coffee break
_
NGI Zero / part 2
Demo
pi-lar GmbH
Kreuzgasse 2-4
50667 Köln
www.pi-lar.net
info@pi-lar.net
eliza@neuropil.org
www.neuropil.org
https://www.gitlab.com/pi-lar/neuropil
neuropil@pi-lar.net
Let's
chat !
Join Our Workshops!