Presented by: Core Dev Team

Technical Walkthrough

BLOCKHAIN LAYER

STREAMING LAYER

STORAGE LAYER

CONSENSUS PROTOCOL

ARCHITECTURE OVERVIEW

NEM Catapult

PeerStream Protocol - Media Routing, Presence and Messaging

IPFS P2P Storage

Proof of Importance, Proof of Storage and Proof of Bandwidth

API Gateway

External DApp Development

IPFS (Interplanetary File System)

IPFS is a protocol and network designed to create a content-addressable, peer-to-peer method of storing and sharing hypermedia in a distributed file system

How IPFS Works

Bob adds a file to IPFS and is assigned a unique fingerprint called a cryptographic hash. Automatic deduplication of data prevents duplicate files.
(example: QmR7GSQM93Cx5e...)

Alice downloads the file using the unique hash.
When looking up files, you're asking the network to find nodes storing the content behind a unique hash

Ken downloads the same file using the hash.

Downloads simultaneously from different nodes as files are segmented into chunks, each with 256kb size

NEM Blockchain Platform

Block
~120 TXNs
Transaction 1..120
Block
~120 TXNs
Transaction 1..120
Block
~120 TXNs
Transaction 1..120
Block
~120 TXNs
Transaction 1..120

 A message of at most 1024 bytes can be attached to each transfer transaction. This makes storing of large files in the blockchain impossible! 

Combining NEM and IPFS

The ProximaX Solution

Block
Transfer Transaction
Message









 
{
   "digest":"04d185c832f6.....",
   "hash":"QmWDQegEhLdCUWF6a...",
   "keywords":"plain,file,text",
   "metaData":"{'name':'some-data'}",
   "name":"filename.txt",
   "timestamp":1521154256963,
   "type":"text/plain"
}
Field Description
digest This is a content based signature of the file for validation
hash The base58 encoded string IPFS reference
keywords A custom field that can be used to tag a specific file. Searchable
metaData JSON structured text for additional meta data. Searchable
name The name of the file (with or without extension)
timestamp The actual time that the file was uploaded to IPFS
type Content type of the file

ProximaX P2P Storage Benefit

High Availability
Files can be replicated across peers to ensure that they are always available to be consumed by any peers on the network.

Fault tolerant
Peers on the network allows operational continuity in the event of single or multiple peer failure

Deduplication
The solution will have a data compression technology that detects duplicate files and avoid copying the same file to the
network.

ProximaX P2P Storage Benefit

Content-addressable storage

A mechanism for storing information that can be retrieved based on its content, not its storage location

 

Scalability

The storage solution will scale based on the number of peers that participates on the network

 

Web Service HTTP API

The P2P storage nodes will have a built in Exposed Web Services endpoints for developers to easily integrate the storage with their decentralised applications.

 

ProximaX Ecosystem

NEM Blockchain Platform Consensus Protocol

Full client ProximaX node - it runs an IPFS locally

Light client ProximaX node - only interfaces through the HTTP REST Endpoint

SDK DEMO
ProximaX Testnet Gateways

Testnet1 Ireland

Testnet2 Singapore

Testnet3 Sydney

NEM

Public IPFS

Testnet Router

Latency Based DNS Routing

Sample Application using ProximaX SDK
Lightweight Client

SDK DEMO 1
Upload Directly to Testnet2 Singapore

Testnet1 Ireland

Testnet2 Singapore

Testnet3 Sydney

NEM

Public IPFS

Sample Application using ProximaX SDK
Lightweight Client

SDK DEMO 2
Download from Testnet3 Sydney

Testnet1 Ireland

Testnet2 Singapore

Testnet3 Sydney

NEM

Public IPFS

Sample Application using ProximaX SDK
Lightweight Client

SDK DEMO 3
Upload using password

Testnet1 Ireland

Testnet2 Singapore

Testnet3 Sydney

NEM

Public IPFS

Sample Application using ProximaX SDK
Lightweight Client

File is symmetrically encrypted using PBKDF2 with HMAC, then SHA256 signed

SDK DEMO 4
Download using password

Testnet1 Ireland

Testnet2 Singapore

Testnet3 Sydney

NEM

Public IPFS

Sample Application using ProximaX SDK
Lightweight Client

File is symmetrically decrypted using PBKDF2 with HMAC, then SHA256 signed

Hands-on SDK DEMO
Upload using Lightweight Client

Testnet1 Ireland

Testnet2 Singapore

Testnet3 Sydney

NEM

Public IPFS

Lightweight Client
Application running locally using the ProximaX SDK

Hands-on SDK DEMO 1
Upload using Lightweight Client

private RemotePeerConnection remotePeerConnection 
    = new RemotePeerConnection("https://testnet.gateway.proximax.io");


/************************************************
 * RemotePeerConnection
 * DEMO 1: Uploads a file, announces a transaction with a message containing IPFS Hash and metadata
 *************************************************/
@RequestMapping(method = RequestMethod.GET, path = "/uploadfile", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> uploadFile() {

    final Upload upload = new Upload(remotePeerConnection);

    try {
        final UploadFileParameter parameter = UploadFileParameter.create()
                .senderOrReceiverPrivateKey(senderPrivateKey)
                .receiverOrSenderPublicKey(receiverPublicKey)
                .file(new File("files_upload/uploadfile.txt"))
                .keywords("plain,file")
                .metadata(Collections.singletonMap("MY_KEY", "MY_VALUE"))
                .build();

        // Upload the file
        final UploadResult uploadResult = upload.uploadFile(parameter);

        JSONObject jsonObject = new JSONObject()
                .appendField("nem_hash", uploadResult.getNemHash());

        return ResponseEntity.ok().body(jsonObject.toJSONString());

    } catch (IOException | UploadException e) {
        return ResponseEntity.badRequest().body(new JSONObject().appendField(
                "error", e.getMessage()).toJSONString());
    }
}

Hands-on SDK DEMO 2
Download using Lightweight Client

private RemotePeerConnection remotePeerConnection 
    = new RemotePeerConnection("https://testnet.gateway.proximax.io");

/************************************************
 * RemotePeerConnection
 * DEMO 2: Downloads the file uploaded from DEMO 1 using the returned nem transaction hash
 *************************************************/
@RequestMapping(method = RequestMethod.GET, path = "/downloadfile/{nemHash}", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> downloadFile(@PathVariable(value = "nemHash") String nemHash) {

    final Download download = new Download(remotePeerConnection);

    try {
        // Download the file
        final DownloadFileResult result = download.downloadFile(DownloadParameter.create()
                .nemHash(nemHash).build());

        Path path = Paths.get("files_download/" + nemHash + ".txt");
        Files.write(path, result.getData());

        return ResponseEntity.ok().build();

    } catch (IOException | DownloadException e) {
        return ResponseEntity.badRequest().body(new JSONObject().appendField(
                "error", e.getMessage()).toJSONString());
    }
}

Hands-on SDK DEMO
NEM Asymmetrical Encryption

Testnet1 Ireland

Testnet2 Singapore

Testnet3 Sydney

NEM

Public IPFS

File is asymmetrically encrypted/decrypted using NEM's Ed25519 private keys, then SHA256 signed.

Lightweight Client
Application running locally using the ProximaX SDK

Hands-on SDK DEMO 3
Upload using
NEM Asymmetrical Encryption

private RemotePeerConnection remotePeerConnection 
    = new RemotePeerConnection("https://testnet.gateway.proximax.io");


/************************************************
 * RemotePeerConnection - Privacy Strategy
 * DEMO 3: Upload an encrypted file - NEM Key asymmetrical encryption
 *************************************************/
@RequestMapping(method = RequestMethod.GET, path = "/uploadsecurefile", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> uploadSecureFile() {

    final Upload upload = new Upload(remotePeerConnection);

    try {
        final UploadFileParameter parameter = UploadFileParameter.create()
                .senderOrReceiverPrivateKey(senderPrivateKey)
                .receiverOrSenderPublicKey(receiverPublicKey)
                .file(new File("files_upload/uploadsecurefile.txt"))
                .keywords("secure,file")
                .metadata(Collections.singletonMap("MY_KEY", "MY_VALUE"))
                .securedWithNemKeysPrivacyStrategy()  // Sets the privacy strategy using NEM Key asymmetrical encryption
                .build();

        // Upload the secure file
        final UploadResult uploadResult = upload.uploadFile(parameter);

        JSONObject jsonObject = new JSONObject()
                .appendField("nem_hash", uploadResult.getNemHash());

        return ResponseEntity.ok().body(jsonObject.toJSONString());

    } catch (IOException | UploadException e) {
        return ResponseEntity.badRequest().body(new JSONObject().appendField(
                "error", e.getMessage()).toJSONString());
    }
}

Hands-on SDK DEMO 4
Download using
NEM Asymmetrical Encryption

private RemotePeerConnection remotePeerConnection 
    = new RemotePeerConnection("https://testnet.gateway.proximax.io");

/************************************************
 * RemotePeerConnection
 * DEMO 4: Downloads the encrypted file uploaded from DEMO 3 using the returned nem transaction hash
 *************************************************/
@RequestMapping(method = RequestMethod.GET, path = "/downloadsecurefile/{nemHash}", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> downloadSecureFile(@PathVariable(value = "nemHash") String nemHash) {

    final Download download = new Download(remotePeerConnection);

    try {
        // Download the file
        final DownloadFileResult result = download.downloadFile(
                DownloadParameter.create()
                        .nemHash(nemHash)
                        .securedWithNemKeysPrivacyStrategy(senderPrivateKey, receiverPublicKey)
                        .build());

        Path path = Paths.get("files_download/" + nemHash + ".txt");
        Files.write(path, result.getData());

        return ResponseEntity.ok().build();

    } catch (IOException | DownloadException e) {
        return ResponseEntity.badRequest().body(new JSONObject().appendField(
                "error", e.getMessage()).toJSONString());
    }
}

Hands-on SDK DEMO 5
Search by Keyword

private RemotePeerConnection remotePeerConnection 
    = new RemotePeerConnection("https://testnet.gateway.proximax.io");

/************************************************
 * RemotePeerConnection
 * DEMO 5: Search by Keyword
 *************************************************/
@RequestMapping(method = RequestMethod.GET, path = "/searchbykeyword/{keyword}",
     produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> searchByKeyword(@PathVariable(value = "keyword") String keyword) {

    Search search = new Search(remotePeerConnection);

    try {
        // Search the transactions associated to the NEM account by keyword
        List<ResourceHashMessageJsonEntity> result = search.searchByKeyword(
            senderPrivateKey, senderPublicKey, keyword);

        JSONObject jsonObject = new JSONObject();
        int index = 0;
        for (ResourceHashMessageJsonEntity r : result) {

            JSONObject jsonTemp = new JSONObject()
                    .appendField("ipfs_hash", r.getHash())
                    .appendField("keywords", r.getKeywords())
                    .appendField("metadata", r.getMetaData())
                    .appendField("digest", r.getDigest())
                    .appendField("name", r.getName());

            jsonObject.appendField(Integer.valueOf(index++).toString(), jsonTemp);
        }

        return ResponseEntity.ok().body(jsonObject.toJSONString());

    } catch (ApiException | InterruptedException | ExecutionException | PeerConnectionNotFoundException e) {
        return ResponseEntity.badRequest().body(
                new JSONObject().appendField("error", e.getMessage()).toJSONString());
    }
}

Hands-on SDK DEMO
Full ProximaX Node (with IPFS)
Local Peer Connection

NEM

Public IPFS

Full ProximaX Node (with IPFS)
Application running locally using the ProximaX SDK with ProximaX Platform IPFS locally installed

Hands-on SDK DEMO 6
Upload using Local Peer Connection

private LocalHttpPeerConnection localHttpPeerConnection = new LocalHttpPeerConnection(
        ConnectionFactory.createNemNodeConnection("http", "104.128.226.60", 7890), // NEM NODE
        ConnectionFactory.createIPFSNodeConnection("/ip4/127.0.0.1/tcp/5001") // IPFS NODE
);

/************************************************
 * LocalHttpPeerConnection
 * DEMO 6: Uploads a file, announces a transaction with a message containing IPFS Hash and metadata
 *************************************************/
@RequestMapping(method = RequestMethod.GET, path = "/uploadfilelocal", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> uploadFileLocalConnection() {

    // Set LocalHttpPeerConnection
    final Upload upload = new Upload(localHttpPeerConnection);

    try {
        final UploadFileParameter parameter = UploadFileParameter.create()
                .senderOrReceiverPrivateKey(senderPrivateKey)
                .receiverOrSenderPublicKey(receiverPublicKey)
                .file(new File("files_upload/uploadfilelocalconnection.txt"))
                .keywords("plain,file,local")
                .metadata(Collections.singletonMap("MY_KEY", "MY_VALUE"))
                .build();

        // Upload the file
        final UploadResult uploadResult = upload.uploadFile(parameter);

        JSONObject jsonObject = new JSONObject()
                .appendField("nem_hash", uploadResult.getNemHash());

        return ResponseEntity.ok().body(jsonObject.toJSONString());

    } catch (IOException | UploadException e) {
        return ResponseEntity.badRequest().body(new JSONObject().appendField(
                "error", e.getMessage()).toJSONString());
    }
}

Hands-on SDK DEMO 7
Download using Local Peer Connection

private LocalHttpPeerConnection localHttpPeerConnection = new LocalHttpPeerConnection(
        ConnectionFactory.createNemNodeConnection("http", "104.128.226.60", 7890), // NEM NODE
        ConnectionFactory.createIPFSNodeConnection("/ip4/127.0.0.1/tcp/5001") // IPFS NODE
);

/************************************************
 * LocalHttpPeerConnection
 * DEMO 7: Downloads the file uploaded from DEMO 6 using the returned nem transaction hash
 *************************************************/
@RequestMapping(method = RequestMethod.GET, path = "/downloadfilelocal/{nemHash}", produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> downloadFileLocalConnection(@PathVariable(value = "nemHash") String nemHash) {

    // Set LocalHttpPeerConnection
    final Download download = new Download(localHttpPeerConnection);

    try {
        // Download the file
        final DownloadFileResult result = download.downloadFile(DownloadParameter.create()
                .nemHash(nemHash).build());

        Path path = Paths.get("files_download/" + nemHash + "-local.txt");
        Files.write(path, result.getData());

        return ResponseEntity.ok().build();

    } catch (IOException | DownloadException e) {
        return ResponseEntity.badRequest().body(new JSONObject().appendField(
                "error", e.getMessage()).toJSONString());
    }
}

Encryption and Privacy Strategies

Plain Content Strategy

Content is stored without any encryption and will be readable by any user downloading it

Secure with NEM Keys

Uses the user's NEM private and public keys to encrypt the file and can only be decrypted by the same set of keys. The file will be downlodable but needs to be decrypted with the same set of keys to get the decrypted file. The most secure way of encrypting the file

Secure with Password

Uses a 50 character passphrase to protect the file. It it set to 50 characters to add difficulty for any brute force attacks

Custom

Developers can use a custom encryption/decryption process

ProximaX P2P and NEM Network Setup

 

  1. Public P2P Storage with Private Mijin blockchain Network
     
  2. Private P2P Storage with Private Mijin blockchain Network
     
  3. Private P2P Storage with Public NEM Network
     
  4. Public P2P Storage with Public NEM Network

Work in Progress

 

  • Explore the SDK for other features
     
  • ProximaX Applications
     
  • Support for other programming languages
    (C#, Typescript, Golang)
     
  • PeerStream Media streaming and messaging
     
  • Consensus Protocol in Catapult
     
  • Developer Site - https://developer.proximax.io

 

ProximaX SDK Walkthrough - (with script)

By Carlo Cayos

ProximaX SDK Walkthrough - (with script)

ProximaX SDK demo deck with script

  • 501