Database GUI

Filter, visualise and compare big data sets

By Jonathan Shimwell

This work was funded by the RCUK Energy Programme
[Grant number EP/P012450/1]

Creating database entries from

combined sources

File1.txt

Supplementary meta data

• from MS Excel files

• from the filename

• arbitrarily added

col1, col2

1,          2.14,

2,          3.56,

3,          4.56

4,          5.43

5,          4.45

File2.txt

col1, col2

1,          2.14,

2,          3.56,

3,          4.56

4,          5.43

5,          4.45

Column format text files

File2.xls

id:             123b

name:       Jon

material:  Steel

date:        1/4/18

time:        12:00

method:   tensile

Flexible and extendable format

File3.txt

col1, col2

1,          2.14,

2,          3.56,

3,          4.56

4,          5.43

5,          4.45

• JSON objects available in:
• binary for speedy access
• ASCII for human readability

Uploading to a local, remote or

distributed database

Cloud servers around the world

Local computer

1. initial creation
2. stress testing
3. performance analysis
4. pre-deployment validation

• Improved accessibility
• Multiple locations to reduce latency
• Access to scalable compute resources

RESTfull APIs

Dormant code waiting for input

--> URL based input -->

<-- JSON base output <--

www.localhost.com/?material=Eurofer&uploader=Jon

Indicates beginning

of arguments

argument name

argument value

indicates start

of new argument

indicates separation between argument name and value

=

&

?

Meta data filters are made dynamically according to the fields present in the database

Interactive graph with hover text, box zoom, pan and much more

Table displaying results of query updated with changes to the filter dropboxs

GUI created on the fly using stateless components

Select the fields for the X and Y axis on the plot

All dropboxes show available contents for that field and can accept text input that filters contents

Graph requires more input before it is available

Required input is highlighted in red to guide user

Plotted items are added to a second table to allow removal after the items are no longer visible in the query results table

Technology used

 A web framework for creating the stateless GUI.

Allows app like performance by efficient update of the render

React.js

MongoDB

NoSQL database offering excellent scaling through horizontal

partition of data (sharding) and web friendly JSON-like format

Serverless functions offer URL endpoints and scalable infrastructure for RESTfull APIs

AWS Lambda

Automatic and dynamic workflow

React.js

MongoDB

RESTfull APSs

File1.txt

col1, col2

1,          2.14,

2,          3.56,

3,          4.56

4,          5.43

5,          4.45

File2.xls

id:             123b

name:       Jon

material:  Steel

date:        1/4/18

time:        12:00

method:   tensile

custom python scripts for each URL endpoint

Interactive GUI

Scalable database

Raw data files + meta data

Version control

Github