An introduction to data

reality

data

information

knowledge

wisdom

raw data

observation

meaning

understanding

decision

data

information

knowledge

insight

wisdom

conspiracy

researched phenomena
published data

not researched phenomena
not published data

2.5 exabytes/day

2500000000000000000 bytes

1.000.000. terabytes

90% of digital data was produced in the last 2 years

530.000.000 mp3 files

150.000.000 32GB Iphone

90 years HD video

5% of useful data

00100011100011011110001101100001100111100101001101001111001101010100001100111110010101110101010010111010101011110100001011010101110101010110111011101110101011011010101110110101011010110100000101010101011111101010101011011111010001010101101101101010101010101010101010011011011010101010111010101011100101010101111101001000101000101001001010101101010111010101000010101111010101011001011011010101010110111010101000101000001010101110101101011010101000010101010110111010110101010000010101010100100101010110111110101010100000101010101011010100000101010111110101001010101010101010100101011010101010101011010110111101001010101010101000001010100000000101010100110111010101101010000

user friendly data

xls
xlsx
csv
tsv
sps
json
xml
kml
geojson
shp
geotiff
pdf

jpeg

Sources of processed data

META GLOBAL SPECIAL MARKET RESEARCH NATIONAL
Google Data UN Global Terrorism DB Kantar Public Procurements
Ourworldindata Worldbank Freedom House Gemius Central Statistical Bureau
Nasdaq Data Eurostat RSF Ipsos MET
Eur Data Portal UNESCO TI TNS Tax Bureau
IMF Company Reg Land ownership
NASA OCCRP ALEPH National funds
OECD Átlátszó
FB, Twitter

crawling

manual collection

The Government yesterday announced that a new swimming pool will be built for the Budapest Water World Cup. The spot will be in the High Street, the estimated total budget is 120m EUR. The construction works were awarded to the Trust Us Ltd.

WHO?

WHEN?

WHAT?

WHY?

WHERE?

WITH WHOM?

HOW MUCH/MANY?

The Government yesterday announced that a new swimming pool will be built for the Budapest Water World Cup. The spot will be in the High Street, the estimated total budget is 120m EUR. The construction works were awarded to the Trust Us Ltd.

WHO? THE GOVERNMENT

WHEN? YESTERDAY

WHAT? SWIMMING POOL

WHY/WHAT PURPOSE? BUDAPEST WATER WORLD CUP

WHERE? HIGH STREET

WITH WHOM? TRUST US LTD.

HOW MUCH/MANY? 120M EUR

WHO? THE GOVERNMENT

WHEN? YESTERDAY, TWO WEEKS AGO

WHAT? SWIMMING POOL, COURT, STADIUM

WHY/WHAT PURPOSE? BUDAPEST WATER WORLD CUP

WHERE? HIGH STREET, GRAND BOULEVARD, PÉCS

WITH WHOM? TRUST US LTD., TRUST us sro., Trustus,

HOW MUCH/MANY? 120M EUR, 710M EUR, 687M USD

date year, H, Q, month, days, hours to seconds
geolocation continent, country, region, county, city, lines, polygons, coordinates
string description - names, qualities, categories
measure whole, fraction, decimal
boolean true/false, yes/no
date string quantity geolocation logic
2021-09-28 Chris 13 000 000 Barcelona true
2019. 09. 28. Suzy 13.000.000 HUN11 0
28. sept 2021. football 13,33 47.89, 29.67 1

SOME RULES

one column, one data category
​consistency - always the same way
more data, more accuracy
keep the data clean