An introduction to data
reality
data
information
knowledge
wisdom
raw data
observation
meaning
understanding
decision
data
information
knowledge
insight
wisdom
conspiracy
researched phenomena
published data
not researched phenomena
not published data
2.5 exabytes/day
2500000000000000000 bytes
1.000.000. terabytes
90% of digital data was produced in the last 2 years
530.000.000 mp3 files
150.000.000 32GB Iphone
90 years HD video
5% of useful data
00100011100011011110001101100001100111100101001101001111001101010100001100111110010101110101010010111010101011110100001011010101110101010110111011101110101011011010101110110101011010110100000101010101011111101010101011011111010001010101101101101010101010101010101010011011011010101010111010101011100101010101111101001000101000101001001010101101010111010101000010101111010101011001011011010101010110111010101000101000001010101110101101011010101000010101010110111010110101010000010101010100100101010110111110101010100000101010101011010100000101010111110101001010101010101010100101011010101010101011010110111101001010101010101000001010100000000101010100110111010101101010000
user friendly data
xls
xlsx
csv
tsv
sps
json
xml
kml
geojson
shp
geotiff
pdf
jpeg
Sources of processed data
| META | GLOBAL | SPECIAL | MARKET RESEARCH | NATIONAL |
|---|---|---|---|---|
| Google Data | UN | Global Terrorism DB | Kantar | Public Procurements |
| Ourworldindata | Worldbank | Freedom House | Gemius | Central Statistical Bureau |
| Nasdaq Data | Eurostat | RSF | Ipsos | MET |
| Eur Data Portal | UNESCO | TI | TNS | Tax Bureau |
| IMF | Company Reg | Land ownership | ||
| NASA | OCCRP ALEPH | National funds | ||
| OECD | Átlátszó | |||
| FB, Twitter |
crawling
manual collection
The Government yesterday announced that a new swimming pool will be built for the Budapest Water World Cup. The spot will be in the High Street, the estimated total budget is 120m EUR. The construction works were awarded to the Trust Us Ltd.
WHO?
WHEN?
WHAT?
WHY?
WHERE?
WITH WHOM?
HOW MUCH/MANY?
The Government yesterday announced that a new swimming pool will be built for the Budapest Water World Cup. The spot will be in the High Street, the estimated total budget is 120m EUR. The construction works were awarded to the Trust Us Ltd.
WHO? THE GOVERNMENT
WHEN? YESTERDAY
WHAT? SWIMMING POOL
WHY/WHAT PURPOSE? BUDAPEST WATER WORLD CUP
WHERE? HIGH STREET
WITH WHOM? TRUST US LTD.
HOW MUCH/MANY? 120M EUR
WHO? THE GOVERNMENT
WHEN? YESTERDAY, TWO WEEKS AGO
WHAT? SWIMMING POOL, COURT, STADIUM
WHY/WHAT PURPOSE? BUDAPEST WATER WORLD CUP
WHERE? HIGH STREET, GRAND BOULEVARD, PÉCS
WITH WHOM? TRUST US LTD., TRUST us sro., Trustus,
HOW MUCH/MANY? 120M EUR, 710M EUR, 687M USD
| date | year, H, Q, month, days, hours to seconds |
| geolocation | continent, country, region, county, city, lines, polygons, coordinates |
| string | description - names, qualities, categories |
| measure | whole, fraction, decimal |
| boolean | true/false, yes/no |
| date | string | quantity | geolocation | logic |
|---|---|---|---|---|
| 2021-09-28 | Chris | 13 000 000 | Barcelona | true |
| 2019. 09. 28. | Suzy | 13.000.000 | HUN11 | 0 |
| 28. sept 2021. | football | 13,33 | 47.89, 29.67 | 1 |
SOME RULES
one column, one data category
consistency - always the same way
more data, more accuracy
keep the data clean
Short introduction to data
By Attila Bátorfy
