Technical Introduction

to Wikidata

Toni Hermoso Pulido

@toniher

Amical Wikimedia

Docker

https://github.com/toniher/docker-debian-python-mediawiki

Wikimedia

Wikidata

Wikidata: Namespaces

  • «Wiki ones»: User, Wikidata, Category, etc.
  • Wikibase ones: Item, Property (both entities)

Special:AllPages

Wikidata: Concepts

Reelin

Wikidata: Concepts

  • Labels
  • Descriptions
  • Aliases
  • Sitelinks
  • Statements

Wikidata Glossary

Wikidata: Statements

https://www.wikidata.org/wiki/Q3454165

Wikidata: Data Types

Data Model

Data Types

Triple

Paul Schuster was born in Dresden

RDF

(Resource Description Framework)

RDF Serialisation formats

  • RDF/XML
  • Turtle
  • JSON-LD
  • etc.

Wikidata: Exports

Wikidata:

Popular properties

List of properties

Top 100 properties

Wikidata and Wikipedia(s)

Used in

  • Interwiki
  • Infoboxes
  • etc.

How?

Wikidata and Wikipedia(s)

Linkage at the API level

Example: Les Misérables

query, pageprops, -> wikibase_item (+ info)

Extra information:

wbentityusage

wblistentityusage

Wikidata API access

MediaWiki API access: Like any other MediaWiki.

https://wikidata.org/w/api.php

Access via JSON (or other) exports

SPARQL endpoint access

Wikidata API clients

Semantic Web

The Semantic Web Stack

Tim Berners-Lee (2001) - Machine-friendly Web (*, *)

Linked (Open) Data

http://lod-cloud.net/

SPARQL

SPARQL Protocol and RDF Query Language

Similar to SQL, but for the Semantic Web

SPARQL: Endpoints and interfaces

Interface to multiple endpoints

Yasgui

Wikidata SPARQL inteface

query.wikidata.org

SPARQL: Prefixes

PREFIX wd: <http://www.wikidata.org/entity/>
PREFIX wds: <http://www.wikidata.org/entity/statement/>
PREFIX wdv: <http://www.wikidata.org/value/>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX p: <http://www.wikidata.org/prop/>
PREFIX ps: <http://www.wikidata.org/prop/statement/>
PREFIX pq: <http://www.wikidata.org/prop/qualifier/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

Some used by Wikidata

SPARQL: Prefixes

Other external ones

SPARQL federation in Wikidata

Example: Universities by Page Rank

SPARQL: Queries

Wikidata Query examples

SPARQL portal in Wikidata

Docs:

Querying Wikidata

Web of data (advanced SPARQL - not Wikidata)

 

SPARQL: Queries

Additional examples:

SPARQL: API Access

Example for Wikidata: https://query.wikidata.org/sparql

SPARQL client in Python

SPARQLWrapper

Wikidata Dumps

  • JSON
  • XML
  • RDF
  • etc.

https://dumps.wikimedia.org/wikidatawiki

Entities

https://dumps.wikimedia.org/wikidatawiki/entities/

Wikidata dumps: Usage

Using Unix tools, regexp, and then processing JSON (one per line)

Convenient tool: wikidata-filter

Toy dataset: public universities in Wikidata

Convenient NodeJS image: https://github.com/toniher/docker-nodejs-wikidata

Wikidata dumps: Usage

Example:

 

Count all public universities in Spain

zcat unipubli.json.gz | wikidata-filter  --claim P17:Q29 | wc -l

Wikidata: Creating bots

Documentation

about creation or permissions

Testing