Wikipedia and

Wikidata access with Python

Toni Hermoso Pulido

@toniher

Amical Wikimedia

Wiki

Wikipedia

Wikimedia

MediaWiki

Wiki CMS

Built mostly in:

  • PHP
  • JavaScript
https://www.mediawiki.org

Working with Docker

https://docker.com/

Getting started with Docker

Docker images used

https://github.com/toniher/docker-SemanticMediaWiki

https://github.com/toniher/docker-debian-python-mediawiki

We build the one above following README instructions, modifying Bash scripts and Dockerfile if desired

We can build this one as well. Alternately we can reuse it from:

https://hub.docker.com/r/toniher/debian-python-mediawiki/

However you need to mount example scripts provided in the repo above

Bots

The hidden ones

https://www.mediawiki.org/wiki/Manual:Bots

MediaWiki API

Accessing MediaWiki programatically

https://ca.wikipedia.org/w/api.php
https://www.mediawiki.org/wiki/API:Main_page

MediaWiki API clients / libraries

  • Python (Mwclient, Pywikibot)

  • Perl (MediaWiki::API, MediaWiki::Bot)

  • JavaScript (nodemw)

  • etc.

https://www.mediawiki.org/wiki/API:Client_code

Mwclient

A MediaWiki API Python client library

http://mwclient.readthedocs.io/en/master/

RESTBase

A sophisticated Wikipedia REST API

https://ca.wikipedia.org/api/rest_v1/

Wikidata

Wikidata: Namespaces

  • «Wiki ones»: User, Wikidata, Category, etc.
  • Wikibase ones: Item, Property (both entities)

Special:AllPages

Wikidata: Concepts

  • Labels
  • Descriptions
  • Aliases
  • Sitelinks
  • Statements

Wikidata Glossary

Wikidata: Concepts

Reelin

Wikidata: Statements

https://www.wikidata.org/wiki/Q3454165

RDF

(Resource Description Framework)

Wikidata API access

 

MediaWiki API access: Like any other MediaWiki.

https://wikidata.org/w/api.php

 

Access via JSON (or other) exports

SPARQL endpoint access

Wikidata API clients

Linked (Open) Data

http://lod-cloud.net/

SPARQL: Endpoints and interfaces

Wikidata SPARQL inteface

query.wikidata.org

Interface to multiple endpoints

Yasgui

Example query:

SPARQL: API Access

Example for Wikidata: https://query.wikidata.org/sparql

SPARQL client in Python

SPARQLWrapper

Wikimedia Hackathon 2018

https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2018

#wmhack

18-20 May