Tim Sherratt PRO
Historian and hacker. All the slide decks available here are licensed under a Creative Commons Attribution 4.0 International License. Fee free to reuse and share!
Jupyter notebooks for web archives
Play along
Asking questions with web archives – introductory notebooks for historians
Project
- a starting point for researchers
- use existing APIs (Memento & CDX)
- no special tools
- complimenting projects like Archives Unleashed
AIMS
Jupyter
- combines text and live code
- use in your browser
- run in the cloud (no software to install)
- both tool and tutorial
Jupyter
text
CODE
RUNNING CODE
- click on a cell
- hit Shift+Enter
EDITING CODE
- click on a cell
- edit the contents
- hit Shift+Enter to run
GLAM Workbench
GLAM Workbench
GLAM Workbench
GLAM Workbench
Works with these archives
Repositories uSED
- Australian Web Archive
- New Zealand Web Archive
- UK Web Archive
- Wayback Machine (Internet Archive)
But notebooks can be adapted to work with other Pywb, Open Wayback, and Memento compliant systems.
GLAM Workbench
View & download
RUN LIVE!
BINDER
- builds a customised computing environment
- opens notebook ready-to-run
BINDER LIMITS
- inactive notebooks are closed
- notebooks and data are not saved!
- use download links in notebooks
MAIN THEMES
- Types of data
- Harvesting data & creating datasets
- Change over time
TYPES OF DATA
Timemaps & Mementos
Harvesting data
- Get the archived version of a page closest to a particular date
- Find all the archived versions of a web page
- Harvesting collections of text from archived web pages
- Harvesting data about a domain using the IA CDX API
- Find and explore Powerpoint presentations from a specific domain
- Exploring subdomains in the whole of gov.au
Harvesting data
Harvesting data
Harvesting data
Harvesting data
CHANGE over time
- Compare two versions of an archived web page
- Observing change in a web page over time
- Create and compare full page screenshots from archived web pages
- Using screenshots to visualise change in a page over time
- Display changes in the text of an archived web page over time
- Find when a piece of text appears in an archived web page
CHANGE over time
APPMODE
- hides all code cells
- runs all code cells automatically
- turns a notebook into an app
CHANGE over time
CHANGE over time
CHANGE over time
Suggestions? Problems?
Suggestions? Problems?
Jupyter notebooks for web archives
By Tim Sherratt
