Play along

https://slides.com/wragge/iipc-jupyter

Asking questions with web archives – introductory notebooks for historians

• IIPC Discretionary Funding Program 2019–2020
• Project description
• Blog post
• Zenodo repository

Project

• a starting point for researchers
• use existing APIs (Memento & CDX)
• no special tools
• complimenting projects like Archives Unleashed

AIMS

Jupyter

• combines text and live code
• use in your browser
• run in the cloud (no software to install)
• both tool and tutorial

Jupyter

text

CODE

RUNNING CODE

• click on a cell
• hit Shift+Enter

EDITING CODE

• click on a cell
• edit the contents
• hit Shift+Enter to run

GLAM Workbench

https://glam-workbench.github.io/

GLAM Workbench

https://glam-workbench.github.io/

GLAM Workbench

https://glam-workbench.github.io/web-archives/

GLAM Workbench

Works with these archives

Repositories uSED

• Australian Web Archive
• New Zealand Web Archive
• UK Web Archive
• Wayback Machine (Internet Archive)

But notebooks can be adapted to work with other Pywb, Open Wayback, and Memento compliant systems.

GLAM Workbench

View & download

RUN LIVE!

BINDER

• builds a customised computing environment
• opens notebook ready-to-run

BINDER LIMITS

• inactive notebooks are closed
• notebooks and data are not saved!
• use download links in notebooks

MAIN THEMES

• Types of data
• Harvesting data & creating datasets
• Change over time

TYPES OF DATA

• Timegates, Timemaps, and Mementos
• Exploring the Internet Archive's CDX API
• Comparing CDX APIs
• Timemaps vs CDX APIs

Timemaps & Mementos

https://glam-workbench.github.io/web-archives/#timegates-timemaps-and-mementos

Harvesting data

• Get the archived version of a page closest to a particular date
• Find all the archived versions of a web page
• Harvesting collections of text from archived web pages
• Harvesting data about a domain using the IA CDX API
• Find and explore Powerpoint presentations from a specific domain
• Exploring subdomains in the whole of gov.au

Harvesting data

https://glam-workbench.github.io/web-archives/#harvesting-collections-of-text-from-archived-web-pages

Harvesting data

https://glam-workbench.github.io/web-archives/#find-and-explore-powerpoint-presentations-from-a-specific-domain

Harvesting data

https://glam-workbench.github.io/web-archives/#exploring-subdomains-in-the-whole-of-govau

Harvesting data

CHANGE over time

• Compare two versions of an archived web page
• Observing change in a web page over time
• Create and compare full page screenshots from archived web pages
• Using screenshots to visualise change in a page over time
• Display changes in the text of an archived web page over time
• Find when a piece of text appears in an archived web page

CHANGE over time

APPMODE

• hides all code cells
• runs all code cells automatically
• turns a notebook into an app

CHANGE over time

https://glam-workbench.github.io/web-archives/#display-changes-in-the-text-of-an-archived-web-page-over-time

CHANGE over time

https://glam-workbench.github.io/web-archives/#create-and-compare-full-page-screenshots-from-archived-web-pages

https://glam-workbench.github.io/web-archives/#using-screenshots-to-visualise-change-in-a-page-over-time

CHANGE over time

https://glam-workbench.github.io/web-archives/#find-when-a-piece-of-text-appears-in-an-archived-web-page

Suggestions? Problems?

https://glam-workbench.github.io/web-archives/

Suggestions? Problems?

https://github.com/GLAM-Workbench/web-archives

Tim Sherratt • @wragge • #glamworkbench