Water damage: 76%; Root damage: 24%

Using Computer Vision to Explore the Human Remains Trade on Social Media

Shawn Graham, Carleton U

scholar.social/@electricarchaeo

follow along at https://tinyurl.com/sg-dec6

images of human remains will be shown

The 19th/20th c 'look' of the

ethnographic museum

is alive and well

on social media

Distant Reading

Statistical patterns in word use over tens of thousands of posts reveal patterns of discourse that characterize how bone trade enthusiasts envision the dead and their 'hobby'

CNN & Vision Models

Text

Distant Viewing

Pretrained models were not trained on human remains; classifying your own images always finds what you thought was already there.

not a real skull, but a rubber toy

Visual similarity

Scene Tagging

Used MS Azure to assign tags (based on Azure's models of tagged imagery generated from captchas etc).
Nodes are tags; edges are photographs. Thus tagA -- tagB represents a photo described with both tags.
Network analysis then lets us explore clusters of tag use within the images of a single user, to characterize the sensorial affect of their feed

Patterns of Differences

Back to the Beginning

Computer vision tech is changing rapidly.
Generative AI might have a role to play in our work
But for now, 'good ol' image classification might be the most productive use

The Canadian Federal Election of 2019

•In 2019, Conservative candidate Claire Ratteé (Skeena-Bulkley Valley Riding, N. British Columbia) bought this skull for her boyfriend

•The boyfriend showed it off on Facebook in February 2019, attracting local media attention, critique (including by us), and law enforcement investigation.

•Ratteé claimed the skull was ‘European’ and from ‘the 1700s’. Local First Nations representatives raised concern over the possibility the remains were Indigenous and recently exhumed.

A taphonomic classifier

What taphonomy is seen in the area circled in red?

•Animal damage (score=0.88087).

• Water damage (score=0.10119).

•Spalling damage (score=0.01356).

•Insect damage (score=0.00413).

•Weathering damage (score=0.00013).

ArchaeoCLIP

A text and image model where both the captions and the images are captured
Retrained on 77k images from Open Context
Other images/text can be embedded in the model
Thus a free-text natural language search model for collections of images
Try it here
Data Browser for LAION

Working with Instagram

You'll always be running to stand still
But this notebook works as of last week: data scraper
Then you'd use this notebook to pull your data into a dataframe
You can also download both notebooks from this github gist for exploration later or on your own machine
If you work through both notebooks, you'll also have imagery that you can feed into ArchaeoCLIP