Latent Spaces of Heritage Crime
Shawn Graham, Carleton U
shawn.graham@carleton.ca | scholar.social/@electricarchaeo
Using Knowledge Graph Embedding Models to Explore the Gaps in What We Think We Know
follow along!
https://slides.com/shawngraham/latent-spaces-of-heritage-crime/
How can we discover and visualize the things we don't know?
... we can do it with a knowledge graph.
- a knowledge graph can have many different kinds of relationships and many different kinds of entities
- linked open data is a special kind of knowledge graph where entities and predicates are tied to authorities
- not mission critical, but if you wish to link your graph across repositories, then absolutely necessary.
- knowledge graphs can be queried by traversal (neo4j)
- knowledge graphs manage data by adding more links
...but wait, there's more!
Knowledge graphs
triples = [
['Red Riding Hood', 'lives_in', 'Village'],
['Red Riding Hood', 'is_child_of', 'Mother'],
['Mother', 'gives_basket_to', 'Red Riding Hood'],
['Red Riding Hood', 'visits', 'Grandma'],
['Grandma', 'lives_in', 'Forest House'],
['Wolf', 'lives_in', 'Forest'],
['Wolf', 'meets', 'Red Riding Hood'],
['Wolf', 'eats', 'Grandma'],
['Wolf', 'disguises_as', 'Grandma'],
['Wolf', 'waits_in', 'Forest House'],
['Woodcutter', 'hunts', 'Wolf'],
['Woodcutter', 'saves', 'Red Riding Hood'],
['Grandma', 'is_at', 'Forest House'],
['Wolf', 'is_at', 'Forest House']
]
good ol' wikipedia, eh? https://en.wikipedia.org/wiki/Knowledge_graph_embedding
Models I used to create the embeddings:
DistMult - uses diagonals; can't represent asymetric relations well
ComplEx - like DistMult but with complex numbers, can handle asymetric relations
RotatE - models relationships as rotations in complex space. Can handle inversion/composition tool.
<- across 3 dimensions, contrasting between actors and places
<- across 3 dimensions, organizing across domestic/wild divide, and who acts upon whom
<- across 3 dimensions, notice the close overlap of wolf/grandma in dimension 0 and dimension 2
Graham, Yates et al 2023
Antiquities Trade Example
Here, we were merely interested in clustering, and we didn't appreciate the difference that model geometry made
2. Cleaning and priors: Watch out for polysemy! What would you expect to see in the result?
1. Extract a knowledge graph. Beware ontology!
How many predicates is too much? Not enough?
3. Project into appropriate models (ie, different shapes attend to different ideas) How many dimensions?
4. Validate the models. Check against prior beliefs!
5. Measure for the latent gaps that are meaningful. Predict missing links (statements of knowledge)
...each step is theory-laden and dangerous.
follow along...
Humans
- Thoughtful process
- Hard to coordinate
- Dictionary of predicates, or resolve afterwards?
- Inconsistent
- Slow
- Return fewer triplets of higher quality
Machines
- LLM as word calculator
- Relentless
- Careful schemas, rules mostly followed but often flummoxed by the passive voice
- Returns more triplets of lower quality
A pattern for building temporally-grounded knowledge graphs from scholarly and documentary sources, optimised for downstream knowledge graph embedding (KGE) training.
---
## The core problem
Most knowledge graph construction from text produces triples: `(subject, predicate, object)`. Applied to scholarly or historical sources, this collapses a critical distinction. A monograph arguing that the Treaty of Westphalia established state sovereignty is doing something categorically different from the same monograph reporting that previous historians have made this argument. The first is a knowledge claim being advanced; the second is a claim being attributed. Embedding models trained on undifferentiated triples will represent both as equivalent facts which could create a fundamental error of provenance.
A second problem: triples are atemporal. `(Napoleon, commanded, Grand Armée)` is true for a bounded interval. Without a temporal anchor, the triple cannot be placed on a timeline, cannot be compared with contradicting triples from different periods, and cannot serve as useful training data for temporally-aware embedding methods such as TComplEx, TNTComplEx, or DE-SimplE. The date is not metadata about the extraction but is instead a first-class component of the knowledge claim itself.
This pattern addresses both problems. Every extracted unit of knowledge is a **quad**, rather than a triple, and every quad carries a **provenance class** distinguishing what the source is *claiming* from what it is *reporting others have claimed*.[...]
The idea is to describe the problem, and describe the tests that would indicate whether or not the problem was being solved, and then instructions to invoke the creation of the relevant code.
This is a hermeneutic approach.
The pattern that results can be fed to your language model (word calculator) of choice. Or graduate students.
Spec Driven Development
Using local model qwen3.5 9b hosted at Carleton University (example is building a knowledge graph from a synoptic archaeological project report).
interface: GUI, TUI, CLI (for batches)
functionally, this is a language manipulation task of translation. We did an experiment back in 2023 with GPT3 comparing a machine-annotated graph versus the human-annotated graph of the antiquities trade and found very similar results.
Choose your model!
Different model geometries, i.e, how they mathematically represent 'knowledge' imply different kinds of attention/value/noticing
so.... generate a variety of models
"...so you've got a knowledge graph. That don't impress me much." - not Shania Twain
import tensorflow as tf
import tf_keras
import ampligraph
import numpy as np
# x_gist is the knowledge graph in subject, predicate, object order, loaded in from a github gist page
# Initialize models
models3 = {
'DistMult': ScoringBasedEmbeddingModel(k=3, eta=5, scoring_type='DistMult'),
'ComplEx': ScoringBasedEmbeddingModel(k=3, eta=5, scoring_type='ComplEx'),
'RotatE': ScoringBasedEmbeddingModel(k=3, eta=5, scoring_type='RotatE'),
'TransE': ScoringBasedEmbeddingModel(k=3, eta=5, scoring_type='TransE')
}
# Train each model
for name, model in models3.items():
model.compile(optimizer='adam', loss='multiclass_nll')
model.fit(X_gist, epochs=200)
-
TransE (Translation): This model is best for simple, hierarchical relationships where you can think of the link between two things as a distance (e.g., "Cerveteri" + "is located in" = "Italy", "looter_a" + "worked_for" = "middleman_1"). It struggles with "one-to-many" relationships eg, "Giacomo_Medici" + "consigned_materials_to" = "auction_house_a" and "auction_house_b" and "auction_house_c"
-
DistMult (Multiplication): This model is best for symmetric relationships where the direction of the arrow doesn't matter (e.g., "Person A is a friend of Person B"). Uses simple multiplication so the math is the same regardless of which entity comes first. Very fast but unable to tell the difference where the direction of the arrow does matter, ie "A is the client of B" does not mean "B is the client of A."
-
ComplEx (Complex Embeddings): This model is best for asymmetric or directed relationships where the order is crucial (e.g., "Smith", "was_patron_of", "MET"; the museum does not patronize the person!). By using complex numbers (which have both a "real" and "imaginary" part), it can mathematically distinguish between a relationship and its reverse, allowing it to model both symmetric and non-symmetric patterns effectively.
-
RotatE (Rotation): This model is best for relational patterns like inversion and composition. An inversion means it 'understands' that if "Aidonia", "source_of", "Mycenaean_Gold", then it can predict "Mycenaean_Gold", "looted_from", "Aidonia". A composition means the model can follow a chain of logic.
k ~ number of dimensions, which should be roughly same order of magnitude as number of predicates.
Attend to the Gaps
Traditionally, with KGE, you either cluster the results or ask the model to complete triplets of interest.
This is how we were tipped to look into the relationship between Leonardo Patterson and the Brooklyn Museum.
But it may be that the gaps show us things our scholarship has not attended to yet.
https://github.com/shawngraham/kge-explorer/
There is no one 'correct' way to explore an embedding space.
so if time allows, here are some other experiments.
In the knowledge graph embeddings, the two 'best' scoring models are TransE and RotatE implying that the underlying data is best represented as chains of simple hierarchical relationships, and also chains of inference and composition
- which could be saying something about the nature of the underlying articles
We then use principle components analysis to reduce to 3 dimensions. We search that 3 dimensional space for gaps where otherwise the distribution of points would suggest there should be something.
The gaps therefore represent ideas/entities that should be present but aren't.
RotatE organized the network primarily by real-world context: cultural geography, market layer, and hierarchical status.
| Dimension | Suggested Label / Meaning | Negative Pole (Top 5) | Positive Pole (Top 5) |
| PC 0 |
Cultural & Geographic Origin (Mediterranean vs. Mesoamerican) |
1. Column krater (-0.554) 2. Bracelet pair (Etruscan) (-0.547) 3. Sicilian plastic vase (-0.532) 4. Red-figure duck askos (-0.526). Donkey-head rhyton (-0.512) | 1. Piedras Negras Stela 26 (+0.761) 2. Piedras Negras Stela 32 (+0.718) 3. Piedras Negras Stela 5 (+0.694) 4. Piedras Negras Stela 2 (+0.692) 5. Piedras Negras Stela 7 (+0.690) |
| PC 1 |
Operational Context (Law Enforcement/Investigations vs. Commercial Art Market) |
1. Italian Investigations of Medici/Hecht (-0.484) 2. Ed Dwyer (-0.428) 3. Johnnie Brown Fell (-0.425) 4. Docemia Looters (-0.416) 5. Hecht Paris Apartment Raids (-0.407) | 1. Peel Park Investigation (+0.407) 2. Antiquaria Romana (+0.389) 3. Hydra Gallery (+0.384) 4. Piedras Negras Stela 7 (+0.378) 5. Tecafin Fiduciaire (+0.373) |
| PC 2 |
Network Hierarchy (Ground-level/Local Actors vs. High-Level Institutions/Galleries) |
1. Community Registry (-0.443) 2. Clive Hollinshead (-0.416) 3. polychrome ceramic vessels (-0.411) 4. Wanborough Temple Activity (-0.410) 5. Steven Berger (-0.396) | 1. Merrin Gallery (+0.424) 2. Mrs Alexi Dupont (+0.371) 3. Piedras Negras Stela 35 (+0.364) 4. Corrupt Archaeologist (+0.350) 5. Denison (+0.347) |
TransE organized the network purely by structural node types (Person, Event, Location, Document), explaining much more of the dataset's variance in just 3 dimensions.
| Dimension | Suggested Label / Meaning | Negative Pole (Top 5) | Positive Pole (Top 5) |
| PC 0 |
Structural Class (Individual Human Masterminds vs. Macro-Level Events) |
1. Marion True (-0.687) 2. Gianfranco Becchina (-0.683) 3. Giacomo Medici (-0.663) 4. N. Koutoulakis (-0.655) 5. Vincenzo Bossi (-0.612) | 1. Atlas Arqueológico Excavation (+0.734) 2. Sipán Seizure Repatriation (+0.717) 3. Silver Hoard of Everbeek Discovery (+0.706) 4. Atru Maithuna Theft Event (+0.702) 5. Denver Art Museum Acquisition (+0.699) |
| PC 1 |
Entity Nature (Ground-Level Labour/Actors vs. Physical Looting Sites/Geographies) |
1. Stela 1 Artifact (-0.588) 2. Guatemala’s Servicios... (-0.561) 3. Val Edwards (-0.556) 4. Anonymous Looters (-0.548)<br>5. Unnamed antiquities dealer (-0.542) | 1. Cerro El Plomo (+0.681) 2. Belitung shipwreck site (+0.616) 3. Río Azul (+0.572) 4. Field on Outskirts of Salisbury (+0.553) 5. polychrome ceramic vessels (+0.549) |
| PC 2 |
Materiality (Documentary/Provenance Evidence vs. Tangible Assets/Physical Rings) |
1. McAlpine Catalogue (-0.595) 2. Treasure Hunting Magazines (-0.592) 3. La Tercera de la Hora... (-0.576) 4. Perge (-0.547) 5. Canadian Export License (-0.544) | 1. Saint Francis Protecting the Laics (+0.649) 2. Wanborough Temple Site (+0.621) 3. Swetnam Drew Kelly Smuggling Ring (+0.619) 4. Val Edwards (+0.608) 5. Village of Coroma (+0.596) |
Gap 1 exists in the space between: Etruscan black-figured kalpis, Piedras Negras Stelae 35 & 26, the raid on Medici's storerooms, Piedras Negras Stela 2
Gap 4 is similar, includes more of the Piedras Negras Stelae;
Gap 8 is defined by Hecht-Medicci-Becchina-Camera on the edges;
This gap exists in Cluster 1, which is defined solely by the Piedras Negras Stelae.
The TC article suggests that the looting of the site is done by drug traffickers and loggers.
The model is suggesting however that mathematically, there is a gap here that is reminiscent of what we see in Italy: so not ad-hoc looting of convenience, but rather something far more systematic.
I'm still working out the method here, which depends on exploring the topology of this 3d space.
For another approach, let's look at the next slide!
This approach surfaces patterns through a kind of tug-o-war between the different principal components
- Dim 0 "Masterminds" (Medici, Becchina, True) at the negative pole, and the "Macro-Events" (repatriations, raids, excavations) at the positive pole.
- Dim 1 Ground-level labour / Looting Sites.
- Dim 2 Documents / Physical Artifacts.
I expected 'significant gaps' at each apex, but this didn't happen (it does in the RotatE model).
The clustering and gaps here shows not a truth about the antiquities trade, but a truth about the underlying source data. The strong clustering in the one pole is the key.
The case studies from TC focus on strong hierarchical networks, and mathematically this shows up by suggesting gaps in our knowledge regarding the low-level sourcing of materials.
Or: TransE picks up on the historiography of the case studies.
Map: a two-dimensional projections of the model’s multidimensional latent space, mapped to a simplex (triangle) using the top three Principal Components. Entities in the corners are most strongly dominated by that particupar dimension / principle component. Entities in the centre bridge concepts.
〞
So what?
– You, in your head,
right now
(always ask: so what!)
Summary
-
Vector algebra lets us play 'what if', and try to find the characteristics of unknown entities/relationships
-
TransE surfaced a kind of computational historiography of the source data, showing gaps in attention and how those gaps then might affect attempts at link prediction or clustering.
-
RotatE suggested similarity of patterns from one geographical region to another
- I've built more models with different geometries but haven't analyzed fully yet. But they will surface other aspects.
So What? Well:
The Trafficking Culture case studies are all well-known episodes.
The fact that the results make sense reassures us that the method sees meaningful patterns. We find things that we would expect, and we find surprises - Leonardo Patterson - that turn out to actually be there in the world.
So imagine what it might surface when fed thousands of newspaper articles, art theft notifications, or other kinds of grey literature...!
Things we know
Things we didn't know we knew
Thanks!
shawn.graham@carleton.ca
- knowledge graph construction can be machine-accelerated
- data cleaning is the single most important step
- different embedding model geometries pick up different facets of the trade
- everything is a direction; strike out in a new direction and you'll surface new ideas in your old data.
- no one embedding model will give you the complete picture
- an embedding model can find clusters, predict unknown relationships, and be queried to reveal gaps in the underlying data
- knowledge graph embedding models can represent all kinds of narrative information, leading to new insights about things we thought we knew