Adventures in Correcting XML Collation with Python and XSLT:

 

Elisa Beshero-Bondar  

          Twitter: @epyllia | GitHub: @ebeshero


Balisage: The Markup Conference
 Monday August 1, 2022 @ 11am



Link to these slides: https://bit.ly/untangle-fv

Background image created by the author from a loom on Reddit and the frontispiece illustration of Frankenstein (1831)

Untangling the Frankenstein Variorum

A Digital Variorum

Our use of the term: A digital edition that investigates change to a work by comparing distinct versions of it.

 

 

 

Machine-assisted collation

  • Our method of conducting the comparison
  • Software helps us to find meaningful moments of comparison and difference
  • Humans teach the software how to interpret the texts to compare them meaningfully

 

 

What we're collating

Print Publications of Frankenstein in MWS's lifetime:  

  • 1818 Edition published anonymously (3 volumes)

  • 1823 Edition printed by MWS's father William Godwin, the first to include her name as the author. (2 volumes)

  • 1831 Edition extensively revised by MWS, bound with Friedrich von Schiller's The Ghost Seer in Bentley's Standard Series of novels) (1/2 of a volume)

Manuscript versions: 

  • Fair copy MS drafts (~1816) at the Bodleian Library, Oxford: Abinger c56, c57, c58  

     

  • Thomas Copy made sometime between 1818 and 1822: MWS's marginal comments on a print copy of 1818

Collaboration effort across institutions since 2017

  • Scott Weingart, Rikk Mulligan, Matt Lincoln, Jon Klancher and team at Carnegie Mellon University:
    • got us started, conceptualize the edition interface, location/prep of source texts + scholarly editorial annotations.
      In flux...several left CMU, some remain connected.
  • Raffaele Viglianti (MITH: Shelley-Godwin Archive):
    • interface development + assist w/ work on the Shelley-Godwin archive
  • Me (Pitt-Greensburg => Penn State Erie):
    • Back-end XML Collation, pipeline to output TEI editions w/ variation data
    • Need as much help as I can get from fresh eyes and ideas! (Thanks, Mia Borgia, Jackie Chan, and Yuying Jin!)
  • Angela Brunk (Kansas State U):
    • new connection; consults on UX / accessibility for the interface

  James Rieger, ed., first new edition of 1818 in 141

  years :   inline collation of "Thomas" w/ 1818,

  1831 variants in endnotes

Legend:

Stuart Curran and Jack Lynch: PA Electronic Edition (PAEE) , collation of 1818 and 1831: HTML

Nora Crook crit. ed of 1818,  variants of "Thomas",   1823, and 1831 in endnotes (P&C MWS collected works)

Romantic Circles TEI conversion of PAEE ; separates the texts of 1818 and 1831; collation via Juxta

1974

~mid-1990s

1996

Charles Robinson, The Frankenstein Notebooks (Garland): print facsimile of 1816 ms drafts

2007

Shelley-Godwin Archive publishes diplomatic/documentary edition of 1816 ms drafts

print edition

digital edition

Legend:

2013

2017

Critical and Documentary Editions Leading to the Frankenstein Variorum Project

Frankenstein Variorum Project :

assembly/proof-correcting of PAEE files; OCR/proof-correcting 1823; "bridge" TEI edition of S-GA notebook files; automated collation; incorporating "Thomas" copy text

 

  • Can we make an edition that conveniently compares the manuscripts to the print publications?
     

  • Can we make a comprehensive variorum to show changes to the novel over time, from 1816 to 1831?

    • Which editorial interventions persist from 1816 to 1831?

      • MWS in the "Thomas" copy: how much of this persists into 1831?

      • PBS's additions: which/how many of these persist to 1831?

      • What parts of the novel were most mutable?  


 

Motivating Questions

Variorum challenge

1. Share a nonlinear, divergent edition history 

 

 

  • 1816 notebooks to 1818: uneven (gaps in notebooks)
  • Thomas divergence:
    • copy with margin notes was left in Italy before 1823, apparently not consulted  later
  • 1823 edits:  largely retained in 1831
  • 1831 major revisions:
    • alteration of character relationships, added chapter and several lengthened passages

2. Introduce textual scholarship to students, fans of Frankenstein as well as text scholars, 19c specialists:

 

 

 

  • Recruit next generation of text scholars
  • Don't tuck variants away into endnotes/footnotes
  • Instead, tell the story of Frankenstein’s ”hot” and ”cool” alterations

The “base markup” for collation

  • Each version deemed a "reading witness"  to the novel at a distinct moment in its textual history.
     
  • Print books
    • simple structural markup for 1818, 1823, and 1831 eds.
       
  • Manuscript versions
     
    • Thomas copy (handwritten notes on 1818 print text): prepped from 1818 files with some simple manuscript coding of handwritten insertions, deletions, and notes
       
    • 1816 MS Notebook: It's complicated...

PA Electronic Edition (mid 1990s): 1818 vs 1831

  • started from base HTML 1.0 files
  • up-converted to clean, simple XML
    • ”on its way” to TEI (structural elements in text)
    • prepared for machine-assisted collation (via CollateX): including element tags
    • deep hierarchy of novel ”flattened” to milestones: <div type="volume"/>, <p/>, etc.
  • corrected against photofacsimiles of 1818 and 1831 print publications

Prepared from OCR new XML of 1823 edition

 

  • prepared by William Godwin, the first edition bearing the name ”Mary Wollstonecraft Shelley” on the title page
     
  • XML syntax matches that of 1818 and 1831 editions

Working with MS versions

Perhaps more to the point, I can say. . .

Well, mostly...
 

...But let's take it one stage at a time...

About that little problem...

  • analyze the markup to determine where it meaningfully intersects with the simpler encoding of the print
  • ignore what markup has no counterpart in the other witnesses

...unless we

Shelley-Godwin Archive’s diplomatic edition of the 1816 Notebooks at http://shelleygodwinarchive.org

  • collection of TEI files, one file per notebook page

  • encoded page surface-by-surface and line-by-line
  • required me to resequence the code to include in the collation*
    • ​*margin notes were not encoded at point of insertion but at the end of each page file
    • Insertion points were marked carefully with @xml:ids and pointers, so resequencing with XSLT worked
    • We produced an alternate version of the S-GA files in order to collate it in reading order.  

“S-GA msColl” 

Shelley-Godwin Archive: sample page surface:

S-GA: resequenced / compressed for collation

<surface lrx="3847" lry="5342" 
partOf="#ox-frankenstein_volume_i" 
ulx="0" uly="0" folio="21r" shelfmark="MS. Abinger c. 56" base="ox-ms_abinger_c56/ox-ms_abinger_c56-0045.xml" 
id="ox-ms_abinger_c56-0045" sID="ox-ms_abinger_c56-0045"/>
      <graphic url="http://shelleygodwinarchive.org/images/ox/ms_abinger_c56/ms_abinger_c56-0045.jp2"/>
      <zone type="main" sID="c56-0045__main"/> 

<lb n="c56-0045__main__17"/> 
         <del rend="strikethrough" sID="c56-0045__main__d2e9811"/>But how<del eID="c56-0045__main__d2e9811"/> How can I describe
      my <lb n="c56-0045__main__18"/> emotion at this catastrophe; or how 

<w ana="start"/>deli<lb n="c56-0045__main__19"/>neate<w ana="end"/> 

the wretch whom with such <lb n="c56-0045__main__20"/> infinite pains and care I had endeavoured <lb n="c56-0045__main__21"/> to form. His limbs were in proportion <lb n="c56-0045__main__22"/> and I had selected his features <del rend="strikethrough" sID="c56-0045__main__d2e9830"/>h<del eID="c56-0045__main__d2e9830"/> as <lb n="c56-0045__main__23"/> 
         <mod sID="c56-0045__main__d2e9835"/>
            <del rend="strikethrough" sID="c56-0045__main__d2e9837"/>handsome<del eID="c56-0045__main__d2e9837"/>
            <mdel>.</mdel>
            <anchor xml:id="c56-0045.01"/>
            <zone corresp="#c56-0045.01" type="left_margin" sID="c56-0045__left_margin"/> 
               <lb n="c56-0045__left_margin__1"/> 
               <add sID="c56-0045__left_margin__d2e9849"/>
                  <mod sID="c56-0045__left_margin__d2e9851"/>
                     <del rend="strikethrough" sID="c56-0045__left_margin__d2e9853"/>handsome<del eID="c56-0045__left_margin__d2e9853"/>
                     <add hand="#pbs" place="superlinear" sID="c56-0045__left_margin__d2e9856"/>beautiful.<add eID="c56-0045__left_margin__d2e9856"/>
                  <mod eID="c56-0045__left_margin__d2e9851"/>
               <add eID="c56-0045__left_margin__d2e9849"/>
            <zone eID="c56-0045__left_margin"/>
         <mod eID="c56-0045__main__d2e9835"/>
         <mod sID="c56-0045__main__d2e9863"/>
            <del rend="strikethrough" sID="c56-0045__main__d2e9865"/>Handsome<del eID="c56-0045__main__d2e9865"/>
            <add hand="#pbs" place="superlinear" sID="c56-0045__main__d2e9868"/>Beautiful<add eID="c56-0045__main__d2e9868"/>
         <mod eID="c56-0045__main__d2e9863"/>; Great God! His <lb n="c56-0045__main__24"/>
  • added word boundary markup to indicate whole words spanning lines
  • resequenced margin zone content: (followed S-GA's pointers to represent semantic reading order for collation)

“Thomas”  (much easier...)

Thomas copy marginalia

  • prepared new XML from 1818 edition, with <add>, <del>, <note> elements

Added new XML: Thomas Copy 

  • Added insertions, deletions, + margin-notes to 1818 edition using <add>, <del>, and <note>
     
  • checked against and respond to/update Nora Crook and James Reiger editions

Frankenstein Variorum: The Collation Process

Gothenburg model : algorithm for computer-assisted collation, developed in 2009 workshop of collateX and Juxta developers.

  1. Tokenization :

    • Break down the smallest unit of comparison: (words--with punctuation, or character-by-character):

    • FV tokenizes words and includes punctuation and tags:
      '&lt;del&gt;the', 'frame', 'on', 'whic&lt;del&gt;', 'my', 'man', 'completeed,.'

  2. Normalization  

    • ​​ '&' = 'and'

    • <p xml:id="novel1_letter4_div4_p2"> = <p/>

  3. Alignment

    • Identify comparable divergence: what makes text sequences comparable units?

    • “Chunking” text into comparable passages (chapters/paragraphs that line up with identifiable start and end points). Collation proceeds chunk by chunk.

  4. Analysis  

    • ​​ Study output, correct, and re-align after machine process, AND refine automated processing

  5. Visualization: ​

    • ​​Critical edition interface, graph displays

Computer-aided collation: Gothenburg Model

Collation is weaving...

  • Source documents supplying “threads”
  • collation software = weaving machine
    • needs to be able to find where the “threads” run together,
    • and where they diverge (and what constitutes divergence)
    • Markup functions as signals to the collation weaving machine

Guiding the machine reading

  • Markup is not the same in all of the editions
  • That's okay, because we hide some of it from collation software
  • Our method:
    • Python script instructs on strings of text that the collation algorithm can just skip over
    • Marking the markup: Identify markup that is not helpful for comparing the documents

With witnesses prepared and inspected, we turn to the Python script that tokenizes and normalizes the source files, preparing the witnesses to be compared by collation software.

Image credits: https://rootsofprogress.org/learning-the-loom,

https://www.oschaslings.com/blog/warp-weft-weave-structure-understanding-how-your-sling-is-made-wrapping-qualities

warp: sets tension or looseness of weave:
normalizing the text "thread" to tell us how  to "pull" it

weft: moves through the warp threads cross-wise:

establishes moments of alignment across the text threads

Which power loom?

XSLT or Python/CollateX?

  • XSLT differentiation driven by XPath string comparison
    • tan:collate() might be desirable if it's searching for the longest possible moments of comparison 
    • Precision/legiblity of normalizing code 
    • Needs a lot of adaptation from the TAN library to incorporate into production pipeline
      • Collation software sometimes prioritizes the normalization as the "source of truth", but that's only an interpretation
      • But we need to output the original source text together with its normalized basis for comparison

 

 

Which power loom?

XSLT or Python/CollateX?

  • Python and collateX
    • Bespoke TEI output: customized in our project pipeline
    • Maybe just tokenization /alignment problem that we can correct?
    • Needs to be rewritten and better documented for sustainability; sharing the code

 

 

 

Tokenization

  • You need to be able to modify this

  • What is the smallest basis for comparison of text streams?
    • Character by character?
    • Word by word? 
      • including punctuation? 
      • including markup around words?
      • isolating the markup?

 

 

 

 

Tokenization

  • Is our tokenization too granular?
    • I'm working on that now with the Python-to-collateX script
    • Not sure how to modify to this degree of precision with tan:collate() yet

 

 

 

 

 

 

 

 

 

 

def tokenize(inputFile):
        return regexLeadingBlankLine.sub('\n', regexBlankLine.sub('\n', 
					extract(inputFile))).split('\n')

reduces multiple `\n`with a single `\n`and separates word and element tokens with newlines. The actual tokens are  built up by the extract() function. . . 

Tokenization: extracting XML with Pulldom

 

 

 

 

 

 

 

 

 

def extract(input_xml):
    """Process entire input XML document, firing on events"""
    # Start pulling; it continues automatically
    doc = pulldom.parse(input_xml)
    output = ''
    for event, node in doc:
        # elements to ignore: xml
        if event == pulldom.START_ELEMENT and node.localName in ignore:
            continue
     
        if event == pulldom.START_ELEMENT and node.localName in inlineVariationEvent:
            doc.expandNode(node)
            output += '\n' + node.toxml() + '\n'
        
        elif event == pulldom.START_ELEMENT and node.localName in blockEmpty:
            output += '\n' + node.toxml() + '\n'
        # ebb: empty inline elements that do not take surrounding white spaces:
        elif event == pulldom.START_ELEMENT and node.localName in inlineEmpty:
            output += node.toxml()
       
        elif event == pulldom.START_ELEMENT and node.localName in inlineContent:
            output += '\n' + regexEmptyTag.sub('>', node.toxml())
        elif event == pulldom.END_ELEMENT and node.localName in inlineContent:
            output += '</' + node.localName + '>' + '\n'
      
        elif event == pulldom.CHARACTERS:
            output += normalizeSpace(node.data)
        else:
            continue
    return output
ignore = ['mod', 'sourceDoc', 'xml', 'comment', 'anchor', 'include',
          'delSpan', 'addSpan','handShift', 'damage', 
          'restore', 'zone', 'surface',
          'graphic', 'unclear', 'retrace']
blockEmpty = ['pb', 'p', 'div', 'milestone',
              'lg', 'l', 'cit', 'quote',
              'bibl', 'ab', 'head']
inlineEmpty = ['sga-add', 'lb', 'gap',  'hi', 'w']
inlineContent = ['del-INNER', 'add-INNER', 'metamark',
                 'mdel', 'shi']
inlineVariationEvent = ['del', 'add', 'note']

In the Python script:

Prepare the warp: create lists of XML element names for special treatment

  • Ignored whole elements need to be screened out of the collation entirely.
  • Other whole elements need to be preserved but normalized.
  • XML Pulldom library helps us with special handling of XML elements.

Normalization

 

 

 

 

 

 

 

 

How should the collation software read the tokens and understand sameness?


                <!-- Should punctuation be ignored? -->
    <xsl:param name="tan:ignore-punctuation-differences" as="xs:boolean" select="false()"/>
    
    <xsl:param name="additional-batch-replacements" as="element()*">
        <!--ebb: normalizations to batch process for collation. NOTE: We want to do these to preserve some markup \\
            in the output for post-processing to reconstruct the edition files. 
            Remember, these will be processed in order, so watch out for conflicts. -->
        <replace pattern="(<.+?>\s*)&gt;" replacement="$1" 
                 message="normalizing away extra right angle brackets"/>
         <replace pattern="&amp;" replacement="and" 
                  message="ampersand batch replacement"/>
        <replace pattern="</?xml>" replacement="" 
                 message="xml tag replacement"/>
        <replace pattern="(<p)\s+.+?(/>)" replacement="$1$2" 
                 message="p-tag batch replacement"/>
        <replace pattern="(<)(metamark).*?(>).+?\1/\2\3" replacement="" 
                 message="metamark batch replacement"/>
      <!--ebb: metamark contains a text node, and we don't want its 
contents processed in the collation, so this captures the entire element. -->
        <replace pattern="(</?)m(del).*?(>)" 
                 replacement="$1$2$3" message="mdel-SGA batch replacement"/> 
      <!--ebb: mdel contains a text node, so this catches both start and end tag.
        We want mdel to be processed as <del>...</del>-->
        <replace pattern="</?damage.*?>" 
                 replacement="" message="damage-SGA batch replacement"/> 
      <!--ebb: damage contains a text node, so this catches both start and end tag. -->
        <replace pattern="</?unclear.*?>" replacement="" 
                 message="unclear-SGA batch replacement"/>
      <!--ebb: unclear contains a text node, so this catches both start and end tag. -->
        <replace pattern="</?retrace.*?>" replacement=""
                 message="retrace-SGA batch replacement"/>
      <!--ebb: retrace contains a text node, so this catches both start and end tag. -->

See Joel Kalvesmaki's Balisage papers on
tan:diff (2021) and tan:collate 2022 (this week)

Normalizing/collating with TAN:Diff XSLT Library

Define regex patterns you need (Python)

RE_PARA = re.compile(r'<p\s.+?/>')
RE_INCLUDE = re.compile(r'<include.*?/>')
RE_HEAD = re.compile(r'<head.*?/>')
RE_AB = re.compile(r'<ab.*?/>')
RE_ADDEND = re.compile(r'</add>')
RE_NOTE_START = re.compile(r'<note.*?>')
RE_NOTE_END = re.compile(r'</note>')
RE_DELSTART = re.compile(r'<del.*?>')
RE_DELEND = re.compile(r'</del>')
RE_SGA_ADDSTART = re.compile(r'<sga-add.+?sID.+?>')
RE_SGA_ADDEND = re.compile(r'<sga-add.+?eID.+?>')
RE_MDEL = re.compile(r'<mdel.*?>.+?</mdel>')
RE_SHI = re.compile(r'<shi.*?>.+?</shi>')
RE_METAMARK = re.compile(r'<metamark.*?>.+?</metamark>')
RE_HI = re.compile(r'<hi\s.+?/>')
RE_PB = re.compile(r'<pb.*?/>')
RE_LB = re.compile(r'<lb.*?/>')
RE_LG = re.compile(r'<lg[^<]*/>')
RE_L = re.compile(r'<l\s[^<]*/>')
RE_CIT = re.compile(r'<cit\s[^<]*/>')
RE_QUOTE = re.compile(r'<quote\s[^<]*/>')
RE_OPENQT = re.compile(r'“')
RE_CLOSEQT = re.compile(r'”')
RE_GAP = re.compile(r'<gap\s[^<]*/>')
RE_sgaP = re.compile(r'<milestone[^<]+?unit="tei:p.+?/>')
RE_MILESTONE = re.compile(r'<milestone.+?>')
RE_MULTI_LEFTANGLE = re.compile(r'<{2,}')
RE_MULTI_RIGHTANGLE = re.compile(r'>{2,}')

Normalize the markup so it's comparable

def normalize(inputText):
    return RE_MULTI_LEFTANGLE.sub('<',\
        RE_MULTI_LEFTANGLE.sub('>', \
        RE_INCLUDE.sub('', \
        RE_AB.sub('', \
        RE_HEAD.sub('', \
        RE_AMP.sub('and', \
        RE_MDEL.sub('', \
        RE_SHI.sub('', \
        RE_HI.sub('', \
        RE_LB.sub('', \
        RE_PB.sub('', \
        RE_PARA.sub('<p/>', \
        RE_sgaP.sub('<p/>', \
        RE_MILESTONE.sub('', \
        RE_LG.sub('<lg/>', \
        RE_L.sub('<l/>', \
        RE_CIT.sub('', \
        RE_QUOTE.sub('', \
        RE_OPENQT.sub('"', \
        RE_CLOSEQT.sub('"', \
        RE_GAP.sub('', \
        RE_DELSTART.sub('<delstart/>', \
        RE_DELEND.sub('<delend/>', \
        RE_ADDSTART.sub('<addstart/>', \
        RE_ADDEND.sub('<addend/>', \
        RE_MOD.sub('', \
        RE_METAMARK.sub('', inputText))))))))))))))))))))))))))).lower()

Find and replace the regex patterns before feeding to collateX

       Tokenization error

 

 


    <app>
		<rdgGrp n="['spot,', 'and', 'endeavoured,']">
			<rdg wit="f1818">spot, and endeavoured, </rdg>
			<rdg wit="f1823">spot, and endeavoured, </rdg>
			<rdg wit="fThomas">spot, and endeavoured, </rdg>
			<rdg wit="f1831">spot, and endeavoured, </rdg>
		</rdgGrp>
		<rdgGrp n="['spotand', 'endeavoured']">
			<rdg wit="fMS">spot& endeavoured </rdg>
		</rdgGrp>
	</app>
  . . . the spot<add eID="c57-0117__main__d3e21951"/> & endeavoured . . .

In the fMS source: 

       Repairing the tokenization error

 

 


 if event == pulldom.START_ELEMENT and node.localName in inlineEmpty:
            output += '\n' + node.toxml() + '\n'

by adding newline characters around markup nodes

       Repairing the tokenization error

 

 

corrected this output...

<app>
		<rdgGrp n="['spot', '&lt;addend/&gt;']">
			<rdg wit="fMS">spot 
              &lt;add eID=&quot;c57-0117__main__d3e21951&quot;/&gt; </rdg>
		</rdgGrp>
		<rdgGrp n="['spot,']">
			<rdg wit="f1818">spot, </rdg>
			<rdg wit="f1823">spot, </rdg>
			<rdg wit="fThomas">spot, </rdg>
			<rdg wit="f1831">spot, </rdg>
		</rdgGrp>
	</app>
	<app>
		<rdgGrp n="['and']">
			<rdg wit="f1818">and </rdg>
			<rdg wit="f1823">and </rdg>
			<rdg wit="fThomas">and </rdg>
			<rdg wit="f1831">and </rdg>
			<rdg wit="fMS">&amp; </rdg>
		</rdgGrp>
	</app>

but created a new problem...

<app>
		<rdgGrp n="['for', 'there']">
			<rdg wit="f1818">for there </rdg>
			<rdg wit="f1823">for there </rdg>
			<rdg wit="fThomas">for there </rdg>
			<rdg wit="f1831">for there </rdg>
			<rdg wit="fMS">for there </rdg>
		</rdgGrp>
	</app>
	<app>
		<rdgGrp n="['', '', '&lt;addstart/&gt;']">
			<rdg wit="fMS">&lt;lb n=&quot;c57-0118__main__4&quot;/&gt; 
              &lt;lb n=&quot;c57-0118__left_margin__1&quot;/&gt; &lt;add hand=&quot;#pbs&quot; 
              sID=&quot;c57-0118__left_margin__d3e21996&quot;/&gt; </rdg>
		</rdgGrp>
	</app>
	<app>
		<rdgGrp n="['was']">
			<rdg wit="f1818">was </rdg>
			<rdg wit="f1823">was </rdg>
			<rdg wit="fThomas">was </rdg>
			<rdg wit="f1831">was </rdg>
			<rdg wit="fMS">was </rdg>
		</rdgGrp>
	</app>
	<app>
		<rdgGrp n="['&lt;addend/&gt;']">
			<rdg wit="fMS">&lt;add eID=&quot;c57-0118__left_margin__d3e21996&quot;/&gt; </rdg>
		</rdgGrp>
	</app>
	<app>
		<rdgGrp n="['no', 'sign', 'of', 'any']">
			<rdg wit="f1818">no sign of any </rdg>
			<rdg wit="f1823">no sign of any </rdg>
			<rdg wit="fThomas">no sign of any </rdg>
			<rdg wit="f1831">no sign of any </rdg>
			<rdg wit="fMS">no sign of any </rdg>
		</rdgGrp>
	</app>
	<app>
		<rdgGrp n="['violence']">
			<rdg wit="fMS">violence </rdg>
		</rdgGrp>
		<rdgGrp n="['violence,']">
			<rdg wit="f1818">violence, </rdg>
			<rdg wit="f1823">violence, </rdg>
			<rdg wit="fThomas">violence, </rdg>
			<rdg wit="f1831">violence, </rdg>
		</rdgGrp> </app>

Solutions

  • Revise the tokenizing algorithm! What do we understand?
    • <add> in S-GA witness is not as meaningful as <add> in Thomas
      • ​Why? Because S-GA shows "genetic" handwriting, insertions over its own text.
      • Thomas text is a purposeful and complete new insertion into the 1818 edition.
    • Redo the inputs to distinguish <sga-add/> elements from Thomas <add>...</add> elements.
    • Tokenize Thomas text adds: "&lt;add&gt; . . . .&lt;/add&gt;"  
      • This is now one complete token and its own unit of contrast from the other editions that can't be broken up.  
    • Tokenize whole <del>...</del> elements too, S-GA + Thomas
      • In both, deleted text needs to be collated as part of deleted passage, not broken up wherever a word or punctuation happens to align.
      • SGA: usually false starts, sometimes longer unwanted passages.
      • Sequestering the deleted passages reliably isolates them in the collation output

Collation Pre-processing vs. Post-processing

When to make the interventions?

  • Brittle: Try to prevent yourself from hand-correcting or XSLT-correcting the outputs; gets complicated fast
  • Pre-process as much as possible...but you can't solve every snag this way, and this also gets complicated fast 
    • Did I really need to make 15 different tiny files out of one collation unit to align around three long added passages in the S-GA and the 1831 edition? That's hand-work, too: brittle.
  • Setting up <add>...</add> and <del>...</del> to be sequestered means they NEVER are allowed to line up, even in parts. But the collation outputs them in strange places 
  • Is it okay to post-process the output with XSLT to adjust the alignments? 
    • Only if you can establish a reliable pattern for doing it.

What we're doing with the collation data
 

  • Making a “stand-off“ Spine (info + pointers to collation data)
     

  • Generating the edition files with collation data marked “inline”

Interface production pipeline

 

 

 

  • Output files are copied into the post-collation repo:
    https://github.com/FrankensteinVariorum/fv-postCollation
     
  • Files in the postColl-workspace/P\d-output/ directories
    • become the finalized versions determined to be ready for publishing in the Variorum digital edition
    • XSLT pipeline "raises" the distinct edition files with their "hotspot" markers of variation from each other
       
  • Before the post-processing pipeline, we need to:
    • try to ensure accuracy of collation!
    • check and correct by:
      • correcting the Python script feeding the collation to handle repeated kinds of problems
      • correcting by hand the errors that are difficult to handle programmatically

 

  • “Spine” data model = standoff use of TEI critical apparatus: 
    • coordinates data on variance: piece by piece (vertebra) which specific passages line up and where they differ.
    • points to specific locations in separate edition files
    • outputs source texts prior to normalization: used to construct the five individual TEI editions holding "hotspot"markup at moments of variance.
    • includes data on normalization: e.g. normalized tokens used in collation process
    • includes processed data, like maximum edit-distance, at each variant location

Tokenization and alignment is a research method.

There will be more than one way to express it.

Your alignment decisions express your theory of your text.  

Reflections on machine-assisted collation and the TEI critical apparatus

We (scholars and programmers, and scholarly programmers) need to share our algorithms for tokenization and alignment, together with the editions and interfaces we design with it.

Launch Phase Completed

  • One third of the collation is being displayed at https://frankensteinvariorum.github.io/
  • This (first) portion of the novel helped us test our ”spine” data model and the Variorum reader web interface

Letter III passage: “Thomas” viewpoint

Letter III passage ("Remember me...”) “Thomas” viewpt

Letter III passage ("Remember me...”): “1831” viewpt

 A heavily revised passage, showing the MS notebook view

”Heatmap” view, showing variation intensity as blocks with circles color-coded by edition. Selecting a circle on the heatmap view displays the edition and its variants.

cu 01 msColl 1818 volume frontmatter PREFACE. T HE event on which this fiction is founded has been supposed, by Dr. Darwin, and some of the physiological writers of Germany, as not of impossible occurrence. I shall not be supposed as according the remotest degree of serious faith to such an imagination; yet, in assuming it as the basis of a work of fancy, I have not considered myself as merely weaving a series of supernatural terrors. The event on Thomas volume frontmatter PREFACE. T HE event on which this fiction is founded has been supposed, by Dr. Darwin, and some of the physiological writers of Germany, as not of impossible occurrence. I shall not be supposed as according the remotest degree of serious faith to such an imagination; yet, in assuming it as the basis of a work of fancy, I have not considered myself as merely weaving a series of supernatural terrors. The event on 1823 volume frontmatter PREFACE. T HE event on which this fiction is founded has been supposed, by Dr. Darwin, and some of the physiological writers of Germany, as not of impossible occurrence. I shall not be supposed as according the remotest degree of serious faith to such an imagination; yet, in assuming it as the basis of a work of fancy, I have not considered myself as merely 1831 frontmatter introduction PREFACE. T HE event on which this fiction is founded has been supposed, by Dr. Darwin, and some of the physiological writers of Germany, as not of impossible occurrence. I shall not be supposed as according the remotest degree of serious faith to such an imagination; yet, in assuming it as the basis of a work of fancy, I have not considered myself as merely weaving a series of supernatural terrors. The event on which the interest of the story depends is exempt from the disadvantages of a mere tale of spectres or enchantment. It was recommended by the novelty of the situations which it developes; and, however impossible as a physical fact, affords a point of view to the imagination for the delineating of human passions more comprehensive and commanding than any which the ordinary relations of existing events can yield. cu 02 msColl 1818 1818V1 LETTER I FRANKENSTEIN; OR, THE MODERN PROMETHEUS. LETTER I LETTER I To Mrs. S AVILLE Thomas FRANKENSTEIN; OR, THE MODERN PROMETHEUS. LETTER I LETTER I To Mrs. S AVILLE 1823 FRANKENSTEIN; OR , THE MODERN PROMETHEUS. LETTER I. To Mrs . S AVILLE 1831 FRANKENSTEIN; OR , THE MODERN PROMETHEUS. LETTER I. To Mrs. Saville, England. St. Petersburgh, Dec. 11th, 17—. Y cu 03 msColl 1818 1818V1 LETTER II LETTER II. To Mrs. S AVILLE Thomas LETTER II. To Mrs. S AVILLE 1823 LETTER II. To Mrs. S AVILLE 1831 LETTER II. To Mrs. Saville, England. Archangel, 28th March, 17—. How slowly the time passes here, encompassed as I am by frost and snow! yet a second step is taken towards my enterprise. I have hired a vessel, and am occupied in collecting my sailors; those whom I have already engaged, appear to be men on whom I can depend, and are certainly possessed of dauntless courage. cu 04 msColl 1818 1818V1 LETTER III LETTER III. To Mrs. S AVILLE Thomas LETTER III. To Mrs. S AVILLE 1823 LETTER III. To Mrs. S AVILLE 1831 LETTER III. To Mrs. Saville, England. July 7th, 17—. M cu 05 msColl 1818 1818V1 LETTER IV LETTER IV. To Mrs. S AVILLE Thomas LETTER IV. To Mrs. S AVILLE 1823 LETTER IV. To Mrs. S AVILLE 1831 LETTER IV. To Mrs. Saville, England. August 5th, 17—. So strange an accident has happened to us, that I cannot forbear recording it, although it is very probable that you will see me before these papers can come into your possession. cu 06 msColl 1818 Thomas 1823 1831 cu 07 msColl Chapt. 2 Those events which materially influence our fu ture 1818 1818V1 CHAPTER I CHAPTER I. I AM by birth a Genevese; and my family is one of the most distinguished of that republic. My ancestors had been for many years counsellors and syndics; and my father had filled several public situations with honour and reputation. He was respected by all who knew him for his integrity and indefatigable attention to public business. He passed his younger days perpetually occupied by the affairs of Thomas CHAPTER I. I AM by birth a Genevese; and my family is one of the most distinguished of that republic. My ancestors had been for many years counsellors and syndics; and my father had filled several public situations with honour and reputation. He was respected by all who knew him for his integrity and indefatigable attention to public business. He passed his younger days perpetually occupied by the affairs of 1823 CHAPTER I. I AM by birth a Genevese; and my family is one of the most distinguished of that republic. My ancestors had been for many years counsellors and syndics; and my father had filled several public situations with honour and reputation. He was respected by all who knew him for his integrity and indefatigable attention to public business. He passed his younger days perpetually occupied by the affairs of his country; 1831 CHAPTER I. I AM by birth a Genevese; and my family is one of the most distinguished of that republic. My ancestors had been for many years counsellors and syndics; and my father had filled several public situations with honour and reputation. He was respected by all who knew him, for his integrity and indefatigable attention to public business. He passed his younger days perpetually occupied by the affairs of his country; a variety of circumstances had prevented his marrying early, nor was it until the decline of life that he became a husband and the father of a family. CHAPTER II. W E were brought up together; there was not quite a year difference in our ages. I need not say that we were strangers to any species of disunion or dispute. Harmony was the soul of our companionship, and the diversity and contrast that subsisted in our characters drew us nearer together. Elizabeth was of a calmer and more concentrated disposition; but, with all my ardour, I was capable of a more intense application, and was more deeply smitten with the thirst for knowledge. She busied herself with following the aerial creations of the poets; and in the majestic and wondrous scenes which surrounded our Swiss home—the sublime shapes of the mountains; the changes of the seasons; tempest and calm; the silence of winter, and the life and turbulence of our Alpine summers,—she found ample scope for admiration and delight. While my companion contemplated with a serious and satisfied spirit the magnificent appearances of things, I delighted in investigating their causes. The world was to me a secret which I desired to divine. Curiosity, earnest research to learn the hidden laws of nature, gladness akin to rapture, as they were unfolded to me, are among the earliest sensations I can remember. cu 08 msColl Chapter 3 When I had attained the age of Chap. 4 The next morning I delivered my letters of introduction and paid a visit to some of the principal 1818 1818V1 CHAPTER II CHAPTER II. W HEN I had attained the age of seventeen, my parents resolved that I should become a student at the university of Ingolstadt. I had hitherto attended the schools of Geneva; but my father thought it necessary, for the completion of my education, that I should be made acquainted with other customs than those of my native country. My departure was therefore fixed at an early date; but, before the day resolved upon could arrive, the first misfortune of my life occurred—an omen, as it were, of my future misery. Thomas CHAPTER II. W HEN I had attained the age of seventeen, my parents resolved that I should become a student at the university of Ingolstadt. I had hitherto attended the schools of Geneva; but my father thought it necessary, for the completion of my education, that I should be made acquainted with other customs than those of my native country. My departure was therefore fixed at an early date; but, before the day resolved upon could arrive, the first misfortune of my life occurred—an omen, as it were, of my future misery. 1823 CHAPTER II. W HEN I had attained the age of seventeen, my parents resolved that I should become a student at the university of Ingolstadt. I had hitherto attended the schools of Geneva; but my father thought it necessary, for the completion of my education, that I should be made acquainted with other customs than those of my native country. My departure was therefore fixed at an early date; but, before the day resolved upon could arrive, the first misfortune of my life occurred—an omen, as it were, of my future misery. 1831 CHAPTER III. W HEN I had attained the age of seventeen, my parents resolved that I should become a student at the university of Ingolstadt. I had hitherto attended the schools of Geneva; but my father thought it necessary, for the completion of my education, that I should be made acquainted with other customs than those of my native country. My departure was therefore fixed at an early date; but, before the day resolved upon could arrive, the first misfortune of my life occurred—an omen, as it were, of my future misery. cu 09 msColl Chap. 5 From this day natural philosophy and parti cularly Chapter 6. When I found this so 1818 1818V1 CHAPTER III CHAPTER III. F ROM this day natural philosophy, and particularly chemistry, in the most comprehensive sense of the term, became nearly my sole occupation. I read with ardour those works, so full of genius and discrimination, which modern inquirers have written on these subjects. I attended the lectures, and cultivated the acquaintance, of the men of science of the university; and I found even in M. Krempe a great deal of sound sense and real information, combined, it is true, with a repulsive physiognomy and manners, but not on that account the less valuable. In M. Waldman I found Thomas CHAPTER III. F ROM this day natural philosophy, and particularly chemistry, in the most comprehensive sense of the term, became nearly my sole occupation. I read with ardour those works, so full of genius and discrimination, which modern inquirers have written on these subjects. I attended the lectures, and cultivated the acquaintance, of the men of science of the university; and I found even in M. Krempe a great deal of sound sense and real information, combined, it is true, with a repulsive physiognomy and manners, but not on that account the less valuable. In M. Waldman I found 1823 CHAPTER III. From this day natural philosophy, and particularly chemistry, in the most comprehensive sense of the term, became nearly my sole occupation. I read with ardour those works, so full of genius and discrimination, which modern inquirers have written on these subjects. I attended the lectures, and cultivated the acquaintance, of the men of science of the university; and I found even in M. Krempe a great deal of sound sense and real information, combined, it is true, with a repulsive physiognomy and manners, but not on that account the less valuable. In M. Waldman I found a true friend. His gentleness was never tinged by dogmatism; and his instructions were given with an air of frankness and good nature, that banished every idea of pedantry. It was, perhaps, the amiable character of this man that inclined me more to that branch of natural philosophy which he professed, than an intrinsic love for the science itself. But this state of mind had place only in the first steps towards knowledge: the more fully I entered into the science, the more exclusively I pursued it for its own sake. That application, which at first had been a matter of duty and resolution, now became so ardent and eager, that the stars often disappeared in the light of morning whilst I was yet engaged in my laboratory. As I applied so closely, it may be easily conceived that I improved ra 1831 CHAPTER IV. F ROM this day natural philosophy, and particularly chemistry, in the most comprehensive sense of the term, became nearly my sole occupation. I read with ardour those works, so full of genius and discrimination, which modern inquirers have written on these subjects. I attended the lectures, and cultivated the acquaintance, of the men of science of the university; and I found even in M. Krempe a great deal of sound sense and real information, combined, it is true, with a repulsive physiognomy and manners, but not on that account the less valuable. In M. Waldman I found a true friend. His gentleness was never tinged by dogmatism; and his instructions were given with an air of frankness and good nature, that banished every idea of pedantry. In a thousand ways he smoothed for me the path of knowledge, and made the most abstruse enquiries clear and facile to my apprehension. My application was at first fluctuating and uncertain; it gained strength as I proceeded, and soon became so ardent and eager, that the stars often disappeared in the light of morning whilst I was yet engaged in my laboratory. cu 10 msColl Chapter 7 It was on a dreary night of November that I beheld the frame on whic Chap. 7 This was the commencement of a nervous fever which confined me for several months. 1818 1818V1 CHAPTER IV CHAPTER IV. I T was on a dreary night of November, that I beheld the accomplishment of my toils. With an anxiety that almost amounted to agony, I collected the instruments of life around me, that I might infuse a spark of being into the lifeless thing that lay at my feet. It was already one in the morning; the rain pattered dismally against the panes, and my candle was nearly burnt out, when, by the glimmer of the half-extinguished light, I saw the dull yellow eye of the creature open; it breathed Thomas CHAPTER IV. I T was on a dreary night of November, that I beheld the accomplishment of my toils. With an anxiety that almost amounted to agony, I collected the instruments of life around me, that I might infuse a spark of being into the lifeless thing that lay at my feet. It was already one in the morning; the rain pattered dismally against the panes, and my candle was nearly burnt out, when, by the glimmer of the half-extinguished light, I saw the dull yellow eye of the creature open; it breathed 1823 CHAPTER IV. I T was on a dreary night of November, that I beheld the accomplishment of my toils. With an anxiety that almost amounted to agony, I collected the instruments of life around me, that I might infuse a spark of being into the lifeless thing that lay at my feet. It was already one in the morning; the rain pattered dismally against the panes, and my candle was nearly burnt out, when, by the glimmer of the half-extinguished light, I saw the dull yellow eye of the creature open; it breathed 1831 CHAPTER V. I T was on a dreary night of November, that I beheld the accomplishment of my toils. With an anxiety that almost amounted to agony, I collected the instruments of life around me, that I might infuse a spark of being into the lifeless thing that lay at my feet. It was already one in the morning; the rain pattered dismally against the panes, and my candle was nearly burnt out, when, by the glimmer of the half-extinguished light, I saw the dull yellow eye of the creature open; it breathed hard, and a convulsive motion agitated its limbs. cu 11 msColl Ch V– 113 He then put the following letter into my hands. "To V. Frankenstien 1818 1818V1 CHAPTER V CHAPTER V. C LERVAL then put the following letter into my hands. Thomas CHAPTER V. C LERVAL then put the following letter into my hands. 1823 CHAPTER V. C LERVAL then put the following letter into my hands. 1831 CHAPTER VI. C LERVAL then put the following letter into my hands. It was from my own Elizabeth:— cu 12 msColl Chap. 8 one of my first duties on my recovery was to introduce Clerval to the 1818 Thomas 1823 1831 cu 13 msColl Chap. 9 On my return I found the following letter from my father. To V.–Frankenstein Chapter 10 Night had closed in Chap. 11 We were soon joined 1818 1818V1 CHAPTER VI CHAPTER VI. O N my return, I found the following letter from my father:— Thomas CHAPTER VI. O N my return, I found the following letter from my father:— 1823 CHAPTER VI. O N , my return, I found the following letter from my father:— 1831 CHAPTER VII. O N my return, I found the following letter from my father:— cu 14 msColl Chapter 1 2 I cannot attempt to describe what I 1818 1818V1 CHAPTER VII CHAPTER VII. W E passed a few sad hours, until eleven o’clock, when the trial was to commence. My father and the rest of the family being obliged to attend as witnesses, I accompanied them to the court. During the whole of this wretched mockery of justice, I suffered living torture. It was to be decided, whether the result of my curiosity and lawless devices would cause the death of two of my fellow-beings: one a smiling babe, full of innocence and joy; the other far more dreadfully murdered, with every aggravation of in volume frontmatter Thomas CHAPTER VII. W E passed a few sad hours, until eleven o’clock, when the trial was to commence. My father and the rest of the family being obliged to attend as witnesses, I accompanied them to the court. During the whole of this wretched mockery of justice, I suffered living torture. It was to be decided, whether the result of my curiosity and lawless devices would cause the death of two of my fellow-beings: one a smiling babe, full of innocence and joy; the other far more dreadfully murdered, with every aggravation of in volume frontmatter 1823 CHAPTER VII. W E passed a few sad hours, until eleven o’clock, when the trial was to commence. My father and the rest of the family being obliged to attend as witnesses, I accompanied them to the court. During the whole of this wretched mockery of justice I suffered living torture. It was to be decided, whether the result of my curiosity and lawless devices would cause the death of two of my fellow-beings: one a smiling babe, full of innocence and joy; the other far more dreadfully murdered, with every aggravation of infamy that could make 1831 CHAPTER VIII. W E passed a few sad hours, until eleven o’clock, when the trial was to commence. My father and the rest of the family being obliged to attend as witnesses, I accompanied them to the court. During the whole of this wretched mockery of justice I suffered living torture. It was to be decided, whether the result of my curiosity and lawless devices would cause the death of two of my fellow-beings: one a smiling babe, full of innocence and joy; the other far more dreadfully murdered, with every aggravation of infamy that could make the murder memorable in horror. Justine also was a girl of merit, and possessed qualities which promised to render her life happy: now all was to be obliterated in an ignominious grave; and I the cause! A thousand times rather would I have confessed myself guilty of the crime ascribed to Justine; but I was absent when it was committed, and such a declaration would have been considered as the ravings of a madman, and would not have exculpated her who suffered through me. cu 15 msColl Chap. 13 Nothing is more painful than 1818 1818V2 CHAPTER I CHAPTER I. N OTHING is more painful to the human mind, than, after the feelings have been worked up by a quick succession of events, the dead calmness of inaction and certainty which follows, and deprives the soul both of hope and fear. Justine died; she rested; and I was alive. The blood flowed freely in my veins, but a weight of despair and remorse pressed on my heart, which Thomas CHAPTER I. N OTHING is more painful to the human mind, than, after the feelings have been worked up by a quick succession of events, the dead calmness of inaction and certainty which follows, and deprives the soul both of hope and fear. Justine died; she rested; and I was alive. The blood flowed freely in my veins, but a weight of despair and remorse pressed on my heart, which 1823 CHAPTER VIII. N OTHING is more painful to the human mind, than, after the feelings have been worked up by a quick succession of events, the dead calmness of inaction and certainty which follows, and deprives the soul both of hope and fear. Justine died; she rested; and I was alive. The blood flowed freely in my veins, but a weight of despair and remorse pressed on my heart, which nothing could remove. Sleep fled from my eyes; I wandered like an evil spirit, for I had committed deeds of mischief 1831 CHAPTER IX. N OTHING is more painful to the human mind, than, after the feelings have been worked up by a quick succession of events, the dead calmness of inaction and certainty which follows, and deprives the soul both of hope and fear. Justine died; she rested; and I was alive. The blood flowed freely in my veins, but a weight of despair and remorse pressed on my heart, which nothing could remove. Sleep fled from my eyes; I wandered like an evil spirit, for I had committed deeds of mischief beyond description horrible, and more, much more (I persuaded myself), was yet behind. Yet my heart overflowed with kindness, and the love of virtue. I had begun life with benevolent intentions, and thirsted for the moment when I should put them in practice, and make myself useful to my fellow-beings. Now all was blasted: instead of that serenity of conscience, which allowed me to look back upon the past with self-satisfaction, and from thence to gather promise of new hopes, I was seized by remorse and the sense of guilt, which hurried me away to a hell of intense tortures, such as no language can describe. cu 16 msColl Chap. 14 he next day, contrary to the prognostics of our guides, was fine although clouded. 1818 1818V2 CHAPTER II CHAPTER II. T HE next day, contrary to the prognostications of our guides, was fine, although clouded. We visited the source of the Arveiron, and rode about the valley until evening. These sublime and magnificent scenes afforded me the greatest consolation that I was capable of receiving. They elevated me from all littleness of feeling; and although they did not remove my grief, they subdued and tranquillized it. In some degree, also, they diverted my mind from the thoughts over which it had Thomas CHAPTER II. T HE next day, contrary to the prognostications of our guides, was fine, although clouded. We visited the source of the Arveiron, and rode about the valley until evening. These sublime and magnificent scenes afforded me the greatest consolation that I was capable of receiving. They elevated me from all littleness of feeling; and although they did not remove my grief, they subdued and tranquillized it. In some degree, also, they diverted my mind from the thoughts over which it had 1823 CHAPTER IX. T HE next day, contrary to the prognostications of our guides, was fine, although clouded. We visited the source of the Arveiron, and rode about the valley until evening. These sublime and magnificent scenes afforded me the greatest consolation that I was capable of receiving. They elevated me from all littleness of feeling; and although they did not remove my grief, they subdued and tranquillized it. In some degree, also, they diverted my mind from the thoughts over which it had 1831 CHAPTER X. I SPENT the following day roaming through the valley. I stood beside the sources of the Arveiron, which take their rise in a glacier, that with slow pace is advancing down from the summit of the hills, to barricade the valley. The abrupt sides of vast mountains were before me; the icy wall of the glacier overhung me; a few shattered pines were scattered around; and the solemn silence of this glorious presence-chamber of imperial Nature was broken only by the brawling waves, or the fall of some vast fragment, the thunder sound of the avalanche, or the cracking, reverberated along the mountains of the accumulated ice, which, through the silent working of immutable laws, was ever and anon rent and torn, as if it had been but a plaything in their hands. These sublime and magnificent scenes afforded me the greatest consolation that I was capable of receiving. They elevated me from all littleness of feeling; and although they did not remove my grief, they subdued and tranquillised it. In some degree, also, they diverted my mind from the thoughts over which it had brooded for the last month. I retired to rest at night; my slumbers, as it were, waited on and ministered to by the assemblance of grand shapes which I had contemplated during the day. They congregated round me; the unstained snowy mountain-top, the glittering pinnacle, the pine woods, and ragged bare ravine; the eagle, soaring amidst the clouds—they all gathered round me, and bade me be at peace. cu 17 msColl Vol. II Chap I "It is with difficulty that I remember the æra of my Chap. 2 As soon as morning dawned I crept from my assylum to 1818 1818V2 CHAPTER III CHAPTER III. “I T is with considerable difficulty that I remember the original æra of my being: all the events of that period appear confused and indistinct. A strange multiplicity of sensations seized me, and I saw, felt, heard, and smelt, at the same time; and it was, indeed, a long time before I learned to distinguish between the operations of my various senses. By degrees, I remember, a stronger light pressed upon my nerves, so that I was obliged to shut my eyes. Darkness then came over me, and troubled me; but Thomas CHAPTER III. “I T is with considerable difficulty that I remember the original æra of my being: all the events of that period appear confused and indistinct. A strange multiplicity of sensations seized me, and I saw, felt, heard, and smelt, at the same time; and it was, indeed, a long time before I learned to distinguish between the operations of my various senses. By degrees, I remember, a stronger light pressed upon my nerves, so that I was obliged to shut my eyes. Darkness then came over me, and troubled me; but 1823 CHAPTER X. “I T is with considerable difficulty that I remember the original æra of my being: all the events of that period appear confused and indistinct. A strange multiplicity of sensations seized me, and I saw, felt, heard, and smelt, at the same time; and it was, indeed, a long time before I learned to distinguish between the operations of my various senses. By degrees, I remember, a stronger light pressed upon my nerves, so that I was obliged to shut my eyes. Darkness then came over me, and troubled me; but hardly had I felt this, when, by opening my eyes, as I now suppose, the light 1831 CHAPTER XI. “I T is with considerable difficulty that I remember the original era of my being: all the events of that period appear confused and indistinct. A strange multiplicity of sensations seized me, and I saw, felt, heard, and smelt, at the same time; and it was, indeed, a long time before I learned to distinguish between the operations of my various senses. By degrees, I remember, a stronger light pressed upon my nerves, so that I was obliged to shut my eyes. Darkness then came over me, and troubled me; but hardly had I felt this, when, by opening my eyes, as I now suppose, the light poured in upon me again. I walked, and, I believe, descended; but I presently found a great alteration in my sensations. Before, dark and opaque bodies had surrounded me, impervious to my touch or sight; but I now found that I could wander on at liberty, with no obstacles which I could not either surmount or avoid. The light became more and more oppressive to me; and, the heat wearying me as I walked, I sought a place where I could receive shade. This was the forest near Ingolstadt; and here I lay by the side of a brook resting from my fatigue, until I felt tormented by hunger and thirst. This roused me from my nearly dormant state, and I ate some berries which I found hanging on the trees, or lying on the cu 18 msColl Chap. 3 It was some time A considerable period elapsed 1818 1818V2 CHAPTER IV CHAPTER IV. “I LAY on my straw, but I could not sleep. I thought of the occurrences of the day. What chiefly struck me was the gentle manners of these people; and I longed to join them, but dared not. I remembered too well the treatment I had suffered the night before from the barbarous villagers, and resolved, whatever course of conduct I might hereafter think it right to pursue, that for the present I would remain quietly in my hovel, watching, and endeavouring to discover the motives which influenced their actions. Thomas CHAPTER IV. “I LAY on my straw, but I could not sleep. I thought of the occurrences of the day. What chiefly struck me was the gentle manners of these people; and I longed to join them, but dared not. I remembered too well the treatment I had suffered the night before from the barbarous villagers, and resolved, whatever course of conduct I might hereafter think it right to pursue, that for the present I would remain quietly in my hovel, watching, and endeavouring to discover the motives which influenced their actions. 1823 CHAPTER XI. “I LAY on my straw, but I could not sleep. I thought of the occurrences of the day. What chiefly struck me was the gentle manners of these people; and I longed to join them, but dared not. I remembered too well the treatment I had suffered the night before from the barbarous villagers, and resolved, whatever course of conduct I might hereafter think it right to pursue, that for the present I would remain quietly in my hovel, watching, and endeavouring to discover the motives which influenced their actions. volume frontmatter 1831 CHAPTER XII. “I LAY on my straw, but I could not sleep. I thought of the occurrences of the day. What chiefly struck me was the gentle manners of these people; and I longed to join them, but dared not. I remembered too well the treatment I had suffered the night before from the barbarous villagers, and resolved, whatever course of conduct I might hereafter think it right to pursue, that for the present I would remain quietly in my hovel, watching, and endeavouring to discover the motives which influenced their actions. cu 19 msColl 1818 1818V2 CHAPTER V CHAPTER V. “I NOW hasten to the more moving part of my story. I shall relate events that impressed me with feelings which, from what I was, have made me what I am. Thomas CHAPTER V. “I NOW hasten to the more moving part of my story. I shall relate events that impressed me with feelings which, from what I was, have made me what I am. 1823 CHAPTER I. “I NOW hasten to the more moving part of my story. I shall relate events, that impressed me with feelings which, from what I had been, have made me what I am. 1831 CHAPTER XIII. “I NOW hasten to the more moving part of my story. I shall relate events, that impressed me with feelings which, from what I had been, have made me what I am. cu 20 msColl another Chapter Sometime elapsed before I became informed of th 1818 1818V2 CHAPTER VI CHAPTER VI. “S OME time elapsed before I learned the history of my friends. It was one which could not fail to impress itself deeply on my mind, unfolding as it did a number of circumstances each interesting and wonderful to one so utterly inexperienced as I was. Thomas CHAPTER VI. “S OME time elapsed before I learned the history of my friends. It was one which could not fail to impress itself deeply on my mind, unfolding as it did a number of circumstances each interesting and wonderful to one so utterly inexperienced as I was. 1823 CHAPTER II. “S OME time elapsed before I learned the history of my friends. It was one which could not fail to impress itself deeply on my mind, unfolding as it did a number of circumstances, each interesting and wonderful to one so utterly inexperienced as I was. 1831 CHAPTER XIV. “S OME time elapsed before I learned the history of my friends. It was one which could not fail to impress itself deeply on my mind, unfolding as it did a number of circumstances, each interesting and wonderful to one so utterly inexperienced as I was. cu 21 msColl Chap. 7 The season winter 1818 1818V2 CHAPTER VII CHAPTER VII. “S UCH was the history of my beloved cottagers. It impressed me deeply. I learned, from the views of social life which it developed, to admire their virtues, and to deprecate the vices of mankind. Thomas CHAPTER VII. “S UCH was the history of my beloved cottagers. It impressed me deeply. I learned, from the views of social life which it developed, to admire their virtues, and to deprecate the vices of mankind. 1823 CHAPTER III. “S UCH was the history of my beloved cottagers. It impressed me deeply. I learned, from the views of social life which it developed, to admire their virtues, and to deprecate the vices of mankind. 1831 CHAPTER XV. “S UCH was the history of my beloved cottagers. It impressed me deeply. I learned, from the views of social life which it developed, to admire their virtues, and to deprecate the vices of mankind. cu 22 msColl Chapt. 8 When my hunger was appeased I directed my steps towards the well known path that conducted to the cottage – All there, The same chapter continued Chap. 9 Chap. 9 I now saw Thus my journey appeared It was evening when I arrived in the outskirts 1818 1818V2 CHAPTER VIII CHAPTER VIII. “C URSED , cursed creator! Why did I live? Why, in that instant, did I not extinguish the spark of existence which you had so wantonly bestowed? I know not; despair had not yet taken possession of me; my feelings were those of rage and revenge. I could with pleasure have destroyed the cottage and its inhabitants, and have glutted myself with their shrieks and misery. Thomas CHAPTER VIII. “C URSED , cursed creator! Why did I live? Why, in that instant, did I not extinguish the spark of existence which you had so wantonly bestowed? I know not; despair had not yet taken possession of me; my feelings were those of rage and revenge. I could with pleasure have destroyed the cottage and its inhabitants, and have glutted myself with their shrieks and misery. 1823 CHAPTER IV. “C URSED , cursed creator! Why did I live? Why, in that instant, did I not extinguish the spark of existence which you had so wantonly bestowed? I know not; despair had not yet taken possession of me; my feelings were those of rage and revenge. I could with pleasure have destroyed the cottage and its inhabitants, and have glutted myself with their shrieks and misery. 1831 CHAPTER XVI. “C URSED , cursed creator! Why did I live? Why, in that instant, did I not extinguish the spark of existence which you had so wantonly bestowed? I know not; despair had not yet taken possession of me; my feelings were those of rage and revenge. I could with pleasure have destroyed the cottage and its inhabitants, and have glutted myself with their shrieks and misery. cu 23 msColl Chap. 9 The creature finished speaking and fixed his eyes looks 1818 1818V2 CHAPTER IX CHAPTER IX. T HE being finished speaking, and fixed his looks upon me in expectation of a reply. But I was bewildered, perplexed, and unable to arrange my ideas sufficiently to understand the full extent of his proposition. He continued— volume frontmatter Thomas CHAPTER IX. T HE being finished speaking, and fixed his looks upon me in expectation of a reply. But I was bewildered, perplexed, and unable to arrange my ideas sufficiently to understand the full extent of his proposition. He continued— volume frontmatter 1823 CHAPTER V. T HE being finished speaking, and fixed his looks upon me in expectation of a reply. But I was bewildered, perplexed, and unable to arrange my ideas sufficiently to understand the full extent of his proposition. He continued— 1831 CHAPTER XVII. T HE being finished speaking, and fixed his looks upon me in expectation of a reply. But I was bewildered, perplexed, and unable to arrange my ideas sufficiently to understand the full extent of his proposition. He continued— cu 24 msColl Chap. 10 Day after day, week after week passed away 1818 1818V3 CHAPTER I CHAPTER I. D AY after day, week after week, passed away on my return to Geneva; and I could not collect the courage to recommence my work. I feared the vengeance of the disappointed fiend, yet I was unable to overcome my repugnance to the task which was enjoined me. I found that I could not compose a female without again devoting several months to profound study and labo Thomas CHAPTER I. D AY after day, week after week, passed away on my return to Geneva; and I could not collect the courage to recommence my work. I feared the vengeance of the disappointed fiend, yet I was unable to overcome my repugnance to the task which was enjoined me. I found that I could not compose a female without again devoting several months to profound study and labo 1823 CHAPTER VI. D AY after day, week after week, passed away on my return to Geneva; and I could not collect the courage to recommence my work. I feared the vengeance of the disappointed fiend, yet I was unable to overcome my repugnance to the task which was enjoined me. I found that I could not compose a female without again devoting several months to profound study and laborious disquisition. I had heard of some discoveries having been made by 1831 CHAPTER XVIII. D AY after day, week after week, passed away on my return to Geneva; and I could not collect the courage to recommence my work. I feared the vengeance of the disappointed fiend, yet I was unable to overcome my repugnance to the task which was enjoined me. I found that I could not compose a female without again devoting several months to profound study and laborious disquisition. I had heard of some discoveries having been made by an English philosopher, the knowledge of which was material to my success, and I sometimes thought of obtaining my father’s consent to visit England for this purpose; but I clung to every pretence of delay, and shrunk from taking the first step in an undertaking whose immediate necessity began to appear less absolute to me. A change indeed had taken place in me: my health, which had hitherto declined, was now much restored; and my spirits, when unchecked by the memory of my unhappy promise, rose proportionably. My father saw this change with pleasure, and he turned his thoughts towards the best method of eradicating the remains of my melancholy, which every now and then would return by fits, and with a devouring blackness overcast the approaching sunshine. At these moments I took refuge in the most perfect solitude. I passed whole days on the lake alone in a little boat, watching the clouds, and listening to the rippling of the waves, silent and listless. But the fresh air and bright sun seldom failed to restore me to some degree cu 25 msColl Chap 11 2. London was Chap 12 Having parted from my friend I deter mined to visit some remote spot of this 1818 1818V3 CHAPTER II CHAPTER II. L ONDON was our present point of rest; we determined to remain several months in this wonderful and celebrated city. Clerval desired the intercourse of the men of genius and talent who flourished at this time; but this was with me a secondary object; I was principally occupied with the means of obtaining the information necessary for the completion of my promise, and quickly availed myself of the letters of introduction that I had brought with me, addressed to the most distinguished natural philosophers. Thomas CHAPTER II. L ONDON was our present point of rest; we determined to remain several months in this wonderful and celebrated city. Clerval desired the intercourse of the men of genius and talent who flourished at this time; but this was with me a secondary object; I was principally occupied with the means of obtaining the information necessary for the completion of my promise, and quickly availed myself of the letters of introduction that I had brought with me, addressed to the most distinguished natural philosophers. 1823 CHAPTER VII. L ONDON was our present point of rest; we determined to remain several months in this wonderful and celebrated city. Clerval desired the intercourse of the men of genius and talent who flourished at this time; but this was with me a secondary object; I was principally occupied with the means of obtaining the information necessary for the completion of my promise, and quickly availed myself of the letters of introduction that I had brought with me, addressed to the most distinguished natural philosophers. 1831 CHAPTER XIX. L ONDON was our present point of rest; we determined to remain several months in this wonderful and celebrated city. Clerval desired the intercourse of the men of genius and talent who flourished at this time; but this was with me a secondary object; I was principally occupied with the means of obtaining the information necessary for the completion of my promise, and quickly availed myself of the letters of introduction that I had brought with me, addressed to the most distinguished natural philosophers. cu 26 msColl Chapter 13 I started forward & exclaimed—Villain, before you sign my death-warrant, be sure that you are yourself safe. I would have seized 1818 1818V3 CHAPTER III CHAPTER III. I SAT one evening in my laboratory; the sun had set, and the moon was just rising from the sea; I had not sufficient light for my employment, and I remained idle, in a pause of consideration of whether I should leave my labour for the night, or hasten its conclusion by an unremitting attention to it. As I sat, a train of reflection occurred to me, which led me to consider the effects of what I was now doing. Three years before I was engaged in the same manner, and had created a fiend whose unparalleled barbarity had desolated my Thomas CHAPTER III. I SAT one evening in my laboratory; the sun had set, and the moon was just rising from the sea; I had not sufficient light for my employment, and I remained idle, in a pause of consideration of whether I should leave my labour for the night, or hasten its conclusion by an unremitting attention to it. As I sat, a train of reflection occurred to me, which led me to consider the effects of what I was now doing. Three years before I was engaged in the same manner, and had created a fiend whose unparalleled barbarity had desolated my 1823 CHAPTER VIII. I SAT one evening in my laboratory; the sun had set, and the moon was just rising from the sea; I had not sufficient light for my employment, and I remained idle, in a pause of consideration of whether I should leave my labour for the night, or hasten its conclusion by an unremitting attention to it. As I sat, a train of reflection occurred to me, which led me to consider the effects of what I was now doing. Three years before I was engaged in the same manner, and had created a fiend whose unparalleled barbarity had desolated my 1831 CHAPTER XX. I SAT one evening in my laboratory; the sun had set, and the moon was just rising from the sea; I had not sufficient light for my employment, and I remained idle, in a pause of consideration of whether I should leave my labour for the night, or hasten its conclusion by an unremitting attention to it. As I sat, a train of reflection occurred to me, which led me to consider the effects of what I was now doing. Three years before I was engaged in the same manner, and had created a fiend whose unparalleled barbarity had desolated my heart, and filled it for ever with the bitterest remorse. I was now about to form another being, of whose dispositions I was alike ignorant; she might become ten thousand times more malignant than her mate, and delight, for its own sake, in murder and wretchedness. He had sworn to quit the neighbourhood of man, and hide himself in deserts; but she had not; and she, who in all probability was to become a thinking and reasoning animal, might refuse to comply with a compact made before her creation. They might even hate each other; the creature who already lived loathed his own deformity, and might he not conceive a greater abhorrence for it when it came before his eyes in the female form? She also might turn with disgust from him to the superior beauty of man; she might quit him, and he be again alone, exasperated by the fresh provocation of being deserted by one of his own species. cu 27 msColl Chap. 14 I was soon int Chap. 15 We were not allowed to converse for any length of time for the precarious state of my health rendered every precaution 1818 1818V3 CHAPTER IV CHAPTER IV. I WAS soon introduced into the presence of the magistrate, an old benevolent man, with calm and mild manners. He looked upon me, however, with some degree of severity; and then, turning towards my conductors, he asked who appeared as witnesses on this occasion. Thomas CHAPTER IV. I WAS soon introduced into the presence of the magistrate, an old benevolent man, with calm and mild manners. He looked upon me, however, with some degree of severity; and then, turning towards my conductors, he asked who appeared as witnesses on this occasion. 1823 CHAPTER IX. I WAS soon introduced into the presence of the magistrate, an old benevolent man, with calm and mild manners. He looked upon me, however, with some degree of severity: and then, turning towards my conductors, he asked who appeared as witnesses on this occasion. 1831 CHAPTER XXI. I WAS soon introduced into the presence of the magistrate, an old benevolent man, with calm and mild manners. He looked upon me, however, with some degree of severity: and then, turning towards my conductors, he asked who appeared as witnesses on this occasion. cu 28 msColl 1818 1818V3 CHAPTER V CHAPTER V. W E had resolved not to go to London, but to cross the country to Portsmouth, and thence to embark for Havre. I preferred this plan principally because I dreaded to see again those places in which I had enjoyed a few moments of tranquillity with my beloved Clerval. I thought with horror of seeing again those persons whom we had been accustomed to visit together, and who might make inquiries concerning an event, the very remembrance of which made me again feel the pang I endured Thomas CHAPTER V. W E had resolved not to go to London, but to cross the country to Portsmouth, and thence to embark for Havre. I preferred this plan principally because I dreaded to see again those places in which I had enjoyed a few moments of tranquillity with my beloved Clerval. I thought with horror of seeing again those persons whom we had been accustomed to visit together, and who might make inquiries concerning an event, the very remembrance of which made me again feel the pang I endured 1823 CHAPTER X. W E had resolved not to go to London, but to cross the country to Portsmouth, and thence to embark for Havre. I preferred this plan principally because I dreaded to see again those places in which I had enjoyed a few moments of tranquillity with my beloved Clerval. I thought with horror of seeing again those persons whom we had been accustomed to visit together, and who might mkae 1831 CHAPTER XXII. T HE voyage came to an end. We landed, and proceeded to Paris. I soon found that I had overtaxed my strength, and that I must repose before I could continue my journey. My father’s care and attentions were indefatigable; but he did not know the origin of my sufferings, and sought erroneous methods to remedy the incurable ill. He wished me to seek amusement in society. I abhorred the face of man. Oh, not abhorred! they were my brethren, my fellow beings, and I felt attracted even to the most repulsive among them, as to creatures of an angelic nature and celestial mechanism. But I felt that I had no right to share their intercourse. I had unchained an enemy among them, whose joy it was to shed their blood, and to revel in their groans. How they would, each and all, abhor me, and hunt me from the world, did they know my unhallowed acts, and the crimes which had their source in me! cu 29 msColl Chap. 16 This letter revived in my memory what I had before forgotten, the mi 1818 Thomas 1823 1831 cu 30 msColl Chap. 17 It was eight o'clock when we landed; we walked 1818 1818V3 CHAPTER VI CHAPTER VI. I T was eight o’clock when we landed; we walked for a short time on the shore, enjoying the transitory light, and then retired to the inn, and contemplated the lovely scene of waters, woods, and mountains, obscured in darkness, yet still displaying their black outlines. Thomas CHAPTER VI. I T was eight o’clock when we landed; 1823 CHAPTER XI. I T was eight o’clock when we landed; we walked for a short time on the shore, enjoying the transitory light, and then retired to the inn, and contemplated the lovely scene of waters, woods, and mountains, obscured in darkness, yet still displaying their black outlines. 1831 CHAPTER XXIII. I T was eight o’clock when we landed; we walked for a short time on the shore, enjoying the transitory light, and then retired to the inn, and contemplated the lovely scene of waters, woods, and mountains, obscured in darkness, yet still displaying their black outlines. cu 31 msColl Chap. 18 Alas! reflection in the 1818 1818V3 CHAPTER VII CHAPTER VII. M Y present situation was one in which all voluntary thought was swallowed up and lost. I was hurried away by fury; revenge alone endowed me with strength and composure; it modelled my feelings, and allowed me to be calculating and calm, at periods when otherwise delirium or death would have been my portion. Thomas CHAPTER VII. M Y present situation was one in which all voluntary thought was swallowed up and lost. I was hurried away by fury; revenge alone endowed me with strength and composure; it modelled my feelings, and allowed me to be calculating and calm, at periods when otherwise delirium or death would have been my portion. 1823 CHAPTER XII M Y present situation was one in which all voluntary thought was swallowed up and lost. I was hurried away by fury; revenge alone endowed me with strength and composure; it moulded my feelings, and allowed me to be calculating and calm, at periods when otherwise delirium or death would have been my portion. 1831 CHAPTER XXIV. M Y present situation was one in which all voluntary thought was swallowed up and lost. I was hurried away by fury; revenge alone endowed me with strength and composure; it moulded my feelings, and allowed me to be calculating cu 32 msColl Walton in continuation. August 1 3 You have read this strange & terrific story, August 1 3 You have read this strange & terrific story, Marg 1818 Thomas 1823 1831 cu 33 msColl September 7 The die is cast. I have consented to return if we are not destroyed. Thus are my hopes blasted– by cowardice and indecision – I come back September 1 2 1818 backmatter Thomas backmatter 1823 backmatter 1831 backmatter

Surveying Frankenstein’s five versions | Source XSLT

Legend

MS

1818

Thm

1823

1831

Alignments, gaps, and comparative lengths of each collation unit

chapter heading or other structural boundary

Mouse over a black box... 

How do we finish this?

  1. Improve and complete the collation​
    • Problem: we rushed the collation refinement to figure out how to deliver the interface!
    • Problem: our lives changed and the pandemic happened; old documentation is patchy
    • Solution: slow, steady review and revision and documentation of our collation algorithms
  2. Test and revise the UX of the navigation and reading interface*
    • *with recursive cycles of error-correction (see stage 1)
       
  3. Document our edition data model for the TEI / Markup community

Revisiting the Gothenberg Model

 

 

Error correction is a cyclical process!

We need to:

  • Inspect our collation results
  • Understand and categorize the kinds of errors we see
  • Revise the Python script OR the collation method
  • Re-run the collation
  • Re-inspect the results
  • Decide when it's okay to hand-correct
  • Minimize hand-correction as much as possible

FV Back-end to Front-end

 

 

 

 

 

 

 

 

 

 

Strengths

  • Data model for production pipeline
  • Collation spine constructs critical edition files in TEI XML
  • Interface delivers collation data for passages in each edition

Weaknesses

  • Error-prone tokenizing problem in the Python feed!
  • Python script is hard to read!
  • Need more legible preparation code

Solutions

  • Rewrite the Python, figure out the tokenization problem
  • Switch to XSLT?: TAN diff (TAN XSLT function library)
  • Try both and see which is better!

Our ongoing collation work

 

 

 

  • Collation projects take much longer to debug than you ever expected
     
  • Correct the input machinery, not the output.
    • Reduce post-processing to correct output errors!
    • Work on the pre-processing.
       
  • Machine-assisted processes need a lot of documentation
    • for project sustainability
    • for reproducibility of data

Adventures in Correcting XML Collation w/ Python and XSLT

By Elisa Beshero-Bondar

Adventures in Correcting XML Collation w/ Python and XSLT

The word by word, comma by comma, and sometimes tag by tag comparison of manuscripts and editions (called “collation”) is notoriously tedious and error-prone. But computer-aided collation is like a power loom that inevitably tangles up threads caught in the machinery. We need new tooling to help us unsnarl the threads. To this point, we aligned variant passages in the Frankenstein Variorum project using a Python script to feed collateX. Now we are experimenting with the Text Alignment Network’s tandiff XSLT to handle the string comparison completely with XPath and XSLT. How far can we take XSLT and Schematron in automating the preparation, collation, and correction of electronic editions?

  • 1,003

More from Elisa Beshero-Bondar