Simplifying Metadata Editing for the Non-Programmer:

Semi-automating metadata documenting using Python's docx Word module

Kim Sundeen

GIS Programmer Analyst, MN Power, MN

work: 218-355-2836218-355-2836

work: ksundeen [at] mnpower.com

personal: kim.h.sundeen [at] gmail.com

@KSundeen

Northern MN GIS Meeting, April 28th, 2016

 

Overview

  • Importance of Metadata
  • Changing our Thinking Towards Metadata
  • Playing Catch-up in Documenting Metadata
  • Using Python for Metadata Import/Export
    • Configuring modules
    • Interactive Demo 

Importance of Metadata?

Isn't it obvious?

 

 

  • knowledge transfer, archiving historical records, record methods, standardize definitions, establish contacts, establish legal limitations for data distribution...etc
  • FGDC:  https://www.fgdc.gov/metadata

 

What are obstacles to simplifying process?

Changing the Culture of Metadata Editing

from FGDC's "Institutionalize Metadata Before It Institutionalizes You" (2005)

Obstacle Potential Solution
Metadata standards are too extensive & difficult to implement Create metadata templates
Metadata production requires time & other resources Identify benefits to individuals who create metadata: 
  1. Reduced data management workload;
  2. Better knowledge of distribution/sharing limitations;
  3. Performance measures
Few tangible benefits & incentives to produce metadata. Technical staff know the data, but management establishes new policies/procedures & allocates resources for policies. "Therefore, those inspired to integrate metadata production must champion an effort to educate administrators." 

How to Play Catch-up with Metadata?

 

Philosophical discussions of new & future policies aside...

 

 

 

 

let's simplify how to document EXISTING GIS layers.

Python Modules Demo

arcpy: http://pro.arcgis.com/en/pro-app/arcpy/get-started/what-is-arcpy-.htm

 

python-docx:  http://python-docx.readthedocs.io/en/latest/index.html

 

arcpy_metadata:  https://github.com/ucd-cws/arcpy_metadata

 

openpyxl:  http://openpyxl.readthedocs.io/en/default/

 

Tips on Installing Python Modules

 

 

  1. Use Windows cmd to navigate to directory where pip was installed.

  2. Then follow instructions to install python packages...

  3. Sometimes doesn't work; try easy install method

 

>pip install python-docx 

 

 

>easy_install python-docx 

 


Thinking Outside the Esri Metadata Box 

  1. How about creating a Word doc formatted using your existing metadata?

  2. How about allowing GIS users to update Word doc/Excel doc instead of any metadata editor?

  3. How about importing updates of Word doc/Excel doc metadata back into Esri's metadata editor?


PseudoCode to Automate

  • 1) Export attribute, domain, subtype, counts, field metadata for review in Excel

  • 2) Export existing metadata into MS Word document for reviewing, updating, & entering new metadata into the Word doc

    • Provides printed version of metadata

  • 3) (Future Addition) - take the newly updated MS Word / Excel file metadata & import directly into Esri's metadata editor.

    • Offers  simple way for multiple users to review & correct metadata.

    • Automates import into metadata editor fields.


PseudoCode to Automate (cont.)

  • 1) Export attribute, domain, subtype, counts, field metadata for review in Excel

Iterating through FC Fields


PseudoCode to Automate (cont.)

  • 1) Export attribute, domain, subtype, counts, field metadata for review in Excel

Iterating through Subtypes


PseudoCode to Automate (cont.)

  • 1) Export attribute, domain, subtype, counts, field metadata for review in Excel

Iterating through Domains


PseudoCode to Automate (cont.)

  • 1) Export attribute, domain, subtype, counts, field metadata for review in Excel

Getting FC Field Value Counts


PseudoCode to Automate (cont.)

  • 1) Export attribute, domain, subtype, counts, field metadata for review in Excel

Initial csv output


PseudoCode to Automate (cont.)

  • 1) Export attribute, domain, subtype, counts, field metadata for review in Excel

then just DATA ->Text to Columns -> Delimited -> Separate by ';'


PseudoCode to Automate (cont.)

  • 2) Export existing metadata into MS Word document for reviewing, updating, & entering new metadata into the Word doc

    • Provides printed version of metadata

Formatting output Word Objects


PseudoCode to Automate (cont.)

  • 2) Export existing metadata into MS Word document for reviewing, updating, & entering new metadata into the Word doc

    • Provides printed version of metadata

Creating Bullet Points for GIS Dataset Names


PseudoCode to Automate (cont.)

  • 3a) (Future Addition)  take the newly updated MS Word doc's metadata and import it directly into Esri's metadata editor.

    • Offers  simple way for multiple users to review & correct metadata.

    • Automates import into metadata editor fields.

 

  • 3b) (Future Addition) or take updated information from Excel & import into metadata editor.


PseudoCode to Automate (cont.)

  • 3a) (Future Addition)  take the newly updated MS Word doc's metadata and import it directly into Esri's metadata editor.

 

Using python's arcpy_metadata module


PseudoCode to Automate (cont.)

  • 3b) (Future Addition) or take updated information from Excel & import into metadata editor.

​

Using python's openpyxl module

Questions?

see python code at GitHub

**will be updated soon​

https://github.com/ksundeen


Made with Slides.com