D8: How to migrate content and be happy

About me

https://www.drupal.org/u/itsekhmistro

https://github.com/itsekhmistro

D7 vs D8

D7:

• Migrate
• Feeds
• Custom

D8:

• Migrate
• Custom

Advantages of Migrate API

• Universal tool for import
• Allows to track changes
  • Hashes
  • Highwater mark
• Has Drush integration
• Flexible Plugins system

Cons

• Only Drush oriented
• Missing advanced examples

Structure

Source

Destination

Process

Migration API

built using Drupal 8 Plugin System

Getting started

• Migrate_tools
• Migrate_plus
• Migrate_example

Practice

WTF?

• Migrations are not plain
• Migrations have dependencies on other migrations
• Advanced cases are not that "well-documented"
• You'd better know how to deal with Paragraphs, Terms, Media

Do it fast

• Prepare Specification
• Prepare Source (and put down an "interface agreement")
• Implement POC
• Add Logs
• Finalize it
• Forget about it

Possible sources

Anything:

• DB
• REST service
• XML
• CSV
• JSON

Specifications

Source DB of another CMS

Download template at the link

Specifications

Source XML file

Download template at the link

Sample Migration

# Migration configuration for offers.
id: foo
label: foo
migration_group: bar
migration_dependencies: {}

destination:
  plugin: entity:node

process:
  type:
    plugin: default_value
    default_value: offer

  title: o_name
  field_description: o_description

  sticky:
    plugin: default_value
    default_value: 0
  uid:
    plugin: default_value
    default_value: 1

source:
  plugin: url
  track_changes: true
  data_fetcher_plugin: file
  data_parser_plugin: xml
  urls: private://import/offer.xml
  item_selector: /client/offer
  ids:
    o_id:
      type: string

  fields:
    -
      name: o_id
      label: 'External id'
      selector: @id
    -
      name: o_name
      label: 'Name'
      selector: titre/text()
    -
      name: o_description
      label: 'Description'
      selector: description/text()

Process plugins, pipeline

"Sometimes, a source value must pass through multiple plugins to end up with the right value and structure for the destination property."

"The second plugin and so on does not need a source as their input is the output of the previous plugin."

Process plugins, pipeline

  field_photos:
    -
      plugin: migration
      migration: program_label
      source: p_labels
      no_stub: true
    -
      plugin: foo
      prefix_source: p_title
      suffix_source: p_labels
      handle_multiples: TRUE
    -
      plugin: iterator
      process:
        target_id: fid
        alt: alt

Hints and tips

#Migration status

$drush ms

#Reset a active migration's status to idle.

$drush migration-reset-status foo

#Reload configuration from .yml config

$drush cim --partial --source=modules/custom/foo/config/install

$drush cr

Maitainance

Hints and tips

#Keep your code clean in foo.install

<?php
/**
 * Implements hook_uninstall().
 */
function migrate_foo_uninstall() {
  // Delete this module's migrations.
  $migrations = [
    'foo',
    'bar',
  ];
  foreach ($migrations as $migration) {
    Drupal::configFactory()
      ->getEditable('migrate_plus.migration.' . $migration)
      ->delete();
  }
}

Uninstall

Hints and tips

#Execute migration (without executing dependencies!).

$drush mi foo

#Execute migration with limited number of items

$drush mi foo --limit=10

#Execute migration limited to Comma-separated list of item IDs to import

$drush mi foo --idlist=45301,45303

#Execute migration (Execute all dependent migrations first.).

$drush mi foo --execute-dependencies

Run

Debug migration

$export XDEBUG_CONFIG="idekey=PHPSTORM"

$drush mi foo --limit=2

See https://www.lullabot.com/articles/debugging-drush-commands-with-xdebug-and-phpstorm

Make sure that param "max. simultaneous connections" is set to 3 or higher

Profile migration

$blackfire run ~/.composer/vendor/bin/drush.php mi foo --limit=2

Profile migration