Feeds and the Feeds ecosystem
Hi. I'm Chris.
d.o & irc: twistor
twitter: chrisleppanen
Feeds
Architecture
- Fetcher
- Parser
- Processor
It fetches!
- HTTP
- File/directory upload
- SOAP :(
- LDAP
- Databases
- But wait, there's more.
It parses!
- CSV
- RSS/Atom
- OPML
- Sitemap files?
- XML via XPath
- XML via XSLT
- JSON via JSONPath
- KML
- Slideshare
- Shape files
- ICCD
- iCal
- LDAP entries
- OAI-PMH
- Clients ?!
- REGEX!!!
- Wordpress
- Sharepoint
- Flickr
- XLS
- Excel
- OpenERP
- KML
- Email headers
- FFFFOUND!
- OFX
- Readability
- Goutte
- GDAL
- Filemaker
- Youtube
It processes!
- Nodes
- Users
- Taxonomy terms
-
Data tables
- Comments
- Commerce products
- Contacts
- RedHen contacts
- RedHen organizations
- Locations
- Arbitrary entities
Mapping!
Mapping
Will it do other things?
Yes.
- Schedule periodic imports via cron
- Lookup/create taxonomy terms
- Pubsubhubbub
- Attach to nodes for user created feeds
- Expires old entries
- Has support for most fields
I want it to do more.
Ok fine.
http://drupal.org/project/issues/feeds
There's probably an existing issue.
Maybe even a patch.
Someone might even respond to it.
Support for other modules should go in those modules.
Feeds Tamper
1.0 Woot!
Perfect and bug free.
Let's modify some data
- String to UNIX timestamp
- Unix timestamp to date
- Keyword filter
- Required filter
- Encode/decode HTML entities
- Make URLs absolute
- Strip HTML tags
- Explode a string into an array
- Remove empty items
- Implode an array into a string
- Make a list unique
- Format numbers
- Convert to numbers
- Math!
- Hashes
- Convert to/from country codes
- Convert to/from US states
- Entity Field Query finder
- Rewrite
- Set default value
- Convert case
- Convert to boolean
- Find-replace
- Find-replace with REGEX
- Format a string
- Pad a string
- Trim a string
- URL encode/decode
But that's not enough things!
I hear you.
Plugins!
Declare that you have plugins.
/**
* Implements hook_ctools_plugin_directory().
*/
function my_module_ctools_plugin_directory($module, $plugin) {
if ($module == 'feeds_tamper') {
return 'feeds_tamper_plugins'; // The directory that contains said plugins.
}
}
Weird global $plugin thingy.
<?php
/**
* @file
* Contains my super sweet Feeds Tamper plugin.
*/
$plugin = array(
'form' => 'my_module_plugin_form',
'callback' => 'my_module_callback',
'name' => 'Super sweet tamper plugin',
'multi' => 'loop',
'category' => 'Other',
);
Add a form.
/**
* Configuration form.
*/
function my_module_plugin_callback($importer, $element_key, $settings) {
$form = array();
$form['help'] = array(
'#markup' => t('It does things to stuff.'),
);
return $form;
}
Add the callback.
/**
* Modify things.
*/
function my_module_callback($result, $item_key, $element_key, &$field, $settings, $source) {
$field = "You've been modified.";
}
All together now.
<?php $plugin = array( 'form' => 'my_module_plugin_form', 'callback' => 'my_module_callback', 'name' => 'Super sweet tamper plugin', 'multi' => 'loop', 'category' => 'Other', );
function my_module_plugin_callback($importer, $element_key, $settings) {
$form = array(); $form['help'] = array( '#markup' => t('It does things to stuff.'), ); return $form; }
function my_module_callback($result, $item_key, $element_key, &$field, $settings, $source) { $field = "You've been modified."; }
Let's do something.
Yay for live demos.
http://www.aperfectbrick.com
It has inspiration quotes and we need inspiration.
- Scrape a website's HTML
- Create some article nodes
- Assign some terms
- Import some images
- Make the article title UPPERCASE
- Get all past articles.
What we'll need:
- Feeds
- Feeds Tamper
- Feeds XPath Parser
- Feeds Crawler
Feeds XPath Parser
Feeds Crawler
Allows us to paginate through a site.
Has some modes:
- URL patterns
- XPath for the "next" link
- Auto-mode for Atom rel="next" links
It's demo time
Entity Field Query finder
It's new and it's awesome.
Help!
The future and beyond...
- Rapidly approaching a stable release.
- 8.x - ported, but broken atm
- 3.0 - backport from 8.x that uses the Entity API module exclusively.
- New UI - Are there any designers in the audience?
Links
- https://drupal.org/project/feeds
- https://drupal.org/project/feeds_tamper
- https://drupal.org/project/feeds_xpathparser
- https://drupal.org/project/feeds_crawler
- https://drupal.org/node/856644 (plugin list)
Questions?
Compliments?
Complaints?
Threats?
Feeds and the Feeds ecosystem
By Chris Leppanen
Feeds and the Feeds ecosystem
- 1,293