Using Generators to Beat Memory Bloat
By: Korvin Szanto
github.com/korvinszanto
@korvinszanto
php.ug/slackinvite
What is a List?
- Includes arrays and Iterators (Traversable)
- Can be finite or infinite
- You can hint for lists with the `iterable` keyword
Working with Lists
- Each: Iterate over the items in a list
- Map: Alter items in a list using a callback
- Filter: Filter out items in a list using a callback
- Slice: Get a segment of a list
Generators
- Added in PHP 5.5
- They are a compromise
- They are way better
/**
* Run a callback on each item in an array
*/
function each(array $array, callable $callback) {
// Loop over items in an array and send them to a callback
foreach ($array as $key => $item) {
$callback($item, $key);
}
// Pass back the given array
return $array;
}
// Create a finite list
$list = [1, 2, 3, 4];
// For each item in the list, echo out the value
custom_each($list, function($value) {
echo "{$value} - ";
});
/**
* Returns an array with values mapped through a callback
*/
function custom_map(callable $callback, array $array) {
$results = [];
// Run all items in the array through a callable
foreach ($array as $key => $item) {
// Store the result
$results[$key] = $callback($item);
}
// Return the mapped array
return $results;
}
// Create a finite list
$list = [1, 2, 3, 4];
// For each item in the list, echo out the value
$mappedList = custom_map(function($value) {
return $value * 2;
}, $list);
// Output the mapped list as json
echo json_encode($mappedList);
/**
* Filter an array through a callback
*/
function custom_filter(array $array, callable $callback) {
$results = [];
// Run all items in the array through a callable
foreach ($array as $key => $item) {
// Test the item with the filter callback
if ($callback($item)) {
// Store the result
$results[$key] = $item;
}
}
// Return the filtered array
return $results;
}
// Create a finite list
$list = [1, 2, 3, 4];
// For each item in the list, echo out the value
$mappedList = custom_filter($list, function($value) {
// Check if the value is odd
return $value % 2;
});
// Output the mapped list as json
echo json_encode($mappedList);
// Make a list of user id's to check
$users = range(1, 20);// Make a list of user id's to check
$users = range(1, 20);
// Filter out non-numeric stuff
$forceNumeric = 'is_numeric';// Make a list of user id's to check
$users = range(1, 20);
// Filter out non-numeric stuff
$forceNumeric = 'is_numeric';
// Map ids to the user inflate method
$inflateUsers = 'User::inflateByID';
// Make a list of user id's to check
$users = range(1, 20);
// Filter out non-numeric stuff
$forceNumeric = 'is_numeric';
// Map ids to the user inflate method
$inflateUsers = 'User::inflateByID';
// Ensure that we don't have any null values
$forceObjects = 'is_object';
// Make a list of user id's to check
$users = range(1, 20);
// Filter out non-numeric stuff
$forceNumeric = 'is_numeric';
// Map ids to the user inflate method
$inflateUsers = 'User::inflateByID';
// Ensure that we don't have any null values
$forceObjects = 'is_object';
// Filter out non-admins
$onlyAdmins = function(User $user) {
return $user->isAdmin();
};
// Make a list of user id's to check
$users = range(1, 20);
// Filter out non-numeric stuff
$forceNumeric = 'is_numeric';
// Map ids to the user inflate method
$inflateUsers = 'User::inflateByID';
// Ensure that we don't have any null values
$forceObjects = 'is_object';
// Filter out non-admins
$onlyAdmins = function(User $user) {
return $user->isAdmin();
};
// filter, map, filter, filter
$filtered = custom_filter($users, $forceNumeric);
$mapped = custom_map($inflateUsers, $filtered);
$filtered = custom_filter($mapped, $forceObjects);
$admins = custom_filter($filtered, $onlyAdmins);// Make a list of user id's to check
$users = range(1, 20);
// Filter out non-numeric stuff
$forceNumeric = 'is_numeric';
// Map ids to the user inflate method
$inflateUsers = 'User::inflateByID';
// Ensure that we don't have any null values
$forceObjects = 'is_object';
// Filter out non-admins
$onlyAdmins = function(User $user) {
return $user->isAdmin();
};
// filter, map, filter, filter
$admins = custom_filter(
custom_filter(
custom_map(
$inflateUsers,
custom_filter($users, $forceNumeric)),
$forceObjects),
$onlyAdmins);There has to be a better way.
Enter Collections
// filter, map, filter, filter
$admins = custom_filter(
custom_filter(
custom_map(
$inflateUsers,
custom_filter($users, $forceNumeric)),
$forceObjects),
$onlyAdmins);// Turn our list into a Collection object
$users = new Collection($users);
// filter, map, filter, filter
$admins = $users
->filter($forceNumeric)
->map($inflateUsers)
->filter($forceObjects)
->filter($onlyAdmins);CODE
We've done it!
We're working with lists now!
You start thinking...
What happens when
your dataset
starts getting
Bigger?
CODE
This Won't Scale
Enter Generators
/**
* Returns an array with values mapped through a callback
*/
function custom_map(callable $callback, iterable $array) {
$results = [];
// Run all items in the array through a callable
foreach ($array as $key => $item) {
// Store the result
$results[$key] = $callback($item);
}
// Return the mapped array
return $results;
}
/**
* Returns an array with values mapped through a callback
*/
function custom_map(callable $callback, iterable $array) {
// Run all items in the array through a callable
foreach ($array as $key => $item) {
// Yield the result
yield $key => $callback($item);
}
}
Generators are confusing
A function that contains a yield statement works differently
You can only iterate over them once
Once they "close" they cannot be reopened
Certain things are backwards
Iterators are harder
Iterators are annoying
They require a lot of code to do simple things
When they get complex, you probably need multiple iterators instead of one
Wrapping your mind around how they iterate takes time
But.. PHP comes with a ton of them already
Iterators already solve a lot of our problems
CODE
Generator Powered Collections
buttress/collecterator
in composer
github.com/korvinszanto
@korvinszanto
php.ug/slackinvite
https://joind.in/talk/54236
Review this talk
Using Generators to Beat Memory Bloat
By Korvin Szanto
