Piotr Żelasko
Research scientist at CLSP, John's Hopkins University. PhD @ AGH-UST in Cracow. My interests are automatic speech recognition, natural language processing, C++ and Python, machine learning and deep learning, and jazz music.
Range-v3: incoming changes in the STL
Piotr Żelasko
Techmo / AGH-UST
Kraków, 18.07.2017
A few words about me
techmo.pl
dsp.agh.edu.pl
Motivation
Conventions
- I typically omit the std:: namespace for improved readability.
- I typically omit the ranges:: namespace when using sub-namespace (e.g.
ranges::view:: becomes view::). - Headers are all omitted (in order to compile the code you can #include <ranges/v3/all.hpp> or algorithm-specific headers).
All source code is available at https://github.com/pzelasko/range-v3-snippets
STL: quick recap
Examples
vector<int> ints(10);
iota(begin(ints), end(ints), 1);
int sum = accumulate(begin(ints), end(ints), 0);
cout << 0;
for_each(begin(ints), end(ints), [](int n) {
cout << " + " << n;
});
cout << " = " << sum;
// console:
// 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 = 55Problems addressed by the STL containers and algorithms
Common container interface
// STL containers can be created from a pair of
// begin() and end() iterators, or an initializer_list.
vector<int> x = {0, 2, 1};
list<int> y = {0, 2, 1};
set<int> z = {0, 2, 1};
// STL algorithms only care about the Range interface
// i.e. begin() and end() methods which return iterators,
// abstracting away from the container implementation.
cout << distance(begin(x), end(x)) << " "
<< distance(begin(y), end(y)) << " "
<< distance(begin(z), end(z)) << "\n";Standardized iterator properties
- Input iterator
- Output iterator
- Forward iterator
- Bidirectional iterator
- Random Access Iterator
- (C++17) Contiguous Iterator
Single pass
Multi-pass
STL limitations
Example
const vector<int> ints = // initialize vector with numbers
// Filter numbers
vector<int> filtered;
copy_if(begin(ints), end(ints), back_inserter(filtered), is_odd);
// Transform numbers
vector<int> odd_squares;
transform(begin(filtered), end(filtered), back_inserter(odd_squares), square);
// Display numbers
for(int n : odd_squares) {
cout << n << " ";
}begin()/end() noise
unnecessary dynamic allocation
not easily composable
Ranges
Ranges TS / Range-v3
The specification based on a proposal to extend the C++ language.
A C++11 library by Eric Niebler which implements Ranges TS + extras
Not necessarily the same as the range-based STL2 introduced in the future
Original proposal (good read!):
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4128.html
Assumptions
Two basic concepts
Iterable (owns the objects)
Range (doesn't own the objects)
SomeType o; // SomeType satisfies Iterable
// These calls have to be valid:
auto it1 = begin(o); // o has begin iterator
auto it2 = end(o); // o has end iterator (note: may have a different type)
it1 == it2; // types of it1 and it2 have to be EqualityComparableOtherType r; // OtherType satisfies Range
// Same as Iterable, plus:
// r is Constructible
// r is Assignable
// r is DestructibleNamespaces
- ranges
(range-based implementations of STL algorithms
and utilities)
- ranges::view
("range adaptors" - lazy ranges and algorithms)
- ranges::action
(non-lazy algorithms)
Range-v3 examples
Range algorithms
// Just like std:: algorithms from <algorithm>,
// but instead of begin() and end() iterators,
// provide an Iterable (i.e. the container).
vector<int> numbers {10, 5, 7, 2, 9, 8, 1};
ranges::sort(numbers); // sorts in-place like std::sort
// numbers contains: [1,2,5,7,8,9,10]
vector<int> numbers2 = {3, 4, 5};
vector<int> output;
ranges::set_intersection(numbers, numbers2, ranges::back_inserter(output));
// output contains: [5]Projections
struct Person {
string name;
int age = 0;
int GetAge() const { return age; }
string GetName() const { return name; }
};
ostream &operator<<(ostream &os, const Person &p);
bool IsNameValid(const string &n);
// Ranges TS brings a new term to STL algorithms: projection.
// Projection is a callable which "preprocesses" the item
// before the main algorithm is applied.
vector<Person> people { {"Ann", 57}, {"Jake", 23}, {"John", 53} };
// Equivalent calls
ranges::sort(people, [](const Person &left, const Person &right) {
return left.GetAge() < right.GetAge();
});
ranges::sort(people, std::less<int>{}, [](const Person &p) { return p.GetAge(); });
ranges::sort(people, std::less<int>{}, &Person::GetAge);
// Equivalent calls
int count = ranges::count_if(people, [](const Person &p) {
return IsNameValid(p.GetName());
});
int count_ = ranges::count_if(people, IsNameValid, &Person::GetName);
Range adaptors
// Unlike std:: algorithms, views are lazily evaluated.
// They operate on ranges and return lightweight range adaptor objects,
// which can be copied/moved and composed together.
// This is accomplished with "smart iterators",
// e.g. transform_iterator, filter_iterator
// Lazily-evaluated range of ints [0, 100)
auto numbers = view::ints(0, 100);
// Range composition (also a lazily-evaluated range)
auto squares = numbers | view::transform([](int i) { return i * i; });
// Another range composition
auto odds = numbers | view::filter([](int i) { return i % 2; });
// Lazy range set_intersection algorithm
auto common = view::set_intersection(squares, odds);
// Limit range evaluation to the first five elements
auto first_five = view::take(common, 5);
// All computation is done in here [before C++17 use RANGES_FOR macro]
for (int num : first_five) {
cout << num << ", ";
}
Range adaptors
// Views can be infinite, e.g. ints beginning at 0.
auto naturals = view::ints(0);
// Views can be piped together in a single expression.
auto numbers = naturals
| view::transform(square)
| view::filter(odd)
| view::drop(3)
| view::take(5);
// Or used as a function calls, which can be awkward.
auto numbers_alt =
view::take(
view::drop(
view::filter(
view::transform(naturals,
square),
odd),
3),
5);
// Display first range ascending and second descending
for (int n : numbers) { cout << n << " "; } cout << '\n';
for (int n : view::reverse(numbers_alt)) { cout << n << " "; } cout << '\n';
Range slices
// Views can be piped together in a single expression.
auto numbers = view::ints(0)
| view::transform(square)
| view::filter(odd);
// [49 81 121 169 225 289 361 441 529 625 729 841 961 1089 1225 1369 1521]
auto sliced = numbers | view::slice(3, 20);
// [49 121 225 361 529 729 961 1225 1521]
auto every_second = view::stride(numbers, 2);
// "Python-like" syntax
for (int n : sliced[{5, end}]) {
cout << n << " ";
}
// Other variants:
sliced[{7, end}];
sliced[{end - 5, end - 1}];
// And a very nice compile error:
//
// numbers[{end - 2, end}];
//
// error: static assertion failed: Can't index from the end of an infinite range!
Views and containers
auto numbers = view::ints(0)
| view::transform(square)
| view::filter(odd)
| view::drop(3)
| view::take(5);
// Range views are convertible to std:: containers.
vector<int> vec_ = numbers;
unordered_set<int> set_ = numbers;
// (49: 0), (81: 1), (121: 2), (169: 3), (225: 4)
map<int, int> map_ = view::zip(numbers, view::ints(0));
// But not to an array.
// int arr[5] = numbers;
// array<int, 5> arr = numbers;
Comparison
int to_drop = 3;
int to_take = 5;
vector<int> numbers;
for (int i = 0; ; ++i) {
int number = square(i);
if (odd(number)) {
if (to_drop > 0) {
--to_drop;
continue;
} else if (to_take-- > 0) {
numbers.push_back(number);
} else {
break;
}
}
}
vector<int> numbers = view::ints(0)
| view::transform(square)
| view::filter(odd)
| view::drop(3)
| view::take(5); Is there a bug?
Is it obvious?
Range actions
// Range views can be subject to actions and algorithms,
// which produce either a single element result,
// or an Iterable (container) which owns its elements.
// Actions are evaluated instantly (i.e. not lazily).
// "Reduce" action performed on a range.
auto sum = ranges::accumulate(numbers, 0);
// Actions performed on elements of a vector,
// which do not modify the source (elements are copied).
// note: ranges::move is analogous, but (obviously) modifies the source.
auto cpy = vec_ | ranges::copy
| action::sort
| action::stride(2)
| action::transform(add_5);
static_assert(is_same_v<decltype(cpy), vector<int>>);Range actions
// Actions performed on a vector in-place.
vector<int> vec_ = numbers; // vec_ contains: [49, 81, 121, 169, 225]
action::sort(vec_); // vec_ contains: [49, 81, 121, 169, 225]
action::stride(vec_, 2); // vec_ contains: [49, 121, 225]
action::transform(vec_, add_5); // vec_ contains: [54, 126, 230]
// Alternative syntax:
vector<int> vec_ = numbers; // vec_ contains: [49, 81, 121, 169, 225]
vec_ |= action::sort
| action::stride(2)
| action::transform(add_5);
// vec_ contains: [54, 126, 230]
Ranges compile errors
auto numbers = view::ints(0)
| view::transform(square)
| view::filter(odd)
| view::drop(3)
| view::take(5);
// Compile time error
auto sorted = action::sort(numbers);
// Compiler message (some noise might be around depending on compiler version):
// error: static assertion failed:
// The iterator of the range passed to action::sort
// must allow its elements to be permuted; that is,
// the values must be movable and the iterator must be mutable.
Dealing with strings
string str {"tom killed bob"};
// [[t,o,m],[k,i,l,l,e,d],[b,o,b]]
auto rng = str | view::split(' ');
// [t,o,m,k,i,l,l,e,d,b,o,b]
auto rng2 = rng | view::join;
// tomkilledbob
string rng3 = view::join(rng);
// tom killed bob
string rng3a = view::join(rng, " ");
// [tom,killed,bob]
auto rng4 = rng | view::for_each([](auto r){ return yield(std::string{r}); });
auto rng5 = rng | view::transform([](auto r){ return std::string{r}; });
// [[t,o,m, ,k,i,l,l,e,d, ,b,o,b]]
auto rng6 = str | view::split(" "); // Problems with raw c-string '\0' char
// [[t,o,m, ,k,i],[e,d, ,b,o,b]]
auto rng7 = str | view::split(view::c_str("ll"));
// [[t,o,m],[k,i,l,l,e,d],[b,o,b]]
auto rng8 = str | view::split(" "s);
Example: vocabulary
auto lines_to_words() {
return view::transform([](const auto &line){
return line
| view::split(" "s)
| view::transform([](auto r){ return std::string{r}; });
}) | view::join;
}
template <typename Range>
unordered_set<string> unique_words(Range &&lines) {
return lines | lines_to_words();
}
const vector<string> sents = { "Tom killed Bob", "Bob killed Tom", "Alice is unhappy" };
// [unhappy,is,Tom,Bob,killed,Alice]
const auto vocab = unique_words(sents);
Range comprehensions
// Alternative implementation of lines_to_words()
auto lines_to_words() {
return view::for_each([](const auto &line) {
return line | view::split(' ') | view::for_each([](auto r) {
return yield(string{r});
});
});
}
unconditional
Range comprehensions
// Find names in sentences
const vector<string> sents = { "Tom killed Bob", "Bob killed Tom", "Alice is unhappy" };
const vector<string> names = { "Tom", "Bob", "Alice" };
auto results = view::for_each(names, [&](const auto &name) {
return view::for_each(sents, [&](const auto &sent) {
return yield_if(sent.find(name) != string::npos,
name + " was found in sentence \"" + sent + "\"");
});
});conditional and nested
Wave streaming demo
Code available at https://github.com/pzelasko/range-v3-snippets
Benchmark
Best of 5 with "-O3 -march=native"
baseline versions: g++5.4, clang3.8
Conclusions
Goes into the standard
- Range-based algorithm variants
- Projection arguments
- Related concepts
I'm not sure about these:
- Range adaptors (i.e. lazy algorithms)
- Range comprehensions (basically depends on range adaptors)
- Actions (additional/alternative algorithms in range-v3)
Source: my understanding of
www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4651.pdf
Pros
- Composable
- More readable
- More semantics with less code
- Generally easier to reason about (correctness)
- Maintained (authors respond quickly)
Cons
- Potentially horrible compile times
- Very scarce documentation (tests help though)
- No API stability guarantees
- Older (?) compilers may crash (g++ 5.4 sometimes did)
Thank you
for your attention
Have a question? Ask away.
Range-v3: incoming changes in the STL
By Piotr Żelasko
Range-v3: incoming changes in the STL
Presentation about range-v3 library for a C++ User Group Kraków meetup
