Range-v3: incoming changes in the STL

Piotr Żelasko

Techmo / AGH-UST

Kraków, 18.07.2017

A few words about me

techmo.pl

dsp.agh.edu.pl

Motivation

Conventions

  • I typically omit the std:: namespace for improved readability.
  • I typically omit the ranges:: namespace when using sub-namespace (e.g. ranges::view:: becomes view::).
  • Headers are all omitted (in order to compile the code you can #include <ranges/v3/all.hpp> or algorithm-specific headers).

All source code is available at https://github.com/pzelasko/range-v3-snippets

STL: quick recap

Examples

vector<int> ints(10);

iota(begin(ints), end(ints), 1);

int sum = accumulate(begin(ints), end(ints), 0);

cout << 0;
for_each(begin(ints), end(ints), [](int n) {
  cout << " + " << n;	
});
cout << " = " << sum;

// console: 
// 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 = 55

Problems addressed by the STL containers and algorithms

Common container interface

// STL containers can be created from a pair of
// begin() and end() iterators, or an initializer_list.
vector<int> x = {0, 2, 1};
list<int>   y = {0, 2, 1};
set<int>    z = {0, 2, 1};

// STL algorithms only care about the Range interface
// i.e. begin() and end() methods which return iterators,
// abstracting away from the container implementation.
cout << distance(begin(x), end(x)) << " " 
     << distance(begin(y), end(y)) << " " 
     << distance(begin(z), end(z)) << "\n";

Standardized iterator properties

  • Input iterator
  • Output iterator
  • Forward iterator
  • Bidirectional iterator
  • Random Access Iterator
  • (C++17) Contiguous Iterator

Single pass

Multi-pass

STL limitations

Example

const vector<int> ints = // initialize vector with numbers

// Filter numbers
vector<int> filtered;
copy_if(begin(ints), end(ints), back_inserter(filtered), is_odd);

// Transform numbers
vector<int> odd_squares;
transform(begin(filtered), end(filtered), back_inserter(odd_squares), square);

// Display numbers
for(int n : odd_squares) {
  cout << n << " ";
}

begin()/end() noise

unnecessary dynamic allocation

not easily composable

Ranges

Ranges TS / Range-v3

The specification based on a proposal to extend the C++ language.

A C++11 library by Eric Niebler which implements Ranges TS + extras

Not necessarily the same as the range-based STL2 introduced in the future

Original proposal (good read!):
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4128.html

Assumptions

Two basic concepts

Iterable (owns the objects)

 



 

Range (doesn't own the objects)

SomeType o;  // SomeType satisfies Iterable

// These calls have to be valid:
auto it1 = begin(o);  // o has begin iterator
auto it2 = end(o);    // o has end iterator (note: may have a different type)
it1 == it2;           // types of it1 and it2 have to be EqualityComparable
OtherType r;  // OtherType satisfies Range

// Same as Iterable, plus:
// r is Constructible
// r is Assignable
// r is Destructible

Namespaces

  • ranges
    (range-based implementations of STL algorithms
    and utilities)
     
  • ranges::view
    ("range adaptors" - lazy ranges and algorithms)
     
  • ranges::action
    (non-lazy algorithms)

Range-v3 examples

Range algorithms

 // Just like std:: algorithms from <algorithm>,
 // but instead of begin() and end() iterators,
 // provide an Iterable (i.e. the container).
 
 vector<int> numbers {10, 5, 7, 2, 9, 8, 1};
 ranges::sort(numbers);  // sorts in-place like std::sort
 
 // numbers contains: [1,2,5,7,8,9,10]

 
 vector<int> numbers2 = {3, 4, 5};
 vector<int> output;
 ranges::set_intersection(numbers, numbers2, ranges::back_inserter(output));
 
 // output contains: [5]

Projections

 struct Person {                                                                                                                                                                                            
     string name;                                                                                                                                                                                           
     int age = 0;                                                                                                                                                                                           
                                                                                                                                                                                                            
     int GetAge() const { return age; }                                                                                                                                                                     
     string GetName() const { return name; }                                                                                                                                                                
 };                                                                                                                                                                                                                                                                                                                                                                                                                     
 ostream &operator<<(ostream &os, const Person &p);                                                                                                                                                                                                                                                                                                                                  
 bool IsNameValid(const string &n);                                                                                                                                                                                           
                                                                                                                                                                                                            
 // Ranges TS brings a new term to STL algorithms: projection.                                                                                                                                          
 // Projection is a callable which "preprocesses" the item                                                                                                                                              
 // before the main algorithm is applied.                                                                                                                                                                                                                                                                                                                                                                       
 vector<Person> people { {"Ann", 57}, {"Jake", 23}, {"John", 53} };                                                                                                                                     
                                                                                                                                                                                                        
 // Equivalent calls                                                                                                                                                                                    
 ranges::sort(people, [](const Person &left, const Person &right) { 
    return left.GetAge() < right.GetAge(); 
 });                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
 ranges::sort(people, std::less<int>{}, [](const Person &p) { return p.GetAge(); });                                                                                                                                                                                                                                                                                                                            
 ranges::sort(people, std::less<int>{}, &Person::GetAge);                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                                                
 // Equivalent calls                                                                                                                                                                                    
 int count  = ranges::count_if(people, [](const Person &p) { 
     return IsNameValid(p.GetName()); 
 });                                                                                                                                                                                                                                                                                                               
 int count_ = ranges::count_if(people, IsNameValid, &Person::GetName);                                                                                                                                  

Range adaptors

 // Unlike std:: algorithms, views are lazily evaluated.
 // They operate on ranges and return lightweight range adaptor objects,
 // which can be copied/moved and composed together.
 // This is accomplished with "smart iterators", 
 // e.g. transform_iterator, filter_iterator
 
 // Lazily-evaluated range of ints [0, 100)
 auto numbers = view::ints(0, 100);
 
 // Range composition (also a lazily-evaluated range)
 auto squares = numbers | view::transform([](int i) { return i * i; });
 
 // Another range composition
 auto odds = numbers | view::filter([](int i) { return i % 2; });
 
 // Lazy range set_intersection algorithm
 auto common = view::set_intersection(squares, odds);
 
 // Limit range evaluation to the first five elements
 auto first_five = view::take(common, 5);
 
 // All computation is done in here [before C++17 use RANGES_FOR macro]
 for (int num : first_five) {
     cout << num << ", ";
 }

Range adaptors

 // Views can be infinite, e.g. ints beginning at 0.
 auto naturals = view::ints(0);
 
 // Views can be piped together in a single expression.
 auto numbers = naturals
     | view::transform(square)
     | view::filter(odd)
     | view::drop(3)
     | view::take(5);
 
 // Or used as a function calls, which can be awkward.
 auto numbers_alt = 
     view::take(
         view::drop(
             view::filter(
                 view::transform(naturals, 
                     square),
                 odd),
             3),
         5);
 
 // Display first range ascending and second descending
 for (int n : numbers) { cout << n << " "; } cout << '\n';
 for (int n : view::reverse(numbers_alt)) { cout << n << " "; } cout << '\n';

Range slices

 // Views can be piped together in a single expression.                                                                                                                                                 
 auto numbers = view::ints(0)                                                                                                                                                                                
     | view::transform(square)                                                                                                                                                                          
     | view::filter(odd);                                                                                                                                                                               
                                                                                                                                                                                                        
 // [49 81 121 169 225 289 361 441 529 625 729 841 961 1089 1225 1369 1521]                                                                                                                             
 auto sliced = numbers | view::slice(3, 20);                                                                                                                                                            
                                                                                                                                                                                                        
 // [49 121 225 361 529 729 961 1225 1521]                                                                                                                                                              
 auto every_second = view::stride(numbers, 2);                                                                                                                                                          
                                                                                                                                                                                                        
 // "Python-like" syntax                                                                                                                                                                                
 for (int n : sliced[{5, end}]) {                                                                                                                                                                       
     cout << n << " ";                                                                                                                                                                                  
 }                                                                                                                                                                                                      
                                                                                                                                                                                                        
 // Other variants:                                                                                                                                                                                     
 sliced[{7, end}];                                                                                                                                                                                      
 sliced[{end - 5, end - 1}];                                                                                                                                                                            
                                                                                                                                                                                                        
 // And a very nice compile error:                                                                                                                                                                      
 //                                                                                                                                                                                                     
 // numbers[{end - 2, end}];                                                                                                                                                                            
 //                                                                                                                                                                                                     
 // error: static assertion failed: Can't index from the end of an infinite range! 

Views and containers

 auto numbers = view::ints(0)
     | view::transform(square)
     | view::filter(odd)
     | view::drop(3)
     | view::take(5);
 
 // Range views are convertible to std:: containers.
 vector<int> vec_        = numbers;
 unordered_set<int> set_ = numbers;
 // (49: 0), (81: 1), (121: 2), (169: 3), (225: 4)
 map<int, int> map_      = view::zip(numbers, view::ints(0));
 
 // But not to an array.
 // int arr[5] = numbers;
 // array<int, 5> arr = numbers;  

Comparison

 int to_drop = 3;
 int to_take = 5;
 vector<int> numbers;
 for (int i = 0; ; ++i) {
     int number = square(i);
     if (odd(number)) {
         if (to_drop > 0) {
             --to_drop;
             continue;
         } else if (to_take-- > 0) {
             numbers.push_back(number);
         } else {
             break;
         }
     }
 }
 vector<int> numbers = view::ints(0)
     | view::transform(square)
     | view::filter(odd)
     | view::drop(3)
     | view::take(5); 

Is there a bug?

Is it obvious?

Range actions

 // Range views can be subject to actions and algorithms,
 // which produce either a single element result,
 // or an Iterable (container) which owns its elements.
 // Actions are evaluated instantly (i.e. not lazily).
 
 // "Reduce" action performed on a range.
 auto sum = ranges::accumulate(numbers, 0);

 // Actions performed on elements of a vector,
 // which do not modify the source (elements are copied).
 // note: ranges::move is analogous, but (obviously) modifies the source.
 auto cpy = vec_ | ranges::copy
     | action::sort
     | action::stride(2)
     | action::transform(add_5);
 
 static_assert(is_same_v<decltype(cpy), vector<int>>);

Range actions

 // Actions performed on a vector in-place.
 vector<int> vec_ = numbers;  // vec_ contains: [49, 81, 121, 169, 225]
 action::sort(vec_);          // vec_ contains: [49, 81, 121, 169, 225]
 action::stride(vec_, 2);        // vec_ contains: [49, 121, 225]
 action::transform(vec_, add_5); // vec_ contains: [54, 126, 230]

 // Alternative syntax:
 vector<int> vec_ = numbers;  // vec_ contains: [49, 81, 121, 169, 225]

 vec_ |= action::sort
      | action::stride(2)
      | action::transform(add_5);

 // vec_ contains: [54, 126, 230]

Ranges compile errors

 auto numbers = view::ints(0)
     | view::transform(square)
     | view::filter(odd)
     | view::drop(3)
     | view::take(5);
 
 // Compile time error
 auto sorted = action::sort(numbers);
 
 // Compiler message (some noise might be around depending on compiler version):

 // error: static assertion failed: 
 // The iterator of the range passed to action::sort 
 // must allow its elements to be permuted; that is, 
 // the values must be movable and the iterator must be mutable.

Dealing with strings

 string str {"tom killed bob"};                                                                                                                                                                         
                                                                                                                                                                                                        
 // [[t,o,m],[k,i,l,l,e,d],[b,o,b]]                                                                                                                                                                     
 auto rng = str | view::split(' ');                                                                                                                                                                     
                                                                                                                                                                                                        
 // [t,o,m,k,i,l,l,e,d,b,o,b]                                                                                                                                                                           
 auto rng2 = rng | view::join;                                                                                                                                                                          
                                                                                                                                                                                                        
 // tomkilledbob                                                                                                                                                                                        
 string rng3 = view::join(rng);                                                                                                                                                                         
                                                                                                                                                                                                        
 // tom killed bob                                                                                                                                                                                      
 string rng3a = view::join(rng, " ");                                                                                                                                                                   
                                                                                                                                                                                                        
 // [tom,killed,bob]                                                                                                                                                                                    
 auto rng4 = rng | view::for_each([](auto r){ return yield(std::string{r}); });                                                                                                                                                                                                                                                                                     
 auto rng5 = rng | view::transform([](auto r){ return std::string{r}; });                                                                                                                               
                                                                                                                                                                                                        
 // [[t,o,m, ,k,i,l,l,e,d, ,b,o,b]]                                                                                                                                                       
 auto rng6 = str | view::split(" ");  // Problems with raw c-string '\0' char                                                                                                                                                                  
                                                                                                                                                                                                        
 // [[t,o,m, ,k,i],[e,d, ,b,o,b]]                                                                                                                                                                       
 auto rng7 = str | view::split(view::c_str("ll"));                                                                                                                                                      
                                                                                                                                                                                                        
 // [[t,o,m],[k,i,l,l,e,d],[b,o,b]]                                                                                                                                                                     
 auto rng8 = str | view::split(" "s);                                                                                                                                                       

Example: vocabulary

auto lines_to_words() {                                                                                                                                                                                    
     return view::transform([](const auto &line){                                                                                                                                                           
         return line                                                                                                                                                                                        
         | view::split(" "s)                                                                                                                                                                                
         | view::transform([](auto r){ return std::string{r}; });                                                                                                                                           
     }) | view::join;                                                                                                                                                                                       
 }                                                                                                                                                                                                          
                                                                                                                                                                                                            
 template <typename Range>                                                                                                                                                                                  
 unordered_set<string> unique_words(Range &&lines) {                                                                                                                                                        
     return lines | lines_to_words();                                                                                                                                                                       
 }                                                                                                                                                                                                          
                                                                                                                                                                                                            
 const vector<string> sents = { "Tom killed Bob", "Bob killed Tom", "Alice is unhappy" };  

 // [unhappy,is,Tom,Bob,killed,Alice]                                                                                                             
 const auto vocab = unique_words(sents);                                                                                                                                                                

Range comprehensions

 // Alternative implementation of lines_to_words()
 auto lines_to_words() {                                                                                                                                                                                    
     return view::for_each([](const auto &line) {                                                                                                                                                           
         return line | view::split(' ') | view::for_each([](auto r) {                                                                                                                                       
             return yield(string{r});                                                                                                                                                                       
         });                                                                                                                                                                                                
     });                                                                                                                                                                                                    
 }     

unconditional

Range comprehensions

 // Find names in sentences
 const vector<string> sents = { "Tom killed Bob", "Bob killed Tom", "Alice is unhappy" };                     
 const vector<string> names = { "Tom", "Bob", "Alice" };                                                                                                                                                

 auto results = view::for_each(names, [&](const auto &name) {                                                                                                                                           
     return view::for_each(sents, [&](const auto &sent) {                                                                                                                                               
         return yield_if(sent.find(name) != string::npos,                                                                                                                                               
                 name + " was found in sentence \"" + sent + "\"");                                                                                                                                     
     });                                                                                                                                                                                                
 });

conditional and nested

Wave streaming demo

Code available at https://github.com/pzelasko/range-v3-snippets

Benchmark

Best of 5 with "-O3 -march=native"

baseline versions: g++5.4, clang3.8

Conclusions

Goes into the standard

  • Range-based algorithm variants
  • Projection arguments
  • Related concepts

I'm not sure about these:

  • Range adaptors (i.e. lazy algorithms)
  • Range comprehensions (basically depends on range adaptors)
  • Actions (additional/alternative algorithms in range-v3)

Source: my understanding of
www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4651.pdf

Pros

  • Composable
  • More readable
  • More semantics with less code
  • Generally easier to reason about (correctness)
  • Maintained (authors respond quickly)

Cons

  • Potentially horrible compile times
  • Very scarce documentation (tests help though)
  • No API stability guarantees
  • Older (?) compilers may crash (g++ 5.4 sometimes did)

Thank you
for your attention

Have a question? Ask away.

Made with Slides.com