Need For Speed: part 2

Reduce heap allocations

This talk is not about:

• Reduce the amount of RAM your application requires

• Special allocators (memory pools, arena allocators,...)

Problem

Some C++ features are SO convenient that the user sometimes forget about the cost of heap allocations

Measure first!

• Valgrind / Callgrind
• Perf ( Hotspot )
• Visual Studio
• Intel VTune

Only if you saw that new and delete

take a relevant amount of CPU, try to reduce heap allocations

Tip #1: use Small Object Optimizations

    class String
    {
    public:
       String(const String& s) { ... }
       String(const char* s)   { ... }
    
       // all the methods of std::string
    private:
        union Data {
    
            struct NonSSO {
                char* ptr;             // 8 bytes 
                std::size_t size;      // 8 bytes   
        	std::size_t capacity;  // 8 bytes 
            } non_sso;
    
            struct SSO {
        	char string[ 23 ];     // 23 bytes 
                uint8_t size;          // 1 byte 
            } sso;
    
        } m_data; 
    };

https://github.com/elliotgoodrich/SSO-23

CppCon 2016: Nicholas Ormrod

“The strange details of std::string at Facebook"

Tip #2: use std::move

• Move resources from one instance to another.
• Usually it invalidates the source object.
• Prevent most of the cost of copying and heap allocations
• Particularly useful to insert into a STL containers (and more intuitive than emplace)

(real world example)

We can do better!

  // This function is now 2X faster than the previous one
  void DeserializePointCloud(const PointCloudMessage& msg)
  {
        // recycle an instance already created instance of 
        PointCloud cloud;

        if( buffer.size() >= MAX_BUFFER_SIZE) {
            cloud = std::move( buffer.front() );
        }
         
        while( buffer.size() >= MAX_BUFFER_SIZE) {
            scan_buffer.pop_front();
        }  

        cloud.features = // deserialize from message
        cloud.points.resize( N_POINTS );
        
        for (int i=0; i < N_POINTS; i++)
        {
            cloud.points[i] = // deserialize from message
        }

        buffer.push_back( std::move(cloud) );
   }
        

Tip #3: think about your API

    std::string IntToString( int num );
    int StringToInt( std::string text );

    std::string IntToString( int num );
    int StringToInt( const std::string& text );

    bool IntToString( int num, std::string& output);
    bool StringToInt( const std::string& text, int& number );

Better...

Best... (?)

    void TakesCharStar(const char* s);         // C convention
    void TakesString(const std::string& s);    // C++ convention


    //----------------------------------------------------
    // your code
    const char* char_array = "helloWorld";
    std::string cpp_string("helloWorld");

    TakesCharStar( char_array );         // OK
    TakesCharStar( cpp_string.c_str() ); // ugly ? 

    TakesString( char_array );    // potentially allocate memory
    TakesString( cpp_string );    // OK

Exceptions to tip #3:

Return Value Optimization

Sometime you do not have to worry, because the compiler calls std::move for you under the hood

    std::vector<Noisy> f()
    {
        std::vector<Noisy> v = std::vector<Noisy>(2); 
        return v; 
    }            
     
    void PrintSize(std::vector<Noisy> arg)
    {
        std::cout << "arg.size() = " << arg.size() << '\n';
    }
    
    int main()
    {
        std::vector<Noisy> v = f();    
        PrintSize(v);                          
        return 0;
    }

Guess the output...

Guess the output...

    std::vector<Noisy> f()
    {
        std::vector<Noisy> v = std::vector<Noisy>(2); 
    
        return v; 
    }            
     
    int main()
    {
        std::vector<Noisy> v = f(); 
        return 0;
    }

    std::vector<Noisy> f()
    {
        std::vector<Noisy> v = std::vector<Noisy>(2); 
        return v; 
    }            
     
    void PrintSize(std::vector<Noisy> arg)
    {
        std::cout << "arg.size() = " << arg.size() << '\n';
    }
    
    int main()
    {
        std::vector<Noisy> v = f();                              
        return 0;
    }

Guess the output...