Programming has all of these little axioms.

Someone observes some kind of behavior from this language or feature, then they try to communicate it, then they make a sweeping generalization that's mostly correct, but when repeating it, nuance tends to get lost.

Heck, I'm probably doing the same right now.

What I also want to talk about is why.

Part of this is to just talk about some neat, overlooked things and tease out more truth out of those axioms.

I'm also trying to figure out something else, some soft skill involved here that's more language-agnostic, since this obviously applies beyond C++.

Virtual Functions

• Virtual Functions in C++ are understood to just be "slow"
• When prompted for an explanation, most people blame the virtual table and the layer of indirection

credit: http://www.drbio.cornell.edu/pl47/programming/TICPP-2nd-ed-Vol-two/html/TicV2.html

Virtual Functions

• The "UberShader" Anecdote that demonstrates this axiom
  • Big particle system project, tech demo for a publisher, design looked great on paper
  • Object-oriented programming, use of inheritance and abstraction clearly delineated how each particle behaved on a conceptual level

Virtual Functions, Cont.

• Performance was poor, about 10 frames per second when actually used in a game
• The performant solution was to throw the OOP approach out the window
• Single particle class with same functionality, but implemented through configuration and composition instead of inheritance
• Still based around an "allergy" to abstraction and virtual functions

Why did that actually work?

• Wrong lesson: "They're just slow!"
• Right lesson: Data locality​ or Cache coherency
  • ​Location of vtable functions for base class not guaranteed to be near each other in memory
  • Fire then Smoke then Water then Fire again instead all Fire, all Water, all Smoke
• If list of particles was well-sorted at runtime, majority of cost (cache misses) would be gone because this would also change data locality

http://gameprogrammingpatterns.com/data-locality.html

Branching Logic

Opposite problem: If I had to guess, people don't think there's a cost, or there's a negligible one

At least, people only looking at software. People that work on hardware see this as a potential optimization gain.

More of an opportunity cost, not that branching is inherently slow. Without branch prediction on the hardware, branching has no cost, but no gain.

Branch Prediction

• "The time that is wasted in case of a branch misprediction is equal to the number of stages in the pipeline from the fetch stage to the execute stage."

Sidenote: Again, I've seen UberShader types of setups also avoid branch prediction. (Also, having one shader avoids state changes on the GPU and changing between active shader types.)

Instead of  branching if/else in a boolean, they usually have some kind of math operation with a toggle in the term.

//branching version, will compile to jump instruction

if(bApplyShaderTerm == true)
{
    texColor += someRandomShaderTerm;
}

//multiplication version -- more cycles, but saves on prediction cost on average
//in case of prediction misses

float applyShaderTerm = 1.0f;
texColor = texColor + someRandomShaderTerm * applyShaderTerm;

Data access

When reading from a hard drive, what's slower:

Reading a 1 MB file, or reading a 1 GB file?

...Or is this even the right question to start from?

Data access

• Two hard drive characteristics: access time and data transfer rate
• Main cost is actually the access time
• Reading from small chunks in separate read operations is, in general, going to be slower than reading the entire binary into RAM