My profession has made me use C# a whole lot, and I’ve forgotten most of what I had learned in C++, so this article makes for a great refresher for me.

Thanks.

Consider that a reference counter has to be updated every time a reference is created or deleted. If a smart pointer points to object a, and you set it to point to b instead, you have to update the reference counters for both objects. And every update has to be done atomically to ensure thread safety as well. That makes it still more costly. When you add it up like that, it’s actually quite a few CPU cycles that are thrown away updating reference counters.

By comparison, a garbage collector doesn’t have to do anything when references are created, modified or destroyed. It only has to step in when the heap has been filled so much that a memory allocation fails. Once that happens, it has to traverse the graph of live objects, marking each as in use. All dead (nonreachable) objects are never even touched by the GC; so they’re effectively free. Traversing this graph does take a bit of time, as does the heap compaction that typically follows. But because it happens so rarely compared to ref counting, it’s still vastly cheaper overall.

Didn’t know that garbage collectors were less resource-intensive than shared_ptrs; I thought they were about the same, since they did about the same thing.

I didn’t really intend this post to be a guide to “useful” optimizations though. It’s just intended to highlight some of the quirks and complexities of low-level optimization. If anything, it should be taken as a warning that trying to optimize by fiddling with individual ASM instructions is going to cause a lot of headaches, and you’ll see some unexpected results in many cases.

I agree that remembering what the combination of OoO and superscalar can do is important, and easy to forget. Witness Mike Pall and LuaJIT2 for the payoff…

All things considered, cache layout is probably the most practical performance-related “thing” to keep in mind for most programmers. It’s much easier to reason about memory access patterns than asm dependency chains. But of course I wish I had measurements to back my intuition up.

My favorite example of the effect of a (relatively) obscure hardware structure on code is that inline caches make more sense than C++ style vtbls on a processor without a BTB, but the BTB helps the vtbl more than the inline cache. So modern processors are, in effect, optimized for C++-style virtual dispatch!

True, but there are still areas where such micro-optimizations may be useful. First, if you know what the CPU actually does to your code, you can make some more general assumptions about what is fast and what isn’t. Taking the example in my post above, the exact latency might vary, but I doubt you’ll find a CPU where integer multiplication has less latency than addition. So we can still reason about what is preferable inside a tight loop where the combined latency is what’s holding us back. Or knowing that almost every modern CPU is superscalar, we can determine that a higher instruction count isn’t necessarily worse for performance. We can try to split up our dependencies so that subexpressions can be evaluated in parallel.

But of course you’re right, at a certain point we get down to the really CPU-specific stuff, and that’s probably dangerous to rely on, unless you know the exact hardware configuration on which the program is going to run. (You might be targeting a Playstation 3 specifically, for example, and then you don’t care that your optimizations wouldn’t work on other CPU’s)