today at 6:45 PM
This article reminds me of performance advice I was starting to see in the 2000s decade. Basically it was to not introduce a bunch of pointer heavy data structures to get lower algorithmic complexity. Stuff it all into a vector. You will use some algorithms that the computer science textbook will say it's slower, but if it fits all in cache it doesn't matter. The cache misses following pointers all over town hurts you more.
today at 7:40 PM
Or even, stuff it into several parallel vectors (structure of arrays instead of array of structures).
today at 7:39 PM
I too am in the "premature optimization bad" camp.
Beyond the low-hanging fruit like ensuring you aren't creating O(n^2) complexity by accident, I think C++ is fast enough/has mature-enough compilers that by the time you're worrying about cache hits materially affecting performance, you're probably also sufficiently staffed and capitalized to pay people to A/B test that performance.
today at 9:02 PM
I don't know if this advice is strictly advocating to avoid premature optimization. Many problems are modeled intuitively with lots of tiny allocations and pointer heavy structures, and this advice is saying to avoid that.
I think it's more like: prioritize cache locality over big O compexity.
today at 7:48 PM
And how would that staff have learned it ?
today at 9:34 PM
I don’t understand the question. Are you implying someone cannot know how to do something in a particular codebase unless they’ve already done it on that same codebase?
today at 9:10 PM
I don't really agree because it's so hard to reform a full application that's been written without regard to performance, after it's been written. You really need to pay attention from the beginning.
today at 7:54 PM
While this advice isn't wrong, it is misleading. In my benchmarks std::map beats vector after 9 elements. Less than that and linear search is better but branch prediction and cache loading is very good.
Run your own benchmarks on your own data of course. Also map is not considered the best key value store.
today at 8:03 PM
Because in your benchmark all std::map nodes were allocated in succession, most likely being placed in adjacent memory locations...
This likely won't be true in a real application with a non-trivial allocation pattern.
today at 8:45 PM
That might or might not be true in the real world. Often in my applications I'm creating at startup and then referencing later.
Still a custom map that allocated a bunch of nodes would be a useful optimization.
today at 8:45 PM
Actually it really depends, because allocators can also be kind of smart (and you don't have to use the default allocator).
And then, on the other hand - I really doubt GP's map beats a vector, with all of those pointers bins and stuff, in a non-contrived benchmark with 10 elements.
Finally - it's not either-or: There are better hash maps whose memory is sequentially allocated and/or are otherwise cache-aware. And there are data structures geared towards parallel execution on multiple threads; and towards SIMD; etc. etc.
today at 10:24 PM
> We know that the C++ keyword volatile is used to prevent a value from being optimized in a way that would allow the compiler to keep it in a processor register rather than fetching it from its original memory location each time.
Actually, we don't know that. The meaning of volatile is rather subtle
today at 8:15 PM
While we're here, has anyone seen any resources related to data-oriented design when GCs are involved? So much of data-oriented design is arena-focused, but that's not always possible, when the lifetime model of the code requires a GC (for whatever reason).
I feel like the DoD movement is a slow-moving, but big, change through how systems programming is done, but that there's still insufficient material for how to do this in different scenarios. I would really like to apply this more to my areas of work, which are also in C++, but there seems to be a gap between what they're presenting and how it can be applied.
More specifically, I'm using C++ to build a dynamic programming language runtime for a Clojure dialect. That runtime is required to be garbage collected, type-erased, and highly polymorphic. So I surely can't just SoA or AoS everything. Yes, I can pack my data, and I can avoid the GC whenever possible, both in compiler/runtime code and in generated code via escape analysis. But what about everything else, which is the 80% or more of the system? It could be that this runtime is too far at odds with DoD, but I generally see things as a gradient rather than black and white.
today at 4:03 PM
I write in C++ almost every day but never have the need to optimize for speed. Even when you write straightforward code it's already blazingly fast.
today at 4:37 PM
It is probably very domain specific. In robotics for example everything is a zero sum game: CPU, memory bandwidth, GPU, battery life etc ... So it is really a topic, probably true for anything embedded actually. Some other offline applications: HFT, Telco etc.. I wish the GUI apps devs respect more the laptop resources they are running on, don't get me started on the 4 instances of chrome I need to run just for discord, signal etc ...
today at 7:52 PM
GUI engine developers need to trade EVERYTHING for execution time, otherwise JavaScript would simply not be fast enough to handle modern applications.
If your device has enough resources to power V8, modern GUIs are certainly very pleasant and snappier than a more minimal GUI like HN. Otherwise they are horrendous and very laggy.
today at 8:06 PM
I don't know if you got my point. Starting a multi gigabyte machinery for the web just for a chatting app is pure insanity sorry. It will be slow to start, slow to react and a battery hog vs a comparable quality QT app. The worse part is usually people use the web stack for desktop app because they don't want to bother giving a good experience to the people on their own native platform.
today at 9:21 PM
I do wish more things were written in Qt, but Qt Widgets is very unpleasant to work in for a modern dynamic app.
QtQuick/QML/JS is very pleasant and I do wish more people would use it, but from I've seen it's 25-50% the resource use of electron, not some multi-order-of-maginute improvement, do I understand why many people still prefer electron for portability in this case.
today at 7:23 PM
today at 5:33 PM
When writing code for end-user applications, I think it's mostly true. When it's writing code for a database engine, a game engine, a 3d renderer, or anything else that involves heavy data processing, optimization is the core "thing" often and it might not even be a good enough solution without it. Although, a lot of time even then C++ is good enough even then when picking reasonable data structures to represent the data.
today at 8:12 PM
These are what I like to call "infinity applications" where the need for speed is essentially infinite.
Even if you write them in hand-optimized assembly they would still clamor for more speed.
today at 7:35 PM
Then why are you using C++? Java/C#/Go are already fast enough for general application development. Why would you accept the footguns if not for performance?
today at 7:57 PM
In my case, about 5% of our code needs the power of C++. Mixing C++ with any other language is a huge pain. Even if we were using C, mixing C with anything else is a pain, and that's despite being the most supported FFI.
Note that we started our project before Rust was an option. These days I would certainly look at rust to see if that would cover our 5% of the needs but now we have a lot of C++ and mixing rust with C++ is a pain.
today at 8:56 PM
I agree that mixing languages adds complexity. And I say that as someone who routinely does it, because many times it's better to reuse a mature/audited component than rewrite it from scratch.
today at 8:47 PM
> Mixing C++ with any other language is a huge pain.
It is less pain than for most other languages, except for C. The pain is in exposing a C API for your C++ code. Then you build a library and you're set - because basically every language has the ability to call C code. Python, Rust, Java, etc. etc.
today at 8:59 PM
Well once you have a C API, it is relatively easy to call it from any other language.
The painful part is to have to go through a C API (modern languages can express much richer APIs and of course there are different constraints on the different runtimes, e.g. GC).
The annoying part is that each language adds overhead (its runtime). I wouldn't call it painful (I don't have much to do about it), I say "annoying" just because I would rather minimise the amount of code I ship.
today at 8:36 PM
Sometimes it's about the libraries. E.g. writing Computer Vision is nicer in C++ right now (IMHO) because most CV libraries are in C++.
Similarly I like to do video stuff in C just because I call gstreamer/ffmpeg directly in C, rather than having to bridge everything.
today at 5:49 PM
Definitely need to optimize a bit for games and huge scale web apps. We've been finding big optimizations in our app recently. App works without them because we can scale horizontally but cutting CPU usage by 30% by eliminating redundant work and reducing copies of big objects? Why wouldn't we want to do that? This isn't even fancy algorithm stuff, mostly just shoddy initial implementations by 100s of eng working on a codebase over 7 years (not even that old). Stuff like that creeps in.
today at 9:17 PM
It really depends a lot on the domain.
In (soft) realtime audio programming, your audio callback might only have a time budget of 1.3 milliseconds. Everytime you exceed that limit, you'll hear a dropout. That's when you'll start to optimize the hell out of your program :)
today at 9:05 PM
"blazingly fast" as in how much trading rules could you apply to 10Gbit/s stream of stock market data on one core? On one socket?
How 100Gbit NICs could your filter through your stateful firewall at line speed? And with 64 byte packets?
today at 5:05 PM
True, but moving from a list of unique polymorphic pointers to a std::variant gains you at least a 2-3x speed up in terms of TLB and cacheline locality. From there, swapping to SOA will net you another 4-8x, so you're looking at nearly 25x improvement by going data first. That may not matter in the unique case of say, games, where rendering a million entities will dwarf the cost of SIMD processing a million entities, but in something like numerical simulations (fluids) or quant it will be warmly welcomed
today at 5:54 PM
are you sure?
https://stackoverflow.com/questions/69444641/c17-stdvariant-...
today at 6:41 PM
To add to sibling comment...
GCC 11 (2021) std::visit was slower than virtual dispatch.
GCC 12 (2022) optimized std::visit so it can be faster than virtual dispatch.
https://shubhankar-gambhir.github.io/posts/your-stdlib-imple...
today at 6:18 PM
If you take the linked benchmark and use the latest compiler version, the std::variant version is faster. The annoying thing about std::variant (and with some other features of modern C++) is that it generates a bunch of code that the compiler has to optimize away.
today at 6:35 PM
Somehow and for some reason rust enums don’t have this problem and are far more ergonomic and easier to work with (not to mention compile times are amazing). I don’t know exactly why it’s better to have it as a first class language primitive and why the compiler has such a problem with std::visit, but clearly c++ meta programming slows down things in a super linear way such that the compiler has problems both from code gen and then optimization.
today at 5:16 PM
Is the improvement from using std:variant vs polymorphism just due to the indirection you save on?
today at 5:57 PM
That, and it frees the compiler from reasoning about virtual inlining, and that the std::variant approach can pack potentially more than one object into a single cacheline. TLBs also work with 4096 byte pages, so 32 polymorphic 128 byte entities may (at the absolute worst case) use 32 distinct pages which requires 32 TLB virtual translations, while the std::variant one uses 1.
The next step of going SOA benefits from all of the above, it just further unlocks you packed quad and oct instructions (AVX256 and 512 depending if you buy AMD or not).
today at 5:51 PM
This is a sign you might be more productive in a higher-level language.
today at 5:55 PM
And the code might be faster, too, like in that 2005 series of articles by Raymond Chen and Rico Mariani (in which one of them wrote a program in C++ and the other wrote the same program in C#).
https://devblogs.microsoft.com/oldnewthing/20060731-15/?p=30...
https://learn.microsoft.com/en-us/archive/blogs/ricom/perfor...
today at 5:54 PM
This follows Rob Pike’s 5 Rules for Programming
https://web.archive.org/web/20250201145327/https://users.ece...
today at 5:34 PM
I started writing a [CPU-only] 3-D rendering library in C++ recently, after having written the equivalent in C as a proof-of-concept and an experiment. The reason I decided to write it in C++ after C, is not only because I wanted to tap into meta-programming which is facilitated much better with C++, or that I wanted niceties like procedure overloading, but because some things with C or C++ aren't automagically optimised -- like if you want to leverage struct-of-array (SoA) memory layouts because it allows fewer SIMD (AVX in my case) instructions in the rendering pipeline. You do _not_ get that "for free" just writing a single procedure in C++, much less with C. Both languages are layout-sensitive, I mean this is in part what gives you the speed -- optimising with memory layout for cache locality etc. But you have to do it yourself. Meaning that if you need array-of-struct (AoS) or in fact don't know which path the CPU would prefer, there's no other way than roll up your sleeves and one way or another implement both.
The kicker is, in my case I chose C++ because templates allow me to reuse most of the code in the rendering pipeline _regardless_ of whether I go for AoS or SoA layout. I leverage operator overloading to do vector by matrix multiplication which is implemented in both variants. I do have to specify the desired variant during building, but I've profiled and for Intel x86 AVX in my case SoA is something like twice as efficient because I process ("shade") 8 vertices with 4-5 instructions instead of 1 vertex at a time (still shaded with vectorisation -- just "rotated", i.e in the pipeline axis and not vertex buffer axis).
TL;DR; C++ gives you plenty fast by default, but it's not always enough. The difference between 5 and 15 frames per second, well, makes all the difference -- our eyes are only fooled once the frames-per-second rate goes sufficiently up, anything below an acceptable threshold and it's completely different experience. You then either sacrifice resolution or level of detail etc, or decide to squeeze more from the language by helping the compiler.
today at 9:04 PM
today at 4:35 PM
I rarely use C++ but when I do it is for speed. It’s not uncommon that carefully crafted intrinsics can 10x the straightforward naive implementation.
today at 8:03 PM
Depends on your field. When one microsecond is considered "hellishly slow", you might reconsider
today at 7:26 PM
There’s code where there exists a concept of “fast enough,” and code where there is no such thing.
today at 5:36 PM
Then you don't work on a product that has any scale <shrug>.
today at 10:26 PM
[delayed]
today at 4:34 PM
There's no mention of branch prediction, or context switching, or synchronisation. Depending on what you're doing, they could be very consequential. There's only very brief mention of parallelisation with threads and with SIMD.
High-performance programming is a big topic. The scope is far too broad for a single blog post, which naturally gives only cursory discussion of C++ and computer architecture. The article isn't bad considering, but I do think it's the wrong format. A blog series, or even a book, would be more fitting.
today at 5:15 PM
They're a bit old and missing some details, but I like Agner Fog's manuals.
https://www.agner.org/optimize/
today at 5:33 PM
I've not read Fog's Optimizing software in C++ but I see it's freely available there as a PDF (182 pages). Looks like a great resource on these topics.
https://www.agner.org/optimize/optimizing_cpp.pdf
today at 4:59 PM
Learn which instructions SIMD nicely (sqrt / fabs, etc). Use ternaries in loops for masking. Use trig identities and lookup tables (don't recompute sin(3t) when you can use two vector multiples using a table of sin(t) eg. sin(t) * sin(t) * sin(t)). Use divisible constexpr constants in loops to eliminate the SIMD tail. Be careful with type casts and floats. `float x; x += 0.5` will introduce *cvt instructions even if the compiler statically knew better otherwise (use 0.5f). Compile with --fast-math and friends so errno doesn't invalidate your SIMD pipeline.
today at 5:11 PM
Most applications (including most applications that care about numerical performance) should not use -ffast-math.
today at 5:14 PM
That has a similar problem to the article, it's trying to fit far too much into too small a format.
What you've written mostly makes sense to someone who already has a solid understanding of SIMD and of C++ (although I can't say I follow all of it), but the target audience is people who don't. For them, each point needs a much lengthier explanation.
today at 5:36 PM
Likely the best tip would to `objdump -d` and inspect the assembly then checking performance counters. Prepending (__attribute__((used)) will allow you to inspect your functions.
A quick restrict example:
#define fn __attribute__((used))
fn void copy1(int* to, const int* from, const int size)
{
for(int i = 0; i < size; i++)
to[i] = from[i];
}
fn void copy2(int* to, const int* from)
{
constexpr int size = 1024;
for(int i = 0; i < size; i++)
to[i] = from[i];
}
fn void copy3(int* restrict to, const int* restrict from)
{
constexpr int size = 1024;
for(int i = 0; i < size; i++)
to[i] = from[i];
}
gcc test.c -c -O3 && objdump -d ./test.o
This is a good starting point for self teaching. The impact of your TLB, L1, and overall instruction count (with IPC) can further be measured with `./perf stat -d -d -d ./a.out`. If you want a quick rule of thumb, no instructions are fast instructions.
today at 6:35 PM
This seems like domain specific advice.
today at 5:56 PM
Do you have any recommended essential reading for this?
today at 6:56 PM
I'm no expert in this stuff but:
creata's comment [0] mentions the works of Agner Fog, which seem very good, and are freely available.
I haven't read C++ High Performance [1] but it looks like it covers the sorts of topics you'd expect, although it looks like it doesn't cover computer architecture in detail e.g. branch prediction. There are books on that too, of course.
[0] https://news.ycombinator.com/item?id=49868657
[1] https://www.packtpub.com/en-us/product/c-high-performance-97...
today at 3:23 PM
"This article was originally published in Polish in issue 4/2013" — a lot of excellent advice. Sad to see C++ have moved in last decade in a direction that makes writing efficient, simple low level code harder and harder :(
today at 5:13 PM
Writing clear, concise, and efficient code in C++ has never been simpler or easier. The improvements in C++ over the last 15 years have been qualitative.
So many complex, esoteric, and difficult to maintain incantations that used to be required for efficient code generation are no longer necessary.
today at 7:11 PM
How do you process read-only mmaped data in C++, in accordance with the language rules? As an example.
today at 8:07 PM
I think there are lots of Unix APIs that are impossible to use without UB, e.g. SCM_RIGHTS (maybe io_uring as well?).
today at 4:17 PM
I think it has become EASIER: for instance, since C++23 Rust-like move semantics can be used, which provides the compiler with extra information that can be leveraged for the generation of better code.
Or take constexpr - it permits to move computations to compile time that are complex and in older versions either had to be done at runtime, or an ugly workaround had to be used (e.g. assigning a mysterious literal pre-computed in another run or by hand).
today at 4:38 PM
> C++23 Rust-like move semantics can be used
What C++23 feature allows that?
today at 6:29 PM
The closest thing I can think of is trivial relocation [0] which matches the bitwise copy + no destructor on moved-from bits of Rust's moves, but that was only added to the draft for C++26 and was removed late in the process anyways [1].
[0]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2025/p27...
[1]: https://herbsutter.com/2025/11/10/trip-report-november-2025-...
today at 5:16 PM
Maybe this one?
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p22...
today at 6:30 PM
I think that's basically clarifying the conditions under which C++11-style moves can be performed.
today at 3:43 PM
How? you can write exactly the same low level code today.
today at 3:53 PM
My thought too.
There are so many things that are expressible in C++ now that could not be without writing much more code or using per-compilation tools back then. The ability to run code at compile time that is not run at runtime is huge, #embed lets us make other tools output available without linker scripts or compiler specific tools that.
Also, most of the code from the past still works(from 10 years ago definitely works)
today at 4:21 PM
Shot in the dark, but maybe the OP is referring to the fact that these code conventions are explicitly discouraged by the C++ core guidelines. The SoA example falls afoul of the rule requiring T* to be used only for singular object pointers, for example.
today at 4:33 PM
Not a regular C++ programmer but wouldn’t you use std::span here instead? Sure it’ll carry a few redundant lengths but it makes using functions that take spans easier. When I do write C++ it’s usually for speed so I’m often working at the intrinsics level, though AI has gotten good enough at it that I now generally delegate this work to an agent.
today at 5:07 PM
Depending on the specific code, the compiler may even eliminate the redundant lengths.
today at 7:08 PM
eh, yes-ish, but only thanks to compiler writers. at one point committee went all-out enforcing their lifetime model, making bit_cast not an option. If your code accesses same data using different types, you are spelunking ruins with snake pits and lava. more and more stuff needs magic support code in std::, making no-lib code less and less possible (it used to be that with no-rtti and no-exceptions, you could use all c++ features and only needed cxa_at_exit, operator delete, and few other little things. NOT ANY MORE).
On one hand, you have consteval and stuff, letting you FINALLY initialize data at compile time (hey, 20 years late but still!)
on other hand, it is done in most non-debuggable way possible. try setting breakpoint or adding print to constexpr function that causes your requires clause to fail...
so no, newer C++ the language is not possible to use for low level work. The dialects that compiler makers support are. We will see for how long
today at 5:16 PM
My latest C++ project is assessment engine covering various actuarial type things like calculates risk for insurance etc. Typical performance for bulk calculation reaches millions to 10s of millions assessments per second on 16 core server. Well there is a trick there that inside it JIT compiles rules from a DSL to an executable code. interpreter mode (used mainly for audit mode) is about 3-5 times slower which is still insanely fast
today at 6:25 PM
i heard a lot of AI and LLM is in python?
today at 6:50 PM
Python is often used as (very useful!) glue for calling CUDA, C, C++, Fortran, etc… codes.
So, if you are thinking about the sort of “business logic” that’s often Python, but the performance comes from the parts that are usually not.
today at 6:39 PM
Yes, but the numerical backend of the python libraries are written in lower-level languages. For example, pytorch uses a C++ backend.
today at 6:37 PM
You heard correctly. This discussion is about C++ though.
today at 6:32 PM
today at 4:05 PM
[dead]