today at 9:41 PM
I am hoping for portable SIMD so much. But I still think that often a manually rolled SIMD will be faster.
Also the state of SIMD in Cranelift is also very WIP. They pretty much just support a subset of 128bit vectors with some rare exceptions.
today at 9:20 PM
I am using my own lib, `lin_alg`, which apes core_simd for floating point values, and extends the concept to vectors and quaternions. I will eventually replace the floating point portions with core::simd upon its arrival in stable Rust.
Downside: It's currently x86 only.
today at 8:12 PM
Hot take: there is no portable SIMD.
You can either have performance (=write manual ASM for each platform), or portability, but not both.
What so-called "portable SIMD" libraries give you is "portable auto-vectorization". "Portable performance" is a global property of the algorithm. Relying on auto-vectorization will result in e.g. sub-optimal register spills in practice. The microbenchmarks will look great, though. ;)
today at 9:31 PM
You've got a point but are overstating it considerably. There is a big gap between just autovectorization and the portable primitives a library like Highway or Fearless SIMD will give you. For example, I haven't seen autovectorization do select or swizzle.
But there's another point in the tradeoff space. One of the explicit design decisions in Fearless SIMD is to support "downcasting," or specialization to a specific microarchitecture. At least for the kind of problems I've worked on, even when you're doing something fancy with arch-specific permutations or what not, the majority of the operations will be pretty vanilla, and can be expressed well in the portable subset.
So you can think of a library like Fearless SIMD as enabling your extreme optimization use case, just more ergonomically.
Of course, this depends on LLVM compiling intrinsics to assembly efficiently. That hasn't always been the case, and is not perfect now (a number of issues have been filed against rustc and LLVM while developing Fearless SIMD), but is pretty good.
As always, though, you do have to measure performance, and I frequently look at the assembler output to double-check that it's doing the right thing. The day of "fire and forget" portable SIMD has not yet arrived.
today at 9:57 PM
The issue with libraries which offer you portable simd is that the auto vectorizer of the compiler will likely generate faster code.
today at 10:29 PM
The autovectorizer afaik rarely emits optimizations for the different vector units to support + efficiently caches the CPUid check to happen once on program start. It’s a good baseline but the continuum (today) is scalar -> auto vectorized -> portable SIMD -> hand rolled explicit. That portable SIMD lets you bridge into hand rolled explicit ergonomically is a power auto-vectorization doesn’t have. Either the compiler does it or doesn’t but you have no way to even have a check that says “fail to build the program if this function isn’t vectorized”. This is important if you’re relying on that property and someone accidentally adds a data dependency and breaks the optimization without you realizing. Portable and explicit SIMD don’t have this problem by definition.
today at 8:30 PM
Getting 2x or 4x performance in your inner loops using a reasonable SIMD library is infinitely better than theoretically getting 8x performance with hand-coded nonportable intrinsics, because the latter is never going to happen in most programs, so the actual point of comparison is scalar code, or autovectorized code at best.
today at 9:34 PM
Sure, in the same way that there is no such thing as portable code at all. The result will be suboptimal, but it will still be better than not having it.
today at 9:21 PM
I agree with that historically auto-vertorization does not seem to work reliably. I'm not sure about your broad claim.
Thoughts on an abstraction over ARM and x86, at 128, 256, and 512-bit widths which, either in a manual or automatic way (The latter more challenging) makes your floating point computations 4-16x faster with minimal restructuring? I think that's doable, and a nice goal of SIMD.
today at 8:23 PM
> there is no portable SIMD
Except in languages with a JIT compiler
today at 9:40 PM
That still just gets you autovectorization, and generally locks you out of the performance you could have with direct SIMD intrinsics.
Granted, the number of cases this distinction matters is relatively small, making a function faster only makes a program appreciably faster if that function is a bottleneck.
today at 8:33 PM
Starts to get a bit philosophical on what constitutes "portable" but JIT compilers would emit an opcode based off of whatever the frontend/IR is saying to do surely?
today at 8:45 PM
It's a continuum. Some things basically all SIMD implementations support. Want to add 2 4xf32 vectors together? That's pretty easy to do portably.
But yeah to be fair if you are at that point, you probably want to go fully non-portable anyway. Especially with AI.
Has anyone even figured out how to do vector stuff (SVE/RVV) without assembly?
today at 8:25 PM
What about numpy, numba, and torch.compile?
today at 8:57 PM
Those are manually optimized per arch, aren't they?
today at 9:43 PM
The libraries, yes, but the code you write is portable (at least until you get into squeezing the last few percent and switch to Triton / Helion in case of GPU, and even those are decently portable).
There's also Halide, where you write the algo but the framework gets you the scheduling and SIMD.
today at 9:28 PM
Can we start using stainless steel instead?
today at 8:05 PM
[flagged]