FMA instruction

[9il]

Hello,

I am looking for a portable way of vectorized FMA operations for BLAS.

Variant 1

fast attribute

double dot(double[] a, double[] b)
{
    typeof(return) s = 0;

    foreach(size_t i; 0..c.length)
    {
        s = inlineIR!(`
        %p = fmul fast double %0, %1
        %r = fadd fast double %p, %2
        ret double %r`, double)(a[i], b[i], s);
    }

    return s;
}

Result: No FMA

LBB0_8:
    vmovupd -224(%rax), %ymm4
    vmovupd -192(%rax), %ymm5
    vmovupd -160(%rax), %ymm6
    vmovupd -128(%rax), %ymm7
    vmulpd  -224(%rsi), %ymm4, %ymm4
    vmulpd  -192(%rsi), %ymm5, %ymm5
    vmulpd  -160(%rsi), %ymm6, %ymm6
    vmulpd  -128(%rsi), %ymm7, %ymm7
    vaddpd  %ymm0, %ymm4, %ymm0
    vaddpd  %ymm1, %ymm5, %ymm1
    vaddpd  %ymm2, %ymm6, %ymm2
    vaddpd  %ymm3, %ymm7, %ymm3
    vmovupd -96(%rax), %ymm4
    vmovupd -64(%rax), %ymm5
    vmovupd -32(%rax), %ymm6
    vmovupd (%rax), %ymm7
    vmulpd  -96(%rsi), %ymm4, %ymm4
    vmulpd  -64(%rsi), %ymm5, %ymm5
    vmulpd  -32(%rsi), %ymm6, %ymm6
    vmulpd  (%rsi), %ymm7, %ymm7
    vaddpd  %ymm0, %ymm4, %ymm0
    vaddpd  %ymm1, %ymm5, %ymm1
    vaddpd  %ymm2, %ymm6, %ymm2
    vaddpd  %ymm3, %ymm7, %ymm3
    addq    $256, %rsi
    addq    $256, %rax
    addq    $-32, %rdx
    jne LBB0_8

Variant 2

llvm_fmuladd function

double dot(double[] a, double[] b)
{
    typeof(return) s = 0;

    foreach(size_t i; 0..a.length)
    {
        s = llvm_fmuladd(a[i], b[i], s);
    }

    return s;
}

Result: No Vectorization

LBB0_7:
    vmovsd  -56(%rcx), %xmm1
    vmovsd  -48(%rcx), %xmm2
    vfmadd132sd -56(%rax), %xmm0, %xmm1
    vfmadd231sd -48(%rax), %xmm2, %xmm1
    vmovsd  -40(%rcx), %xmm0
    vfmadd132sd -40(%rax), %xmm1, %xmm0
    vmovsd  -32(%rcx), %xmm1
    vfmadd132sd -32(%rax), %xmm0, %xmm1
    vmovsd  -24(%rcx), %xmm0
    vfmadd132sd -24(%rax), %xmm1, %xmm0
    vmovsd  -16(%rcx), %xmm1
    vfmadd132sd -16(%rax), %xmm0, %xmm1
    vmovsd  -8(%rcx), %xmm2
    vfmadd132sd -8(%rax), %xmm1, %xmm2
    vmovsd  (%rcx), %xmm0
    vfmadd132sd (%rax), %xmm2, %xmm0
    addq    $64, %rax
    addq    $64, %rcx
    addq    $-8, %rdx
    jne LBB0_7

[9il]

ping @JohanEngelen

[JohanEngelen]

I will have to figure out how LLVM handles "fast", and how to trigger optimizations... looking through LLVM's testcases now.

[9il]

Thanks!

[JohanEngelen]

Does @target("+fma") help?

[JohanEngelen]

In LLVM tests, the flag -enable-unsafe-fp-math is passed to llc to get the optimization. But we want to apply it per function. I see that some tests pass "unsafe-fp-math"="false" as a function attribute (separate from e.g. "target-features"="+sse,+sse2", so probably @target("+fma") does nothing).

[9il]

so probably @target("+fma") does nothing

Yes

[9il]

But we want to apply it per function.

This is very important for a generic library

[JohanEngelen]

https://github.com/llvm-mirror/clang/blob/master/lib/CodeGen/CGCall.cpp#L1708

So we have to somehow allow people to specify this with LDC. Do you know how clang does this? Can you specify this per function in clang, or only on the commandline ?

[JohanEngelen]

(and how do you think it is best? commandline or per function?)

[9il]

In LLVM level this can be implemented with attribute groups.

commandline or per function

Only per function. For example, mir.las.sum contains summation algorithms, for fast this attribute should be added but not for pairwise. Furthermore both this functions are generic, this means that user flag in dub would affect both of functions whatever module they are located.

[JohanEngelen]

I could add a very generic @(ldc.attributes.llvmattr("...","...")) UDA, that would put attributes on functions so that we don't need maintenance on each new special attribute that LLVM invents. But it would be quite a low-level thing, that is perhaps not portable to different LLVM versions for which the name of an attribute may change.
Random examples from LLVM tests: "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+sse,+sse2" "unsafe-fp-math"="false" "use-soft-float"="false".
In this particular case, it would look like: @(ldc.attributes.llvmattr("unsafe-fp-math","true")).

The other option is to add an ldc.attributes.... special UDA for each thing we want to pass to LLVM.
Something like @(ldc.attributes.unsafeFPMath) might pass these three to LLVM: "no-infs-fp-math"="true" "no-nans-fp-math"="true" "unsafe-fp-math"="true".

@klickverbot @redstar Which solution would you prefer?

[9il]

@(ldc.attributes.llvmattr("...","..."))

Yes, please! This would be awesome for BLAS ^_^

[JohanEngelen]

@9il what other attributes would you want to attach to functions?
I don't know how stable the exact attribute names are in LLVM, and how they apply to all the different targets. If we hide the exact name(s) behind @(ldc.attributes.unsafeFPMath), LDC can apply different LLVM attributes depending on the compile target.

[9il]

@9il what other attributes would you want to attach to functions?
I don't know how stable the exact attribute names are in LLVM, and how they apply to all the different targets. If we hide the exact name(s) behind @(ldc.attributes.unsafeFPMath), LDC can apply different LLVM attributes depending on the compile target

I want as much as possible. I will write code that would generate other code. Note, that this is no big problem with portability because I would define aliases in a configuration file, which would be hidden from user.

Please add ability to define attributes for inlineIR too.

[9il]

BTW, how can i get LLVM version during CT?