0.3.8 tests returns fatal errors on Skylake with GCC 9.2.0

[akesandgren]

We get multiple fatal errors when running the tests on Skylake systems.
CFLAGS = -O2 -ftree-vectorize -march=native -fno-math-errno

For instance:

 ******* FATAL ERROR - COMPUTED RESULT IS LESS THAN HALF ACCURATE *******
           EXPECTED RESULT   COMPUTED RESULT
       1      0.267033          0.267033    
       2     -0.335365         -0.335365    
       3     -0.308392         -0.308392    
       4      0.347952          0.347952    
       5     -0.476523E-01     -0.476523E-01
       6     -0.200500         -0.200500    
       7      0.276024          0.276024    
       8     -0.416284         -0.416284    
       9      0.419880          0.466533    
      10      0.383916          0.426573    
      11      0.410889          0.456543    
 ******* DGEMV  FAILED ON CALL NUMBER:
   2176: DGEMV ('N', 11,  7, 0.0, A, 12, X, 1, 0.9, Y, 2)         .

 DGBMV  PASSED THE TESTS OF ERROR-EXITS

 ******* FATAL ERROR - COMPUTED RESULT IS LESS THAN HALF ACCURATE *******
           EXPECTED RESULT   COMPUTED RESULT
       1      0.276024          0.276024    
       2     -0.416284         -0.416284    
       3      0.419880          0.419880    
       4      0.872128E-01      0.872128E-01
       5      0.383916          0.383916    
       6      0.168132          0.168132    
       7     -0.344356         -0.344356    
       8     -0.227473         -0.227473    
       9     -0.380320         -0.422577    
      10      0.962038E-01      0.106893    
      11      0.240060          0.266733    
 ******* DGBMV  FAILED ON CALL NUMBER:
   8656: DGBMV ('N', 11,  7,  0,  0, 0.0, A,  2, X, 1, 0.9, Y, 2) .

Any thoughts on that?

[akesandgren]

PS, a build on Broadwell with the same config settings works as it should

[akesandgren]

Currently rebuilding without vectorize...

[lexming]

I confirm the same error on a Intel Xeon Gold 6126 CPU using GCC-9.2.0.

The build command is make BINARY='64' CC='gcc -ftree-vectorize' FC='gfortran -ftree-vectorize' USE_OPENMP='1' USE_THREAD='1'
Specifically, the compilation option triggering those errors is -ftree-vectorize.

List of failed tests:

• DGEMV
• DGBMV
• DSYMV
• DSBMV
• DSPMV
• cblas_dgemv

[akesandgren]

confirmed, removing -ftree-vectorize makes the problem go away.

[akesandgren]

Would be nice if there was a proper fix for this.

[martin-frbg]

Disabling the dgemv_n microkernel for SkylakeX in kernel/x86_64/dgemv_n_4.c would be my first bet. (Unless 0.3.7 or earlier worked with -ftree-vectorize - that file is unchanged from 0.3.4 or so)

[akesandgren]

It didn't work with vectorize before either, just bringing this up again, since we accidentally forgot to turn if off at the first build.

[bartoldeman]

0.3.7 fails too, but not with GCC 8.3.0, only with GCC 9.2.0.

[Diazonium]

Similar issues happened before I think, when a newer GCC version started shuffling/clobbering different registers. @wjc404 did a lot of work on the AVX-512 assembly, maybe he has a better idea what goes wrong.

[bartoldeman]

The test fails with alpha=0, which means it's actually an issue with DSCAL I suspect.

[bartoldeman]

Indeed putting the generic
DSCALKERNEL = ../arm/scal.c
in kernel/x86_64/KERNEL.SKYLAKEX
fixes the failures. Now digging deeper.

[martin-frbg]

Passes with a snapshot of gcc 10 ... possible gcc9 bug ? Trying attribute(no_tree_vectorize) on the dscal_kernel_8 in dscal_microk_skylakex-2.c now... works.
Now why would the gcc tree vectorizer fall over
https://github.com/xianyi/OpenBLAS/blob/cb6ef49857719b64c5f882e32957f5de2fb1d302/kernel/x86_64/dscal_microk_skylakex-2.c#L35-L44 ?

[bartoldeman]

noinline does the trick too. What is weird is that the increment is 2, and if I read the code correctly dscal_kernel_8 should not even be invoked for that case.