Compares involving NaN are done wrong

[tydeman]

Bugzilla Link	43727
Version	8.0
OS	Linux
CC	@topperc,@DougGregor,@zygoloid,@rotateright

Extended Description

Compares (<, <=, >=, >) that involve a floating-point NaN are done wrong with
respect to raising invalid and/or being false. This is on Intel hardware.

[topperc]

I know we don't trap for signalling nans. There is an ongoing effort to support that. But I'd be surprised if we don't return false for nans with those compares. Do you have an example?

[tydeman]

This is NOT a problem with signaling NaNs. It is a problem with quiet NaNs.

#include <stdio.h>
#include <math.h>
#include <float.h>
#include <fenv.h>

int main(void){
if( !(NAN > 0.f) ){
(void)printf("Got expected true branch\n");
}else{
(void)printf("Got bad false branch\n");
}
if( FE_INVALID & fetestexcept( FE_ALL_EXCEPT ) ){
(void)printf("Got expected invalid\n");
}else{
(void)printf("Did not get expected invalid\n");
}

return 0;
}

[rotateright]

See C11 7.6.1/2: "If part of a program tests floating-point status flags, sets floating-point control modes, or runs under non-default mode settings, but was translated with the state for the FENV_ACCESS pragma ‘‘off’’, the behavior is undefined."

...and you should see that clang does not support that pragma if you add it to the source example. So (unfortunately) possibly a duplicate of bug #8472 ?

[arsenm]

Technically need to enable FENV_ACCESS, which is only semi-supported. Duplicate #8472

[llvmbot]

@llvm/issue-subscribers-clang-frontend