y2038: tests should not be failing if system date is set to after 2038

[kanavin]

To test the readiness of Yocto stack for Y2038 we run qemu virtual machines with RTC set to some day in 2040. This causes a few of openssl's tests to fail on both 32 bit and 64 bit systems: the reason is that test data, certificates in particular, seem to set their expiry date to 2035 or so.

I would propose to set the expiry date to far enough in the future that it won't have to be tweaked in our lifetimes: this way real Y2038 issues in openssl (or in things it depends on) can be exposed and fixed (it's well possible there are none, but that needs confirmation too).

Failures seen:

Test Summary Report
-------------------
80-test_cmp_http.t               (Wstat: 1280 (exited 5) Tests: 6 Failed: 5)
  Failed tests:  1-5
  Non-zero exit status: 5
80-test_cms.t                    (Wstat: 256 (exited 1) Tests: 16 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
90-test_sslapi.t                 (Wstat: 256 (exited 1) Tests: 3 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1

If there's agreement on this, I can prepare the patch, but I need guidance as to how to regenerate the data correctly (there seem to be no way of doing it with a script from metadata, rather the data is simply stored in git).

[kanavin]

Ping, please.

[levitte]

For anything that's in test/certs, it should be quite easy, just do this:

$ cd test/certs
$ ./setup.sh

Although, I'm not sure that it should be necessary, as the expiry has been set to 100 years forward, and the files in there weren't generated that long ago...

Regarding test_cmp_http and test_cms, it's a different story. The diverse certs are in test/recipes/80-test_cmp_http_data/Mock and test/recipes/80-test_cms_data, unfortunately without a script to recreate those files. @DDvO, you created those, right? Would you mind adding a script to recreate them?

[mattcaswell]

Can you provide verbose test output for the sslapi test failure?

make TESTS=test_sslapi V=1 test

[kanavin]

Can you provide verbose test output for the sslapi test failure?

make TESTS=test_sslapi V=1 test

We package the tests into the target image, and use ./test/run_tests.pl directly. With that, I get:

90-test_sslapi.t ........................ 
    # INFO:  @ /usr/src/debug/openssl/3.2.0/test/helpers/ssltestlib.c:1246
    # SSL_connect() failed -1, 12
    # INFO:  @ /usr/src/debug/openssl/3.2.0/test/helpers/ssltestlib.c:1246
    # SSL_connect() failed -1, 1
    # 403726FFFE7E0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:/usr/src/debug/openssl/3.2.0/ssl/statem/statem_clnt.c:2091:
    # INFO:  @ /usr/src/debug/openssl/3.2.0/test/helpers/ssltestlib.c:1280
    # SSL_accept() failed -1, 1
    # 403726FFFE7E0000:error:0A000415:SSL routines:ssl3_read_bytes:ssl/tls alert certificate expired:/usr/src/debug/openssl/3.2.0/ssl/record/rec_layer_s3.c:861:SSL alert number 45
    # ERROR: (bool) 'create_ssl_connection(serverssl, clientssl, SSL_ERROR_NONE) == true' failed @ /usr/src/debug/openssl/3.2.0/test/sslapitest.c:613
    # false
    # 403726FFFE7E0000:error:0A000197:SSL routines:SSL_shutdown:shutdown while in init:/usr/src/debug/openssl/3.2.0/ssl/ssl_lib.c:2766:
    # 403726FFFE7E0000:error:0A000197:SSL routines:SSL_shutdown:shutdown while in init:/usr/src/debug/openssl/3.2.0/ssl/ssl_lib.c:2766:
    # OPENSSL_TEST_RAND_SEED=-2083212775
    not ok 23 - test_client_cert_verify_cb
# ------------------------------------------------------------------------------
    # ERROR: (bool) 'SSL_build_cert_chain(ssl, SSL_BUILD_CHAIN_FLAG_NO_ROOT | SSL_BUILD_CHAIN_FLAG_CHECK) == true' failed @ /usr/src/debug/openssl/3.2.0/test/sslapitest.c:659
    # false
    # 403726FFFE7E0000:error:0A000086:SSL routines:ssl_build_cert_chain:certificate verify failed:/usr/src/debug/openssl/3.2.0/ssl/ssl_cert.c:1050:Verify error:certificate has expired
    # OPENSSL_TEST_RAND_SEED=-2083212775
    not ok 24 - test_ssl_build_cert_chain
# ------------------------------------------------------------------------------
    # ERROR: (bool) 'SSL_CTX_build_cert_chain(ctx, SSL_BUILD_CHAIN_FLAG_NO_ROOT | SSL_BUILD_CHAIN_FLAG_CHECK) == true' failed @ /usr/src/debug/openssl/3.2.0/test/sslapitest.c:698
    # false
    # 403726FFFE7E0000:error:0A000086:SSL routines:ssl_build_cert_chain:certificate verify failed:/usr/src/debug/openssl/3.2.0/ssl/ssl_cert.c:1050:Verify error:certificate has expired
    # OPENSSL_TEST_RAND_SEED=-2083212775
    not ok 25 - test_ssl_ctx_build_cert_chain
# ------------------------------------------------------------------------------
../../util/wrap.pl ../../test/sslapitest ../../test/certs ../../test/recipes/90-test_sslapi_data/passwd.txt /tmp/yk2HQq8yoQ default ../../test/default.cnf ../../test/recipes/90-test_sslapi_data/dhparams.pem => 1
not ok 1 - running sslapitest
90-test_sslapi.t ........................ 1/? ----------------------------------
90-test_sslapi.t ........................ Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/4 subtests 
	(less 2 skipped subtests: 1 okay)

If that's not verbose enough, I can try again with extra verbosity if you tell me how.