Disable hypothesis deadlines

[eric-czech]

I saw a one-off build failure today related to the hypothesis deadlines associated with each test: https://github.com/eric-czech/sgkit/runs/1275478210?check_suite_focus=true.

I believe these should be disabled by convention for our project. I say that because exceeding the 200ms default deadline is not something I've been able to reproduce and there is very little value in hypothesis failing a whole build because one test parameterized one way and run at one point in time on a VM took a little longer. We could increase the deadlines but I don't see much value in that since the execution times probably have some long tail and it would only be a matter of time before it happens with a higher threshold.

I think having a benchmark suite (https://github.com/pystatgen/sgkit/issues/68) is a better way to catch performance problems/regressions, even if we are not benefiting from hypothesis potentially finding parameterizations that are particularly slow.

[eric-czech]

For posterity, the hypothesis numpy integration has been problematic most times that I've tried to use it. I had this problem in the past which was closed but not fixed: HypothesisWorks/hypothesis#2518. In the case with the failing build above, I found that the tests themselves are taking longer than they really should to run but it appears that much of that time is actually related to Hypothesis trying to generate data. Here is the hypothesis statistics summary from the slowest parameterization for the test_max_str_len test:

sgkit/tests/test_utils.py::test_max_str_len[DataArray-5-U]:

  - during reuse phase (0.00 seconds):
    - Typical runtimes: < 1ms, ~ 39% in data generation
    - 3 passing examples, 0 failing examples, 0 invalid examples

  - during generate phase (0.61 seconds):
    - Typical runtimes: 0-29 ms, ~ 73% in data generation
    - 22 passing examples, 0 failing examples, 56 invalid examples

  - Stopped because settings.max_examples=25

This is apparently indicating that 73% of the time that the test takes is dedicated to generating the data. The whole test takes .61 seconds (which adds up across the different parameterizations). I don't understand why it takes hypothesis so long to generate arrays of random strings with at most ~3k elements.

[tomwhite]

I hit this problem too, where jit compilation was causing an intermediate error. This was in #378, but I've now switched to a guvectorize version that doesn't have the same issue.

@eric-czech is this fixed now, or do we have to do something to turn it off globally?

[eric-czech]

I only changed it in the test at https://github.com/pystatgen/sgkit/blob/cf33b0132dc2bafa3f5e492c270443fefc219d23/sgkit/tests/test_utils.py#L167.

Good to know you hit it too. Looks like profiles are the right way to control this globally?