Skip to content

Commit 4e4ea3b

Browse files
jcpetruzzameta-codesync[bot]
authored andcommitted
Simplify logic for handling testcases that didn't run
Summary: # Context When running a suite, there are various reason why a testcase may end up not running. Most likely because some previous init_per function failed, but it is also possible that the user explicitly skipped it. The test runner needs to figure out what case we are in, and mark it as either a failure or an omission. # Problem The logic to do this is unnecessarily convoluted. We have two functions that do the exact same thing, one calling the other for some cases. The only actual differences are: * One of the two uses `timeout` for the testcase if the init failure was a `timeout`, but we don't emit `timeout` internally, so this is mostly theoretical * The error messages of one of the functions are more clear than the other, but in practice, we use the less clear ones that talk about "results not being recorded" which for the user is going to be rather cryptic # This diff We simplify by keeping only one of the two functions, the one with the more clear error messages. Reviewed By: TD5 Differential Revision: D87076370 fbshipit-source-id: c7e68352a2b2722b5c3f9c62ab5fc3244abfbc81
1 parent 12ce4bb commit 4e4ea3b

File tree

1 file changed

+6
-47
lines changed

1 file changed

+6
-47
lines changed

erlang/common_test/cth_hooks/src/cth_tpx_test_tree.erl

Lines changed: 6 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -367,57 +367,16 @@ get_missing_result(Inits, QualifiedName) ->
367367
details => ~"no results for this test were recorded",
368368
std_out => ""
369369
},
370-
handle_missing_results(Inits, MainResult).
370+
handle_skipped_result(Inits, MainResult).
371371

372372
-doc """
373373
Generates an user informative message in the case of the missing result by attempting to find the right init to blame.
374-
""".
375-
-spec handle_missing_results(Inits :: [method_result()], method_result()) -> method_result().
376-
handle_missing_results([], MainResult) ->
377-
MainResult;
378-
handle_missing_results([Init | Inits], MainResult) ->
379-
InitStdOut = io_lib:format(~"~ts stdout: ~ts", [maps:get(name, Init), maps:get(std_out, Init)]),
380-
case maps:get(outcome, Init) of
381-
failed ->
382-
MainResult#{
383-
details =>
384-
io_lib:format(
385-
~"no results for this test were recorded because init ~ts failed with error message:\n ~ts",
386-
[maps:get(name, Init), maps:get(details, Init)]
387-
),
388374
389-
std_out => InitStdOut
390-
};
391-
timeout ->
392-
MainResult#{
393-
details =>
394-
io_lib:format(
395-
~"no results for this test were recorded because init ~ts timed-out with error message:\n ~ts",
396-
[maps:get(name, Init), maps:get(details, Init)]
397-
),
398-
399-
std_out => InitStdOut
400-
};
401-
skipped ->
402-
handle_skipped_result([Init | Inits], MainResult);
403-
omitted ->
404-
MainResult#{
405-
details =>
406-
io_lib:format(
407-
~"no results for this test were recorded because init ~ts was omitted with message:\n ~ts",
408-
[maps:get(name, Init), maps:get(details, Init)]
409-
),
410-
411-
std_out => InitStdOut
412-
};
413-
passed ->
414-
handle_skipped_result([Init | Inits], MainResult)
415-
end.
416-
417-
%% A result can be erlang skipped if it is either user skipped or skipped because of an init failure.
418-
%% Skip is an error state in tpx. If it is user skipped, the test is reported as omitted, which is not an error state.
419-
%% In the case where it is skipped because of init failure, it is reported as failed with appropriate user message reporting
420-
%% to the init to be blamed.
375+
Notice that an Erlang test-result can be `skipped` if it is either skipped by the user or was skipped because of an init failure.
376+
As `skipped` is an error state in tpx, if it was skipped by the user, the test is reported as omitted, which is not an error state.
377+
In the case where it is skipped because of init failure, it is reported as failed with appropriate user message reporting
378+
to the init to be blamed.
379+
""".
421380
-spec handle_skipped_result(Inits :: [method_result()], MainResult :: method_result()) -> method_result().
422381
handle_skipped_result([], MainResult) ->
423382
MainResult;

0 commit comments

Comments
 (0)