Add annotation to skip all but first test

[matthewmorgan]

I'm not sure if this behavior is desired, but it's pretty common in the other exercism tracks I've seen:

1. Test suite arrives to the user with only the first test enabled. All others are skipped.
2. After getting the first test passing, user 'unskips' the next test.
3. pass test, rinse, repeat...

I implemented the change in check_exercises.py to 'unskip' the skipped tests for CI.

If this isn't wanted, no big deal, I'll just close it. LMK!

Closes #391

NB I made a change to this to use Python file methods instead of calling sed.

[matthewmorgan]

@m-a-ge @behrtam anything you want changed on this PR, or do you intend to merge it as is?

[behrtam]

I don't think this is a good idea. This just pollutes the tests and asks for manual changes of the test suit. This is something that should instead be handled by tools like pytest -x isogram_test.py (fail fast after first) which is already described in the docs.

[matthewmorgan]

@behrtam thanks for your response. I respectfully disagree, and hope you'll consider my point of view.

I'm surprised that you're against the idea of skipping the tests, as it's a common pattern on other exercism tracks (Ruby, Java, JavaScript, C++, EcmaScript for example,) and (IMO) a fundamental piece of the TDD workflow, which is one of the core principles of exercism.

Be that as it may, I do see an open issue in your backlog to 'unskip' optional tests for certain suites. The changes I made to check_exercises.py would accomplish that. If you would prefer to see that change in a separate PR, just let me know and I can do that instead.

[behrtam]

I'm not "against the idea of skipping the tests" quite the contary I love the TDD flow (red - green - refactor). I'm just against polluting the test suit to do this when there are tools that can do this cleaner and nicer for you.

[matthewmorgan]

@behrtam I understand. Do you want me to put the check_exercises.py charges in another PR, or modify this one, or are you not interested in those changes either?

[matthewmorgan]

@behrtam on thinking about this further: I don't agree that adding the skip annotations 'pollutes' the test suite. Quite the contrary-- I think it is declarative, and conveys with better clarity to the user the intention that the tests should be approached and passed one by one.