Formalize the process for deprecating exercises

[kytrinyx]

Sometimes we need to deprecate exercises. For example:

• we decide that the exercise is not interesting enough.
• we want to rename the exercise (deprecating the old name).
• we realize that there are two exercises that are too similar, and want to deprecate one of them.

Probably other good reasons, too.

Anyway, doing this is a bit tricky, because:

• several language tracks typically implement any given exercise. We want to be able to deprecate things without creating dependencies between tracks, or making it tricky to deploy (or easy to break).
• if people have submitted a solution then this affects what they are able to see. If we are renaming an exercise, they should see all the solutions that come in with the new name. Also, people who are solving the new version of the exercise should see solutions that were solved against the old version. We have previously written migrations to rename the old solutions to the new ones, and update the access control lists (acls table in the database).

Note that when we deprecate an exercise, it means that we don't automatically deliver it when someone says exercism fetch $TRACK_ID. However, people can always fetch it explicitly by saying exercism fetch $TRACK_ID $PROBLEM_SLUG. If they submit it, then it will go to the site. That said, nobody will see it, unless we migrate it.

We should probably come up with a reproducible step-by-step process for deprecating problems and document it.

[kytrinyx]

Relevant issues:

• The protein-translation and nucleotide-codons exercises are the same problem-specifications#268
• Almost identical exercises: binary, trinary, octal, hexadecimal problem-specifications#276
• Duplicate Hamming distance exercises problem-specifications#184

[NobbZ]

So things I do see, which needs to be altered in the overal progress:

• Old exercise needs to be mapped to the new one, but nut necesseraly¹
• When an exercise gets consolidated like hamming and point-mutations before, we need a mechanism that merges iterations if both have been available in a single track. To not let things happen that might probably happened in My progress of Common lisp is 30/29 (103%)  common-lisp#119.
• When trying to fetch a deprecated version, the CLI should error out and give an error message that explains how to fetch the replacement or how to send the old thing under the new name (do not try to resolve transparently, that will cause new confusion!). For older clients which don't support displaying this info, we need at least a “exercise not available” and if possible a “please update to newest CLI”

This are the things I do see right now which we need to handle at least, perhaps other people see other things?

I can't tell much about how to do that stuff, since I am not involved in exercisms codebase and architecture at all. And also I am missing the necessary spare time to read into the source :(

1: In the process of adding all-your-base (exercism/problem-specifications#279) we need to deprecate 4 exercises which are specialised versions of the new one, and having solved the old ones wouldn't make one able to solve the replacements, as I already said in the discussion about all-your-base, it is more difficult to solve than the old exercises.

[ErikSchierboom]

@NobbZ has some very good point. I think that the CLI should indeed error.

I don't really see how to merge iterations, the exercises are not exactly the same of course.

[NobbZ]

Merging submissions and iterations is only valid for simple renames and exercises which happens to be the same (as happened with hamming and point-mutations). Of course we can't do that with the current all-your-base monster. Still it is something to consider in the overal process.

A thing that really bothers me, is that in situation as with all-your-base former submissions could be totally lost.

I will generalize it a bit and describe the overal situation that bothers me:

• User U1 submits solutions to X1 and X2.
• Both gets deprecated in favor of X3.
• User U1 decides that X3 is so similar to X1 and X2, that he simply skips X3.
• User U2 solves X3.
• User U2 searches for other solutions to X3.

What will U2 see, assuming that only this two users exist?

• An empty list, since he was the first who solved X3? Very a bad thing.
• One of X1 and X2? But which exactly and who decides? What if another User U3 has only solved that one that is not displayed?
• A list including all sumbissions that have been deprecated by X3 (recursively!) where deprecated ones are marked by a slightly different color or a deprecation icon in the list, as well as a banner on the submissions page showing it is deprecated by X3, additionally a link to a full explanation of deprecation.

I really like that last one. It remains full discoverability of all submissions that have been happened so far and will be happen in the future. Also it does ensure that on deprecation of a deprecator its deprecationees(?) are still findable.

[Insti]

An empty list, since he was the first who solved X3

This does not seem unreasonable to me. Any new problem will have a point at which it has no solutions. The readme for X3 should reference X1 and X2 which should still be individually accessible if the user wishes to go hunting for solutions.

[NobbZ]

OK, but if we reference X1 and X2 in the readme of X3, we need to maintain them as completely separate and unrelated activity streams, which weakens the meaning of a deprecation.

For me "deprecating X in favor of Y" can mean 2 things. Either we pretend X has never existed and we delete every reference to it, perhaps even poping it from users that solved it while not giving credits ofr Y, effectively forcing them to do Y again. The other way to read this, is to value former applications of X and tell on that pages that there is now Y prefered. This makes transition much more smooth (it can happen before all tracks have implemented Y) and values the time and effort people have already put on X.

[kytrinyx]

What if we make it so we don't translate old exercises, but automatically give people access to both the deprecated and the new exercise:

• people who solved the old exercise would be given access to the new exercise
• when someone solves the new exercise, they're given access to both the new one and the deprecated ones

[NobbZ]

Which is more or less what I've said in

• A list including all sumbissions that have been deprecated by X3 (recursively!) where deprecated ones are marked by a slightly different color or a deprecation icon in the list, as well as a banner on the submissions page showing it is deprecated by X3, additionally a link to a full explanation of deprecation.

I think it is really important to not have any magic or silent conversion happening there but to be verbose about deprecation.

It has to be the choice of a reviewer if he wants to take a look at submissions of a deprecated exercise, as well as it has to be the choice of a trainee to redo a deprecated exercise in form of its successor.

[kytrinyx]

@NobbZ I like the suggestions around the visual representations. Agreed on not having magic or silent conversion (though we have done that before in the case of point-mutations -> hamming. In that case it wasn't too terrible, I think, since it really was just a rename.

[kytrinyx]

I've added this issue to work towards supporting global deprecation: https://github.com/exercism/x-api/issues/139

[tejasbubane]

The support for global deprecation is implemented in exercism/DEPRECATED.x-api#140.

Since this is still unclear to many eg. exercism/javascript#250 (comment), should we document the entire deprecation process somewhere?

Something which covers stuff like:

• What global deprecation means and how to deprecate any exercise.
• What should each track do?
• What happens to previously served exercises?

[kytrinyx]

In addition to the various issues linked to above, I also found this one exercism/problem-specifications#129

[kytrinyx]

From @stkent:

Confirming that deprecation of strain is planned (1, 2).

[kytrinyx]

I think we've got this sorted, since the new site handles deprecation a lot more elegantly.