Order exercises by increasing difficulty

[ErikSchierboom]

At the moment, the exercises are kinda haphazardly ordered. Ideally, we'd list the exercises in progressive difficulty order, as that proveds the best experience for the user.

In the Rust track, they went about this problem by looking at the concepts that are required. This might be a bit infeasable for this track, as there are so many exercises.

I suggest we first start by categorizing the exercises in three categories:

1. Easy
2. Moderate
3. Difficult

We can then look at the individual categories to determine the ordering within that category. What do you think about this approach?

[jwood803]

I wonder if that could be something we can add to the config...

[ErikSchierboom]

Well, the config file is JSON, so that means no comments.

[jovaneyck]

This makes a lot of sense.

I'm only about 1/3rd in the F# track and experience this. Some exercises require understanding of a lot of concepts if you want to solve them elegantly (partial active patterns, options, list functions, piping,...), while the follow-up exercise is a walk in the park again. I'll see if I can find the time to pick out the ones I found especially tricky.

I found the ones that have a non-functional API (I'm looking at you Clock & Bank Account!) add complexity that isn't necessary to grasp F# FP basics. When working in the real world/C# interop those things become very relevant but these topics aren't that relevant if you want a taste of idiomatic F#.
I'd push those exercises further down the track.
Especially Bank Account, that one even had me pulling out MailBoxProcessor 😄

[ErikSchierboom]

@jovaneyck Thanks a lot in advance already! There are some exercises with a non-functional API. Do you think we should be converting them to proper functional exercises?

[rmunn]

@ErikSchierboom I feel like I should try to complete all the exercises before I can really rank them, but I do think that the ones with a non-functional API are worth converting.

[ErikSchierboom]

@rmunn Sure, that totally makes sense! I'll rework the non-functional exercises.

[ErikSchierboom]

@jovaneyck PR #152 was just merged, which converts the six remaining OO exercises to functional ones :)

[choiaa]

@ErikSchierboom I do prefer the approach of tagging each exercise with a list of topics that need to be understood beforehand, and from there grouping the topics themselves by difficulty to determine the order of exercises. With difficulty being based on personal opinion there will be more room for subjectivity and the influence of each person's programming background.

I can start tagging each exercise as I do it, plus the finished ones. I was thinking of turning this into a little fsharp program, maybe something along these lines:

type Topic    = Topic of string
type Exercise = { title : string; topics : Topic list }
let Exercises = [ ("bob", ["PatternMatching", "Conditionals"]) // ...
                ] |> List.map (fun title topics -> { title = title; topics = List.map Topic topics})

Once all the data is in then it will be easy to write a sorting function and adjust the algorithm as needed.

[ErikSchierboom]

Great idea! I'll try to work on my list of tags as well.

[choiaa]

@ErikSchierboom How about using the header topics in the fsharp language reference for the tags?

Also, I assume we will be using the example solutions in the xfsharp repo?

[ErikSchierboom]

@choiaa The F# language reference is a very convenient help. I do think that we should also have a way of marking the difficulty of an exercise. One exercise that uses pattern-matching might be a lot more difficult than another.

The example solutions are just that: examples. There are probably far better solutions to many of the problem than what I came up with. I think we can use the examples as a guide to see what concepts will probably be used to solve them, but they shouldn't be the definitive guide.

[choiaa]

@ErikSchierboom You're right, I did notice the difference in difficulty even for a single topic. Do you suggest adding a single Difficulty field to the Exercise record or adding three different fields as so?

Easy : Topic list
Medium : Topic list 
Advanced : Topic list

Also, where in the repo do you recommend placing this sorting program? Maybe it can be turned into an exercise with a test suite of its own?

[ErikSchierboom]

I think the first option (single field) is better, because even though one exercise might have three advanced topics and the other just one, the latter might still be more difficult.

As for where to put this code, maybe we should create a tools or helpers directory and then put the program in a sub-directory within that directory. We can then modify the build script to allow us to run the application.

To make it an exercise would require it to be added to the x-common repository. If you think you can create a suitable exercise from it, you should create an issue in that repository.

[ErikSchierboom]

These are all the header topics from the F# language reference page:

• Functions
• Values
• Literals
• Type Inference
• Primitive Types
• Unit Type
• Strings
• Tuples
• Lists
• Options
• Sequences
• Arrays
• Generics
• Records
• Discriminated Unions
• Enumerations
• Reference Cells
• Type Abbreviations
• Classes
• Structures
• Inheritance
• Interfaces
• Abstract Classes
• Members
• Type Extensions
• Parameters and Arguments
• Operator Overloading
• Flexible Types
• Delegates
• Object Expressions
• Copy and Update Record Expressions
• Casting and Conversions
• Access Control
• Conditional Expressions: if...then...else
• Match Expressions
• Pattern Matching
• Active Patterns
• Loops: for...to Expression
• Loops: for...in Expression
• Loops: while...do Expression
• Assertions
• Exception Handling
• Attributes
• Resource Management: The use Keyword
• Namespaces
• Modules
• Import Declarations: The open Keyword
• Signatures
• Units of Measure
• XML Documentation
• Lazy Computations
• Computation Expressions
• Asynchronous Workflows
• Query Expressions
• Code Quotations
• Compiler Directives

Not all of them are useful in the context of grading our exercises. Perhaps we should omit the ones we think won't apply to our examples? For example, I don't think we need Compiler Directives as a topic.

Furthermore, some topics are missing, such as Recursion.

[lucafuelbier]

As a long-term solution you could implement a system where the user can rate a finished problem with easy/appropriate/hard and change the order of the problems based on that feedback. Codewars uses a similar system to grade their new Kata entries. Implementing a solution like this would introduce a couple of other problems tho, mainly how you rank the feedback of your users and how to make the system effective without changing around too much all the time. Also this would mean implementing it for the whole site, not only the F# problem track, which might be undesirable.

[kytrinyx]

As a long-term solution you could implement a system where the user can rate a finished problem with easy/appropriate/hard and change the order of the problems based on that feedback.

Yeah, I think it would help a lot to get feedback from people about difficulty.

[kytrinyx]

Also this would mean implementing it for the whole site, not only the F# problem track, which might be undesirable.

No, very desirable, I think!

A couple of somewhat related discussions:

• Problem ordering, topics covered, and understanding of concepts required discussions#60
• Exercise feedback option after exercise completion exercism#1081

There's been so much to do on Exercism for so long, but I think that the time is probably ripe for something like this now.

[ErikSchierboom]

There has been quite some progress lately on this issue. See this issue in particular. I think the next step for us will be to specify the topics for all exercises. After that, we can try to assign difficulties and then finally we can try to come up with a better order for the exercises.

Once the new format has been approved, I'll start with the groundwork. Hopefully, you'll be able to help assigning the topics.

[ErikSchierboom]

Since merging this PR yesterday, all exercises relevant to the F# track that can be added have been added. The next point of focus is the re-ordering of the exercises, which means we have to do two things:

1. Assign topics to the exercises.
2. Assign a difficulty to the exercise.

For point 1, we can use this list as a starting point. It is quite comprehensive, but if we find any topics lacking, we can add them (although we should prefer to use the default topics).

For point 2, we we should assign each exercise a difficulty, which is a number from 1 (easiest) to 10 (hardest).

Once we have assigned the topics and difificulty for all exercises, we can then re-order the exercises to make for a better learning curve.

While I am happy to start working on this, I would really like your help too. What might be hard for me, might be easy for others, and vice versa.

I know that some people are farther along the track than others, but that's perfectly fine. I've set things up to discuss in batches of 10 exercises, in order of them being served by the track. That means we'll start with the first 10 exercises, which most of you will have completed I think. Once we reach agreement on how to classify those 10 exercises, we'll move on to the next 10.

Hopefully you (@jwood803 @jovaneyck @rmunn @choiaa @Xehanort94 amongst others) are interested in helping make the F# track even better and give your invaluable feedback! Thanks in advance.

[ErikSchierboom]

This has been closed by this PR.