Increasing amount and quality of participation in code review / conversations

[kytrinyx]

For the engines of Exercism to really work, people need to see comments and code based on their solutions.

There are two overarching ways to tackle this, and we should investigate/pursue both.

The first is machine learning / AI to use all of the existing data (both code and comments) to show people something useful. We have an open issue about this here: #65

The second thing is to get people to contribute to the conversations more, and to do so in ways that are more valuable.

People might resist giving feedback for the a number of reasons:

• They don’t realize the benefit. You learn more from giving feedback than getting it. It forces you to:
  • Read code. Like actually, really read it.
  • Recognize code smells.
  • Articulate your thoughts and opinions about readability, simplicity, performance, design, idioms, conventions. This pushes you to a much deeper understanding of concepts, and uncovers those areas where you don’t actually understand something (which can lead you to go learn about it).
• They are intimidated.
  • They might be wrong. People might be smarter than them. They might get laughed at or ridiculed. People might say mean things.
  • For code newbies especially, they are like “who am I to tell someone else what to do?” which is a totally valid objection, but which is kind of irrelevant. A code newbie could look at some code and ask questions: How does this work? Why did you do it this way? I don’t understand this bit here.
• They simply don’t know what to say. They look at code and just see… code.

Other relevant discussions:

• Idea: Require nits before continuing exercism#1956
• Upvote/downvote, flagging and reply system for nitpicks exercism#1927
• Idea: "Labelvoting" exercism#1957
• Under current model, "nits" are far more likely to come from beginners than from experts exercism#2016
• Up-/Downvoting of nitpicks exercism#2711

[GustavoCaso]

@kytrinyx Totally agree with you on everything.

Maybe we could include some basic guides on how to comment code. A rating system for comments, so the user could see how his or her code reviews skills improve by the more he or she review code.

What about after you submit a exercise, you are assign a exercise to review, from another user but same exercise of course that way you have clear idea of what the exercises was about. This will help to get more comments as the same time, you will create more comments.

[GustavoCaso]

Anyone?

[dvberkel]

I don't really like the assigned exercise to review idea. For me it feels like an obligation that I am maybe not willing to commit to. I would rather want to opt-in to such a mechanism and therefore have a choice.

Having said that, I do acknowledge the value of feedback. Both in receiving good feedback and providing good feedback. But the problems and resistance people feel against receiving and giving feedback is as much a cultural thing.

I would love for Exercism.io to instigate a change, but I am not sure if "forcing" people to review others is the best way to start

[nilbus]

There's a nice middle-ground between forcing and what we do now (which is to say, “See related solutions and get involved here: [url]” (source)).

Instead, let's spoon-feed them the the actions that we'd like to motivate them to take, and sell the benefit. For example:

Now that you've completed this exercise, let's see how your solution compares to
others: [url]
To benefit the most from this exercise, find 3 or more submissions that you can
learn something from, have questions about, or have suggestions for.
Post your thoughts and questions in the comments, and start a discussion.
Revise your solution to incorporate what you learned.

After that's done, you can grab the next exercise: exercism fetch <track>

You could probably tweak the wording all day long. Some things I like about this:

• It plays on the innate sense of competition: “let's see how yours compares”
• It describes reviewing code as the natural next step before moving on to the next exercise
• It reminds you that you're really here to learn, and that these are important steps toward that goal
• It suggests review reactions appropriate for people at all levels: learn, ask, incorporate, suggest

Thoughts?

[GustavoCaso]

This feels more natural.

Help avoid the concept of obligation and plant the idea of the importance of code review, as @nilbus the wording can be changed all the day long but the core concepts is clear.

I really like it.

[kytrinyx]

I've added the "design-research" label as I think this is a really interesting and meaty discussion for us to explore as we think about the user experience on the site.

I appreciate all the thought and detail here.

To benefit the most from this exercise, find 3 or more submissions that you can learn something from, have questions about, or have suggestions for. Post your thoughts and questions in the comments, and start a discussion. Revise your solution to incorporate what you learned.

I love this. I have found that a really powerful usage pattern is when people do exactly this: and thank people when they find something that they learn from.

[tleen]

We could always go down the reward incentivization route: add a review leaderboard somewhere, perhaps split into daily/weekly/..., this is also the type of thing you would see with badges, trophies, etc. The downside is that it could discourage quality over quantity, the latter easy to calculate the former not so much.

Along the lines of @nilbus comment, perhaps having the site actually select a few (recent, not commented) entries and link explicitly to those. So instead of

find 3 or more submissions

it would be more like:

It looks like @GustavoCaso, @tleen, and @kytrinyx have also completed this exercise and could use some feedback.

Where we could link directly to their solutions. This would save the user a step and make sure that recent uncommented solutions would get first billing. Its a way to avoid the sort of Bystander effect users may feel. While not an explicit assignment, it would help in perhaps cutting down the sheer (overwhelming?) mass of solutions from which to choose.