command/format: Restructure diagnostic presentation#30331
command/format: Restructure diagnostic presentation#30331apparentlymart wants to merge 1 commit intomainfrom
Conversation
Based on how folks have asked their questions, it seems that our current ordering of summary/snippet/values/detail is causing those skimming the error messages to believe that the "values" portion is the error message, and not read the "detail" part. We originally placed the snippet immediately after the summary so that it would allow quickly seeing the location of the error, but that goal can still be met by including only a reference to the filename and line number up top, and then keeping the full snippet for later in the message. This does make the diagnostic messages slightly longer, by introducing a few new lines where one of them is slightly redundant, but it also makes the most relevant information -- the details of the diagnostic -- closer to the top and thus hopefully more likely to be read first by those who are skimming the messages, rather than reading them in detail.
|
Over in #31299 we did a less severe change to just hide any mentions of unknown values or sensitive values unless the problem is actually related to the presence of those, and I'd hoped that would be a good compromise to address the concern that originally motivated this. However, today I saw a question on Stack Overflow where someone read the context about what type a variable had as the error message and completely ignored the actual error message below it, which shows that this problem isn't limited only to unknown values and sensitive values. This PR is a bit stuck because our feedback loop on this is so long and unpredictable that we don't have any good way to prove that reformatting the messages in this way would actually make the problem better: it's possible that it would just cause the same confusion for a different set of people who bias towards reading the information closest to the bottom of the output because they read upwards from the command prompt (which is the assumption that motivated how Terraform currently presents these.) |
|
Just this week I've seen five separate people ask for help and share only the second half of the error message, so clearly our current presentation is not making it clear to folks where the error messages start and end. I still have no way to prove that what I tried here is an improvement, but the current presentation doesn't seem to be working for people. |
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
Based on how folks have asked their questions, it seems that our current ordering of summary/snippet/values/detail is causing those skimming the error messages to believe that the "values" portion is the error message, and not read the "detail" part.
We originally placed the snippet immediately after the summary so that it would allow quickly seeing the location of the error, but that goal can still be met by including only a reference to the filename and line number up top, and then keeping the full snippet for later in the message.
This does make the diagnostic messages slightly longer, by introducing a few new lines where one of them is slightly redundant, but it also makes the most relevant information -- the details of the diagnostic -- closer to the top and thus hopefully more likely to be read first by those who are skimming the messages, rather than reading them in detail.
This is just a draft for now, because if we decide to go forward with it we'll need to spend a non-trivial amount of effort updating all of the tests for this, and so we want to make sure it's a good direction first before spending that time.