Put dunder global variables in their own scope

[karthiknadig]

Ideally the following should go into their own group.

• builtin dunder variables
• data
• functions, classes etc

[fabioz]

Note: I'm taking a look at this.

[fabioz]

Also related: #763

[fabioz]

Also related: #118

[fabioz]

@karthiknadig @int19h I'm still working this issue (along with the other related issues I commented above) -- I'm in the process of updating test cases now and checking for corner cases.

I separated entries in 4 groups (dunder variables, protected variables, function variables, class variables) -- anything that doesn't enter one of those groups appears directly below the variable (note that a variable goes into the group it matches first there even if it could appear at more than one group).

So, I'd like to gather comments on whether you think that's ok or if you think something should be changed in that organization...

The current result is in the screenshot below.

[int19h]

It looks great. We might want to fiddle with what exactly goes into which category if there's conflicts, but that can be a separate enhancement.

@brettcannon Is there some better name than "dunder variables" for that category?

[int19h]

One other thing we'll need is a boolean property in debug configuration to enable/disable grouping, something like "groupVariables". This will allow vscode-python to gradually roll it out via the experiment framework, and for users to explicitly opt out afterwards.

[brettcannon]

The ones in the example are technically global variables. They are also all standard module attributes. So it really depends on what you're trying to filter out: things that just happen to be named with a leading and trailing __ or specifically module attributes that exist on all/most modules?

[int19h]

I think technically we want to filter out the "magic" variables. It doesn't have anything to do with whether it's globals or not - as you can see on the screenshot, self also has that category.

[fabioz]

@brettcannon

For me it's a way to show variables that are seldomly used (and this includes both heuristics based on the name and variable type) without cluttering the UI too much.

As a bit of history, until now the debugger was always filtering out all of those (so, they'd only be accessible by having the user evaluate them on the cases he actually needed them) -- you can see the actual logic in https://github.com/microsoft/debugpy/blob/0dd33dd250a884bfa2b4f5025b15d2a29575646c/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_resolver.py#L187 -- with the proposed changes, the user would be able to access those through a group that can be expanded.

@int19h
When that setting is turned on, those variables would disappear (as was the case until now) or would they show along all other variables?

[int19h]

I was thinking of the setting that covers this whole functionality (i.e. also protected, function etc). But we might want to think about something more fine-grained than that - e.g. some users might not like separating functions into a different category, and some might not like protected, but will still want dunder. So, maybe something like:

"groupVariables": ["function", "class", "dunder"]

It's unfortunate that we have to do this on the debugger side. Really, what it should be doing is just reporting variables tagged with categories, and the client should then show them in groups (or not) according to their configuration. Users shouldn't have to edit launch.json for this, because it's not a session-specific preference, but user-specific. Which means that the extension will have to surface it in user settings, and then inject it into debug configuration, much like we do with "pythonPath".

So, basically like DAP scopes, but applicable on every level of the hierarchy. This needs DAP and client support though.

[int19h]

Thinking about this more - since we'd want to also accommodate the existing behavior of hiding dunder entirely, how about:

"variablePresentation": {
    "function": "show",
    "dunder": "hide",
    "class": "group",
    ...
}