Improve Performance of Intellisense further

[MysticRyuujin]

This is happening on both the stable and the preview version of the extension, using all latest release versions including VS Code as of today.

I have a redirected My Documents folder, so by default my PSModulePath includes a UNC path.

When I open a .ps1 file or start a new PowerShell file it opens the PowerShell Integrated Console (which starts very slowly, like upwards of a minute with high CPU use).

Then whenever I proceed to start typing commands in the console it flashes "Searching available modules" followed by the UNC path. It completes in about 1 second but it does this over and over and over. Basically every time I press a key on the keyboard and intellisense is trying to figure out what command I'm trying tor run, it flashes in the terminal.

During that 1 second where it's searching for available modules the intellisese is locked and doesn't display anything.

I completely uninstalled VS Code, all settings and extensions, etc. re-installed and it's still happening. My VS Code PowerShell Profile is empty, and all settings are default.

Name                           Value
----                           -----
PSVersion                      5.1.17763.1007
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.1007
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Windows 10 x64 Build 1809

[rjmholt]

This is actually a PowerShell feature that writes the progress message.

Intellisense requires module discovery, so it's running this (intellisense has a tendency to load modules, due to PowerShell's own implementation decisions). The commands and parameters that appear with intellisense depend on what modules are available, and that can only be determined by using the module path.

This isn't really desired behaviour, but it's unlikely to be something that's easy to change overnight.

[MysticRyuujin]

While I understand the overall behavior of looking for modules in the path, I don't understand why it has to search for available modules almost every single time I press a keystroke...and with the slowness of reading the entire UNC directory...which is where the 'az' module is on my machine (and a few others), it's incredibly slow.

I would think that it could cache the available modules, or load them once then refresh on demand, or something.

Now that I look at it, it does appear that ISE does this as well...maybe this isn't a VS Code / PowerShell Extension issue, however, ISE seems significantly faster and/or doing it less.

[rjmholt]

I don't understand why it has to search for available modules almost every single time I press a keystroke...and with the slowness of reading the entire UNC directory...which is where the 'az' module is on my machine (and a few others), it's incredibly slow.

The availability of a command depends on the current state of the filesystem, and PowerShell itself, in its own intellisense engine (which we hook directly into) manages all of that. PowerShell even does its best to cache, but it still checks to see if the module's there every time. But if you're doing something where the command or module availability changes, like install a new module or command, or are developing a script module, caching is going to make intellisense wrong.

But the bottom line is probably that this is a direct consequence of PowerShell behaviours like the one causing #715

[rjmholt]

Some possibilities for alleviating this:

• Suppressing progress notifications when running intellisense operations (may have a performance improvement similar to the Invoke-WebRequest speedup in PS 6)
• Caching intellisense to exchange freshness for performance (@TylerLeonhardt has made some awesome progress here)
  • Caching modules on UNC paths only
• Modifying the PSModulePath under the hood to ignore UNC paths (bad for concurrency, may corrupt state)
• Investigating CommandInvocationIntrinsics.GetCommand as an alternative completion API

[MysticRyuujin]

I think I can survive with changing $ProgressPreference in my VSCode profile, it does indeed seem to help speed things up.

I was also able to move a significant number of modules out of my CurrentUser scope UNC path to the 'AllUsers' path on the local system, though I imagine not everyone has this luxury.

As for the auto-loading of modules referenced in #715, I don't think that's an issue here, running Get-Module Az doesn't return anything so it's not 'loaded' which is good. Maybe it's otherwise related idk.

Appreciate the feedback though.

[rjmholt]

Good feedback on your performance workarounds and the Get-Module point -- thanks for letting us know

[SydneyhSmith]

@MysticRyuujin thanks for all the info, this morning we released a new version of the extension which introduced some caching for intellisense...it would be really great to know if this improved your experience at all?

[MysticRyuujin]

I disabled the preview extension, re-enabled the ProgressPreference, and reinstalled the stable extension.

Once the Integrated Terminal is up and running, I'll start typing something like Get-NetTPC and I'll see the progress bar flash. I'll then complete that command, move to the next line and type Get-NetUDP and see it flash again. However, the 'flash' is much faster, I had to focus on that spot to see that it's searching to UNC path.

Without doing more controlled testing, it's hard to say if it's made a huge difference. It SEEMS faster, in that the Progress Bar is only up for fraction of a second now as opposed to a full second.

I'd say it currently feels like a big improvement?

[SydneyhSmith]

@MysticRyuujin thanks for testing this out, and for confirming that there was a performance improvement...it seems that at this point the fix will likely need to come from the PowerShell engine (improvements to Import-Module) but I will leave this issue open to track the request for faster intellisense

[HoCGeek]

As for the auto-loading of modules referenced in #715, I don't think that's an issue here, running Get-Module Az doesn't return anything so it's not 'loaded' which is good. Maybe it's otherwise related idk.

The "AZ" module is mostly a container for sub-modules, so if you run an AZ cmdlet you may not see the AZ module loaded with get-module AZ. You would probably see the sub-module loaded with get-module AZ*.

[HoCGeek]

If I understand this article correctly ("https://docs.microsoft.com/en-us/powershell/module/powershellget/install-module?view=powershell-5.1"), PowerShell modules get installed to $env:ProgramFiles\WindowsPowerShell\Modules except if you use the -Scope currentuser option which installs the modules to $home\Documents\WindowsPowerShell\Modules. In my case it is "$home" that causes the issue because I can't change this variable and it points to a folder redirected to a UNC path. If MS would give us the option to specify a local path for the "currentuser" repository instead of assuming "$home", then we could make sure it was a local folder and avoid the UNC issue.

[SydneyhSmith]

@HoCGeek you may want to weigh in on this issue for more about the ModulePath behavior PowerShell/PowerShell#7082 (comment)

[regulator-g]

Having the same issue where my Documents/profile is on a network drive and tabbing to complete words causes lag.

My workaround at the moment

1. create PS profile file
2. in the file $env:PSModulePath adds a local folder
3. cut and paste my modules to the local folder

works for now without making any major changes

[brwilkinson]

This issue is fairly old... we should evaluate if it's still an issue...

I have a related question.

While I am running a script via the PowerShell vscode extension, I cannot work on my code, since intellisense does not work.

Is there any way to make the extension multithreaded so that I can deploy and work at the same time?

[andyleejordan]

Sadly there is not, as the PowerShell pipeline is single-threaded, so that's an architecture limitation beneath PSES.

[JustinGrote]

@brwilkinson to expand, most intellisense stuff uses a PowerShell-provided API to do the completion, and that API runs in the same pipeline thread where your script runs (for good reason, that runspace has the most current state). This would have to change inside PowerShell engine itself, and even if it did, we would now need forked support for old PS version vs new ps versions.

While we could do things like caching and assembly type lookups outside that runspace that don't block on that (and there have been PRs to help with that), things like variables and the AST state would inherently potentially be inaccurate as a result, and without having a way to present to the user "hey this stuff is a cached result until your script finished", users will perceive the results as "buggy".

[brwilkinson]

Sadly there is not, as the PowerShell pipeline is single-threaded, so that's an architecture limitation beneath PSES.

Thanks @andschwa I figured so. Would be so nice...

[ghost]

This issue has been marked as external. It has been automatically closed for housekeeping purposes.

[boston51]

This is still an active issue, unfortunately. Running the most recent versions of VSCode and the PowerShell Extension (not preview versions). My intellisense stops working completely, and doesn't start working again until I visit the Terminal for the PowerShell Extension and hit the Enter key a few times to get it to stop searching for available modules, it really is a horrible experience.

I think my best option is to attempt the workaround listed above from @regulator-g on June 24 2020.

1. create PS profile file
2. in the file $env:PSModulePath adds a local folder
3. cut and paste my modules to the local folder

[ghost]

Thank you for your comment, but please note that this issue has been closed for over a week. For better visibility, consider opening a new issue with a link to this instead.

[JustinGrote]

My intellisense stops working completely, and doesn't start working again until I visit the Terminal for the PowerShell Extension and hit the Enter key a few times to get it to stop searching for available modules, it really is a horrible experience.

@boston51 Why is it searching for available modules and why is it taking so long? In my experience I don't have this issue, it's usually pretty fast. Can you put together a simple reproduction?

[boston51]

@JustinGrote That's an excellent question. This is from a work environment, so I want to limit what details I share of course. If you have any instructions on how I can get the information you're looking for that would be great. I've been working with PowerShell for several years, but must admit my knowledge is still lacking in some areas. What would you need from this 'simple reproduction'?

[boston51]

I believe part of it is that the modules I have are loaded on a network drive, which would be annoying to have to copy them down locally to each machine I work on, but it might be a workaround that's doable.

[JustinGrote]

It certainly would help if your searching for modules takes forever (shouldn't take that long unless you have a really slow file server or you are working over a high latency connection such as a VPN).

Alternatively you can modify your PSModulePath to exclude your network modules folder and add it "as needed" if you don't use those modules day to day, otherwise local copy is going to be your only option.

This same "annoyance" is going to be present in a normal powershell terminal too, there's nothing special to vscode specifically here.

It is not generally recommended to have PSModules loaded directly from a network share, instead, make the network share a repository and use install/update module to keep them up to date on the local computer.
https://4sysops.com/archives/how-to-create-a-file-share-powershell-repository/