Virtual File and Physical File get out of sync

[fendor]

I am encountering more often the following problem:

I did not find a way to reliably reproduce it, yet.

[mpickering]

hie is pretty unusable when working on GHC with this issue.

[mpickering]

I just tried using it again, and it worked this time without any real issues.

[mpickering]

I wonder if replacing all the module loading logic with ghcide would be faster than debugging this..

[fendor]

The issue is volatile, sometimes it happens all the time and then it doesn't happen for hours.

[Avi-D-coder]

I wonder if replacing all the module loading logic with ghcide would be faster than debugging this..

Is anyone working on this? If not I may take a crack at it.

[fendor]

We did try to solve it, the branch mpickering/wip/vfs tries to reduce races in haskell-lsp, but the issue is still arising.

Issue #42 describes an idea how we considered debugging concurrent events, but at least I can not read anything from the reports.

The first step would be to actually have reproducible set of steps. Personally, to reproduce the issue, I just start coding. At some point, I see that diagnostics are wrong, but the logs are huge in debug mode, thus, hard to debug. After capturing an lsp session, where the issue happens, you can not really replay it, since timings are completely off, which may or may not be interesting.

In https://gist.github.com/fendor/86fa32194052ebcdbd3fbb3997a31ab4 I actually logged the contents of the virtual file from one change to another and minified noise, which may hide the actual issue. Note the lines https://gist.github.com/fendor/86fa32194052ebcdbd3fbb3997a31ab4#file-virtual-file-hs-L56 and https://gist.github.com/fendor/86fa32194052ebcdbd3fbb3997a31ab4#file-virtual-file-hs-L115 which are changes to the virtual file that should not have happened.
The line https://gist.github.com/fendor/86fa32194052ebcdbd3fbb3997a31ab4#file-virtual-file-hs-L59 ought to describe the actual changes, but they do not explain the actual changes to the virtual files.

This race may or may not be already fixed by mpickering/wip/vfs in Haskell-lsp. The logs have been produced before that pr. Moreover, mpickering worked on making haskell-lsp virtual file system functions atomic, see the branch https://github.com/mpickering/haskell-lsp/tree/wip/virtual-file-fix for the details.

@mpickering did I miss something?

EDIT: we also tried to put additional stress on the system with stress. However, neither CPU intensive tasks nor IO intensive tasks made the occurrence of the bug more or less likely, at least for me.

[mpickering]

See #45 for the places where I found potential races.

[jneira]

Not sure if it is related but i am continue getting the lintCmd: no access to the persisted file. every time i open or save a file. Maybe it is related with the fact that i don't see hlint suggestions.

With master i see the hlint suggestions but they ge removed almost immediately.

[Note: Tested in windows 7 and 10 as always 😉]

full log only opening vscode in a file which should trigger hlint suggestions but throw the lintCmd: no access to the persisted file.error: https://gist.github.com/jneira/0f7ddd27e3e20b4f2890fdb57f27d866

[fendor]

@jneira what you are observing is in my opinion unrelated, it is in general acceptable behaviour. Rather these warnings should be turned into empty results.
no hlint suggestions also sound like a different issue.

[mpickering]

I constructed a test case using dejafu which proves that persistVirtualFile can lead to races - https://github.com/mpickering/haskell-lsp/tree/wip/dejafu

[fendor]

dejafu is a great idea in general to check for races!
Hopefully we can build up on this!

[mpickering]

Rebased branch (where the same test passes)

https://github.com/mpickering/haskell-lsp/tree/wip/dejafu-rebased