PermissionError: [WinError 5] Access is denied if using build multithreads > 3

[shijunjing]

Hello，
I'm enabling the clcache in the Uefi firmware MSVC build toolchain. I've enable thed ccache in our GCC/CLang toolchains and I hope the clcache can work in our MSVC toolchain as well.
I built the clcache through the pyinstaller and I directly hard code the clcache_main.exe as my firmware VS2015 build CL and build in VS2015 cmd environment. My firmware build is driven by Uefi python build script. In my testing, I found if I build with single thread or threads <= 3, the clcache can work well. However, if I build with threads > 4, the clcache will raise the PermissionError: [WinError 5] Access is denied, below is an example. The more threads I use, the easier this issue raise.

Is there any missing configure in my build steps? Any suggestion is appreciated.

[11116] Failed to execute script clcache_main
Traceback (most recent call last):
File "clcache_main.py", line 2, in
File "site-packages\clcache_main_.py", line 1589, in main
File "site-packages\clcache_main_.py", line 1616, in processCompileRequest
File "site-packages\clcache_main_.py", line 1668, in scheduleJobs
File "concurrent\futures_base.py", line 425, in result
File "concurrent\futures_base.py", line 384, in get_result
File "concurrent\futures\thread.py", line 57, in run
File "site-packages\clcache_main.py", line 1689, in processSingleSource
File "site-packages\clcache_main.py", line 1719, in processDirect
File "site-packages\clcache_main_.py", line 1507, in processCacheHit
File "site-packages\clcache_main_.py", line 749, in exit
File "site-packages\clcache_main_.py", line 659, in save
File "contextlib.py", line 119, in exit
File "site-packages\atomicwrites_init_.py", line 159, in open
File "site-packages\atomicwrites_init.py", line 190, in commit
File "site-packages\atomicwrites_init_.py", line 93, in replace_atomic
File "site-packages\atomicwrites_init_.py", line 75, in replace_atomic
File "site-packages\atomicwrites_init.py", line 70, in _handle_errors
PermissionError: [WinError 5] Access is denied.
NMAKE : fatal error U1077: 'C:\steven\clcache-fork\pyinstaller\dist\clcache_main\clcache_main.exe' : return code '0xffffffff'
Stop.

[izmmisha]

Hello, take a look to #339, same problem here.

[shijunjing]

Moving clcache dir out of user profile dir doesn't work for me. I moved the C:\steven\clcache-fork\pyinstaller\dist\clcache_main\ to C:\clcache_main, and the build still fail with threads > 4. Below is the failure log:

[7000] Failed to execute script clcache_main
Traceback (most recent call last):
File "clcache_main.py", line 2, in
File "site-packages\clcache_main_.py", line 1589, in main
File "site-packages\clcache_main_.py", line 1616, in processCompileRequest
File "site-packages\clcache_main_.py", line 1668, in scheduleJobs
File "concurrent\futures_base.py", line 425, in result
File "concurrent\futures_base.py", line 384, in get_result
File "concurrent\futures\thread.py", line 57, in run
File "site-packages\clcache_main.py", line 1689, in processSingleSource
File "site-packages\clcache_main.py", line 1719, in processDirect
File "site-packages\clcache_main_.py", line 1507, in processCacheHit
File "site-packages\clcache_main_.py", line 749, in exit
File "site-packages\clcache_main_.py", line 659, in save
File "contextlib.py", line 119, in exit
File "site-packages\atomicwrites_init_.py", line 159, in open
File "site-packages\atomicwrites_init.py", line 190, in commit
File "site-packages\atomicwrites_init_.py", line 93, in replace_atomic
File "site-packages\atomicwrites_init_.py", line 75, in replace_atomic
File "site-packages\atomicwrites_init.py", line 70, in _handle_errors
PermissionError: [WinError 5] Access is denied.
NMAKE : fatal error U1077: 'C:\clcache_main\clcache_main.exe' : return code '0xffffffff'
Stop.

[izmmisha]

you should move clcache dir, not clcache binary. It can be configured through CLCACHE_DIR environment

[shijunjing]

Thank you! It works with threads > 4 after I set set CLCACHE_DIR=C:\clcache_dir.

Is this issue a known limitation by design? Do we have plan to fix it?

[frerich]

I think this might warrant an entry in the 'Known Issues' section of the Wiki.

[benjaminfuchs]

@frerich I added an idea to fix the issue in this comment #334 (comment)

[boris-papevis]

I can confirm that the retry workaround from @shijunjing is working great. @frerich, please consider merging this one.