[BUG] npm start suddenly started giving `npm ERR! Error: EACCES: permission denied, mkdir '/home/myuser'` as of yesterday (2023-11-30)

[Foosvald]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

We do npm start in a Docker container with a user which doesn't have a home directory. We've been doing this for several years without problems, but as of yesterday (2023-11-30) all containers suddenly get npm ERR! Error: EACCES: permission denied, mkdir '/home/myuser'. Setting NPM_CONFIG_UPDATE_NOTIFIER=false solves this problem, but we've already deployed a lot of containers without the update notifier disabled which causes a lot of problems for us.

Did something change in the response from the npm servers or do you know why this problem suddenly started happening yesterday?

Expected Behavior

npm start should start the script and not exit due to being unable to create a home folder.

Steps To Reproduce

Given the following code as repro.sh:

#!/bin/bash

cat <<EOF > Dockerfile
FROM node:20.10.0-buster-slim as production
WORKDIR /usr/src/my

COPY package.json ./

RUN groupadd -g 999 myuser && \
    useradd -r -u 999 -g myuser myuser

USER myuser

WORKDIR /usr/src/my/backend

CMD [ "npm", "start", "--verbose" ]
EOF

cat <<EOF > package.json
{
    "name": "myservice",
    "version": "1.0.0",
    "private": true,
    "scripts": {
        "start": "sleep 3 && echo hello!"
    }
}
EOF

docker build -f Dockerfile -t myservice:tag1 .
docker run -e NPM_CONFIG_UPDATE_NOTIFIER=$NPM_CONFIG_UPDATE_NOTIFIER --rm -it myservice:tag1

./repro.sh or NPM_CONFIG_UPDATE_NOTIFIER=true ./repro.sh causes npm to exit with the following log:

[+] Building 0.1s (10/10) FINISHED                                                                                                                                                                                                                                                                                                                   docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                                                 0.0s
 => => transferring dockerfile: 274B                                                                                                                                                                                                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/node:20.10.0-buster-slim                                                                                                                                                                                                                                                                                          0.0s
 => [1/5] FROM docker.io/library/node:20.10.0-buster-slim                                                                                                                                                                                                                                                                                                            0.0s
 => [internal] load build context                                                                                                                                                                                                                                                                                                                                    0.0s
 => => transferring context: 180B                                                                                                                                                                                                                                                                                                                                    0.0s
 => CACHED [2/5] WORKDIR /usr/src/my                                                                                                                                                                                                                                                                                                                                 0.0s
 => CACHED [3/5] COPY package.json ./                                                                                                                                                                                                                                                                                                                                0.0s
 => CACHED [4/5] RUN groupadd -g 999 myuser &&     useradd -r -u 999 -g myuser myuser                                                                                                                                                                                                                                                                                0.0s
 => CACHED [5/5] WORKDIR /usr/src/my/backend                                                                                                                                                                                                                                                                                                                         0.0s
 => exporting to image                                                                                                                                                                                                                                                                                                                                               0.0s
 => => exporting layers                                                                                                                                                                                                                                                                                                                                              0.0s
 => => writing image sha256:1c9ddb6a1864b4ad4e88b0c503ae3d645b55b8f184c45afdb27cc5e99d54bd92                                                                                                                                                                                                                                                                         0.0s
 => => naming to docker.io/library/myservice:tag1                                                                                                                                                                                                                                                                                                                    0.0s

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@10.2.3
npm info using node@v20.10.0
npm verb cache could not create cache: Error: EACCES: permission denied, mkdir '/home/myuser'
npm verb logfile could not create logs-dir: Error: EACCES: permission denied, mkdir '/home/myuser'
npm verb title npm start
npm verb argv "start" "--loglevel" "verbose"
npm verb logfile logs-max:10 dir:/home/myuser/.npm/_logs/2023-12-01T12_42_57_202Z-
npm verb logfile could not be created: Error: ENOENT: no such file or directory, open '/home/myuser/.npm/_logs/2023-12-01T12_42_57_202Z-debug-0.log'
npm verb logfile no logfile created

> myservice@1.0.0 start
> sleep 3 && echo hello!

npm verb stack Error: EACCES: permission denied, mkdir '/home/myuser'
npm verb cwd /usr/src/my/backend
npm verb Linux 6.4.16-linuxkit
npm verb node v20.10.0
npm verb npm  v10.2.3
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /home/myuser
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/home/myuser'
npm ERR!  [Error: EACCES: permission denied, mkdir '/home/myuser'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/home/myuser'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
npm verb exit -13
npm verb unfinished npm timer command:start 1701434577260
npm verb unfinished npm timer command:run-script 1701434577267
npm verb code -13

npm ERR! Log files were not written due to an error writing to the directory: /home/myuser/.npm/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal

But NPM_CONFIG_UPDATE_NOTIFIER=false ./repro.sh works:

[+] Building 0.1s (10/10) FINISHED                                                                                                                                                                                                                                                                                                                   docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                                                 0.0s
 => => transferring dockerfile: 274B                                                                                                                                                                                                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/node:20.10.0-buster-slim                                                                                                                                                                                                                                                                                          0.0s
 => [1/5] FROM docker.io/library/node:20.10.0-buster-slim                                                                                                                                                                                                                                                                                                            0.0s
 => [internal] load build context                                                                                                                                                                                                                                                                                                                                    0.0s
 => => transferring context: 180B                                                                                                                                                                                                                                                                                                                                    0.0s
 => CACHED [2/5] WORKDIR /usr/src/my                                                                                                                                                                                                                                                                                                                                 0.0s
 => CACHED [3/5] COPY package.json ./                                                                                                                                                                                                                                                                                                                                0.0s
 => CACHED [4/5] RUN groupadd -g 999 myuser &&     useradd -r -u 999 -g myuser myuser                                                                                                                                                                                                                                                                                0.0s
 => CACHED [5/5] WORKDIR /usr/src/my/backend                                                                                                                                                                                                                                                                                                                         0.0s
 => exporting to image                                                                                                                                                                                                                                                                                                                                               0.0s
 => => exporting layers                                                                                                                                                                                                                                                                                                                                              0.0s
 => => writing image sha256:1c9ddb6a1864b4ad4e88b0c503ae3d645b55b8f184c45afdb27cc5e99d54bd92                                                                                                                                                                                                                                                                         0.0s
 => => naming to docker.io/library/myservice:tag1                                                                                                                                                                                                                                                                                                                    0.0s

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@10.2.3
npm info using node@v20.10.0
npm verb cache could not create cache: Error: EACCES: permission denied, mkdir '/home/myuser'
npm verb logfile could not create logs-dir: Error: EACCES: permission denied, mkdir '/home/myuser'
npm verb title npm start
npm verb argv "start" "--loglevel" "verbose"
npm verb logfile logs-max:10 dir:/home/myuser/.npm/_logs/2023-12-01T12_43_50_048Z-
npm verb logfile could not be created: Error: ENOENT: no such file or directory, open '/home/myuser/.npm/_logs/2023-12-01T12_43_50_048Z-debug-0.log'
npm verb logfile no logfile created

> myservice@1.0.0 start
> sleep 3 && echo hello!

hello!
npm verb exit 0
npm info ok

Environment

• npm: 10.2.3
• Node.js: v20.10.0
• OS Name: Debian buster slim
• System Model Name: Macbook Pro (but happens on Linux hosts as well)
• npm config:

; node bin location = /usr/local/bin/node
; node version = v20.10.0
; npm local prefix = /usr/src/my
; npm version = 10.2.3
; cwd = /usr/src/my/backend
; HOME = /home/myuser

[moznion]

I also encountered the same issue on December 1st at 00:00 UTC when the application restarted. The system had not upgraded the Node and npm versions, and there were no changes to the application code while running the application. I am also wondering about the reason for this issue.

[moznion]

For your information, another workaround for this problem is to specify the cache directory in your .npmrc file like this:

cache=/path/to/accessible/.npm

[karlbohlmark]

For people who are running this in containers they can't easily rebuild and redeploy it also worked for me to override the dns entry for registry.npmjs.org (I tried setting it to 127.0.0.1 in /etc/hosts).

Please report this issue to npm through the npm website as well. If you log in to your npm account you can report an issue with the registry. Maybe someone will get through to someone who understands the problem.

[dglozic]

IBM products that use npm start have also started to experience this. It is unsettling that npm start that we thought will just use package.json is actually making an outbound request to the npm public servers. Images with a version of Node.js and npm that we didn't touch started having issues starting for this exact reason.

How do we lobby for the change to be modified or reverted (and to consider a npm start variant that does not make an outbound request)?

[karlbohlmark]

Yes it’s pretty scary that images that have been running fine for a year suddenly breaks. It should probably be easy for npm to fix this if it was possible to get their attention. We have already moved away from using npm in the docker images going forward but these old containers are running in a lot of places where we don’t have access to upgrade them.

@dglozic You can avoid the outgoing request if you set the config variable as mentioned in the ticket.

[dglozic]

I understand that (thanks!) but that requires changes to the images.

[karlbohlmark]

Since it is an environment variable it should be possible to change it without changing the image assuming you have access to relaunch the container. We did this in our kubernetes environments as a hot-fix yesterday.

[dglozic]

Ah sorry, was not paying attention, I was seeing changes to .npmrc and didn't see that an env var can do the same.

[yannizhang2019]

Can anyone from the community explain a bit on what is causing this problem? We are getting more and more reports from customers on this issue.

[texkelly]

I assume blocking all egress by default, and then defining only required exceptions (using network policies) would also work, as long as you don't add registry.npmjs.org to that exception list. Container images are generally assumed to be static, ideally they shouldn't be updating themselves over the internet at start time.