Mount volume with rw permissions

[aleksey925]

Describe the bug
It is not possible to create a container with a mounted volume.

Steps to reproduce

1. Create file docker-compose.yml with below content:

services:

  db:
    image: postgres:14-alpine
    restart: on-failure:10
    environment:
      - POSTGRES_USER=admin
      - POSTGRES_PASSWORD=admin
      - POSTGRES_DB=db_name
    volumes:
      - ./.volumes/db:/var/lib/postgresql/data
    ports:
      - '5432:5432'

2. Execute command finch compose up

Expected behavior
Container started successful

Logs

WARN[0000] Ignoring: service db: restart="on-failure:10" (unknown) 
WARN[0000] Ignoring: volume: Bind: [CreateHostPath]     
INFO[0000] Ensuring image postgres:14-alpine            
INFO[0000] Creating container runtime-config_db_1       
FATA[0000] error while creating container runtime-config_db_1: exit status 1 
FATA[0000] exit status 1

Finch version: v0.1.1
limactl version: fc3f2a9
qemu version: 7.1.0

[ningziwen]

Duplicate with #107

[aleksey925]

@ningziwen It's not duplicate. I tried to place docker-compose.yml in the root of the user directory. It did not help.

[ningziwen]

The mount path also needs to be in home directory. What is the absolute path of "./.volumes/db" in your compose yaml? Is it in home directory? @aleksey925

[aleksey925]

@ningziwen

Is it in home directory?

Yes.
The path to the folder is ~/.volumes/db.

[monirul]

Hi @aleksey925,
I have reproduced this error.
This error occurs when the folder .volumes/db does not exist. It works for me when i created the folder .volumes/db folder.

My folder structure looks like:

~/finch-issue
|- docker-compose.yml
|- .volumes/db

I run the compose command from ~/finch-issue folder:

finch compose up

The output looks like:

WARN[0000] Ignoring: service db: restart="on-failure:10" (unknown) 
WARN[0000] Ignoring: volume: Bind: [CreateHostPath]     
INFO[0000] Ensuring image postgres:14-alpine            
INFO[0000] Creating container git-issue-131_db_1        
INFO[0000] Attaching to logs                            
db_1 |chown: /var/lib/postgresql/data: Permission denied
INFO[0000] Container "git-issue-131_db_1" exited        
INFO[0000] All the containers have exited               
INFO[0000] Stopping containers (forcibly)               
INFO[0000] Stopping container git-issue-131_db_1 

To fix the db_1 |chown: /var/lib/postgresql/data: Permission denied error pls refer to this github issue in postgres
docker-library/postgres#116

To fix the permission issue I used following compose yml:

services:
  db:
    image: postgres:14-alpine
    restart: on-failure:10
    environment:
      - POSTGRES_USER=admin
      - POSTGRES_PASSWORD=admin
      - POSTGRES_DB=db_name
      - PGDATA=/var/lib/postgresql/pgdata
    volumes:
      - ./.volumes/db:/var/lib/postgresql/data:wr
    ports:
      - '5432:5432'

Output

WARN[0000] Ignoring: service db: restart="on-failure:10" (unknown) 
WARN[0000] Ignoring: volume: Bind: [CreateHostPath]     
INFO[0000] Ensuring image postgres:14-alpine            
INFO[0000] Creating container git-issue-131_db_1        
INFO[0000] Attaching to logs                            
db_1 |The files belonging to this database system will be owned by user "postgres".
db_1 |This user must also own the server process.
db_1 |
db_1 |The database cluster will be initialized with locale "en_US.utf8".
db_1 |The default database encoding has accordingly been set to "UTF8".
db_1 |The default text search configuration will be set to "english".
db_1 |
db_1 |Data page checksums are disabled.
db_1 |
db_1 |fixing permissions on existing directory /var/lib/postgresql/pgdata ... ok
db_1 |creating subdirectories ... ok
db_1 |selecting dynamic shared memory implementation ... posix
db_1 |selecting default max_connections ... 100
db_1 |selecting default shared_buffers ... 128MB
db_1 |selecting default time zone ... UTC
db_1 |creating configuration files ... ok
db_1 |running bootstrap script ... ok
db_1 |sh: locale: not found
db_1 |2023-01-03 20:11:19.853 UTC [31] WARNING:  no usable system locales were found
db_1 |performing post-bootstrap initialization ... ok
db_1 |syncing data to disk ... ok
db_1 |
db_1 |initdb: warning: enabling "trust" authentication for local connections
db_1 |
db_1 |Success. You can now start the database server using:
db_1 |You can change this by editing pg_hba.conf or using the option -A, or
db_1 |
db_1 |--auth-local and --auth-host, the next time you run initdb.
db_1 |    pg_ctl -D /var/lib/postgresql/pgdata -l logfile start
db_1 |
db_1 |waiting for server to start....2023-01-03 20:11:24.248 UTC [37] LOG:  starting PostgreSQL 14.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
db_1 |2023-01-03 20:11:24.252 UTC [37] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 |2023-01-03 20:11:24.266 UTC [38] LOG:  database system was shut down at 2023-01-03 20:11:21 UTC
db_1 |2023-01-03 20:11:24.280 UTC [37] LOG:  database system is ready to accept connections
db_1 | done
db_1 |server started
db_1 |CREATE DATABASE
db_1 |
db_1 |
db_1 |/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1 |
db_1 |2023-01-03 20:11:25.652 UTC [37] LOG:  received fast shutdown request
db_1 |waiting for server to shut down....2023-01-03 20:11:25.656 UTC [37] LOG:  aborting any active transactions
db_1 |2023-01-03 20:11:25.661 UTC [37] LOG:  background worker "logical replication launcher" (PID 44) exited with exit code 1
db_1 |2023-01-03 20:11:25.674 UTC [39] LOG:  shutting down
db_1 |2023-01-03 20:11:25.701 UTC [37] LOG:  database system is shut down
db_1 | done
db_1 |server stopped
db_1 |
db_1 |PostgreSQL init process complete; ready for start up.
db_1 |
db_1 |2023-01-03 20:11:25.924 UTC [1] LOG:  starting PostgreSQL 14.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
db_1 |2023-01-03 20:11:25.924 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1 |2023-01-03 20:11:25.924 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1 |2023-01-03 20:11:25.930 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 |2023-01-03 20:11:25.939 UTC [52] LOG:  database system was shut down at 2023-01-03 20:11:25 UTC
db_1 |2023-01-03 20:11:25.959 UTC [1] LOG:  database system is ready to accept connections

[aleksey925]

Hi @monirul, I tried to replicate this. The container started, but the database files are still missing on my local machine. Unfortunately, this is not a solution and the problem is somewhere deeper.

[aleksey925]

Hi @monirul. Do you have any news?

[monirul]

Hi @aleksey925,
I am still investigating to find out the root cause. I will get back to you asap.

There was a typo in the yml i provided.

services:
  db:
    image: postgres:14-alpine
    restart: on-failure:10
    environment:
      - POSTGRES_USER=admin
      - POSTGRES_PASSWORD=admin
      - POSTGRES_DB=db_name
      - PGDATA=/var/lib/postgresql/db/pgdata
    volumes:
      - ./.volumes/db:/var/lib/postgresql/db:rw
    ports:
      - '5432:5432'

However, for this yml, the compose up fails with an error db_1 |chown: /var/lib/postgresql/db: Permission denied.

For executing the compose command, Finch passes the compose command to nerdctl. This error is returned from the nerdctl. I am investigating why it is failed. I will get back to you as soon as I get the root cause.

Thanks,
Monirul.

[monirul]

Found the root cause of this issue. Thanks @djdongjin for helping me with the investigation.

Finch uses lima for virtualization. It seems this error is a known limitation of the sshfs server in lima. sshfs server runs without the root privilege, due to that, chown command fails with permission error - chown: /var/lib/postgresql/db: Permission denied . Same error occurs in rancher as well. You can also reproduce it by shelling into the vm and run chown command.

Similar lima issues:
lima-vm/lima#231
rancher-sandbox/rancher-desktop#1209

Here is a lima-issue where lima maintainer suggested to use volume create as workaround.
lima-vm/lima#505

I manage to run the postgres using...

$ finch volume create pgdata

compose yaml file

services:
  db:
    image: postgres:14-alpine
    environment:
      - POSTGRES_USER=admin_user
      - POSTGRES_PASSWORD=admin12345
      - POSTGRES_DB=db_name
      - PGDATA=/var/lib/postgresql/db/pgdata
    volumes:
      - pgdata:/var/lib/postgresql/db:rw
    ports:
      - '5432:5432'
volumes:
  pgdata:

then run following command:

$ finch compose up
INFO[0000] Creating volume git-issue-131_pgdata
INFO[0000] Ensuring image postgres:14-alpine
INFO[0000] Re-creating container git-issue-131_db_1
INFO[0000] Attaching to logs
...
db_1                |2023-01-17 01:48:35.429 UTC [1] LOG:  starting PostgreSQL 14.6 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
db_1                |2023-01-17 01:48:35.429 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1                |2023-01-17 01:48:35.429 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1                |2023-01-17 01:48:35.436 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1                |2023-01-17 01:48:35.441 UTC [53] LOG:  database system was shut down at 2023-01-17 01:48:35 UTC
db_1                |2023-01-17 01:48:35.449 UTC [1] LOG:  database system is ready to accept connections

Btw, As per lima road map, currently lima supports 9p as experimental feature and plans to switch to 9p instead of reverse-sshfs as default mount driver in v1.0(tentative).

[pendo324]

Confirmed that the workaround that @monirul posted works. I did not have to run a separate volume command either, just finch compose up with the compose file example that he provided.

For a permanent long term fix, we are tracking moving off of reverse sshfs file sharing to 9p and virtiofs (when available, so Monterrey and above). We were hoping that the latest QEMU version (7.2) would resolve the performance issues of 9p, but it seems like it still needs some more investigation (see the latest comments in lima-vm/lima#971).

I think we can leave this issue open until the long term fix is viable. Please reach out if Monirul's comment/workaround still doesn't solve your issue for now.

[djdongjin]

I did not have to run a separate volume command either, just finch compose up with the compose file example that he provided.

Yeah it's expected. Volumes in compose.yaml will be created automatically.

volumes:
  pgdata: