AOF rewrite with docker volume mapped to host crashes on Windows

[rogerioyuuki]

Minimal Repro

Platform: Windows 10 (doesn't happen with Linux)

docker-compose.yml

version: '2.3'
services:
  redis:
    image: "redis:5-alpine"
    volumes:
      - ./data:/data # This line causes crash
      - ./redis.conf:/redis.conf
    command: redis-server /redis.conf

redis.conf

dir /data
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
auto-aof-rewrite-percentage 1
auto-aof-rewrite-min-size 1k

Exec redis-cli in the container and run a few SET commands to trigger the rewrite.

Result

Redis enters an error loop trying to rename the temporary AOF file

redis_1  | 1:M 01 Mar 2019 13:11:29.526 * Starting automatic rewriting of AOF on 117100% growth
redis_1  | 1:M 01 Mar 2019 13:11:29.526 * Background append only file rewriting started by pid 888
redis_1  | 1:M 01 Mar 2019 13:11:29.561 * AOF rewrite child asks to stop sending diffs.
redis_1  | 888:C 01 Mar 2019 13:11:29.561 * Parent agreed to stop sending diffs. Finalizing AOF...
redis_1  | 888:C 01 Mar 2019 13:11:29.561 * Concatenating 0.00 MB of AOF diff received from parent.
redis_1  | 888:C 01 Mar 2019 13:11:29.567 * SYNC append only file rewrite performed
redis_1  | 888:C 01 Mar 2019 13:11:29.567 * AOF rewrite: 0 MB of memory used by copy-on-write
redis_1  | 1:M 01 Mar 2019 13:11:29.626 * Background AOF rewrite terminated with success
redis_1  | 1:M 01 Mar 2019 13:11:29.629 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
redis_1  | 1:M 01 Mar 2019 13:11:29.632 # Error trying to rename the temporary AOF file temp-rewriteaof-bg-888.aof into appendonly.aof: Permission denied

If you try to exec into the container to look at the AOF you get this weird result:

/data # ls
appendonly.aof
/data # rm appendonly.aof
rm: can't remove 'appendonly.aof': No such file or directory

[wglambert]

You could try enabling shared drives https://docs.docker.com/docker-for-windows/#shared-drives which might work, otherwise use a Docker named volume.

MariaDB/mariadb-docker#152 (comment)

The only solution on Docker for Windows is to use a Docker controlled volume, rather than mapping it to your host.

[tianon]

I've seen a lot of weird IO issues with Docker for Windows (especially when used with Active Directory and thus needing to reset credentials periodically to allow Docker to continue to access the home directory), so it's likely something due to that.

For further assistance debugging your environment, I'd recommend trying the Docker Community Forums, the Docker Community Slack, or Stack Overflow.