GRUB fails to resolve canonical path to device, uses invalid partition and fails to detect zfs

[seletskiy]

Let's suppose following scenario:

I've want to create zpool that use entire device /dev/sda:

# ls -al /dev/disk/by-id/ata-QEMU_HARDDISK_QM00001   
lrwxrwxrwx 1 root root 9 Dec 31 15:46 /dev/disk/by-id/ata-QEMU_HARDDISK_QM00001 -> ../../sda

I'm using by-id path to disk while creating zpool:

# zpool create zroot /dev/disk/by-id/ata-QEMU_HARDDISK_QM00001
# zpool status
  pool: zroot
 state: ONLINE
  scan: none requested
config:

    NAME                         STATE     READ WRITE CKSUM
    zroot                        ONLINE       0     0     0
      ata-QEMU_HARDDISK_QM00001  ONLINE       0     0     0

errors: No known data errors

# zfs list
NAME    USED  AVAIL  REFER  MOUNTPOINT
zroot   110K  3.91G    30K  /zroot

zroot is mounted into /zroot.

So, trying to grub-probe:

# grub-probe /zroot
grub-probe: error: failed to get canonical path of `/dev/ata-QEMU_HARDDISK_QM00001'.

Wut? So, GRUB detected that /zroot is a ZFS (otherwise, how did it know about ata-QEMU stuff?), but if fails to resolve correct path to device.

OK, let's try to fix it in a dirty way:

# ln -s /dev/sda /dev/ata-QEMU_HARDDISK_QM00001
# grub-probe /zroot
grub-probe: error: unknown filesystem.
# grub-probe -vv /zroot
grub-core/kern/fs.c:56: Detecting zfs...
grub-core/osdep/hostdisk.c:319: opening the device `/dev/sda' in open_device()
grub-core/fs/zfs/zfs.c:1183: label ok 0
grub-core/fs/zfs/zfs.c:1183: label ok 1
...
grub-core/kern/fs.c:78: zfs detection failed.

Didn't work. Hmmm... Looks like grub-probe tries to read from /dev/sda.
Take a look:

# zdb -l /dev/sda 
--------------------------------------------
LABEL 0
--------------------------------------------
failed to unpack label 0
--------------------------------------------
LABEL 1
--------------------------------------------
failed to unpack label 1
--------------------------------------------
...

 # fdisk -l /dev/sda
Device           Start          End Size Type
/dev/sda1         2048      8370175   4G Solaris /usr & Apple ZFS
/dev/sda9      8370176      8386559   8M Solaris reserved 1

# zdb -l /dev/sda1

--------------------------------------------
LABEL 0
--------------------------------------------
    version: 5000
    name: 'zroot'
    state: 0
    txg: 4
    pool_guid: 16263322471539432696
    hostname: 'archiso'
    top_guid: 9383719665143350581
    guid: 9383719665143350581
...

Indeed!

# ln -sf /dev/sda1 /dev/ata-QEMU_HARDDISK_QM00001
# grub-probe /zroot
zfs

After that series of hacks I can do grub-install and it finishes successfully.

So, my suggestion is that GRUB uses incorrect device path when zpool is created over entire device.

[FransUrbo]

On Dec 31, 2013, at 5:48 PM, Stanislav Seletskiy wrote:

grub-probe /zroot

grub-probe: error: failed to get canonical path of `/dev/ata-QEMU_HARDDISK_QM00001'.
Wut? So, GRUB detected that /zroot is a ZFS (otherwise, how did it know about ata-QEMU stuff?), but if fails to resolve correct path to device.

Better question still: Why is it interested in that device in the first place?

The man page for grub-probe say:

Probe device information for a given path

But for a zfs filesystem, this should be the dataset, not the/a physical device in the pool!

I'm specifying the device to write to on the grub-install command line, so it shouldn't even care about
that device...

PS. I got the same error with the latest grub. Ever since I started trying to use a ZFS root in April, I've been having trouble with grub (mostly grub-probe). I had to resort to a hacked grub-probe (a shell script with hardcoded values.

[seletskiy]

Better question still: Why is it interested in that device in the first place?

Truth! I was experimenting on a single vdev configuration, where this approach to access a device can be somehow doable, but in stripped or raidz configuration this would be utterly wrong...

[0x54726F6E]

I can confirm this is still an issue with debian wheezy which grub-mkconfig 2.01-22debian1+zfs3~wheezy

Running update-grub with set -x/+x set in grub-probe and /etc/grub/00_header will show that the problem is that grub-probe runs on /dev/sda1 to probe for the filesystem. Which is "unknown" to grub-probe. Some purging and reinstalling later the error still is

grub-probe: error: failed to get canonical path of udev. So no booting from a zfs pool for now.

[Logos01]

I am also experiencing the behavior of failed grub installations. Ubuntu 14.04 w/ latest (as of 2014-04-30). I am also receiving the "grub-probe: error: failed to get canonical path of 'udev'." error.

[maci0]

i have a similar problem
grub2-probe fails when 'zpool' is not in the path

sh-4.2# /usr/sbin/grub2-probe --target=device /
/usr/sbin/grub2-probe: error: failed to get canonical path of ‘ZoL-2316/ROOT/rhel7’.
sh-4.2# export PATH=$PATH:/usr/sbin
sh-4.2# /usr/sbin/grub2-probe --target=device /
/dev/nbd0p2

[erpadmin]

I use ryao's method of full disk zfs which is a bit different to your approach (will be shown later)

anyway this is not an issue for me on raring (more specifically Mint 15), but the error the OP reported cropped up when testing out saucey (Mint 16). I had to use rarings zfs grub ppa as there doesn't seem to be a version specific to saucey.

Anyway here's some output on the new system:

.# grub-probe -V
grub-probe (GRUB) 2.00-19ubuntu2.1

.# grub-probe /
grub-probe: error: failed to get canonical path of /dev/ata-ST31500341AS_9VS40QN8.

.# grub-probe --target=device /
grub-probe: error: failed to get canonical path of /dev/ata-ST31500341AS_9VS40QN8.

.# /tmp/grub-probe -V
/tmp/grub-probe (GRUB) 2.00-13ubuntu3+zfs3~raring

.# /tmp/grub-probe /
/tmp/grub-probe: error: unknown filesystem

.# /tmp/grub-probe --target=device /
/dev/sdc1

On the older system grub-probe works fine:

olivia ~ # zpool status
  pool: pool_one
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
    still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
    pool will no longer be accessible on software that does not support
    feature flags.
  scan: resilvered 102G in 0h42m with 0 errors on Sun Oct 28 13:51:14 2012
config:

    NAME        STATE     READ WRITE CKSUM
    pool_one    ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        sdb     ONLINE       0     0     0
        sda     ONLINE       0     0     0

errors: No known data errors
olivia ~ # grub-probe /
zfs
olivia ~ # gdisk -l /dev/sda
GPT fdisk (gdisk) version 0.8.5

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 976773168 sectors, 465.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 91F0CCEC-AC16-F746-883C-909DF568C93B
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 976773134
Partitions will be aligned on 16-sector boundaries
Total free space is 29 sectors (14.5 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048       976756735   465.8 GiB   BF01  zfs
   2              48            2047   1000.0 KiB  EF02  BIOS boot partition
   9       976756736       976773119   8.0 MiB     BF07  

[dasjoe]

As a workaround I have udev create the required /dev/ata-* symlinks:

# cat > /etc/udev/rules.d/70-zfs-grub-fix.rules << 'EOF'
KERNEL=="sd*[a-z]1|cciss*[a-z]1", ENV{DEVTYPE}=="partition", ENV{ID_SERIAL}=="?*", SYMLINK+="$env{ID_BUS}-$env{ID_SERIAL}"
EOF
# udevadm trigger

[Logos01]

This workaround doesn't achieve correction of the problem behavior for me.

On Thu, May 29, 2014 at 5:38 AM, dasjoe notifications@github.com wrote:

As a workaround I have udev create the required /dev/ata-* symlinks:

cat > /etc/udev/rules.d/70-zfs-grub-fix.rules << 'EOF'KERNEL=="sd_[a-z]1|cciss_[a-z]1", ENV{DEVTYPE}=="partition", ENV{ID_SERIAL}=="?*", SYMLINK+="$env{ID_BUS}-$env{ID_SERIAL}"

EOF# udevadm trigger

—
Reply to this email directly or view it on GitHub
#5 (comment).

[storepeter]

The rule above did not work for me, here is a rule, which works for me

KERNEL=="sd*[0-9]", IMPORT{parent}=="ID_*", ENV{ID_FS_TYPE}=="zfs_member", SYMLINK+="$env{ID_BUS}-$env{ID_SERIAL}-part%n"

[nelsoncs]

14.04 broke for me at an update to kernel 3.13.0-30. Originally I had used the raring zfs-grub version, but it now had failed. After remiving zfs-grub and going back to trusty grub version (see https://groups.google.com/a/zfsonlinux.org/forum/#!topic/zfs-discuss/050qD_mvMAg). I was having problem with the incorrect /dev/.....

So, thank you very much for this fix. I am back in business.

[maci0]

recently ran into the same error when using whole devices (on rhel7)

[seletskiy]

Do anyone know, is this repo even alive? It's get ignored for about an year...

[maci0]

with gpt disks maybe using /dev/disk/by-partuuid helps ?

[maci0]

ok, it didnt help...

also when using /dev/sd* and not /dev/disk/by-id the same problems occurs.

when using /dev/disk/by-id and @dasjoe 's udev rule it partly works. but fails when the udev rule is not included in the initram image.