[regression] Terminates with segfault in InitializeGmm/InitContext

[paulmenzel]

After updating Debian’s libigdgmm12 package from 22.0.2+ds1-1 to 22.1.1+ds1-1, all program’s using hardware acceleration terminate with segmentation fault:

1. Firefox/Nigthly
2. mpv

Installing the package with debug symbols, shows the trace below for mpv --hwdec=vaapi /usr/share/help/pt/gnome-help/figures/display-dual-monitors.webm:

#0  0x00007f05e81449ae _ZN6GmmLib7Context11InitContextERK12PLATFORM_STRPK18_SKU_FEATURE_TABLEPK9_WA_TABLEPK14GT_SYSTEM_INFO15GMM_CLIENT_ENUM (libigdgmm.so.12 + 0x709ae)
#1  0x00007f05e81520bc InitializeGmm (libigdgmm.so.12 + 0x7e0bc)                 
#2  0x00007f05ce13064a n/a (iHD_drv_video.so + 0x49c64a)
#3  0x00007f05ce130a2f __vaDriverInit_1_14 (iHD_drv_video.so + 0x49ca2f)
#4  0x00007f060f873f2c n/a (libva.so.2 + 0x3f2c)
#5  0x00007f060f8750a6 vaInitialize (libva.so.2 + 0x50a6)
#6  0x0000561696f52c75 va_initialize (mpv + 0x157c75)
#7  0x0000561696f252fd init (mpv + 0x12a2fd)
#8  0x0000561696f0b171 ra_hwdec_load_driver (mpv + 0x110171)
#9  0x0000561696f16904 load_add_hwdec (mpv + 0x11b904)
#10 0x0000561696f20204 gl_video_load_hwdecs_all (mpv + 0x125204)
#11 0x0000561696f3598b request_hwdec_api (mpv + 0x13a98b)
#12 0x0000561696f31460 run_control (mpv + 0x136460)
#13 0x0000561696e98d3f mp_dispatch_queue_process (mpv + 0x9dd3f)
#14 0x0000561696f32b37 vo_thread (mpv + 0x137b37)
#15 0x00007f06124ecd80 start_thread (libpthread.so.0 + 0x7d80)
#16 0x00007f060f24d76f __clone (libc.so.6 + 0xfa76f)

[calestyo]

Just for the records, my downstream bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007992

Also, vlc is affected, too,... if it's really dependant on VAAPI, then that must select it automatically (cause I didn't configure it to specifically).

[Niko2040]

Confirm bug on Debian Unstable with libigdgmm12 version 22.1.1+ds1-1 , even vainfo crashes with segfault. Downgraded to previous version as workaround.

bt log:

#0  GmmCreateLibContext (Platform=..., pSkuTable=<optimized out>, pWaTable=0x300000200, pGtSysInfo=0xa800000018, sBdf=...)
    at ./Source/GmmLib/GlobalInfo/GmmInfo.cpp:155
#1  0x00007ffff55ef0bc in InitializeGmm (pInArgs=0x7ffffffef590, pOutArgs=0x7ffffffef508)
    at ./Source/GmmLib/GlobalInfo/GmmLibDllMain.cpp:57
#2  0x00007ffff5a9964a in DdiMedia_InitMediaContext (ctx=0x55555555f600, devicefd=<optimized out>, 
    major_version=<optimized out>, minor_version=<optimized out>, apoDdiEnabled=<optimized out>)
    at ./media_driver/linux/common/ddi/media_libva.cpp:1937
#3  0x00007ffff5a99a2f in DdiMedia__Initialize (minor_version=0x0, major_version=0x0, ctx=0x55555555f600)
    at ./media_driver/linux/common/ddi/media_libva.cpp:1691
#4  __vaDriverInit (ctx=0x55555555f600) at ./media_driver/linux/common/ddi/media_libva.cpp:7411
#5  __vaDriverInit_1_14 (ctx=0x55555555f600) at ./media_driver/linux/common/ddi/media_libva.cpp:7437
#6  0x00007ffff7e20f2c in va_openDriver (dpy=dpy@entry=0x55555555f490, driver_name=<optimized out>) at ./va/va.c:526
#7  0x00007ffff7e220a6 in vaInitialize (dpy=dpy@entry=0x55555555f490, major_version=major_version@entry=0x7fffffffdd40, 
    minor_version=minor_version@entry=0x7fffffffdd48) at ./va/va.c:746
#8  0x000055555555642e in main (argc=<optimized out>, argv=0x7fffffffde78) at vainfo.c:520

[johnmach]

Hi paulmenzel,

Thank you for raising this issue.
We have started debugging this issue and hopefully we will fix this asap.

Thanks.

[johnmach]

Hi paulmenzel,

We have root caused the issue and fix is on its way. We will release the fix asap.

-Thanks

[paulmenzel]

Awesome. Thank you for the update.

It’d be great, if you added the line

Resolves: #95

at the end of the commit message. Then GitHub is going to automatically close the issue.

[Sherry-Lin]

@paulmenzel what's the media-driver version you used? Have you re-build driver after GmmLib updated?

[rra]

Just in case it helps save anyone else debugging time, this also affects Google Chrome (via the google-chrome-stable or google-chrome-unstable packages) with Debian unstable. The symptom of the Google Chrome problems is the rather generic error message:

[9033:9033:0321/195424.214284:ERROR:gpu_process_host.cc(972)] GPU process exited unexpectedly: exit_code=11

followed by all WebGL pages failing to load.

[paulmenzel]

@paulmenzel what's the media-driver version you used?

$ dpkg -l intel-media-va-driver-non-free
intel-media-va-driver-non-free:amd64 22.2.1+ds1-1

Have you re-build driver after GmmLib updated?

I do not build anything myself, and just install the Debian packages. But no, it looks like [it was not rebuilt] as the upload is from February 23rd, 2022.

But why should it be rebuilt? The library interface should be compatible, as the SO name was not increased.

$ ldd /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
    linux-vdso.so.1 (0x00007ffe9eb5b000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbe43592000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbe4358c000)
    libigdgmm.so.12 => /lib/x86_64-linux-gnu/libigdgmm.so.12 (0x00007fbe43501000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fbe432e4000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbe431a0000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fbe43180000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbe42fa5000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fbe4593e000)

PS: You can check the Debian versions at https://packages.debian.org, and even download the packages. Installing Debian in a VM is also very easy with grml-debootstrap for example.

[sebastinas]

Just to reiterate what @paulmenzel already said as the Debian maintainer of the media driver stack: if a rebuild fixes the issue, then gmmlib broke its ABI and needs to bump the SONAME.

[jwilk]

Rebuilding intel-media-driver fixes it for me.

[johnmach]

We identified that one of the Gmmlib patch caused break in backward compatibility.

The fix is merged in 29152fc.

The release is in progress. We will post the release tag soon.

[paulmenzel]

Thank you for analyzing this.

I would have hoped you heeded my suggestions regarding the commit. Fix Regression on debian (#97) is a misleading commit message, as the regression probably is not Debian specific at all, and lacks any details, what commit introduced it, and how it is fixed, and why that fix is correct now.

[johnmach]

@paulmenzel,

Indeed, We will definitely heed to the aspects which you have brought about.

One of the patch in intel-gmmlib-22.0.3 has caused the backward compatibly break in Gmmlib. This got introduced as a bug during the implementation of some feature. So the current version intel-gmmlib-22.1.1 is unstable. Kindly refrain from using this.
Use intel-gmmlib-22.0.2 for older ABI/API until the next release.

The fix 29152fc for this is already merged and release tag will be posted soon.

Usually, We fix the regression first, merge it to the external repo following standard flow and create a release tag, then update the external PR/issue with the details and then close it. And currently, we are waiting to release the tag after which we would update all details in this issue and close.

As in reply for #88 (comment), Going forward will make sure to incorporate the addressable suggestions w.r.t the better readability of the commit details.