Some videos cause complete system hang with 0.4.0

[rathann]

This Apple trailer for Arrival (http://trailers.apple.com/movies/paramount/arrival/arrival-final-trailer_h1080p.mov) causes mplayer -vo vdpau -vc ffh264vdpau, (or mpv --vo=vdpau --hwdec=vdpau) to freeze my entire machine. Downgrading to 0.3.6 "fixes" the issue. Also, playing the same video via VA-API directly (for example, using mpv --vo=vaapi --hwdec=vaapi) works just fine even with 0.4.2.
Which information do you need to debug this?

[i-rinat]

Well, I tried that on my machine, and it crashed the Linux kernel. So it's actually a kernel bug. (No application should be able to crash kernel whatever it does.)

I'll try to figure out differences between vdpau path and vaapi paths in mpv + libvdpau-va-gl. Perhaps, some workarounds are possible.

[rathann]

It happens even with 0.4.0. Updated title to reflect this. It seems to be a bug in the i915 driver. I already reported this in Fedora's bug tracker: https://bugzilla.redhat.com/show_bug.cgi?id=1392164 and I'll try to report it in kernel.org bug tracker as well (it usually works better 😄)

[rathann]

Reported in kernel.org bugzilla as well: https://bugzilla.kernel.org/show_bug.cgi?id=187751 .

[rathann]

Just noting that the correct upstream bug tracker is at FDO, where I reported this as well, https://bugs.freedesktop.org/show_bug.cgi?id=98760 . Although with the recent change to use the modesetting driver instead of intel/i915 in Debian (https://tjaalton.wordpress.com/2016/07/23/intel-graphics-gen4-and-newer-now-defaults-to-modesetting-driver-on-x/) and Fedora (https://lists.fedoraproject.org/archives/list/[email protected]/message/AUPYPJMFJZMHPEKN24LBABZKIEOV4NU5/), this might be moot. I haven't tested with the modesetting driver yet, though.

[i-rinat]

Oh, thanks.

I think, I tested with modesetting driver, with the same whole system hang, so switching to modesetting won't help. Most probably, the cause is incomplete parsing in libvdpau-va-gl, which in turn passes broken data to libva, and then to the hardware through kernel driver. So it could be mitigated in libvdpau-va-gl, but it still would be nice to have stronger validation in VA-API driver itself.

Unfortunately, I'm still unable to find time to figure out what goes wrong.

[rathann]

You're right, it's still reproducible with the modesetting driver. I've just checked myself, too. For now, I'm staying with libvdpau-va-gl 0.3.6 as a work around.

[rathann]

Looks like the freeze (actually: consumption of large amounts of memory which triggers oom-killer eventually) is gone with kernel 4.11.x. However, the video image is almost all green now.