GEM objects / shmem file leakage

[yanko-yankulov]

Hi all,

There is a GEM objects leakage after an mmap. The cause I believe is in the linuxkpi.
In the linux_cdev_pager_ctor function we call vm_ops->open(), but this seems to not be required. The open call has a misleading name - it would be better called reopen. Linux uses it only in two places __split_vma (for partial unmap) and copy_vma( for mremap ). In both cases it modifies a preexisting vma by first copying it and then closing the old one.

So when we call vm_ops->open() from the linux_cdev_pager_ctor for a drm_gem_object VMA we are acutually doing double increment of the object refcount and leave the object unfreeable as a result.

Just removing the "vmap->vm_ops->open(vmap);" from the linux_cdev_pager_ctor seems to solve this. I've been running for some time now with such a patch and everything seems OK. I have also done a debug instrumentation to atomically count the created/destroyed shmem pseudo files and with the patch the leakage is gone.

Best,
Yanko

[hselasky]

This issue should be fixed. Please re-open if still present.