Why won't USB flash drive be unmounted in Linux?

Question

linux

Centos 6. I mount the flash drive:
mount /dev/sdb4 /mnt/usb
Everything is fine, I see the files, the mount command shows the mount point.
When unmounting, it swears that the device is busy, unmount it:
umount -l /mnt/usb
After that, the mount command shows that sdb4 is not mounted anywhere, but there are still files in the /mnt/usb directory, and they can be changed, and the changes are written to the USB flash drive. If you pull it out, everything will be saved.
What am I doing wrong?

Answer 1

lsof | grep '/mnt/usb'
Should show what's holding her there.