How to make Pillow.Image.alpha_composite logic run on GPU?

Question

Python

Hey!

In the Pillow library, the Image class has a wonderful function - alpha_composite. If she slips two Images in the RGBA profile, then she will glue the two images very correctly, namely:
1. If both images had transparent areas, then the resulting image will also be transparent.
2. If the one that is placed on top has a translucent area, then the one below will correctly "shine through" through the top
3. If it is opaque on top, then it ignores the underlying pixels.

The problem is that she does it rather slowly. I ran in multiprocessing on all cores, but it seems that this is the limit for the CPU, even Pillow-SIMD does not give a noticeable gain.

So I turned my attention to the GPU. But then another problem arose - no other library that I tried has a mode in which Pillow.Image.alpha_composite works - or I did not figure it out. OpenCV in all modes in which I tried and according to all the advice from SO does not do what is needed - the transparent pixels of the upper image are either not taken into account, or black is replaced with a transparent color in the image itself. pyVips in Image.composite mode does the same as OpenCV.

Can you please tell me how to glue images like Pillow.Image.alpha_composite on GPU? Maybe not even in Python.

Answer 1

even Pillow-SIMD does not give a noticeable gain

If Pillow-SIMD does not give a win, then the problem is something else.
Judging by the benchmarks, the performance gain is up to 4 times with AVX2
https://python-pillow.org/pillow-perf/

Please note that you need to compile Pillow-SIMD with AVX2 support yourself

$ pip uninstall pillow
$ CC="cc -mavx2" pip install -U --force-reinstall pillow-simd

Also, compressing an image while saving it consumes a significant amount of resources.
PNG images can be compressed less aggressively like this: To save as JPEG if you don't need transparency after merging, you need to install the SIMD version of libjpeg - libjpeg-turbo https://fastai1.fast.ai/performance.html#is-jpeg-c. ..
im.save('png.png', compress_level=1)

Link content

Is JPEG compression SIMD-optimized?
libjpeg-turbo replacement for libjpeg is SIMD-optimized. In order to get Pillow or its faster fork Pillow-SIMD to use libjpeg-turbo, the latter needs to be already installed during the former’s compilation time. Once Pillow is compiled/installed, it no longer matters which libjpeg version is installed in your virtual environment or system-wide, as long as the same libjpeg library remains at the same location as it was during the compilation time (it’s dynamically linked).
However, if at a later time something triggers a conda or pip update on Pillow it will fetch a pre-compiled version which most likely is not built against libjpeg-turbo and replace your custom built Pillow or Pillow-SIMD.
Here is how you can see that the PIL library is dynamically linked to libjpeg.so:
cd ~/anaconda3/envs/fastai/lib/python3.6/site-packages/PIL/
ldd _imaging.cpython-36m-x86_64-linux-gnu.so | grep libjpeg
libjpeg.so.8 => ~/anaconda3/envs/fastai/lib/libjpeg.so.8
and ~/anaconda3/envs/fastai/lib/libjpeg.so.8 was installed by conda install -c conda-forge libjpeg-turbo. We know that from:
cd ~/anaconda3/envs/fastai/conda-meta/
grep libjpeg.so libjpeg-turbo-2.0.1-h470a237_0.json
If I now install the normal libjpeg and do the same check on the jpeg’s package info:
conda install jpeg
cd ~/anaconda3/envs/fastai/conda-meta/
grep libjpeg.so jpeg-9b-h024ee3a_2.json
I find that it’s lib/libjpeg.so.9.2.0 (~/anaconda3/envs/fastai/lib/libjpeg.so.9.2.0).
Also, if libjpeg-turbo and libjpeg happen to have the same version number, even if you built Pillow or Pillow-SIMD against libjpeg-turbo, but then later replaced it with the default jpeg with exactly the same version you will end up with the slower version, since the linking happens at build time. But so far that risk appears to be small, as of this writing, libjpeg-turbo releases are in the 8.x versions, whereas jpeg’s are in 9.x’s.
How to tell whether Pillow or Pillow-SIMD is using libjpeg-turbo?
You need Pillow>=5.4.0 to accomplish the following (install from github until then: pip install git+https://github.com/python-pillow/Pillow).
python -c "from PIL import features; print(features.check_feature('libjpeg_turbo'))"
True
And a version-proof check:
from PIL import features, Image
from packaging import version
try: ver = Image.__version__ # PIL >= 7
except: ver = Image.PILLOW_VERSION # PIL < 7
if version.parse(ver) >= version.parse("5.4.0"):
if features.check_feature('libjpeg_turbo'):
print("libjpeg-turbo is on")
else:
print("libjpeg-turbo is not on")
else:
print(f"libjpeg-turbo' status can't be derived - need Pillow(-SIMD)? >= 5.4.0 to tell, current version {ver}")