Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to selectively build libvips for windows? #42

Closed
galeone opened this issue Apr 13, 2022 · 13 comments
Closed

How to selectively build libvips for windows? #42

galeone opened this issue Apr 13, 2022 · 13 comments
Labels
question Further information is requested

Comments

@galeone
Copy link

galeone commented Apr 13, 2022

Hi! I'm trying to integrate libvips inside a project that already has lots of the dependencies that libvips also has.
Moreover, I'm only interested in integrating the pdf import feature.

On linux I can easily build libvips in this way:

PKG_CONFIG_PATH=$(pwd)/lib/pkgconfig/ meson setup build-dir --prefix=/tmp/vips \
    -Dcfitsio=disabled \
    -Dcgif=disabled \
    -Dexif=disabled \
    -Dfftw=disabled \
    -Dfontconfig=disabled \
    -Dgsf=disabled \
    -Dheif=disabled \
    -Dheif-module=disabled \
    -Dimagequant=disabled \
    -Djpeg=disabled \
    -Djpeg-xl=disabled \
    -Djpeg-xl-module=disabled \
    -Dlcms=disabled \
    -Dmagick=disabled \
    -Dmagick-module=disabled \
    -Dmatio=disabled \
    -Dnifti=disabled \
    -Dopenexr=disabled \
    -Dopenjpeg=disabled \
    -Dopenslide=disabled \
    -Dopenslide-module=disabled \
    -Dorc=disabled \
    -Dpangocairo=disabled \
    -Dpdfium=disabled \
    -Dpng=disabled \
    -Dquantizr=disabled \
    -Drsvg=disabled \
    -Dspng=disabled \
    -Dtiff=disabled \
    -Dwebp=disabled \
    -Dzlib=disabled \
    -Dpoppler-module=disabled \
    -Dpoppler=disabled \
    -Dpdfium=enabled \
    --buildtype=release

and having a very minimal libvips instance with pdfium support.

For windows, I'd like to do the same but I'm completely lost in this build system.
I've seen that pdfium is not an option (in fact in the build you provide there's no pdfium), but poppler is a valid alternative.

So here's the question: do you have some hint on how to proceed to build a very minimal version of libvips with only the poppler suppìort - on windows?

@kleisauke kleisauke added the question Further information is requested label Apr 13, 2022
@kleisauke
Copy link
Member

I think the easiest way to exclude unneeded dependencies is to adapt the vips-all_DEPS variable. For example, you could change:

$(PKG)_DEPS := cc libwebp librsvg glib pango libgsf \
libjpeg-turbo tiff lcms libexif libheif libpng \
libspng libimagequant orc imagemagick matio openexr \
cfitsio nifticlib poppler fftw openslide libjxl cgif

to:

$(PKG)_DEPS     :=  cc glib expat poppler

To ensure only libvips (+ required dependencies) and Poppler (+ sub-dependencies) are build. Though, you may also need to do the same for the poppler_DEPS variable.

poppler_DEPS := cc cairo libjpeg-turbo freetype glib openjpeg lcms libpng tiff zlib

However, this may still cause libvips to link against the sub-dependencies that Poppler brings in. To avoid this, you can build libvips with:

--without-lcms \
--without-libopenjp2 \
--without-zlib \
--without-libwebp \
--without-fontconfig \
--without-tiff \
--without-png \
--without-libexif \
--without-jpeg \
--without-ppm \
--without-analyze \
--without-radiance \
--disable-modules \

in build/vips-all.mk.

@galeone
Copy link
Author

galeone commented Apr 13, 2022

Thank you for the quick answer! I test it and I'll get back to you soon

@kleisauke
Copy link
Member

kleisauke commented Apr 13, 2022

Note that within the Meson build system, available in the future libvips v8.13 (see the meson branch), this will be much easier by using the -Dauto_features=disabled option to disable all features and explicitly enable only the ones you need. For example:

-Ddeprecated=false \
-Dintrospection=false \
-Dauto_features=disabled \
-Dpoppler=enabled \
-Dnsgif=false \
-Dppm=false \
-Danalyze=false \
-Dradiance=false \

@galeone
Copy link
Author

galeone commented Apr 13, 2022

That will be a huge improvement in the building configuration for sure, writing that configuration took me some time.

Anyway, this is the patch I created for my configuration

diff --git a/build/vips-all.mk b/build/vips-all.mk
index 6edc806..0e57bfb 100644
--- a/build/vips-all.mk
+++ b/build/vips-all.mk
@@ -8,10 +8,7 @@ $(PKG)_PATCHES  := $(realpath $(sort $(wildcard $(dir $(lastword $(MAKEFILE_LIST
 $(PKG)_GH_CONF  := libvips/libvips/releases,v
 $(PKG)_SUBDIR   := vips-$($(PKG)_VERSION)
 $(PKG)_FILE     := vips-$($(PKG)_VERSION).tar.gz
-$(PKG)_DEPS     := cc libwebp librsvg glib pango libgsf \
-                   libjpeg-turbo tiff lcms libexif libheif libpng \
-                   libspng libimagequant orc imagemagick matio openexr \
-                   cfitsio nifticlib poppler fftw openslide libjxl cgif
+$(PKG)_DEPS     := cc glib expat poppler
 
 define $(PKG)_PRE_CONFIGURE
     # Copy some files to the packaging directory
@@ -80,6 +77,19 @@ define $(PKG)_BUILD
         --enable-debug=no \
         $(if $(IS_LLVM),, --without-libjxl) \
         --without-pdfium \
+               --without-lcms \
+               --without-libopenjp2 \
+               --without-zlib \
+               --without-libwebp \
+               --without-fontconfig \
+               --without-tiff \
+               --without-png \
+               --without-libexif \
+               --without-jpeg \
+               --without-ppm \
+               --without-analyze \
+               --without-radiance \
+               --disable-modules \
         --disable-introspection \
         --disable-deprecated \
         --with-heif=$(if $(IS_HEVC),module,yes) \

I invoked the scrip with:

./build.sh --nightly all x86_64 shared

Is it an expected behavior the usage of only 4 threads while compiling? It's taking a very long time, in fact, I'm still waiting for this step, this is the output

== General overrides: /data/overrides.mk
[plugin]      /data/plugins/nightly/
[plugin]      /data/plugins/mozjpeg/
[plugin]      /data/plugins/zlib-ng/
[plugin]      /data/plugins/llvm-mingw/
[plugin]      /data/plugins/proxy-libintl/
[build]       mxe-conf                 x86_64-w64-mingw32.shared.posix.all
[done]        mxe-conf                 x86_64-w64-mingw32.shared.posix.all        12 KiB         0m1.691s
[build]       ccache                   x86_64-w64-mingw32.shared.posix.all
[done]        ccache                   x86_64-w64-mingw32.shared.posix.all        2264 KiB       0m1.700s
[build]       gendef                   x86_64-w64-mingw32.shared.posix.all
[done]        gendef                   x86_64-w64-mingw32.shared.posix.all        115756 KiB     0m3.235s
[build]       llvm                     x86_64-pc-linux-gnu

@kleisauke
Copy link
Member

By default, it will run as many parallel jobs as there are available CPU cores. The first time build will take a long time indeed, since the whole of LLVM must be built from source.

@galeone
Copy link
Author

galeone commented Apr 13, 2022

Got it.

It reached the and but, unfortunately, it fails during the linking phase

014 libtool: link: x86_64-w64-mingw32.shared.posix.all-gcc -O3 -fdata-sections -ffunction-sections -Wl,--gc-sections -Wl,-s -Wl,--gc-sections -Wl,-s -o .libs/vips.exe vips.o -mms-bitfields      -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/include/poppler/glib -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/include/poppler -I/data/mxe/usr/x86_64-w64-mingw32.shared.     posix.all/include/cairo -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/include/freetype2 -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/include/libpng16 -I/data/mxe/usr/x86_6     4-w64-mingw32.shared.posix.all/include/harfbuzz -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/include/glib-2.0 -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/lib/glib-2.0/in     clude -mms-bitfields -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/include/pixman-1 -DG_DISABLE_ASSERT -DG_DISABLE_CHECKS -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/incl     ude/glib-2.0 -I/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/lib/glib-2.0/include -mms-bitfields  ../libvips/.libs/libvips.dll.a /data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/l     ib/libexpat.dll.a -lpoppler-glib -lcairo -lgmodule-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -L/data/mxe/usr/x86_64-w64-mingw32.shared.posix.all/lib

d.lld: error: undefined symbol: g_libintl_gettext
referenced by vipsthumbnail.o:(main)

I don't really know where that symbol comes from - google is not that helpful in this case. Any guess?

kleisauke added a commit that referenced this issue Apr 13, 2022
- Update OpenEXR to 3.1.5.
- Fix build with GraphicsMagick (#41).
- Fix nightly build (#42).

MXE Updates:
- Update MinGW-w64 to 10.0.0.
@kleisauke
Copy link
Member

It looks like we still need to include -lintl in glib-2.0.pc if we're building libvips from tip-of-tree. I just fixed this with commit 8f930b9.

You may need to invalidate GLib's build cache by removing build/mxe/usr/x86_64-w64-mingw32.shared.posix.all/installed/glib.

@galeone
Copy link
Author

galeone commented Apr 14, 2022

Thank you for your support, I've been able to build it.

Anyway, it doesn't work. When I try to load the library, it automatically searches for the various DLLs compiled, and when it tries to preload libharfbuzz-0.dll the loading stops, and the application crashes.
I'm trying to write an unreal engine plugin, so that's the report I'm getting

[2022.04.14-14.42.02:400][  0]LogWindows: Failed to preload 'C:/project/Plugins/VIPS4UE/Binaries/Win64/libharfbuzz-0.dll' (GetLastError=126)
[2022.04.14-14.42.02:400][  0]LogWindows:   Missing import: libfreetype-6.dll

I used dependencies for understanding what's going on, and libharfbuzz-0.dll, libfreetype-6.dll (that's in the same directory) as well as all the other libraries looks ok.

If it would be able to compile with pdfium I won't have these problems, because all these libraries are dependencies of poppler.
Integrating libvips on Linux has been really straightforward, but I guess the integration on this build tool for windows would be complicated - even though I'm interested in the libvips C library (for avoiding [reducing] ABI compatibility issues).

Anyway, you've been really helpful so far, and thank you for pointing me in the right direction for compiling libvips on Windows.
If you have some other hint about how this error during preloading may come from I would really appreciate it.

@kleisauke
Copy link
Member

Does the pre-built -all or -static binaries (as released on https://github.com/libvips/build-win64-mxe/releases) work on Unreal Engine? If not, I'm not sure what's causing that preload error.

I'm not sure if a PDFium build (see #20) would resolve this, since it also depends on FreeType. However, PDFium does have a permissive license that allows the distribution of static-linked binaries (this not the case with Poppler which is licensed under the GPL).

@galeone
Copy link
Author

galeone commented Apr 15, 2022

With the -static package it works. Or better, it can be linked with the software, but of course, at runtime it fails because I use the vips_pdfload that always returns NULL, not having a valid implementation under the hood. This would be the ideal solution together with pdfium. NOTE: I link all the .lib and .dll, except for libvips-cpp (dll and lib), because I want to use the C API and reduce the ABI compatibility problems that might be present using C++.

With the -all instead, I endup with the very same error:

LogWindows: Failed to preload 'C:/project/Plugins/VIPS4UE/Binaries/Win64/libharfbuzz-0.dll' (GetLastError=126)
LogWindows:   Missing import: libfreetype-6.dll

maybe it's because libfreetype is already shipped by unreal engine and loaded before, and perhaps this creates a conflict

@galeone
Copy link
Author

galeone commented Apr 15, 2022

I managed to build a very minimal version of livbips (with pdfium support!) in shared configuration and it works!

It's a very manual process (because I'm not familiar with the build tool) but after the holidays I plan to write some lines about this, so maybe it can help you in doing it in the right way 🥳

@galeone
Copy link
Author

galeone commented Apr 20, 2022

@kleisauke I wrote some lines about the pdfium support here #20 (comment)

I hope it can help you automatize this manual process - at least there's the proof that it works and it's possible to integrate pdfium on windows build

@kleisauke
Copy link
Member

Closing in favor of #20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants