Repository navigation
Would you accept a PR for automatic macOS and Linux wheel generation? #149
Description
I'm interested in switching my project, OCRmyPDF, to PyMuPDF as the PDF backend since it's far superior to PyPDF2 and the other alternatives, and would probably eliminate my dependency on Ghostscript as well.
However, PyMuPDF installation on non-Windows is a bit of a pain right now, ultimately because Artifex doesn't provide shared libraries for MuPDF.
Would you accept a PR that added a Travis CI compile and deploy step to automatically build binary wheels for PyMuPDF? Basically I want to make sure you don't have some philosophical objection or already have work in progress before I spend time on it. Would boil down if, there is a tagged release, installed the appropriate Homebrew packages, generate a binary wheel and upload it to PyPI. The benefit is that this would enable mac users to "pip install pymupdf" without needing homebrew + gcc + mupdf-tools installed.
Activity
[-]Would you accept a PR for automatic macOS wheel generation?[/-][+]Would you accept a PR for automatic macOS and Linux wheel generation?[/+]jbarlow83 commentedon Mar 18, 2018
Updated initial comment because I found a straightforward way to make manylinux1 wheels too.
JorjMcKie commentedon Mar 18, 2018
Hi there,
I would be more than happy if we could incorporate an elegant way of installing on non-Windows systems!
I am not the repo owner, but I am sure @rk700 will have no objections either!
My skills on non-Windows systems is too low to contribute much - except maybe testing any new installlation procedure on Linux virtual machines (Oracle Virtual Box).
jbarlow83 commentedon Mar 19, 2018
Great!
rk700 commentedon Mar 20, 2018
@jbarlow83 Thank you so much for the help! Now Travis CI is integrated and the wheels can be built successfully.
However it failed when uploading wheels to PyPI: https://travis-ci.org/rk700/PyMuPDF/builds/355674679
I'll take a deeper look at the issue.
jbarlow83 commentedon Mar 20, 2018
I have one of my projects uploading to legacy PyPI since it saved hassle. May be this will help.
You could change the upload so that non-tagged builds go to PyPI, as a way of confirming things work without having to tag everything, e.g.
jbarlow83 commentedon Mar 21, 2018
I checked the Travis logs again. Did you set the environment variable
TWINE_PASSWORDin Travis settings to the password for rk700's PyPI account? That would explain the auth failure.On this screen:
JorjMcKie commentedon Mar 21, 2018
First of all, thank you very much for your contribution to our repo!
It gave me a kick to look around for other options on how to use Travis.
A first observation on the generated wheels (whether they have yet been successfully passed on to PyPI or not): they look really big compared to my homegrown Windows wheels: Linux 26.x MB versus less than 3 MB on Windows!
Once I can get my hands on one of the Linux / MAC wheels I will curiously investigate what is going on.
My first guess:
I am configuring my local MuPDF via
/mupdf/include/mupdf/fitz/config.h, before I generate MuPDF object and PyMuPDF binaries. This mainly boils down to / kicking out the plethora of NOTO fonts present in the original MuPDF source. They are of no use in PyMuPDF. And I know from my Windows environment, that massagingconfig.hbrings down the PyMuPDF binary size down to below 5 MB.As you have been so resourceful: would you have an idea on how to force Travis to also use a modified
config.h?jbarlow83 commentedon Mar 21, 2018
You're welcome... I've learned a lot about Travis on a few different projects so it really didn't take that long.
Yes, fonts could easily explain the size difference.
I should probably explain how it actually works.
The
install:block in .travis.yml downloads$MUPDFand extracts it to a folder namedmupdf/. Thencibuildwheelkicks off a Docker container running on Travis, which has access the current directory, and runs the build, leaving the results in the same folder. The reason for the Docker container is that Travis runs Ubuntu 14.04, and manylinux1 wheels must be built on CentOS 5 for compatibility. TheCIBW_BEFORE_BUILDenvironment passes commands to the Docker container (i.e. before it build the wheels, runmakeon mupdf).So right after this command
...you can patch mupdf any way you see fit, and cibuildwheel will build the modified code the same way for macOS and Linux.
I think the easiest option will be to include a copy of
config.hin PyMuPDF. Travis will have a git checkout of PyMuPDF in the current working directory, so you just copy it to themupdf/directory we downloaded and set up.Or you could do a
diff/patchofconfig.h, so it's more robust to upstream changes.