sqlite3 is broken under Python 3.6

[ogrisel]

Here is the relevant snippet from the build log:

*** WARNING: renaming "_sqlite3" since importing it failed: build/lib.linux-x86_64-3.6/_sqlite3.cpython-36m-x86_64-linux-gnu.so: undefined symbol: sqlite3_stmt_readonly
The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
atexit                pwd                   time               

Following modules built successfully but were removed because they could not be imported:
_sqlite3                                                       

This issue is being discussed here: ghaering/pysqlite#85

The problem is that apparently Python 3.6 requires a more recent version of libsqlite3 that the one available on old centos. We could build or own sqlite3 from source in the docker image but the configure script currently does not make it possible to pass a path to a custom sqlite3 install.

[rmcgibbo]

My first thought is that we should just build the most recent libsqlite into the docker image.

[rmcgibbo]

And if we just put it at the regular location (/usr/lib), there's no need to pass a custom location to the Python build system.

[njsmith]

It's possible that (a) no-one will ever notice b/c people don't usually use sqlite in their build system, (b) by the time they do notice, it will already be fixed upstream. So just ignoring the problem for now also seems like a reasonable option...

[ogrisel]

Alright, let close this for now. If a user of the manylinux docker image finds this issue by googling because she / he is impacted by the issue and upstream has not fixed it yet then we can decide to implement #90 (comment) (build newest libsqlite3 and install it in /usr/lib) as a stopgap.

Users: please feel free to voice your opinion on this issue by commenting here (or opening new issue and linking to this one).

[NathanDunfield]

I just got hit by this. My Python module uses Sphinx's "autodoc" feature to build its documentation, which means it imports my module, which in turn imports "sqlite3". This causes my usual build script to fail for Python 3.6 on the manylinux1 container, even though it works fine for Python 2.7, 3.4, and 3.5. As a stopgap, I can just reuse the documentation that Python 3.5 generates in my Python 3.6 wheel, but a real fix would be great, since I also can't run my doctest suite.

More broadly, the fact that sqlite3 on this container is so old causes other problems for me, because my module uses some sqlite3 features that were added in SQLite 3.6 (which was released in 2008, so not really bleeding edge at this point). My current work-around is to install SQLite 3.6.5 from source over the containers default one; this works fine Python 2.7, 3.4, and 3.5, but not Python 2.6 for some reason. (I'm sure it would work for Python 3.6 as well if "_sqlite3" was being built there.) Here is my Dockerfile if you are curious. It might be worth a shot just installing SQLite 3.7.* in the same way and building all the Pythons against that...

[NathanDunfield]

I went ahead and created pull request #93 which installs the most recent release of SQLite 3 prior to building the Pythons. It fixes both the problem of sqlite3 not working with Python 3.6 as well as the broader issue I raised above.