Consider use of libicu

[NWilson]

I wonder if any customers would be interested in using libicu for Unicode data?

The Unicode tables we currently use are statically-linked in to libpcre2-8, and are moderately large. If the OS already had libicu installed, or the application is otherwise using it, it may be nice to share the data.

There is the chance this would incur a performance penalty, in which case, I suppose no-one will want it. libicu however does provide an (undocumented?) libicudata.so file containing all the data tables. It may be possible to read into those tables without a function call via the public interface.

The old static Unicode support would probably have to stay. So, this would only increase complexity for our project.

I wonder if this would be worth it.

[zherczeg]

I suspect libicu use function calls, while we use memory tables. Function calls are much slower in general.

[NWilson]

$ nm -D /usr/lib/x86_64-linux-gnu/libicudata.so
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 w __cxa_finalize
                 w __gmon_start__
0000000000002000 R icudt74_dat

It looks like we can grab the data tables directly by linking to libicudata.so, and the format of that blob is even documented (sparsely). I can't find evidence that many applications are using it however.

Perhaps this is a "won't fix". I thought it was worth considering, however.