Missing symbol on assets conversion.

[Daniel-Wieczorek]

Hi guys, I want to contribute but I encountered an error while running the openage project. The error occurs when attempting to convert assets after launching. Below is the terminal output:

daniel@DanielPC:~/workareas/openage$ make run
cd bin && ./run main
INFO [py] launching openage 
INFO [py] compiled by GNU 13.3.0
INFO [py] running in DEVMODE
INFO [py] openage nyan API modpack is up to date
  Do you want to convert assets? [Y/n]
> Y

openage requires a local game installation for conversion
but no local installation could be found automatically.
  Do you want to download the AoC trial version? [Y/n]
> Y
Downloading AoC trial version from https://archive.org/download/AgeOfEmpiresIiTheConquerorsDemo/Age2XTrial.exe
Extracting game files to [Directory(/tmp/tmp17ck0xzt)]:...
converting from '/tmp/tmp17ck0xzt'
INFO [py] Looking for compatible games to convert...
INFO [py] Compatible game edition detected:
INFO [py]  * Age of Empires 2: The Conqueror's Trial Version
INFO [py] converting metadata
INFO [py] using cached wrapper: /tmp/AOC_empires2_x1.dat.pickle
INFO [py] Finished metadata read (1.20 seconds)
INFO [py] Starting conversion...
INFO [py] Extracting Genie data...
INFO [py] Creating API-like objects...
INFO [py] Linking API-like objects...
INFO [py] Generating auxiliary objects...
INFO [py] Creating nyan objects...
INFO [py] Creating requests for media export...
INFO [py] Finished data conversion (1.46 seconds)
INFO [py] Starting export...
INFO [py] Dumping info file...
INFO [py] Dumping data files...
INFO [py] Exporting media files...
INFO [py] -- Exporting graphics files...
Exception in thread Thread-3 (_handle_results):
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.12/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/daniel/workareas/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/export/media_exporter.py", line 819, in _export_texture
    texture = Texture(image, palettes)
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/daniel/workareas/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/entity_object/export/texture.py", line 97, in __init__
    from ...value_object.read.media.smx import SMX
ImportError: /home/daniel/workareas/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/value_object/read/media/smx.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZL89__pyx_f_7openage_7convert_12value_object_4read_5media_3smx_8SMXLayer_process_drawing_cmdsP69__pyx_obj_7openage_7convert_12value_object_4read_5media_3smx_SMXLayerP76__pyx_obj_7openage_7convert_12value_object_4read_5media_3smx_SMXOutlineLayerR18__Pyx_memviewsliceRSt6vectorI64__pyx_t_7openage_7convert_12value_object_4read_5media_3smx_pixelSaIS6_EElllim
"""
-- Files done:   0/275
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.12/multiprocessing/pool.py", line 595, in _handle_results
    cache[job]._set(i, obj)
  File "/usr/lib/python3.12/multiprocessing/pool.py", line 781, in _set
    self._error_callback(self._value)
  File "/home/daniel/workareas/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/processor/export/media_exporter.py", line 260, in error_callback
    raise exception
ImportError: /home/daniel/workareas/openage/.bin/g++-debug-Oauto-sanitize-none/openage/convert/value_object/read/media/smx.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZL89__pyx_f_7openage_7convert_12value_object_4read_5media_3smx_8SMXLayer_process_drawing_cmdsP69__pyx_obj_7openage_7convert_12value_object_4read_5media_3smx_SMXLayerP76__pyx_obj_7openage_7convert_12value_object_4read_5media_3smx_SMXOutlineLayerR18__Pyx_memviewsliceRSt6vectorI64__pyx_t_7openage_7convert_12value_object_4read_5media_3smx_pixelSaIS6_EElllim
-- Files done:   0/275

This error seems to be related to an undefined symbol in the compiled Python extension (smx.cpython-312-x86_64-linux-gnu.so). I am unsure if this is expected behavior or a bug?

I tried it on WSL2 and virtual machine with Ubuntu 24.04 LTS, same result. My setup:

### Python Version:
Python 3.12.3
### Installed Python Packages:
Package             Version
------------------- -------------
attrs               23.2.0
Automat             22.10.0
Babel               2.10.3
bcrypt              3.2.2
blinker             1.7.0
certifi             2023.11.17
chardet             5.2.0
click               8.1.6
cloud-init          24.4.1
colorama            0.4.6
command-not-found   0.3
configobj           5.0.8
constantly          23.10.4
cryptography        41.0.7
Cython              3.0.8
dbus-python         1.3.2
distro              1.9.0
distro-info         1.7+build1
gyp                 0.1
httplib2            0.20.4
hyperlink           21.0.0
idna                3.6
incremental         22.10.0
Jinja2              3.1.2
jsonpatch           1.32
jsonpointer         2.0
jsonschema          4.10.3
launchpadlib        1.11.0
lazr.restfulclient  0.14.6
lazr.uri            1.0.6
lz4                 4.0.2+dfsg
Mako                1.3.2.dev0
markdown-it-py      3.0.0
MarkupSafe          2.1.5
mdurl               0.1.2
netifaces           0.11.0
numpy               1.26.4
oauthlib            3.2.2
olefile             0.46
packaging           24.0
pillow              10.2.0
pip                 24.0
pyasn1              0.4.8
pyasn1-modules      0.2.8
pycairo             1.25.1
pycurl              7.45.3
Pygments            2.17.2
PyGObject           3.48.2
PyHamcrest          2.1.0
PyJWT               2.7.0
pyOpenSSL           23.2.0
pyparsing           3.1.1
pyrsistent          0.20.0
pyserial            3.5
python-apt          2.7.7+ubuntu4
pytz                2024.1
PyYAML              6.0.1
requests            2.31.0
rich                13.7.1
service-identity    24.1.0
setuptools          68.1.2
six                 1.16.0
systemd-python      235
toml                0.10.2
Twisted             24.3.0
typing_extensions   4.10.0
ubuntu-pro-client   8001
unattended-upgrades 0.1
urllib3             2.0.7
wadllib             1.3.6
wheel               0.42.0
zope.interface      6.1
### System Information:
Linux DanielPC 5.15.167.4-microsoft-standard-WSL2 #1 SMP Tue Nov 5 00:21:55 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
### CMake Version:
cmake version 3.28.3

Do you have any idea what's wrong and how to fix it? What python version do you use to build? 3.12.3 shall not be supported then?

[heinezen]

Hey can you try to upgrade your Cython version, e.g. by installing it with pip?

[Daniel-Wieczorek]

Yes, sure - updated to latest 3.0.12 and that seems to be working, thanks!

[Daniel-Wieczorek]

Hi @heinezen if that's ok for you I can push PR with updated information about cython version. I tested it on Ubuntu LTS 24.04 on VM and WSL, both works fine with cython==3.0.12.

[heinezen]

@Daniel-Wieczorek Yes, that would be good. It's a known bug that got fixed in Cython 3.0.10

[Daniel-Wieczorek]

PR created: #1767, @heinezen feel free to review.
I updated also docs related to docker 😉