Poetry dependency resolution fails

[JonathanWoollett-Light]

sparse version checks

• I checked that this issue has not been reported before list of issues.

• I have confirmed this bug exists on the latest version of sparse.

• I have confirmed this bug exists on the main branch of sparse.

Describe the bug

Attempting to use the latest version of sparse inconjunction with numpy produces an error.

Steps or code to reproduce the bug

PS C:\Users\Jonathan\documents> poetry new testing-sparse
Created package testing_sparse in testing-sparse
PS C:\Users\Jonathan\documents> cd .\testing-sparse\
PS C:\Users\Jonathan\documents\testing-sparse> poetry add sparse
Using version ^0.17.0 for sparse

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 0 installs, 1 update, 0 removals

  - Downgrading numpy (2.3.1 -> 2.2.6)

Writing lock file
PS C:\Users\Jonathan\documents\testing-sparse> poetry add numpy
Using version ^2.3.1 for numpy

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 0 installs, 3 updates, 0 removals

  - Downgrading llvmlite (0.44.0 -> 0.34.0): Failed

PEP517 build of a dependency failed

Backend subprocess exited when trying to invoke build_wheel

    | Command '['C:\\Users\\Jonathan\\AppData\\Local\\Temp\\tmpt6npz20u\\.venv\\Scripts\\python.exe', 'C:\\Users\\Jonathan\\pipx\\venvs\\poetry\\Lib\\site-packages\\pyproject_hooks\\_in_process\\_in_process.py', 'build_wheel', 'C:\\Users\\Jonathan\\AppData\\Local\\Temp\\tmph3lk4mr6']' returned non-zero exit status 1.
    |
    | running bdist_wheel
    | C:\Users\Jonathan\AppData\Local\Temp\tmpt6npz20u\.venv\Scripts\python.exe C:\Users\Jonathan\AppData\Local\Temp\tmp8ulx86jj\llvmlite-0.34.0\ffi\build.py
    | CMake Error: Could not create named generator Visual Studio 15 2017 Win64
    | Using platforms in Visual Studio generator names is not supported in CMakePresets.json.
    |
    | Generators
    | * Visual Studio 17 2022        = Generates Visual Studio 2022 project files.
    |                                  Use -A option to specify architecture.
    |   Visual Studio 16 2019        = Generates Visual Studio 2019 project files.
    |                                  Use -A option to specify architecture.
    |   Visual Studio 15 2017 [arch] = Generates Visual Studio 2017 project files.
    |                                  Optional [arch] can be "Win64" or "ARM".
    |   Visual Studio 14 2015 [arch] = Generates Visual Studio 2015 project files.
    |                                  Optional [arch] can be "Win64" or "ARM".
    |   Borland Makefiles            = Generates Borland makefiles.
    |   NMake Makefiles              = Generates NMake makefiles.
    |   NMake Makefiles JOM          = Generates JOM makefiles.
    |   MSYS Makefiles               = Generates MSYS makefiles.
    |   MinGW Makefiles              = Generates a make file for use with
    |                                  mingw32-make.
    |   Green Hills MULTI            = Generates Green Hills MULTI files
    |                                  (experimental, work-in-progress).
    |   Unix Makefiles               = Generates standard UNIX makefiles.
    |   Ninja                        = Generates build.ninja files.
    |   Ninja Multi-Config           = Generates build-<Config>.ninja files.
    |   Watcom WMake                 = Generates Watcom WMake makefiles.
    |   CodeBlocks - MinGW Makefiles = Generates CodeBlocks project files
    |                                  (deprecated).
    |   CodeBlocks - NMake Makefiles = Generates CodeBlocks project files
    |                                  (deprecated).
    |   CodeBlocks - NMake Makefiles JOM
    |                                = Generates CodeBlocks project files
    |                                  (deprecated).
    |   CodeBlocks - Ninja           = Generates CodeBlocks project files
    |                                  (deprecated).
    |   CodeBlocks - Unix Makefiles  = Generates CodeBlocks project files
    |                                  (deprecated).
    |   CodeLite - MinGW Makefiles   = Generates CodeLite project files
    |                                  (deprecated).
    |   CodeLite - NMake Makefiles   = Generates CodeLite project files
    |                                  (deprecated).
    |   CodeLite - Ninja             = Generates CodeLite project files
    |                                  (deprecated).
    |   CodeLite - Unix Makefiles    = Generates CodeLite project files
    |                                  (deprecated).
    |   Eclipse CDT4 - NMake Makefiles
    |                                = Generates Eclipse CDT 4.0 project files
    |                                  (deprecated).
    |   Eclipse CDT4 - MinGW Makefiles
    |                                = Generates Eclipse CDT 4.0 project files
    |                                  (deprecated).
    |   Eclipse CDT4 - Ninja         = Generates Eclipse CDT 4.0 project files
    |                                  (deprecated).
    |   Eclipse CDT4 - Unix Makefiles= Generates Eclipse CDT 4.0 project files
    |                                  (deprecated).
    |   Kate - MinGW Makefiles       = Generates Kate project files (deprecated).
    |   Kate - NMake Makefiles       = Generates Kate project files (deprecated).
    |   Kate - Ninja                 = Generates Kate project files (deprecated).
    |   Kate - Ninja Multi-Config    = Generates Kate project files (deprecated).
    |   Kate - Unix Makefiles        = Generates Kate project files (deprecated).
    |   Sublime Text 2 - MinGW Makefiles
    |                                = Generates Sublime Text 2 project files
    |                                  (deprecated).
    |   Sublime Text 2 - NMake Makefiles
    |                                = Generates Sublime Text 2 project files
    |                                  (deprecated).
    |   Sublime Text 2 - Ninja       = Generates Sublime Text 2 project files
    |                                  (deprecated).
    |   Sublime Text 2 - Unix Makefiles
    |                                = Generates Sublime Text 2 project files
    |                                  (deprecated).
    |
    | Trying generator 'Visual Studio 15 2017 Win64'
    | Traceback (most recent call last):
    |   File "C:\Users\Jonathan\AppData\Local\Temp\tmp8ulx86jj\llvmlite-0.34.0\ffi\build.py", line 191, in <module>
    |     main()
    |   File "C:\Users\Jonathan\AppData\Local\Temp\tmp8ulx86jj\llvmlite-0.34.0\ffi\build.py", line 179, in main
    |     main_win32()
    |   File "C:\Users\Jonathan\AppData\Local\Temp\tmp8ulx86jj\llvmlite-0.34.0\ffi\build.py", line 88, in main_win32
    |     generator = find_win32_generator()
    |                 ^^^^^^^^^^^^^^^^^^^^^^
    |   File "C:\Users\Jonathan\AppData\Local\Temp\tmp8ulx86jj\llvmlite-0.34.0\ffi\build.py", line 84, in find_win32_generator
    |     raise RuntimeError("No compatible cmake generator installed on this machine")
    | RuntimeError: No compatible cmake generator installed on this machine
    | error: command 'C:\\Users\\Jonathan\\AppData\\Local\\Temp\\tmpt6npz20u\\.venv\\Scripts\\python.exe' failed with exit code 1

Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with llvmlite (0.34.0)

  - not supporting PEP 517 builds
  - not specifying PEP 517 build requirements correctly
  - the build requirements are incompatible with your operating system or Python version
  - the build requirements are missing system dependencies (eg: compilers, libraries, headers).

You can verify this by running pip wheel --no-cache-dir --use-pep517 "llvmlite (==0.34.0)".

  - Updating numpy (2.2.6 -> 2.3.1)
PS C:\Users\Jonathan\documents\testing-sparse>

Expected results

It should find a valid version of numpy which works.

Actual results

Attempting to add numpy results in an error.

Please describe your system.

OS and version:

PS C:\Users\Jonathan> systeminfo

Host Name:                     DESKTOP-UTVQJHQ
OS Name:                       Microsoft Windows 11 Home
OS Version:                    10.0.26100 N/A Build 26100
OS Manufacturer:               Microsoft Corporation
OS Configuration:              Standalone Workstation
OS Build Type:                 Multiprocessor Free
Registered Owner:              Jonathan
Registered Organization:       N/A
Product ID:                    00325-80734-51633-AAOEM
Original Install Date:         08/06/2025, 22:16:53
System Boot Time:              24/06/2025, 02:16:07
System Manufacturer:           Micro-Star International Co., Ltd.
System Model:                  MS-7D75
System Type:                   x64-based PC
Processor(s):                  1 Processor(s) Installed.
                               [01]: AMD64 Family 25 Model 97 Stepping 2 AuthenticAMD ~3701 Mhz
BIOS Version:                  American Megatrends International, LLC. 1.60, 30/05/2023
Windows Directory:             C:\WINDOWS
System Directory:              C:\WINDOWS\system32
Boot Device:                   \Device\HarddiskVolume1
System Locale:                 en-gb;English (United Kingdom)
Input Locale:                  en-gb;English (United Kingdom)
Time Zone:                     (UTC+00:00) Dublin, Edinburgh, Lisbon, London
Total Physical Memory:         64,673 MB
Available Physical Memory:     48,415 MB
Virtual Memory: Max Size:      173,464 MB
Virtual Memory: Available:     144,852 MB
Virtual Memory: In Use:        28,612 MB
Page File Location(s):         C:\pagefile.sys
Domain:                        WORKGROUP
Logon Server:                  \\DESKTOP-UTVQJHQ
Hotfix(s):                     5 Hotfix(s) Installed.
                               [01]: KB5054979
                               [02]: KB5050575
                               [03]: KB5059093
                               [04]: KB5060842
                               [05]: KB5059502
Network Card(s):               4 NIC(s) Installed.
                               [01]: Realtek 8811CU Wireless LAN 802.11ac USB NIC
                                     Connection Name: WiFi
                                     DHCP Enabled:    Yes
                                     DHCP Server:     192.168.0.1
                                     IP address(es)
                                     [01]: 192.168.0.18
                                     [02]: fe80::3656:309b:edfd:49d5
                               [02]: Realtek Gaming 2.5GbE Family Controller
                                     Connection Name: Ethernet
                                     Status:          Media disconnected
                               [03]: Bluetooth Device (Personal Area Network)
                                     Connection Name: Bluetooth Network Connection
                                     Status:          Media disconnected
                               [04]: RZ616 Wi-Fi 6E 160MHz
                                     Connection Name: WiFi 2
                                     Status:          Media disconnected
Virtualization-based security: Status: Running
                               Required Security Properties:
                                     Base Virtualization Support
                               Available Security Properties:
                                     Base Virtualization Support
                                     DMA Protection
                                     UEFI Code Readonly
                                     SMM Security Mitigations 1.0
                                     Mode Based Execution Control
                               Services Configured:
                                     Hypervisor enforced Code Integrity
                               Services Running:
                                     Hypervisor enforced Code Integrity
                               App Control for Business policy: Enforced
                               App Control for Business user mode policy: Audit
                               Security Features Enabled:
Hyper-V Requirements:          A hypervisor has been detected. Features required for Hyper-V will not be displayed.

sparse version

sparse (>=0.17.0,<0.18.0)

NumPy version

It cannot find a working version.

Numba version

NA

Relevant log output

[hameerabbasi]

This seems to be an issue with the poetry dependency resolver; with pip or uv; this works. For example; we don't require such an old NumPy, llvmlite version:

sparse/pyproject.toml

Line 10 in befe38a

dependencies = ["numpy>=1.17", "numba>=0.49"]

I'm not 100% certain why Poetry chose these incompatible NumPy/llvmlite versions.

[JonathanWoollett-Light]

@hameerabbasi Do you know if there is a version or versions I could specify that would circumvent this issue for now?

[hameerabbasi]

Could you try NumPy >= 2 and Numba >= 0.61

[JonathanWoollett-Light]

So adding the restriction of "numpy (>=2.0.0)" before re-running poetry install fixes the issue.

[hameerabbasi]

Cool; I'll mark this issue as resolved!

[JonathanWoollett-Light]

It also sometimes requires specifying a llvmlite version e.g. "llvmlite (==0.44.0)" to prevent poetry attempting to downgrade it which then causes the error.

[hameerabbasi]

I'd pin to >= 0.44 instead of ==; that's likely to not work long-term.