Skip to content

python 3.13.1 giving import error over nfs [windows] #129333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Ganeshkumhar1 opened this issue Jan 27, 2025 · 11 comments
Open

python 3.13.1 giving import error over nfs [windows] #129333

Ganeshkumhar1 opened this issue Jan 27, 2025 · 11 comments
Labels
OS-windows type-bug An unexpected behavior, bug, or error

Comments

@Ganeshkumhar1
Copy link

Ganeshkumhar1 commented Jan 27, 2025

Bug report

Bug description:

I built python 3.13.1 from source using MSVC , using call build.bat -e -v -p x64 , had no issues in building . After building it I installed numpy 2.1.0 whl from pypi. then I tried using this python over nfs on remote host I am getting numpy import error -

ImportError: DLL load failed while importing _multiarray_umath: The parameter is incorrect.

same set of steps works for me with python3.12.6. seems something is changed in 3.13.1 when trying import over nfs.
could someone please help As I am unable to figure out.

raised issue with numpy dev as well numpy/numpy#28216 , but it seems issue with the way how new python is working. any help is appreciated.

CPython versions tested on:

3.13

Operating systems tested on:

Windows

Linked PRs

@Ganeshkumhar1 Ganeshkumhar1 added the type-bug An unexpected behavior, bug, or error label Jan 27, 2025
@Ganeshkumhar1
Copy link
Author

can this be reviewed please.

@zooba
Copy link
Member

zooba commented Jan 28, 2025

The actual numpy issue is numpy/numpy#28216

And the relevant information (missing from this report) seems to be:

on 3.12.6 I get exact paths -

stdlib: \\<NFS_DIR>\python\Lib
platstdlib: \\<NFS_DIR>\python\Lib
purelib: \\<NFS_DIR>\python\Lib\site-packages
platlib: \\<NFS_DIR>\python\Lib\site-packages
include: \\<NFS_DIR>\python\Include
platinclude: \\<NFS_DIR>\python\Include
scripts: \\<NFS_DIR>\python\Scripts
data: \\<NFS_DIR>\python

on 3.13.1 I get \?\UNC\ added to these paths -

stdlib: \\?\UNC\<NFS_DIR>\python\Lib
platstdlib: \\?\UNC\<NFS_DIR>\python\Lib
purelib: \\?\UNC\<NFS_DIR>\python\Lib\site-packages
platlib: \\?\UNC\<NFS_DIR>\python\Lib\site-packages
include: \\?\UNC\<NFS_DIR>\python\Include
platinclude: \\?\UNC\<NFS_DIR>\python\Include
scripts: \\?\UNC\<NFS_DIR>\python\Scripts
data: \\?\UNC\<NFS_DIR>\python

@barneygale @nineteendo Does this ring a bell to either of you? Did we change something that would add \\?\UNC here? Or fail to remove it?

@ChrisDenton
Copy link

Or fail to remove it?

I would just note that getpath_realpath only attempts to strip \\?\ from drive paths and doesn't handle \\?\UNC at all:

cpython/Modules/getpath.c

Lines 544 to 554 in 180ee43

if (0 == wcsncmp(p, L"\\\\?\\", 4)) {
if (GetFileAttributesW(&p[4]) != INVALID_FILE_ATTRIBUTES) {
p += 4;
len -= 4;
}
}
if (CompareStringOrdinal(path, (int)pathlen, p, len, TRUE) == CSTR_EQUAL) {
result = Py_NewRef(pathobj);
} else {
result = PyUnicode_FromWideChar(p, len);
}

Whereas the realpath in ntpath handles both cases:

cpython/Lib/ntpath.py

Lines 745 to 754 in 180ee43

# For UNC paths, the prefix will actually be \\?\UNC\
# Handle that case as well.
if path.startswith(unc_prefix):
spath = new_unc_prefix + path[len(unc_prefix):]
else:
spath = path[len(prefix):]
# Ensure that the non-prefixed path resolves to the same path
try:
if _getfinalpathname(spath) == path:
path = spath

@nineteendo
Copy link
Contributor

nineteendo commented Jan 28, 2025

Does this ring a bell to either of you? Did we change something that would add \?\UNC here? Or fail to remove it?

I don't recall changing UNC prefixes, but let me do a bisect first.

@nineteendo

This comment has been minimized.

@nineteendo
Copy link
Contributor

nineteendo commented Jan 29, 2025

Looks like it started emitting this error in 500b0f4, but right before that it simply crashed when trying to import numpy. So, I'm not sure if it's only Python which is to blame. I'm going to try again with an earlier numpy version.

UPDATE: the error doesn't occur on numpy 1.25.0.

@Ganeshkumhar1
Copy link
Author

Ganeshkumhar1 commented Jan 29, 2025

@nineteendo did you try numpy 1.25.0 on python 3.13.1?
as per numpy dev numpy 2.1.0 is first version to support on python 3.13.1

numpy/numpy#28207 (comment)

@nineteendo
Copy link
Contributor

I only checked if the import succeeded. I tried again by disabling from collections.abc import Buffer, it now starts crashing in de64e75.

@zooba
Copy link
Member

zooba commented Jan 29, 2025

I would just note that getpath_realpath only attempts to strip \\?\ from drive paths and doesn't handle \\?\UNC at all:

Yeah, I suspect this is probably it. It changed in #86179, which looks like it only landed in 3.13 but pretty early on. Clearly we don't have any tests for running from UNC mounts.

@Ganeshkumhar1
Copy link
Author

@zooba @nineteendo , Is there any workaround I can use to fix this issue without setting PYTHONHOME.

@zooba
Copy link
Member

zooba commented Jan 31, 2025

The only other workaround might be setting PYTHONEXECUTABLE to the full path to python.exe, but that may have adverse impact on virtual environments, and it might not even work.

Alternatively, copy Python onto the local machine and use PYTHONPATH to import modules from the network.

If you're building from source, you could add the extra check for \\?\UNC\ into getpath.c and handle it (and submit a PR, which will get it fixed in the next release if nobody else gets to it)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OS-windows type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants