Skip to content

windows系统下加载模型错误 #2004

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Lyn0222 opened this issue Mar 22, 2025 · 0 comments
Open

windows系统下加载模型错误 #2004

Lyn0222 opened this issue Mar 22, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@Lyn0222
Copy link

Lyn0222 commented Mar 22, 2025

在windows系统下,通过mindnlp库加载模型的时候会发生报错:

import winfcntlock as fcntl # pylint: disable=import-error
ModuleNotFoundError: No module named 'winfcntlock'

经检查,修改utils/download.py里的import winfcntlock as fcntl为from . import winfcntlock as fcntl可以解决该问题。但是,继续运行之后仍有报错:

Traceback (most recent call last):
  File "E:\git\biogpt\infer_mindspore.py", line 8, in <module>
    tokenizer = BioGptTokenizer.from_pretrained(model_name)
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\transformers\tokenization_utils_base.py", line 2115, in from_pretrained
    resolved_config_file = cached_file(
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 527, in cached_file
    resolved_file = download(
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 655, in download
    raise exp
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 651, in download
    pointer_path = threads_exclusive_http_get(url, storage_folder, download_file_name=relative_filename, proxies=proxies, headers=headers)
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 162, in threads_exclusive_http_get
    fcntl.flock(fd, fcntl.LOCK_UN)
AttributeError: module 'mindnlp.utils.winfcntlock' has no attribute 'flock'. Did you mean: 'lock'?

这是因为,download.py中引用winfcntlock.py中的函数名称和实际名称对不上,有两处不对:①download.py中引用的是fcntl.flock(),但是winfcntlock.py中实际名称为lock;②download.py中引用的是fcntl.LOCK_UN,但是winfcntlock.py中没有这个变量声明;

个人参考网上资料,找到了一个解决方案:修改winfcntlock.py中的lock函数名称为flock,并添加变量LOCK_UN = 0x08 。
修改后,为了解决报错:

Traceback (most recent call last):
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 160, in threads_exclusive_http_get
    raise exp
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 156, in threads_exclusive_http_get
    fcntl.flock(fd, fcntl.LOCK_EX)
  File "D:\install\anaconda\envs\biogpt\lib\site-packages\winfcntl\winfcntl.py", line 180, in flock
    handle = msvcrt.get_osfhandle(fd)
TypeError: '_io.TextIOWrapper' object cannot be interpreted as an integer

修改了winfcntlock.py中的hfile = win32file._get_osfhandle(file.fileno())为fd = hfile = win32file._get_osfhandle(file),不然对int变量取file.fileno()会报错。
修改后,下载模型能够正常进行,但是文件夹里会有很多.lock临时文件:

Image

修改download.py文件,在下载完成后删除.lock临时文件:

lock_file.close()
try:
    os.remove(lock_file_path)
except Exception as delete_exp:
    logging.error(f"Failed to delete lock file: {delete_exp}")
Image

现在在windows系统上可以正常运行了,对于download.py文件没有做很多修改,所以linux的运行应该不受影响。

@Lyn0222 Lyn0222 added the bug Something isn't working label Mar 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant