Possible lazy import opportunities in pathlib

[ncoghlan]

BPO	33440
Nosy	@ncoghlan, @pitrou, @njsmith, @serhiy-storchaka
PRs	bpo-33440: Defer imports in pathlib to reduce its import time. #6820

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-05-07.12:28:40.679>
labels = ['3.8', 'library', 'performance']
title = 'Possible lazy import opportunities in `pathlib`'
updated_at = <Date 2018-05-14.20:03:34.394>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2018-05-14.20:03:34.394>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-05-07.12:28:40.679>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 33440
keywords = ['patch']
message_count = 4.0
messages = ['316266', '316276', '316567', '316570']
nosy_count = 4.0
nosy_names = ['ncoghlan', 'pitrou', 'njs', 'serhiy.storchaka']
pr_nums = ['6820']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue33440'
versions = ['Python 3.8']

Linked PRs

• GH-77621: Delay some imports from pathlib #112244

[ncoghlan]

Due to a python-ideas discussion about reducing boilerplate for __file__-relative path calculations, I was running "./python -X importtime -S -c 'import pathlib'" and noticed three potential candidates where it may be worthwhile deferring the imports until the modules are actually needed:

• re (used in _WildcardSelector)
• fnmatch (used in PurePath.match and _WildcardSelector)
• urllib.parse (used in PurePath.as_uri, by way of self._flavour.make_uri)

Using an optimised Python 3.7 on an SSD with warm disk caches, commenting out those 3 imports reduced my typical import times for pathlib from 12-13 ms to 7-8 ms)

[njsmith]

Also it's a smaller win, but it might be worth considering whether we can avoid the import of 'nt' on posix platforms and vice-versa.

[serhiy-storchaka]

Did you mean the idea about module-level alias to dirname(file)? I think this is a dead horse. See bpo-33277.

[serhiy-storchaka]

Personally I support moving imports of rarely used modules into functions that need them. But this is against PEP-8 and should be discussed more widely. Perhaps this rule should be weaken, we have to state what exceptions are permitted.

[barneygale]

I've moved the urllib imports into as_uri() / from_uri() as their use is uncommon. Everything else is likely to be hit in general usage, I think.

[AlexWaygood]

You could probably make this one lazy:

cpython/Lib/pathlib.py

Line 17 in 607b5e3

import warnings

Users probably shouldn't pay the cost of importing warnings unless they're actually using a deprecated function. There are quite a few uses of warnings in pathlib, though, which is why I held off from changing warnings to be a lazy import in pathlib as part of #110286 -- maybe making it a lazy import in pathlib just adds too much ugliness for it to be worthwhile.

The import-time speedup from making warnings a lazy import is small, but measurable.

[barneygale]

Three of four uses of warnings will be removed in 3.14, so perhaps we could look again when main becomes 3.14?

[AlexWaygood]

Three of four uses of warnings will be removed in 3.14, so perhaps we could look again when main becomes 3.14?

SGTM!

[barneygale]

Resolving this issue as I believe there's nothing left to do!

• The urllib import was made lazy in GH-107465: Add pathlib.Path.from_uri() classmethod. #107640
• The fnmatch, glob and re imports were made lazy in GH-77621: Delay some imports from pathlib #112244
• The top-level warnings import will be removed when we branch for Python 3.14
• Everything else is cheap or needed at a module level