Skip to content

Commit 2f2d8a9

Browse files
author
Adrien Di Mascio
committed
BUG: close hdf store on any error
In `read_hdf`, if the `store.select()` call throws either a `ValueError`, a `TypeError` or a `KeyError` then the store is closed. However, if any other exception is thrown (e.g. an `AssertionError` if there are gaps in blocks ref_loc) , the store is not closed and some client code could end up trying to reopen the store and hit an error like: `the file XXX is already opened. Please close it before reopening in write mode`. Furthermore, the exception is re-raised in all cases. This commit just closes store in all cases. Closes #28430
1 parent 0c4404b commit 2f2d8a9

File tree

3 files changed

+26
-9
lines changed

3 files changed

+26
-9
lines changed

doc/source/whatsnew/v1.0.0.rst

+1
Original file line numberDiff line numberDiff line change
@@ -314,6 +314,7 @@ I/O
314314
- Bug in :func:`read_hdf` closing stores that it didn't open when Exceptions are raised (:issue:`28699`)
315315
- Bug in :meth:`DataFrame.read_json` where using ``orient="index"`` would not maintain the order (:issue:`28557`)
316316
- Bug in :meth:`DataFrame.to_html` where the length of the ``formatters`` argument was not verified (:issue:`28469`)
317+
- :meth:`read_hdf` now closes the store on any error thrown (:issue:`28430`)
317318

318319
Plotting
319320
^^^^^^^^

pandas/io/pytables.py

+6-9
Original file line numberDiff line numberDiff line change
@@ -395,15 +395,12 @@ def read_hdf(path_or_buf, key=None, mode="r", **kwargs):
395395
)
396396
key = candidate_only_group._v_pathname
397397
return store.select(key, auto_close=auto_close, **kwargs)
398-
except (ValueError, TypeError, KeyError):
399-
if not isinstance(path_or_buf, HDFStore):
400-
# if there is an error, close the store if we opened it.
401-
try:
402-
store.close()
403-
except AttributeError:
404-
pass
405-
406-
raise
398+
finally:
399+
# if there is an error, close the store if we opened it.
400+
try:
401+
store.close()
402+
except AttributeError:
403+
pass
407404

408405

409406
def _is_metadata_of(group, parent_group):

pandas/tests/io/pytables/test_store.py

+19
Original file line numberDiff line numberDiff line change
@@ -448,6 +448,25 @@ def check_default_mode():
448448
check("w")
449449
check_default_mode()
450450

451+
def test_store_closed_on_error(self, mocker):
452+
"""check hdf store is closed whichever error occurs"""
453+
mocker.patch("pandas.io.pytables.HDFStore.select").side_effect = AssertionError(
454+
"Gaps in blk ref_locs"
455+
)
456+
df = tm.makeDataFrame()
457+
with ensure_clean_path(self.path) as path:
458+
df.to_hdf(path, "df")
459+
try:
460+
pd.read_hdf(path, "df")
461+
except AssertionError: # this is expected, because of our mock
462+
pass
463+
try:
464+
HDFStore(path, mode="w")
465+
except ValueError as exc:
466+
pytest.fail(
467+
"store not closed properly, got error {exc}".format(exc=exc)
468+
)
469+
451470
def test_reopen_handle(self, setup_path):
452471

453472
with ensure_clean_path(setup_path) as path:

0 commit comments

Comments
 (0)