Skip to content

Fix missing stats when migrating from 1.13.0 to 1.14.x #937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Theelx wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Fix missing stats when migrating from 1.13.0 to 1.14.x #937

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e865b70
Allow one-time migration of old experiment stats from older H2O-LLMSt…
Theelx Sep 18, 2025
4fdd8f1
Loosen Python version restrictions and re-add sqlitedict dependency
Theelx Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions llm_studio/app_utils/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
from diskcache import Cache
from h2o_wave import Choice, Q, ui
from pandas.core.frame import DataFrame
from sqlitedict import SqliteDict

from llm_studio.app_utils.db import Experiment
from llm_studio.python_configs.base import DefaultConfigProblemBase
Expand Down Expand Up @@ -1643,9 +1644,21 @@ def get_experiments_info(df: DataFrame, q: Q) -> defaultdict:
loss_function = ""

charts_db_path = os.path.join(row.path, "charts_cache")
old_db_path = os.path.join(row.path, "charts.db")
if os.path.exists(charts_db_path):
with Cache(charts_db_path) as cache:
logs = {key: cache.get(key) for key in cache}

if not logs and os.path.exists(old_db_path):
# migrate the old DB over to the new diskcache, should only need to be run once
# we could delete the old charts.db, but we don't want to risk the user stopping
# the service during the migration and therefore causing permanent data loss
with SqliteDict(old_db_path) as sqlite_logs:
keys = list(sqlite_logs.iterkeys())
for key in keys:
cache.add(key, sqlite_logs[key])
logs = {key: cache.get(key) for key in cache}

if "internal" in logs.keys():
if "current_step" in logs["internal"].keys():
curr_step = int(logs["internal"]["current_step"]["values"][-1])
Expand Down
3 changes: 2 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "h2o-llmstudio"
version = "1.14.4"
readme = "README.md"
license = {file = "LICENSE"}
requires-python = "==3.10.*"
requires-python = ">=3.10, <=3.13"
dependencies = [
"torch==2.7.1",
"tqdm>=4.65.0,<5.0.0",
Expand Down Expand Up @@ -45,6 +45,7 @@ dependencies = [
"deepspeed==0.17.5",
"keyring>=25.2.1, <26.0.0",
"pydantic>=2.8.2, <3.0.0",
"sqlitedict>=2.1.0, <3.0.0",
]

[dependency-groups]
Expand Down
2 changes: 2 additions & 0 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,8 @@ soupsieve==2.8
# via beautifulsoup4
sqlalchemy==2.0.43
# via h2o-llmstudio
sqlitedict==2.1.0
# via h2o-llmstudio
starlette==0.47.3
# via h2o-wave
swagger-spec-validator==3.0.4
Expand Down
Loading