Skip to content

Commit cea03a4

Browse files
authored
Fix CI pipeline + open-source cleanup (symlinks, py3.10, README badges, citation)
- Remove 4 dead absolute symlinks to private /root/hf_cache that broke setup-python's pip-cache glob with EACCES (fixes tests + baselines-drift). - arc_state.py: tomllib backport via tomli for Python 3.10 (fixes the 3.10 matrix leg). - README/repo metadata: drop stale 'private mirror' / invite-only access notes; set homepage to appliedscientific.ai; add topics. - README: badge row (MIT, DOI, HF leaderboard, HF models/data, CI, lab) + License/Citation sections. - CITATION.cff + README citation: real title and author list (Weidener, Brkić, Jovanović, Ulgac, Meduri), preprint DOI 10.64898/2026.06.18.733146.
1 parent 9890bca commit cea03a4

8 files changed

Lines changed: 67 additions & 12 deletions

File tree

CITATION.cff

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,22 @@ title: "VCBench: a capability-stratified benchmark for single-cell foundation mo
33
message: "If you use this benchmark in your research, please cite both the preprint and the software release."
44
type: software
55
authors:
6-
- name: "VCBench contributors"
6+
- family-names: Weidener
7+
given-names: L.
8+
affiliation: "Applied Scientific Intelligence, Inc."
9+
email: lukas@appliedscientific.ai
10+
- family-names: Brkić
11+
given-names: M.
12+
affiliation: "Applied Scientific Intelligence, Inc."
13+
- family-names: Jovanović
14+
given-names: M.
15+
affiliation: "Applied Scientific Intelligence, Inc."
16+
- family-names: Ulgac
17+
given-names: E.
18+
affiliation: "Applied Scientific Intelligence, Inc."
19+
- family-names: Meduri
20+
given-names: A.
21+
affiliation: "Applied Scientific Intelligence, Inc."
722
repository-code: "https://github.com/AppliedScientific/VCBench"
823
license: MIT
924
keywords:
@@ -16,9 +31,18 @@ keywords:
1631
- reproducibility
1732
preferred-citation:
1833
type: article
19-
title: "VCBench: a capability-stratified benchmark for single-cell foundation models"
34+
title: "VCBench: A Multi-Dimensional Benchmark for Single-Cell Foundation Models"
2035
authors:
21-
- name: "VCBench contributors"
22-
journal: "bioRxiv"
36+
- family-names: Weidener
37+
given-names: L.
38+
- family-names: Brkić
39+
given-names: M.
40+
- family-names: Jovanović
41+
given-names: M.
42+
- family-names: Ulgac
43+
given-names: E.
44+
- family-names: Meduri
45+
given-names: A.
2346
year: 2026
24-
notes: "Preprint DOI and Zenodo software DOI available from the repository."
47+
doi: "10.64898/2026.06.18.733146"
48+
url: "https://doi.org/10.64898/2026.06.18.733146"

README.md

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,15 @@
11
# VCBench
22

3+
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
4+
[![DOI](https://img.shields.io/badge/DOI-10.64898%2F2026.06.18.733146-blue.svg)](https://doi.org/10.64898/2026.06.18.733146)
5+
[![Leaderboard](https://img.shields.io/badge/%F0%9F%8F%86%20leaderboard-HF%20Space-ff9d00.svg)](https://huggingface.co/spaces/appliedscientific/vcbench-leaderboard)
6+
[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20models%20%26%20data-appliedscientific-ffce1c.svg)](https://huggingface.co/appliedscientific)
7+
[![tests](https://github.com/AppliedScientific/VCBench/actions/workflows/test.yml/badge.svg)](https://github.com/AppliedScientific/VCBench/actions/workflows/test.yml)
8+
[![Applied Scientific Intelligence](https://img.shields.io/badge/lab-appliedscientific.ai-000000.svg)](https://appliedscientific.ai)
9+
310
VCBench is a capability-stratified benchmark for single-cell foundation models, evaluating five models against pre-registered baselines across five dimensions.
411

512
> **First-time visitors — start here.** The v1.0.0 release reconciles VCBench's evaluator with upstream cell-eval to numerical precision via an explicit anchor-convention parameter. The Arc State checkpoint lives publicly at [`huggingface.co/appliedscientific/arc-state-norman-gears-corrected`](https://huggingface.co/appliedscientific/arc-state-norman-gears-corrected) — with paste-able reproduction snippets that recover the headline numbers in <5 min on CPU.
6-
>
7-
> *Repository access:* The canonical company-org repository at [`AppliedScientific/VCBench`](https://github.com/AppliedScientific/VCBench) is private during peer review. Contact the corresponding author for read access (URLs of the form `github.com/AppliedScientific/VCBench/...` will return 404 without an invite). The companion HuggingFace artefact + reproduction snippets are public and require no access request.
813
914
## What it produces
1015

@@ -145,3 +150,28 @@ snapshot_download("appliedscientific/vcbench-embeddings", repo_type="dataset")
145150
- **GPU:** A100 80GB recommended (required for UCE; 40GB sufficient for others)
146151
- **Baseline construction:** CPU-only, no GPU needed
147152
- **Estimated GPU time:** ~75 GPU-hours total
153+
154+
## License
155+
156+
VCBench is released under the [MIT License](LICENSE).
157+
158+
## Citation
159+
160+
If you use VCBench in your research, please cite both the preprint and the
161+
software release. Machine-readable metadata is in [`CITATION.cff`](CITATION.cff).
162+
163+
**Preprint** — Weidener, L., Brkić, M., Jovanović, M., Ulgac, E., Meduri, A.
164+
*VCBench: A Multi-Dimensional Benchmark for Single-Cell Foundation Models*,
165+
Applied Scientific Intelligence, Inc. (2026).
166+
[doi:10.64898/2026.06.18.733146](https://doi.org/10.64898/2026.06.18.733146)
167+
168+
```bibtex
169+
@article{weidener2026vcbench,
170+
title = {VCBench: A Multi-Dimensional Benchmark for Single-Cell Foundation Models},
171+
author = {Weidener, L. and Brki\'{c}, M. and Jovanovi\'{c}, M. and Ulgac, E. and Meduri, A.},
172+
year = {2026},
173+
doi = {10.64898/2026.06.18.733146},
174+
url = {https://doi.org/10.64898/2026.06.18.733146},
175+
note = {Preprint}
176+
}
177+
```

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ dependencies = [
2929
"scipy>=1.10",
3030
"pydantic>=2.0",
3131
"pyyaml>=6.0",
32+
"tomli>=2.0; python_version < '3.11'",
3233
]
3334

3435
[project.optional-dependencies]

results/dim_c/geneformer/predicted_edges.csv

Lines changed: 0 additions & 1 deletion
This file was deleted.

results/dim_c/scgpt/predicted_edges.csv

Lines changed: 0 additions & 1 deletion
This file was deleted.

results/dim_c/transcriptformer/predicted_edges.csv

Lines changed: 0 additions & 1 deletion
This file was deleted.

results/dim_c/uce/predicted_edges.csv

Lines changed: 0 additions & 1 deletion
This file was deleted.

src/vcbench/models/arc_state.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,14 @@
1414
from __future__ import annotations
1515

1616
import os
17-
import tomllib
1817
from pathlib import Path
1918
from typing import TYPE_CHECKING
2019

20+
try: # tomllib is stdlib on Python >= 3.11; fall back to the tomli backport
21+
import tomllib
22+
except ModuleNotFoundError: # Python 3.9 / 3.10
23+
import tomli as tomllib
24+
2125
from vcbench.models.base import FoundationModel
2226

2327
if TYPE_CHECKING: # pragma: no cover

0 commit comments

Comments
 (0)