Skip to content

Backport Inkling-2-Small with attention/unembedding multi-LoRA - #45

Open
kailash109 wants to merge 2 commits into
spindle-miles-v0.2.0from
codex/inkling2-attention-multilora
Open

kailash109 wants to merge 2 commits into
spindle-miles-v0.2.0from
codex/inkling2-attention-multilora

Conversation

@kailash109

@kailash109 kailash109 commented Nov 17, 2026 •

Copy link
Copy Markdown

Summary

Backport radixark#4819 (eac7f8cc79c09e7b3dcbb6bef3029dc8323390f0) onto spindle-miles-v0.2.0, including the Inkling2 model, checkpoint conversion, native LoRA specs, quantization support, launchers, and tests.

Add native multi-LoRA for explicitly selected attention and unembedding targets (--target-modules attn,unembed). Construct output-head adapters through the slot builder and reject routed-expert multi-LoRA at startup. Derive the restricted architecture spec per target selection; default single-adapter LoRA retains the complete attention/expert/unembedding layout and shared-outer experts.

Spindle PR 342 pins full-model-validated commit e186d97710dcc319da659f8bc25dbb674e227909, which remains an ancestor after merging the updated v0.2 base. Training uses TP=CP=PP=1 and expert parallelism. This recipe uses separate trainer and sampler nodes.

Validation

Merged PR head d23367105d09833dad3c91f490c90aefa18b80c2, in the matched Inkling image on B300:

  • 251 native-LoRA tests passed, including slot forward/gradient parity, exports, sink freezing, default expert layout, expert multi-LoRA rejection, and existing model regressions.
  • 10 Inkling launcher/model snapshots passed.
  • Full launcher suite: 679 passed, 23 failed, 4 errors. Unchanged base e8142459617f8220825ddbacf054b8b17cabdd82: 669 passed, 23 failed, 4 errors. The failing/error case sets are identical; no new failures. The full launcher suite is not green.
  • All changed-file pre-commit checks passed. Inkling regression tests are CPU-only; no CUDA CI registration was added.

Full-checkpoint E2E applies to frozen Miles e186d977 with Spindle 3e0139d, not the newer merged Miles head. It used separate eight-B300 trainer and sampler nodes with exclude_reserved_capacity=True:

  • Stock Tinker SDK 0.26.0: two rank-32 adapters, forward/backward, Adam updates, weights/optimizer save and restore, publication, and SGLang sampling passed. A's loss fell from 2.8943 to 2.6545; B stayed exactly at 2.8943. A's serving logprobs changed by up to 0.866795 nats; B stayed unchanged within 0.00001 nats. Restore returned A's exact saved loss.
  • 8,192-token backward, optimizer step, and publication passed. Long-prompt sampling accepted 8,185 prompt tokens and returned five finite-logprob tokens.
  • Stock tinker-cookbook 0.7.0 with SDK 0.39.1 restored the attention-only checkpoint, trained one step on 32 tokens with finite mean NLL 3.41838, and saved final training and sampler checkpoints. Only proxy authentication headers came from the existing helper; SDK and cookbook were unmodified.
  • Warm short backward: 2.62 seconds; optimizer step: 0.49 seconds; warm 8K backward: 29.64 seconds. Earlier uncached backward calls took 327.61 and 716.61 seconds including compilation. These are observations, not performance comparisons.

The Spindle PR includes required sampler precision and target-alias fixes, a reproducible stock-SDK check, and sanitized validation evidence. Expert adapters, B200, colocation, image inputs, restart, convergence, and full-model mixed-client batches were not validated. The frozen runtime also passed 208 native-LoRA tests and three focused CPU/CUDA adapter checks before the base merge.

Temporary validation apps were stopped; a container listing confirmed no remaining validation GPU containers. Shared model assets and checkpoints were retained.

Backport radixark#4819 at eac7f8c onto spindle-miles-v0.2.0. Keep the complete single-adapter layout and derive an attention/unembedding spec for explicitly restricted targets. Slot the output head and reject routed-expert multi-LoRA at startup.
@kailash109
kailash109 marked this pull request as ready for review November 18, 2026 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant