Skip to content

feat(hslm): Phase 3 — PBT hyperparameter search#582

Merged
gHashTag merged 1 commit intomainfrom
feat/295-phase3-pbt-hyperparam-search
Apr 30, 2026
Merged

feat(hslm): Phase 3 — PBT hyperparameter search#582
gHashTag merged 1 commit intomainfrom
feat/295-phase3-pbt-hyperparam-search

Conversation

@gHashTag
Copy link
Copy Markdown
Owner

Summary

Phase 3: Population-Based Training for automated hyperparameter optimization.

New file

  • src/b2t/pbt_search.zig — 253 LOC

PBT Algorithm

  1. Initialize population of 5 members with random hyperparameters
  2. Train each member for N steps
  3. Evaluate fitness (negative loss)
  4. Exploit: Replace bottom 40% with copies of top performers
  5. Explore: Mutate hyperparameters (lr, label_smoothing, tau, weight_decay)
  6. Repeat until convergence

Hyperparameter search space

  • LR, batch_size, label_smoothing, tau_init, weight_decay, grad_clip
  • Crossover: uniform selection from two parents
  • Mutation: Gaussian perturbation with configurable strength

Tests (5)

  • Population init, fitness tracking, exploit/explore, crossover, mutation

Closes #295

- Add src/b2t/pbt_search.zig
- HyperParams: lr, batch_size, label_smoothing, tau_init,
  weight_decay, grad_clip with crossover + mutation
- PBTMember: individual with fitness tracking
- PBTTracker: population-based training
  initPopulation, updateFitness, exploitAndExplore
  Truncation selection (bottom 40% replaced by top)
  Crossover + mutation for exploration
  Best-ever tracking across generations
- 5 tests: population init, fitness tracking,
  exploit/explore, crossover, mutation

Closes #295
@gHashTag gHashTag merged commit 04c2706 into main Apr 30, 2026
9 of 19 checks passed
@gHashTag gHashTag deleted the feat/295-phase3-pbt-hyperparam-search branch April 30, 2026 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Phase 3: Modal Labs GPU + PBT hyperparameter search (+20-30% speedup)

1 participant