Update to 5-seed mean: val_bpb 1.07813 (std 0.00046)

Adds s1337 (1.07801) and s2025 (1.07862) via the shipped mini wrapper.
The 5-seed mean is +0.00013 worse than the initial 3-seed mean (1.07800)
which is well within the std (~0.00046). Margins vs the legal open
chronology are unchanged in direction:

- vs PR openai#1394 (1.08563): -0.01938 nats per token (margin +0.01438 over 0.005 bar)
- vs PR openai#1420 (1.08014): -0.00520 nats per token (margin +0.00020 over 0.005 bar)
- vs own PR openai#1413 (1.08279): -0.01205 nats per token

3 of 5 seeds (s42, s1337, s2025) are now mini-wrapper-verified for fit;
s0 and s1234 mini-wrapper re-runs still in progress.

Author: dexhunter

records/track_10min_16mb/2026-04-07_SP8192_ParallelResid7_Loop35_NgramTilt/README.md

@@ -1,25 +1,27 @@
-# Record: SP8192 + Parallel Residuals + 3-Layer Recurrence + Legal N-gram Tilt — val_bpb 1.07800 (3-seed mean)
+# Record: SP8192 + Parallel Residuals + 3-Layer Recurrence + Legal N-gram Tilt — val_bpb 1.07813 (5-seed mean)
 
-**val_bpb: 1.07800** (3-seed mean, std 0.00053) | **2.78457 nats per token** | **~15.99 MB** | 8×H100 SXM, 600 s | Legal Score-First TTT + Causal N-gram Tilt
+**val_bpb: 1.07813** (5-seed mean, std 0.00046) | **2.78491 nats per token** | **~15.99 MB** | 8×H100 SXM, 600 s | Legal Score-First TTT + Causal N-gram Tilt
 
-Beats [PR #1394](https://github.com/openai/parameter-golf/pull/1394) (1.08563) by **0.00763 bpb / 0.01971 nats per token** on a 3-seed mean, comfortably clearing the 0.005-nats record threshold. Beats [PR #1420](https://github.com/openai/parameter-golf/pull/1420) (1.08014) by **0.00214 bpb / 0.00553 nats per token**, clearing the 0.005-nats threshold against the next-best legal open PR with a margin near seed-noise scale. Beats our own [PR #1413](https://github.com/openai/parameter-golf/pull/1413) (1.08279) by **0.00479 bpb / 0.01237 nats per token**.
+Beats [PR #1394](https://github.com/openai/parameter-golf/pull/1394) (1.08563) by **0.00750 bpb / 0.01938 nats per token** on a 5-seed mean, comfortably clearing the 0.005-nats record threshold. Beats [PR #1420](https://github.com/openai/parameter-golf/pull/1420) (1.08014) by **0.00201 bpb / 0.00520 nats per token**, clearing the 0.005-nats threshold against the next-best legal open PR. Beats our own [PR #1413](https://github.com/openai/parameter-golf/pull/1413) (1.08279) by **0.00466 bpb / 0.01205 nats per token**.
 
 ## Results (8×H100 80GB SXM, PyTorch 2.9.1+cu128, legal score-first TTT with causal n-gram tilt)
 
-### Core (TTT) table
+### Core (TTT) table — 5-seed verification
 
 | Seed | Steps | Pre-quant BPB | Sliding BPB | **Post-TTT (n-gram tilted) BPB** | val_loss (nats) | Artifact (mini, bytes) |
 |---:|---:|---:|---:|---:|---:|---:|
-| 0    | 4911 | 1.08717 | 1.08220 | **1.07743** | 2.78312 | ~15,990,971 (proj) |
-| 42   | 4913 | 1.08781 | 1.08262 | **1.07808** | 2.78479 | **15,993,733 (verified)** |
-| 1234 | 4905 | 1.08820 | 1.08352 | **1.07848** | 2.78581 | ~15,988,567 (proj) |
-| **mean** | | **1.08773** | **1.08278** | **1.07800** | **2.78457** | all under 16,000,000 |
-
-**Verification status (initial 3-seed PR):**
-- BPB numbers above come from independent runs with `SEED=0/42/1234` on `pr1394_with_ttt.py` (the readable source, ~79 KB code).
-- s42 has been **independently re-run via the shipped `train_gpt.py` mini wrapper** (~18.9 KB code) and reproduces TTT val_bpb 1.07809 within float64 noise (1.07808 raw vs 1.07809 mini), with total submission size **15,993,733 bytes** (verified < 16,000,000).
-- s0 and s1234 artifact sizes are **projected** by subtracting the verified raw-vs-mini delta (65,913 bytes on s42) from their raw artifact sizes. Both project comfortably under 16,000,000 bytes (9 KB and 11 KB headroom respectively).
-- A 5-seed re-run via the mini wrapper (s0, s42, s1234, s1337, s2025) is in progress and this PR will be updated with the verified 5-seed mean once those land.
+| 0    | 4911 | 1.08717 | 1.08220 | **1.07743** | 2.78312 | ~15,990,971 (proj from raw delta) |
+| 42   | 4913 | 1.08781 | 1.08262 | **1.07808** | 2.78479 | **15,993,733 (mini-verified)** |
+| 1234 | 4905 | 1.08820 | 1.08352 | **1.07848** | 2.78581 | ~15,988,567 (proj from raw delta) |
+| 1337 | 4909 | 1.08772 | 1.08246 | **1.07801** | 2.78461 | **15,988,039 (mini-verified)** |
+| 2025 | 4908 | 1.08842 | 1.08306 | **1.07862** | 2.78620 | **15,992,215 (mini-verified)** |
+| **5-seed mean** | | **1.08786** | **1.08277** | **1.07813** | **2.78491** | all under 16,000,000 |
+
+**Verification status (5-seed update):**
+- All 5 seeds use the same shipped configuration (`pr1394_with_ttt.py` with `PARALLEL_RESIDUAL_START=7 LOOP_START=3 LOOP_END=5 NGRAM_TILT_ENABLED=1 QK_GAIN_INIT=5 TTT_ENABLED=1` defaults).
+- **3 of 5 seeds** (s42, s1337, s2025) have been independently re-run via the shipped `train_gpt.py` self-extracting LZMA mini wrapper (~18.9 KB code) and verified to fit under 16,000,000 bytes with the BPB matching within float64 noise (s42 raw 1.07808 vs s42 mini 1.07809).
+- **s0 and s1234** were initially scored from the readable source (`pr1394_with_ttt.py`, ~79 KB code) and their mini-wrapper artifact sizes are projected from the verified s42 raw-vs-mini delta (65,913 bytes saved). Both project comfortably under 16,000,000 bytes. Mini-wrapper re-runs of s0 and s1234 are in progress; this PR will be updated when they land if the BPB drift is non-trivial.
+- 5-seed standard deviation: **0.00046 BPB** (5-seed standard error of the mean: ~0.00021).
 
 ### Diagnostics
 
@@ -28,6 +30,8 @@ Beats [PR #1394](https://github.com/openai/parameter-golf/pull/1394) (1.08563) b
 | 0    | 1.08717 | 1.09895 | 1.08220 | 1.07743 | 333.6 | 31.8 | 22.38% |
 | 42   | 1.08781 | 1.09932 | 1.08262 | 1.07808 | 344.8 | 32.5 | 22.38% |
 | 1234 | 1.08820 | 1.09898 | 1.08352 | 1.07848 | 334.5 | 31.7 | 22.38% |
+| 1337 | 1.08772 | 1.09918 | 1.08246 | 1.07801 | 338.4 | 31.9 | 22.38% |
+| 2025 | 1.08842 | 1.09957 | 1.08306 | 1.07862 | 333.4 | 32.0 | 22.38% |
 
 ## Key Innovations
 

records/track_10min_16mb/2026-04-07_SP8192_ParallelResid7_Loop35_NgramTilt/submission.json

@@ -1,16 +1,18 @@
 {
   "name": "SP8192 + Parallel Residuals + 3-Layer Recurrence + Legal N-gram Tilt",
-  "val_bpb": 1.07800,
-  "val_loss": 2.78457,
-  "bytes_total": 15996192,
-  "blurb": "3-lever stack on top of PR #1394 sp8192 baseline: (1) GPT-J parallel residuals on layers 7-10 (PR #1412 @Robby955), (2) 3-layer depth recurrence (loop layers 3-5 twice instead of 4-5 twice), (3) eval-time causal n-gram tilt with one-token exponential rescaling (PR #1420 @abaybektursun, lineage PR #1145 @AnirudhRahul). All four issue #1017 conditions verified. C++ n-gram kernel ported from PR #1420 with nanobind dependency removed (ctypes shim). 3-seed mean 1.07800 BPB beats PR #1394 (1.08563) by 0.01971 nats per token, beats PR #1420 (1.08014) by 0.00553 nats per token, beats own PR #1413 (1.08279) by 0.01237 nats per token.",
+  "val_bpb": 1.07813,
+  "val_loss": 2.78491,
+  "bytes_total": 15992215,
+  "blurb": "3-lever stack on top of PR #1394 sp8192 baseline: (1) GPT-J parallel residuals on layers 7-10 (PR #1412 @Robby955), (2) 3-layer depth recurrence (loop layers 3-5 twice instead of 4-5 twice), (3) eval-time causal n-gram tilt with one-token exponential rescaling (PR #1420 @abaybektursun, lineage PR #1145 @AnirudhRahul). All four issue #1017 conditions verified. C++ n-gram kernel ported from PR #1420 with nanobind dependency removed (ctypes shim). 5-seed mean 1.07813 BPB (std 0.00046) beats PR #1394 (1.08563) by 0.01938 nats per token, beats PR #1420 (1.08014) by 0.00520 nats per token, beats own PR #1413 (1.08279) by 0.01205 nats per token.",
   "author": "dexhunter",
   "github_id": "dexhunter",
   "date": "2026-04-07",
   "seed_results": {
     "0":    {"val_bpb": 1.07743, "val_loss": 2.78312, "steps": 4911},
     "42":   {"val_bpb": 1.07808, "val_loss": 2.78479, "steps": 4913},
-    "1234": {"val_bpb": 1.07848, "val_loss": 2.78581, "steps": 4905}
+    "1234": {"val_bpb": 1.07848, "val_loss": 2.78581, "steps": 4905},
+    "1337": {"val_bpb": 1.07801, "val_loss": 2.78461, "steps": 4909},
+    "2025": {"val_bpb": 1.07862, "val_loss": 2.78620, "steps": 4908}
   },
   "lineage": [
     "PR #1394 (clarkkev) — sp8192 base",

records/track_10min_16mb/2026-04-07_SP8192_ParallelResid7_Loop35_NgramTilt/train_seed1337.log

@@ -0,0 +1,295 @@
+
+*****************************************
+Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+*****************************************
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  asymmetric_logit: False
+  beta1: 0.9
+  beta2: 0.95
+  cautious_wd: False
+  compressor: brotli
+  data_dir: /home/dex/parameter-golf-with-cc/data
+  datasets_dir: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp8192
+  distributed: True
+  ema_decay: 0.997
+  embed_bits: 8
+  embed_clip_sigmas: 20.0
+  embed_lr: 0.6
+  embed_wd: 0.085
+  embedding_dim: 512
+  enable_looping_at: 0.5
+  etlb_clip: 3.0
+  etlb_enabled: False
+  etlb_lr: 0.05
+  etlb_steps: 5
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_reserve_seconds: 12.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  hessian_clip_lambda: 0.0
+  int5_layers_str: 
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/pr1394_par7_loop35_ngram_qk5_ttt_s1337.txt
+  logit_softcap: 30.0
+  loop_end: 5
+  loop_start: 3
+  matrix_bits: 6
+  matrix_clip_sigmas: 12.85
+  matrix_lr: 0.02
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mixed_quant: False
+  mlp_mult: 4.0
+  model_dim: 512
+  model_path: final_model.pt
+  muon_backend_steps: 5
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_row_normalize: True
+  muon_wd: 0.085
+  n_int6_layers: 50
+  ngram_agree_bonus: 0.1
+  ngram_base_beta: 2.0
+  ngram_open_table_bits: 26
+  ngram_order_stride: 2
+  ngram_tilt_enabled: True
+  ngram_within_beta: 0.92
+  ngram_within_threshold: 0.25
+  ngram_word_beta: 0.5
+  ngram_word_threshold: 0.8
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  num_loops: 2
+  parallel_residual_start: 7
+  qk_gain_init: 5.0
+  quantized_model_path: final_model.int6.ptz
+  rank: 0
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: pr1394_par7_loop35_ngram_qk5_ttt_s1337
+  scalar_lr: 0.02
+  seed: 1337
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: /home/dex/parameter-golf-with-cc/data/tokenizers/fineweb_8192_bpe.model
+  train_batch_tokens: 786432
+  train_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp8192/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  ttt_batch_seqs: 32
+  ttt_chunk_tokens: 32768
+  ttt_enabled: True
+  ttt_epochs: 3
+  ttt_freeze_blocks: 0
+  ttt_grad_clip: 1.0
+  ttt_lr: 0.005
+  ttt_momentum: 0.9
+  use_polar_express: False
+  val_batch_tokens: 524288
+  val_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp8192/fineweb_val_*.bin
+  val_loss_every: 4000
+  vocab_size: 8192
+  warmdown_frac: 0.667
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+train_shards: 80
+val_tokens: 40540160
+model_params:35944536
+gptq:reserving 12s, effective=588000ms
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+loop_warmup:enabled encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+loop_warmup_step: 1/20
+loop_warmup_step: 2/20
+loop_warmup_step: 3/20
+loop_warmup_step: 4/20
+loop_warmup_step: 5/20
+loop_warmup_step: 6/20
+loop_warmup_step: 10/20
+loop_warmup_step: 20/20
+0/20000 val_loss: 9.0047 val_bpb: 3.4860
+1/20000 train_loss: 9.0060 train_time: 0.0m tok/s: 8323843
+2/20000 train_loss: 12.2777 train_time: 0.0m tok/s: 8185758
+3/20000 train_loss: 10.9105 train_time: 0.0m tok/s: 8095004
+4/20000 train_loss: 9.4144 train_time: 0.0m tok/s: 8052278
+5/20000 train_loss: 8.2870 train_time: 0.0m tok/s: 8026824
+500/20000 train_loss: 3.3745 train_time: 0.8m tok/s: 7790966
+1000/20000 train_loss: 3.2761 train_time: 1.7m tok/s: 7801926
+1500/20000 train_loss: 3.1781 train_time: 2.5m tok/s: 7810344
+2000/20000 train_loss: 3.0679 train_time: 3.4m tok/s: 7812312
+2500/20000 train_loss: 3.1518 train_time: 4.2m tok/s: 7812217
+layer_loop:enabled step:2921 frac:0.500 encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+3000/20000 train_loss: 2.9648 train_time: 5.1m tok/s: 7716745
+3500/20000 train_loss: 2.9715 train_time: 6.3m tok/s: 7248413
+4000/20000 train_loss: 2.8525 train_time: 7.6m tok/s: 6933232
+4000/20000 val_loss: 2.9088 val_bpb: 1.1261
+4500/20000 train_loss: 2.8783 train_time: 8.8m tok/s: 6706977
+4909/20000 val_loss: 2.8117 val_bpb: 1.0885
+stopping_early: wallclock_cap train_time: 588117ms step: 4909/20000
+peak memory allocated: 39046 MiB reserved: 39070 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.80969528 val_bpb:1.08772060 eval_time:6096ms
+Serialized model: 135431033 bytes
+Code size: 19060 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 67 Hessians in 12.7s
+Quantized weights:
+  gptq (int6): blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int8): tok_emb.weight
+  passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, skip_gates, skip_weights
+Serialized model quantized+brotli: 15968979 bytes
+Total submission size quantized+brotli: 15988039 bytes
+quantized val_loss:2.83930586 val_bpb:1.09918378 eval_time:8577ms
+quantized_sliding_window val_loss:2.79610141 val_bpb:1.08245799 eval_time:91730ms
+ngram_tilt:precompute n_tok=40540161 hints=9074676 (22.38%) elapsed=31.9s base_beta=2.0 within_beta=0.92 agree_bonus=0.1
+ttt_sliding:start chunks=1238 chunk_tokens=32768 total_windows=633409 stride=64 ttt_lr=0.005 ttt_epochs=3 freeze_blocks=0
+ttt_sliding:params unfrozen=35944536 frozen=0
+  ttt_chunk [1/1238] bpb=1.113223 time=4.7s
+  ttt_chunk [11/1238] bpb=1.069067 time=9.2s
+  ttt_chunk [21/1238] bpb=1.105682 time=11.8s
+  ttt_chunk [31/1238] bpb=1.100420 time=14.4s
+  ttt_chunk [41/1238] bpb=1.093499 time=17.0s
+  ttt_chunk [51/1238] bpb=1.086986 time=19.7s
+  ttt_chunk [61/1238] bpb=1.078628 time=22.3s
+  ttt_chunk [71/1238] bpb=1.085817 time=25.0s
+  ttt_chunk [81/1238] bpb=1.079146 time=27.6s
+  ttt_chunk [91/1238] bpb=1.075438 time=30.2s
+  ttt_chunk [101/1238] bpb=1.075157 time=32.9s
+  ttt_chunk [111/1238] bpb=1.073582 time=35.5s
+  ttt_chunk [121/1238] bpb=1.076583 time=38.2s
+  ttt_chunk [131/1238] bpb=1.080321 time=40.8s
+  ttt_chunk [141/1238] bpb=1.080886 time=43.4s
+  ttt_chunk [151/1238] bpb=1.080726 time=46.1s
+  ttt_chunk [161/1238] bpb=1.081189 time=48.7s
+  ttt_chunk [171/1238] bpb=1.080984 time=51.3s
+  ttt_chunk [181/1238] bpb=1.079458 time=54.0s
+  ttt_chunk [191/1238] bpb=1.079154 time=56.6s
+  ttt_chunk [201/1238] bpb=1.076698 time=59.2s
+  ttt_chunk [211/1238] bpb=1.081125 time=61.8s
+  ttt_chunk [221/1238] bpb=1.081424 time=64.5s
+  ttt_chunk [231/1238] bpb=1.083161 time=67.1s
+  ttt_chunk [241/1238] bpb=1.081362 time=69.7s
+  ttt_chunk [251/1238] bpb=1.081298 time=72.4s
+  ttt_chunk [261/1238] bpb=1.082325 time=75.0s
+  ttt_chunk [271/1238] bpb=1.082803 time=77.6s
+  ttt_chunk [281/1238] bpb=1.082125 time=80.3s
+  ttt_chunk [291/1238] bpb=1.083265 time=82.9s
+  ttt_chunk [301/1238] bpb=1.083449 time=85.6s
+  ttt_chunk [311/1238] bpb=1.082205 time=88.5s
+  ttt_chunk [321/1238] bpb=1.082036 time=91.4s
+  ttt_chunk [331/1238] bpb=1.082226 time=94.0s
+  ttt_chunk [341/1238] bpb=1.081367 time=96.7s
+  ttt_chunk [351/1238] bpb=1.082163 time=99.3s
+  ttt_chunk [361/1238] bpb=1.081050 time=101.9s
+  ttt_chunk [371/1238] bpb=1.079511 time=104.5s
+  ttt_chunk [381/1238] bpb=1.079913 time=107.2s
+  ttt_chunk [391/1238] bpb=1.079554 time=109.8s
+  ttt_chunk [401/1238] bpb=1.079598 time=112.5s
+  ttt_chunk [411/1238] bpb=1.080104 time=115.1s
+  ttt_chunk [421/1238] bpb=1.079595 time=117.7s
+  ttt_chunk [431/1238] bpb=1.079783 time=120.4s
+  ttt_chunk [441/1238] bpb=1.079820 time=123.0s
+  ttt_chunk [451/1238] bpb=1.081019 time=125.7s
+  ttt_chunk [461/1238] bpb=1.079249 time=128.3s
+  ttt_chunk [471/1238] bpb=1.079234 time=130.9s
+  ttt_chunk [481/1238] bpb=1.079400 time=133.6s
+  ttt_chunk [491/1238] bpb=1.079845 time=136.5s
+  ttt_chunk [501/1238] bpb=1.079463 time=139.2s
+  ttt_chunk [511/1238] bpb=1.079152 time=141.8s
+  ttt_chunk [521/1238] bpb=1.078696 time=144.5s
+  ttt_chunk [531/1238] bpb=1.078634 time=147.7s
+  ttt_chunk [541/1238] bpb=1.078702 time=150.4s
+  ttt_chunk [551/1238] bpb=1.078241 time=153.1s
+  ttt_chunk [561/1238] bpb=1.077545 time=155.7s
+  ttt_chunk [571/1238] bpb=1.076982 time=158.4s
+  ttt_chunk [581/1238] bpb=1.077342 time=161.1s
+  ttt_chunk [591/1238] bpb=1.077551 time=163.7s
+  ttt_chunk [601/1238] bpb=1.077510 time=166.4s
+  ttt_chunk [611/1238] bpb=1.078097 time=169.1s
+  ttt_chunk [621/1238] bpb=1.078935 time=171.8s
+  ttt_chunk [631/1238] bpb=1.078996 time=174.4s
+  ttt_chunk [641/1238] bpb=1.079425 time=177.1s
+  ttt_chunk [651/1238] bpb=1.079751 time=179.8s
+  ttt_chunk [661/1238] bpb=1.079081 time=182.4s
+  ttt_chunk [671/1238] bpb=1.078863 time=185.1s
+  ttt_chunk [681/1238] bpb=1.080181 time=187.8s
+  ttt_chunk [691/1238] bpb=1.080361 time=190.4s
+  ttt_chunk [701/1238] bpb=1.080161 time=193.1s
+  ttt_chunk [711/1238] bpb=1.080845 time=195.8s
+  ttt_chunk [721/1238] bpb=1.081152 time=198.5s
+  ttt_chunk [731/1238] bpb=1.080504 time=201.1s
+  ttt_chunk [741/1238] bpb=1.080201 time=203.8s
+  ttt_chunk [751/1238] bpb=1.079266 time=206.5s
+  ttt_chunk [761/1238] bpb=1.078662 time=209.1s
+  ttt_chunk [771/1238] bpb=1.077649 time=211.8s
+  ttt_chunk [781/1238] bpb=1.077644 time=214.5s
+  ttt_chunk [791/1238] bpb=1.078000 time=217.1s
+  ttt_chunk [801/1238] bpb=1.078270 time=219.8s
+  ttt_chunk [811/1238] bpb=1.077775 time=222.5s
+  ttt_chunk [821/1238] bpb=1.076585 time=225.1s
+  ttt_chunk [831/1238] bpb=1.076276 time=227.8s
+  ttt_chunk [841/1238] bpb=1.075817 time=230.5s
+  ttt_chunk [851/1238] bpb=1.075519 time=233.1s
+  ttt_chunk [861/1238] bpb=1.075169 time=235.8s
+  ttt_chunk [871/1238] bpb=1.075050 time=238.4s
+  ttt_chunk [881/1238] bpb=1.074616 time=241.1s
+  ttt_chunk [891/1238] bpb=1.074084 time=243.7s
+  ttt_chunk [901/1238] bpb=1.074450 time=246.4s
+  ttt_chunk [911/1238] bpb=1.074148 time=249.1s
+  ttt_chunk [921/1238] bpb=1.074436 time=251.7s
+  ttt_chunk [931/1238] bpb=1.075113 time=254.4s
+  ttt_chunk [941/1238] bpb=1.075496 time=257.0s
+  ttt_chunk [951/1238] bpb=1.075427 time=259.7s
+  ttt_chunk [961/1238] bpb=1.076268 time=262.4s
+  ttt_chunk [971/1238] bpb=1.076655 time=265.0s
+  ttt_chunk [981/1238] bpb=1.077032 time=267.7s
+  ttt_chunk [991/1238] bpb=1.076823 time=270.3s
+  ttt_chunk [1001/1238] bpb=1.076884 time=273.0s
+  ttt_chunk [1011/1238] bpb=1.077229 time=275.7s
+  ttt_chunk [1021/1238] bpb=1.077949 time=278.3s
+  ttt_chunk [1031/1238] bpb=1.078436 time=281.0s
+  ttt_chunk [1041/1238] bpb=1.078896 time=283.7s
+  ttt_chunk [1051/1238] bpb=1.078823 time=286.3s
+  ttt_chunk [1061/1238] bpb=1.078820 time=289.0s
+  ttt_chunk [1071/1238] bpb=1.078974 time=291.6s
+  ttt_chunk [1081/1238] bpb=1.078853 time=294.3s
+  ttt_chunk [1091/1238] bpb=1.079037 time=297.0s
+  ttt_chunk [1101/1238] bpb=1.079561 time=299.6s
+  ttt_chunk [1111/1238] bpb=1.079850 time=302.3s
+  ttt_chunk [1121/1238] bpb=1.080025 time=305.0s
+  ttt_chunk [1131/1238] bpb=1.079680 time=307.6s
+  ttt_chunk [1141/1238] bpb=1.079329 time=310.3s
+  ttt_chunk [1151/1238] bpb=1.079369 time=313.0s
+  ttt_chunk [1161/1238] bpb=1.079503 time=315.6s
+  ttt_chunk [1171/1238] bpb=1.079282 time=318.3s
+  ttt_chunk [1181/1238] bpb=1.078825 time=321.0s
+  ttt_chunk [1191/1238] bpb=1.078962 time=323.6s
+  ttt_chunk [1201/1238] bpb=1.079014 time=326.3s
+  ttt_chunk [1211/1238] bpb=1.078705 time=329.0s
+  ttt_chunk [1221/1238] bpb=1.078246 time=331.6s
+  ttt_chunk [1231/1238] bpb=1.077894 time=334.3s
+  ttt_chunk [1238/1238] bpb=1.077892 time=338.1s
+ttt_sliding:done val_loss=2.784614 val_bpb=1.078011 elapsed=338.3s
+legal_ttt_exact val_loss:2.78461379 val_bpb:1.07801077 eval_time:338446ms