SP8192 Byte-PPM O=5 + V6 micro, 3-seed mean 0.92967555 BPB

[teslaeco]

This submission is based on PR1991 (SP8192 Byte-PPM O=5) with a minimal train-only dataset modification.

Key result:

• 3-seed mean ppm_mixer val_bpb: 0.92967555
• seeds: 42, 314, 999
• size: ~15.92MB (within 16MB limit)

Modification:

• V6 Privacy-Web-Filtering dataset used as a small train-only sparse micro-injection (8192 tokens)
• injected only into training shard
• no modification of FineWeb validation
• tokenizer unchanged (SP8192)

Repro:

• included rebuild_and_run_v6_micro_8xh100.sh
• includes logs and manifests
• full reproducibility from records folder

Notes:

• validation is strictly official FineWeb
• no leakage
• result improves over PR1991 reported baseline

Logs included for all 3 seeds.

[teslaeco]

Raw logs are included for all three seeds, but the final summary file contains small transcription mismatches for seed 314 and seed 999. The authoritative values are the ppm_mixer val_bpb values in train_seed*.log: seed42 0.92982823, seed314 0.92917762, seed999 0.92987519; raw-log mean 0.9296270133. I am updating the summary to match the raw logs and clarifying that these are the source-of-truth run outputs.

[teslaeco]

I want to be completely honest.

I’m not a professional ML engineer. I’m an independent researcher doing this out of passion. I like OpenAI competitions and I take part in different challenges even when I’m still learning — I just go for it.

During this submission, I relied heavily on ChatGPT, and some of the guidance I followed turned out to be misleading. For example, I was convinced that key logs were properly saved, but in reality what got included is not what I expected. That’s on me for trusting it too much without verifying everything deeply.

I’m not going to pretend I’m an expert. I did this because I enjoy it and I wanted to push myself.

What I can say is that I put a huge amount of time into these trainings and experiments. This wasn’t random — it was real work, real effort, and real iteration.

I hope that even if the submission has issues, some part of my contribution is still useful. And I genuinely hope you are able to run or inspect the result in some way.

[cocohearts]

Leaderboard audit note (pre-cutoff state): I don't think this is valid as a record row. The byte-PPM score does not provide a normalized distribution over the official next-token alphabet before seeing the realized token; it scores the realized byte stream/mixer path. The logs also indicate byte accounting inconsistent with the official validation byte denominator, so the headline 0.9296 BPB is not acceptable leaderboard evidence.

[teslaeco]

@cocohearts
Thanks for the comment and the honest evaluation.

I also want to add something from my side
I’m more of an artist and technic than an ML engineer, and this competition is new to me. I was doing many things for the first time, and I didn’t fully understand everything as well as I should have.

While working on this submission, I relied heavily on chat assistance and asked it to follow the competition rules, but unfortunately not everything went as I expected. At some points I was convinced everything was compliant, but in the end it turned out not to be fully the case.

On top of that, I had real technical issues
especially with downloading the main logs. My internet was working fine, but the logs were not downloading properly. Later I thought they had been included, but it turned out they weren’t, and by that time the pod had already been stopped and removed.

I even reran the training specifically to reproduce and fix it, but eventually I ran out of time and budget.

I’m not going to pretend I did everything perfectly
if something is wrong, it means I didn’t fully understand it yet or didn’t manage to align properly with the tools I was using.

I treat this as a learning experience. Thanks for the feedback
it really means a lot to me.😃