State migration

[bkchr]

We need to migrate all parachains and relay chain to state version 1. There is a pallet for doing this. With 1.0.0 Kusama will enable the state migration. After that we also need to migrate the parachains and Polkadot and its parachains. This issue works as a tracking issue.

• Polkadot [x], [x], [ ]
  • Asset-Hub [ ], [ ], [ ]
  • Bridge-Hub [ ], [ ], [ ], [ ]
  • Collectives [ ], [ ], [ ]
• Kusama [x], [x], [x]
  • Asset-Hub [x], [ ], [ ]
  • Bridge-Hub [ ], [ ], [ ], [ ]
  • Coretime [ ], [ ], [ ], [ ]
  • Encointer [ ], [ ], [ ]
  • Glutton [ ], [ ], [ ], [ ]
  • People [ ], [ ], [ ], [ ]

Three check marks: migration deployed, RPC reports done, migration removed from the runtime.

[bkchr]

@cheme could you please post some status on what needs to be done for Kusama? Aka does the migration works on its own or is that driven by some offchain thingy?

After Kusama was successful, we should also directly start doing it for Polkadot.

[cheme]

Looks ready to go with next kusuma runtime.

This line set at 1 (switch to hybrid state when new runtime released) :

runtimes/relay/kusama/src/lib.rs

Line 146 in 94b2798

state_version: 1,

Start of migration added to Unreleased

runtimes/relay/kusama/src/lib.rs

Line 1731 in 94b2798

init_state_migration::InitMigrate,

Unrealeased set as runtime migration https://github.com/polkadot-fellows/runtimes/blob/94b2798b69ba6779764e20a50f056e48db78ebef/relay/kusama/src/lib.rs#L1654C35-L1654C35

Not that if there is many migration runing together, might be an idea to lower the limit per block:

runtimes/relay/kusama/src/lib.rs

Line 2714 in 94b2798

AutoLimits::<Runtime>::put(Some(MigrationLimits { item: 4_800, size: 204800 * 2 }));

here up to 4800 item or 408000 octet which in case of max out blocks can add :
db_weight_reads_writes(1, 1) = (25_000 + 100_000) * 1000.
(using rocksdb cst).
125_000_000 * 4_800 + 408_000 * 1_139 + some_fix_weight
600 * 10^9 + 464_712_000 out of 2_000_000_000_000, so if I check correctly about 1 third of a block weight.
Note that in case of relay chain, consuming weight is not a must have from my point of view.

runtimes/relay/kusama/src/lib.rs

Line 1468 in 94b2798

parameter_types! {

Warning if the start line get remove from unreleased, the line 146 must be set to 0 (to avoid hybrid state).

[cheme]

@bkchr the weight use in each block is an important point that should be in release note I think.

As a relay chain we could also ignore this weight to prevent any issue.

[cheme]

actually I did use 2 * 10 ^ 12 for block weight but on relay it may be 6*10^12 so would not be worrying then

[bkchr]

The migration will run entirely on chain and doesn't require any external interactions?

[cheme]

chain and doesn't require any exte

on chain, no possible external interactions.

[ggwpez]

Kusama is done? I queried StateTrieMigration.MigrationProcess and got:

 {
  "progress_top": {
    "name": "Complete",
    "values": []
  },
  "progress_child": {
    "name": "ToStart",
    "values": []
  },
  "size": 239581990,
  "top_items": 881284,
  "child_items": 1373
}

The Events also look fine. Just ~480 blocks to migrate everything? Looks like it did about 4800 keys in some blocks, nice 😳

[cheme]

4800 was the limit indeed.
This rpc can be call (paritytech/cumulus#1424) to double check the state did migrate correctly (requires runing it locally as it is unsafe).
Ultimately another good check is to run a warp sync (if the state is not migrated warp sync should not be working).
(cannot run these right now I am ooo)

[ggwpez]

Ultimately another good check is to run a warp sync (if the state is not migrated warp sync should not be working).

I just tried and it still works.

[bkchr]

@cheme so did it finished successfully?

[cheme]

Yes, but if warp sync did pass, all is fine.
(warp sync is a guaranteed fail during migration, and the counter on chain do state the migration is finished).

[bkchr]

@cheme can you prepare the changes for Polkadot?

[cheme]

Will do (tomorrow most likely), I think since block time is longer on polkadot, keeping the same config as kusuma should be fine.

[bkchr]

Ping @cheme

[bkchr]

BTW, we also need to migrate the system chains.