Cannot connect to blockchain after upgrading node and plugin

[bafu]

Describe the bug

After upgrading avalanchego (1.7.14 -> 1.7.16) and subnet-evm (commit 1a7f57e -> 0.2.7), I cannot connect RPC to one of my two EVMs (on different subnets separately).

1. Can-work EVM
   1. with a plain genesis
   2. can connect RPC and transfer native coins
2. Cannot-work EVM

   1. with the genesis containing contractNativeMinterConfig and feeManagerConfig

   2. cannot connect RPC and avalanchego showed the error below

      [07-30|09:47:43.692] ERROR chains/manager.go:293 error creating chain "2StUZ1a5omGxvMRm7b2LHjuoDKTmsVEoCEwNeDRZouWvFZwWir": error while creating new snowman vm rpc error: code = Unknown desc = failed to create backend: database contains incompatible genesis (have af912611960895cc0c3f533ec6b87c55966714cb877632491a2c7b83309a395e, new d108973f4d32737daa30b06c67d8de7c40dc846b2ea326fbb61dc1f87f0ab930)
      

To Reproduce

Steps to reproduce the behavior.

1. Download the pre-build avalanchego and subnet-evm.

2. Copy subnet-evm to avalanchego-v1.7.16/plugins/<vmid>

3. Stop the running avalanchego v1.7.14.

4. Run avalanchego v1.7.16

   cd avalanchego-v1.7.16
   ./avalanchego --whitelisted-subnets=CXPiHVHEutq3cs9ddNxxhW5C6trRTXzfGZsNEvpNm6vyuLugj,2fQBahhq3F9eip8KobMgjbvBEahW3153kvAy6YPDrGMTceZcGG --network-id=fuji --http-host=0.0.0.0
   

Expected behavior

MetaMask can connect to both blockchains via RPC and transfer native coins.

Screenshots

N/A

Logs

avalanchego-log.zip

Metrics

N/A

Operating System

1. OS: Ubuntu 20.04.4 LTS (AWS instance)
2. Avalanchego: v1.7.16
3. subnet-evm: v0.2.7

Additional context

N/A

[bafu]

@Julian-dev28 Could you help double check if my steps to update subnet-evm are correct?

I followed the steps in the Node update document but cannot find the subnet-evm update document. Thanks!

[ceyonur]

can you send the 2fQBahhq3F9eip8KobMgjbvBEahW3153kvAy6YPDrGMTceZcGG.log logs? The one provided in logs is not the problematic chain.

[Julian-dev28]

@bafu Thanks again for bringing this up

I've replied on Discord when this issue was first posted and I hope you were able to solve it.

Due to inactivity, I will go ahead and close the issue for now 🙂

Please feel free to ping a member of the team on discord if you run into any more trouble.

Cheers

[bafu]

@ceyonur @Julian-dev28 Sorry for the late reply because my testnet vaoidator has become non-bootstrapped with unknown issue. I'm figuring out the root cause and will go back to this issue soon. Thanks for following this issue up!

[ceyonur]

We suspect that if you have created your chain before you upgrade your subnet-evm, the chain genesis might have stored incorrectly (without feeConfigManager) with the old version of subnet-evm. After you updated the subnet-evm feeConfigManager gets recognized by the new version and being activated in the genesis, which changes the genesis hash. This could create a genesis not compatible error.

[bafu]

After re-creating a subnet with

1. avalanchego v1.7.14 (pre-built binary)
2. subnet v0.2.8 (pre-built binary)

I can get/set fee configs with FeeManager correctly

> fm = await ethers.getContractAt("IFeeManager", "0x0200000000000000000000000000000000000003")

> await fm.getFeeConfig()
[
  BigNumber { value: "20000000" },
  BigNumber { value: "2" },
  BigNumber { value: "1000000000" },
  BigNumber { value: "100000000" },
  BigNumber { value: "48" },
  BigNumber { value: "0" },
  BigNumber { value: "10000000" },
  BigNumber { value: "500000" },
  gasLimit: BigNumber { value: "20000000" },
  targetBlockRate: BigNumber { value: "2" },
  minBaseFee: BigNumber { value: "1000000000" },
  targetGas: BigNumber { value: "100000000" },
  baseFeeChangeDenominator: BigNumber { value: "48" },
  minBlockGasCost: BigNumber { value: "0" },
  maxBlockGasCost: BigNumber { value: "10000000" },
  blockGasCostStep: BigNumber { value: "500000" }
]

> await fm.setFeeConfig(20000000, 2, 1000000000, 100000000, 48, 0, 10000000, 500000)
{
  hash: '0xd29ee59c9a9a0e752ff2369d09b08653d0fc4433aa8769c5982cfe5de5ce1652',
  type: 0,
  accessList: null,
  blockHash: '0x2ee0f10f1ae7733c80d761b9b760074a205a38f8fb414713d7c25826aceac138',
  blockNumber: 6,
  transactionIndex: 0,
  confirmations: 1,
  from: '0x63B7076FC0A914Af543C2e5c201df6C29FCC18c5',
  gasPrice: BigNumber { value: "1000000000" },
  gasLimit: BigNumber { value: "202292" },
  to: '0x0200000000000000000000000000000000000003',
  value: BigNumber { value: "0" },
  nonce: 6,
  data: '0x8f10b5860000000000000000000000000000000000000000000000000000000001312d00000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000773594000000000000000000000000000000000000000000000000000000000005f5e100000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000989680000000000000000000000000000000000000000000000000000000000007a120',
  r: '0x353be061d8098f894c8f84e209b653431b905636129073071a748ab64c326f8d',
  s: '0x7febdf074517e715d0b3d03c2f73688a6eec9bcf785abc5353a277c64d109967',
  v: 21051,
  creates: null,
  chainId: 10508,
  wait: [Function (anonymous)]
}

> await fm.getFeeConfig()
[
  BigNumber { value: "20000000" },
  BigNumber { value: "2" },
  BigNumber { value: "2000000000" },
  BigNumber { value: "100000000" },
  BigNumber { value: "48" },
  BigNumber { value: "0" },
  BigNumber { value: "10000000" },
  BigNumber { value: "500000" },
  gasLimit: BigNumber { value: "20000000" },
  targetBlockRate: BigNumber { value: "2" },
  minBaseFee: BigNumber { value: "2000000000" },
  targetGas: BigNumber { value: "100000000" },
  baseFeeChangeDenominator: BigNumber { value: "48" },
  minBlockGasCost: BigNumber { value: "0" },
  maxBlockGasCost: BigNumber { value: "10000000" },
  blockGasCostStep: BigNumber { value: "500000" }
]

[bafu]

@ceyonur @Julian-dev28 I want to consult you on the last question related to this issue:

If I want to mint native coins and config dynamic fees on a subnet whose genesis did not contain the precompile configs, how can I add the precompile functionalities? Many thanks!

[yulin-dong]

@ceyonur @Julian-dev28 I want to consult you on the last question related to this issue:

If I want to mint native coins and config dynamic fees on a subnet whose genesis did not contain the precompile configs, how can I add the precompile functionalities? Many thanks!

Is https://docs.avax.network/subnets/customize-a-subnet#network-upgrades-enabledisable-precompiles what you are looking for?

[bafu]

Thanks for your sharing, Yulin.

For anyone looking for the details of config.json in the doc above:

1. The default filepath is $HOME/.avalanchego/configs/chains/<blockchainId>/config.json.
2. You can specify a different fileapth by the --chain-config-dir parameter of avalanchego.
3. References
   1. Network Upgrades: Enable/Disable Precompiles
   2. Subnet Chain Configs
   3. Chain Configs
   4. AvalancheGo Config and Flags