Stack Overflow While Syncing

[aurexav]

0|icefrog  | 2020-02-03 19:16:39 New epoch 4076 launching at block 0xf116…95b9 (block slot 526793793 >= start slot 526793792).
0|icefrog  | 2020-02-03 19:16:39 Next epoch starts at slot 526793992

0|icefrog  | thread 'import-queue-worker-0' has overflowed its stack
0|icefrog  | fatal runtime error: stack overflow

[aurexav]

Sync Trace:

0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli  Darwinia IceFrog Node
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli  Version: 0.4.6-17acb1d-x86_64-linux-gnu
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli    _____                      _       _
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli   |  __ \                    (_)     (_)
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli   | |  | | __ _ _ ____      ___ _ __  _  __ _
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli   | |  | |/ _` | '__\ \ /\ / / | '_ \| |/ _` |
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli   | |__| | (_| | |   \ V  V /| | | | | | (_| |
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli   |_____/ \__,_|_|    \_/\_/ |_|_| |_|_|\__,_|
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli  Chain specification: Darwinia IceFrog Testnet
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli  Node name: Revive GCP
0|icefrog  | 2020-02-03 19:19:19.157 main INFO node_cli::cli  Roles: "AUTHORITY"
0|icefrog  | 2020-02-03 19:19:19.270 main INFO substrate_service::builder  Highest known block at #753265
0|icefrog  | 2020-02-03 19:19:19.270 main INFO sub-libp2p  Local node identity is: QmTGDwiRMdX9VcqiXDFmnFrUgubaF3BMzNoQDPfJGFNcwd
0|icefrog  | 2020-02-03 19:19:19.304 main INFO babe  Starting BABE Authorship worker
0|icefrog  | 2020-02-03 19:19:20.124 main-tokio-1 TRACE sync  Connecting QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq
0|icefrog  | 2020-02-03 19:19:20.124 main-tokio-1 TRACE sync  New peer QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq Status { version: 5, min_supported_version: 3, roles: AUTHORITY, best_number: 832706, best_hash: 0x03e3cb37a218161d7f774cebb293d24ce55cd422ed717b63af4e7b598155a03e, genesis_hash: 0xa7de3b38d9a0a71cdc15acf5cf7e6d352350e221c5e3222dbba92b6f10497c54, chain_status: [] }
0|icefrog  | 2020-02-03 19:19:20.124 main-tokio-1 DEBUG sync  Connected QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq
0|icefrog  | 2020-02-03 19:19:20.125 main-tokio-1 DEBUG sync  New peer with unknown best hash 0x03e3…a03e (832706), searching for common ancestor.
0|icefrog  | 2020-02-03 19:19:20.126 main-tokio-1 TRACE sync  Peer QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq is busy
0|icefrog  | 2020-02-03 19:19:20.126 main-tokio-1 INFO sub-libp2p  Discovered new external address for our node: /ip4/35.234.33.88/tcp/30333/p2p/QmTGDwiRMdX9VcqiXDFmnFrUgubaF3BMzNoQDPfJGFNcwd
0|icefrog  | 2020-02-03 19:19:20.344 main-tokio-0 TRACE sync  BlockResponse 0 from QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq with 1 blocks  (753265)
0|icefrog  | 2020-02-03 19:19:20.344 main-tokio-0 TRACE sync  Got ancestry block #753265 (0x7278…b923) from peer QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq
0|icefrog  | 2020-02-03 19:19:20.344 main-tokio-0 TRACE sync  Ancestry search complete. Ours=0x7278…b923 (753265), Theirs=0x03e3…a03e (832706), Common=Some(0x727856507873f90b09af467b853873c046a0d2415451850fcbaa711c6bd3b923) (753265)
0|icefrog  | 2020-02-03 19:19:20.344 main-tokio-0 TRACE sync  New block request for QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq, (best:832706, common:753265) BlockRequest { id: 0, fields: HEADER | BODY | JUSTIFICATION, from: Number(753266), to: None, direction: Ascending, max: Some(128) }
0|icefrog  | 2020-02-03 19:19:20.483 main-tokio-0 TRACE sync  BlockResponse 1 from QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq with 128 blocks  (753266..753393)
0|icefrog  | 2020-02-03 19:19:20.483 main-tokio-0 TRACE sync  Drained 128 blocks
0|icefrog  | 2020-02-03 19:19:20.483 main-tokio-0 TRACE sync  Accepted 128 blocks (0x3756fc203770b3f5b047a6003c900013b448aa43cacf8d3b1216051993cbff16) with origin NetworkInitialSync
0|icefrog  | 2020-02-03 19:19:20.483 main-tokio-0 TRACE sync  Updating peer QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq info, ours=753393, common=753265->753393, their best=832706
0|icefrog  | 2020-02-03 19:19:20.483 main-tokio-0 TRACE sync  New block request for QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq, (best:832706, common:753393) BlockRequest { id: 0, fields: HEADER | BODY | JUSTIFICATION, from: Number(753394), to: None, direction: Ascending, max: Some(128) }
0|icefrog  | 2020-02-03 19:19:20.483 main-tokio-0 TRACE sync  Scheduling 128 blocks for import
0|icefrog  | 2020-02-03 19:19:20.483 import-queue-worker-0 TRACE sync  Starting import of 128 blocks  (753266..753393)
0|icefrog  | 2020-02-03 19:19:20.483 import-queue-worker-0 TRACE sync  Header 0xf116…95b9 has 5 logs
0|icefrog  | 2020-02-03 19:19:20.631 import-queue-worker-0 INFO babe  New epoch 4076 launching at block 0xf116…95b9 (block slot 526793793 >= start slot 526793792).
0|icefrog  | 2020-02-03 19:19:20.631 import-queue-worker-0 INFO babe  Next epoch starts at slot 526793992
0|icefrog  | thread 'import-queue-worker-0' has overflowed its stack
0|icefrog  | fatal runtime error: stack overflow

[aurexav]

paritytech/substrate#4527

[aurexav]

0|icefrog  | 2020-02-04 06:01:13.178 main-tokio-1 DEBUG sync  Received block announcement 0x85bad6c76d0b9df6182a2e25527e74908083c00a61ec5aad9b32a76f5644d0b4 with number 845544 from QmPBHf9928T3T9c5dsgSyFUcfQbJ1aD8p9iJUykYB7Tcmq
0|icefrog  | 2020-02-04 06:01:13.178 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.220 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.220 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.265 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.267 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.268 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.269 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.269 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.294 main-tokio-1 INFO substrate  Syncing  0.3 bps, target=#845544 (1 peers), best: #435114 (0xc17f…71a8), finalized #334166 (0x886e…944f), ⬇ 3.6kiB/s ⬆ 7.0kiB/s
0|icefrog  | 2020-02-04 06:01:13.295 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.381 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.418 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.420 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.472 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.472 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.473 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.473 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:13.830 import-queue-worker-0 TRACE sync  Block imported successfully Some(435115) (0x721a…fa4b)
0|icefrog  | 2020-02-04 06:01:13.830 import-queue-worker-0 TRACE sync  Header 0x7753…0b65 has 2 logs
0|icefrog  | 2020-02-04 06:01:14.008 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.008 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.010 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.010 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.130 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.132 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.211 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.405 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.405 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.405 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.405 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.405 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.405 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.405 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.406 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.407 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.407 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.407 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.413 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.414 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.415 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.415 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.415 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.416 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.417 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.417 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.685 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.686 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.773 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.773 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.911 main-tokio-1 DEBUG sync  Propagating extrinsics
0|icefrog  | 2020-02-04 06:01:14.912 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.966 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.966 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.966 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.969 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:14.969 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.010 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.010 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.010 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.056 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.056 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.147 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.193 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.193 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.195 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.195 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.207 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.208 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.238 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.238 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.311 main-tokio-0 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.328 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.328 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.330 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.330 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.375 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.376 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.500 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.501 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.501 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.513 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.513 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.593 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.593 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.624 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.625 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.689 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.689 main-tokio-1 TRACE sync  Too many blocks in the queue.
0|icefrog  | 2020-02-04 06:01:15.690 main-tokio-1 TRACE sync  Too many blocks in the queue.

[aurexav]

Workaround:

// by default we disable pruning if the node is an authority (i.e.
// `ArchiveAll`), otherwise we keep state for the last 256 blocks. if the
// node is an authority and pruning is enabled explicitly, then we error
// unless `unsafe_pruning` is set.

Set unsafe-pruning to avoid stack overflow.

[aurexav]

fixed in #268