Forever pending wallet balance after channel closing. #1056

vegardengen · 2018-04-09T08:10:56Z

Background

I closed a channel, cooperatively, 2018-04-08 13:28:50

The TXIDs for this never appeared in any blockchain explorers. Maybe my full node never accepted it? There's no useful logs from bitcoind, however, from this point...

Later in the evening, I restarted my node. At 23:17:14, a new TX, with ID 83aa451a5cdbe82542abc3007d5231f61b6c1d102f12b58f64cb7514332799f9, was sent, for closing the same channel point. That one I can see is confirmed pretty instantly.

After this, the channel is completely closed in LND, but there is unconfirmed wallet balance corresponding to my share of the channel.

bitcoin@thundroid:~/.bitcoin$ date;lncli walletbalance
ma. 09. april 10:08:04 +0200 2018
{
"total_balance": "3394015",
"confirmed_balance": "1394198",
"unconfirmed_balance": "1999817"
}

lnd.log

Your environment

lnd version 0.4.1-beta commit=13945de8060ad42e077988c1cf69e205dc11e528
Linux thundroid 4.14.24-113 Fix name typo in README #1 SMP PREEMPT Mon Mar 5 12:21:35 UTC 2018 armv7l armv7l armv7l GNU/Linux
*bi tcoin@thundroid:~/log/lnd/bitcoin/mainnet$ bitcoind --version
Bitcoin Core Daemon version v0.16.0
any other relevant environment details

Steps to reproduce

Expected behaviour

Funds appear confirmed in wallet after channel is confirmed closed.

Actual behaviour

walletbalance having an eternal "unconfirmed balance".

halfik · 2018-04-09T18:38:11Z

I have same issue. Closed channels almost 2 days ago. One of the txid i got in response coudnt be found in block explorers. Now after i restareted ln i see this 2 channels as pending closing. Thing is they both have 351+ conf atm.

Ps. I was closing channels with closeallchannels command.

transactions:
https://blockchain.info/tx/6b0ee5443582fbb72afd0080f165bf68b62d05ee001aac64d38b2e66de38449e
https://blockchain.info/tx/8df6849fa44065c35aefb14bcc1f6e3056a90e2eb1b5dd53c03bb1115ea64691

Using lnd: 0.4.1
Bitcoin core 15.1
os: Linux raspberrypi 4.9.59-v7+ #1047 SMP Sun Oct 29 12:19:23 GMT 2017 armv7l GNU/Linux

{
"total_limbo_balance": "369503",
"pending_open_channels": [
{
"channel": {
"remote_node_pub": "024a2e265cd66066b78a788ae615acdc84b5b0dec9efac36d7ac87513015eaf6ed",
"channel_point": "01ac4ebf2cd9b4b89423d6b01e3459abc54952b315fe525081ce88d547e08203:1",
"capacity": "250000",
"local_balance": "248190",
"remote_balance": "0"
},
"confirmation_height": 0,
"commit_fee": "1810",
"commit_weight": "600",
"fee_per_kw": "2500"
}
],
"pending_closing_channels": [
{
"channel": {
"remote_node_pub": "0237fefbe8626bf888de0cad8c73630e32746a22a2c4faa91c1d9877a3826e1174",
"channel_point": "07b7f24ecf0b6e7a2bbd213df6bbc42aab4adaca691406026609e6e5d8a4027e:1",
"capacity": "200000",
"local_balance": "199819",
"remote_balance": "0"
},
"closing_txid": "8df6849fa44065c35aefb14bcc1f6e3056a90e2eb1b5dd53c03bb1115ea64691"
},
{
"channel": {
"remote_node_pub": "024a2e265cd66066b78a788ae615acdc84b5b0dec9efac36d7ac87513015eaf6ed",
"channel_point": "3c6ff49e3932730e5f6906452b877b3e1fccdeda0f9899916f87d4e24d07339b:1",
"capacity": "170000",
"local_balance": "169684",
"remote_balance": "0"
},
"closing_txid": "6b0ee5443582fbb72afd0080f165bf68b62d05ee001aac64d38b2e66de38449e"
}
],
"pending_force_closing_channels": [
]
}

Restarting lnd dosent help. Lnd log after restart:
http://wklej.org/id/3404606/

Plus i think lnd stucked. It stayd for like 8-10min with nothing new in logs (look link above) untill i made payinvoice, then it start syncing with network. Maybe im wrong but normaly it starts syncing pretty fast after restart.

Roasbeef · 2018-04-09T19:43:54Z

2018-04-09 20:43:29.746 [DBG] LNWL: Could not resend transaction a9a334622b8fddcf7eb578545ee808f7aa8ed1d03734a0152bdca21c7a82695b: -25: Missing inputs

Are you sure you're running with a segwit enabled bitcoind node? This would look like bitcoind is unable to actually parse the transaction as segwit.

Roasbeef · 2018-04-09T19:46:08Z

If you connect to a btcd backend, is the channel detected as being closed?

Roasbeef · 2018-04-09T19:47:01Z

The TXIDs for this never appeared in any blockchain explorers. Maybe my full node never accepted it? There's no useful logs from bitcoind, however, from this point...

Does your bitcoind node show the transaction as being accepted?

What's the minrelay fee on your node?

Roasbeef · 2018-04-09T19:52:15Z

Looking at the logs, it looks like we paid a very low fee, which possibly bitcoind rejected or all of its peers did. But then the remote peer ends up re-publishing another version of the closing transaction with a slightly higher transaction fee, which eventually gets confirmed.

halfik · 2018-04-09T20:33:13Z

Strange thing happens after lnd restart and resync with network. Look at unconfirmed balance and pending channels balance: http://wklej.org/id/3404754/

Are you sure you're running with a segwit enabled bitcoind node? This would look like bitcoind is unable to actually parse the transaction as segwit.

How to check if? All opening and closing transactions as far were bench32 addresses.

Now after few more restarts of btcd and lnd after both syncs i have this:
http://wklej.org/id/3404871/

vegardengen · 2018-04-09T20:57:13Z

Yah, I believe bitcoind might not have relayed it....

Is there any way to fix this? And should I set a new minrelay fee in bitcoind to prevent it happening again?

Roasbeef · 2018-04-09T23:53:58Z

Is there any way to fix this? And should I set a new minrelay fee in bitcoind to prevent it happening again?

It seems that bitcoind rejected the closing transaction as the fees were very low (which is just the case on mainnet). I think the default settings on bitcoind were set at a time in the past where fees were much higher, you can try lowering it even further in order for it to accept your transaction.

vegardengen · 2018-04-10T06:40:49Z

These are the relevant transactions:

Sent at 13:28:50. Not shown in my full node:
bitcoin@thundroid:~/.bitcoin$ bitcoin-cli gettxout "16e65ccb372f3ed7d430a75723a60c555193cf20eaca4b659028ce0a02d08faa" 0

Sent 23:17:14
bitcoin@thundroid:~/.bitcoin$ bitcoin-cli gettxout "83aa451a5cdbe82542abc3007d5231f61b6c1d102f12b58f64cb7514332799f9" 0
{
"bestblock": "00000000000000000006911151d954fc9821628005e8c722b18bbbe3ae981741",
"confirmations": 284,
"value": 0.01999638,
"scriptPubKey": {
"asm": "0 09485f780c8235e13bfbc2710e1c7cac4cfc2333",
"hex": "001409485f780c8235e13bfbc2710e1c7cac4cfc2333",
"reqSigs": 1,
"type": "witness_v0_keyhash",
"addresses": [
"bc1qp9y977qvsg67zwlmcfcsu8ru43x0cgen5z6gve"
]
},
"coinbase": false
}

Roasbeef · 2018-04-11T02:21:59Z

Have you tried lowering your mintxfee parameter on bitcoind?

vegardengen · 2018-04-11T05:25:06Z

Yes - after the fact, of course.

I can't see any of the transactions I was missing, but that is "fine", because I guess the funds are all in
2018-04-08 23:17:14.535 [INF] NTFN: New confirmations subscription: txid=83aa451a5cdbe82542abc3007d5231f61b6c1d102f12b58f64cb7514332799f9, numconfs=1
and deposited at address bc1qp9y977qvsg67zwlmcfcsu8ru43x0cgen5z6gve...

Problem is, my wallet knows nothing about this address :)

Roasbeef · 2018-04-11T06:25:46Z

Focusing on the OP now...

Looking at the logs, that channel was fully resolved:

2018-04-08 23:17:14.535 [INF] CNCT: Waiting for txid=83aa451a5cdbe82542abc3007d5231f61b6c1d102f12b58f64cb7514332799f9 to close ChannelPoint(5536825a875a56d8152ad9da32f7d58f3e989e29cc629c90f02dc126ae866b55:1) on chain
2018-04-08 23:17:14.535 [INF] NTFN: New confirmations subscription: txid=83aa451a5cdbe82542abc3007d5231f61b6c1d102f12b58f64cb7514332799f9, numconfs=1
2018-04-08 23:17:14.619 [INF] NTFN: Dispatching 1 conf notification for 83aa451a5cdbe82542abc3007d5231f61b6c1d102f12b58f64cb7514332799f9
2018-04-08 23:17:14.619 [INF] CNCT: ChannelPoint(5536825a875a56d8152ad9da32f7d58f3e989e29cc629c90f02dc126ae866b55:1) is fully closed, at height: 517219
2018-04-08 23:17:14.619 [INF] CNCT: Marking ChannelPoint(5536825a875a56d8152ad9da32f7d58f3e989e29cc629c90f02dc126ae866b55:1) fully resolved

Upon startup, if you set LNWL=trace, then which transactions are retransmitted?

…op variable In this commit, we fix a long standing bug where at times a co-op channel closure wouldn't be properly marked as fully closed in the database. The culprit was a re-occurring code flaw we've seen many times in the codebase: a closure variable that closes over a loop iterator variable. Before this instance, I assumed that this could only pop up when goroutines bind to the loop iterator within a closure. However, this instance is the exact same issue, but within a regular closure that has _delayed_ execution. As the closure doesn't execute until long after the loop has finished executing, it may still be holding onto the _last_ item the loop iterator variable was assigned to. The fix for this issue is very simple: re-assign the channel point before creating the closure. Without this fix, we would go to call db.MarkChanFullyClosed on a channel that may not have yet actually be in the pending close state, causing all executions to fail. Fixes lightningnetwork#1054. Fixes lightningnetwork#1056. Fixes lightningnetwork#1075.

vegardengen · 2018-04-18T19:39:18Z

lnd.log

meshcollider added the channel closing Related to the closing of channels cooperatively and uncooperatively label Apr 9, 2018

Roasbeef mentioned this issue Apr 13, 2018

contractcourt: fix co-op chan close issue by not closing over loop iterator variable #1088

Merged

Roasbeef closed this as completed in #1088 Apr 13, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Forever pending wallet balance after channel closing. #1056

Forever pending wallet balance after channel closing. #1056

vegardengen commented Apr 9, 2018

halfik commented Apr 9, 2018 •

edited

Loading

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

halfik commented Apr 9, 2018 •

edited

Loading

Uh oh!

vegardengen commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

vegardengen commented Apr 10, 2018

Uh oh!

Roasbeef commented Apr 11, 2018

Uh oh!

vegardengen commented Apr 11, 2018

Uh oh!

Roasbeef commented Apr 11, 2018

Uh oh!

vegardengen commented Apr 18, 2018

Uh oh!

Forever pending wallet balance after channel closing. #1056

Forever pending wallet balance after channel closing. #1056

Comments

vegardengen commented Apr 9, 2018

Background

Your environment

Steps to reproduce

Expected behaviour

Actual behaviour

halfik commented Apr 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

halfik commented Apr 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vegardengen commented Apr 9, 2018

Uh oh!

Roasbeef commented Apr 9, 2018

Uh oh!

vegardengen commented Apr 10, 2018

Uh oh!

Roasbeef commented Apr 11, 2018

Uh oh!

vegardengen commented Apr 11, 2018

Uh oh!

Roasbeef commented Apr 11, 2018

Uh oh!

vegardengen commented Apr 18, 2018

Uh oh!

halfik commented Apr 9, 2018 •

edited

Loading

halfik commented Apr 9, 2018 •

edited

Loading