Skip to content

ChannelManager Payment Retries #1916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jan 26, 2023
Merged

ChannelManager Payment Retries #1916

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
948ebc6
Copy PaymentAttempts from invoice module to outbound_payment module
valentinewallace Dec 18, 2022
cbb75e2
Copy Retry from invoice module to outbound_payment module
valentinewallace Dec 18, 2022
ae34533
Parameterize add_new_pending_payment with retry strategy and route pa…
valentinewallace Dec 18, 2022
c0a22f7
Store retry data in PendingOutboundPayment::Retryable
valentinewallace Dec 18, 2022
6351a99
Decode onion fail outside of outbound_payments lock
valentinewallace Jan 24, 2023
686ef08
Generate PendingHTLCsForwardable upon retryable payment
valentinewallace Jan 6, 2023
72a7da8
Remove AllPathsFailed outbounds at send_payment_internal callsites in…
valentinewallace Dec 19, 2022
d776dee
Retry HTLCs in process_pending_htlc_forwards
valentinewallace Jan 7, 2023
acf9292
Support sending payments with a retry strategy in ChannelManager
valentinewallace Dec 19, 2022
2f49c81
Test ChannelManager automatic retries
valentinewallace Dec 19, 2022
ad486a4
Payment retries: copy tests from InvoicePayer
valentinewallace Jan 4, 2023
6d81979
Disambiguate send_payment_internal from pay_internal
valentinewallace Jan 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 25 additions & 4 deletions lightning/src/ln/channelmanager.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,15 @@ use crate::ln::features::{ChannelFeatures, ChannelTypeFeatures, InitFeatures, No
#[cfg(any(feature = "_test_utils", test))]
use crate::ln::features::InvoiceFeatures;
use crate::routing::gossip::NetworkGraph;
use crate::routing::router::{DefaultRouter, InFlightHtlcs, PaymentParameters, Route, RouteHop, RoutePath, Router};
use crate::routing::router::{DefaultRouter, InFlightHtlcs, PaymentParameters, Route, RouteHop, RouteParameters, RoutePath, Router};
use crate::routing::scoring::ProbabilisticScorer;
use crate::ln::msgs;
use crate::ln::onion_utils;
use crate::ln::onion_utils::HTLCFailReason;
use crate::ln::msgs::{ChannelMessageHandler, DecodeError, LightningError, MAX_VALUE_MSAT};
#[cfg(test)]
use crate::ln::outbound_payment;
use crate::ln::outbound_payment::{OutboundPayments, PendingOutboundPayment};
use crate::ln::outbound_payment::{OutboundPayments, PaymentAttempts, PendingOutboundPayment, Retry};
use crate::ln::wire::Encode;
use crate::chain::keysinterface::{EntropySource, KeysManager, NodeSigner, Recipient, Sign, SignerProvider};
use crate::util::config::{UserConfig, ChannelConfig};
Expand Down Expand Up @@ -388,7 +388,7 @@ impl MsgHandleErrInternal {
/// Event::PendingHTLCsForwardable for the API guidelines indicating how long should be waited).
/// This provides some limited amount of privacy. Ideally this would range from somewhere like one
/// second to 30 seconds, but people expect lightning to be, you know, kinda fast, sadly.
const MIN_HTLC_RELAY_HOLDING_CELL_MILLIS: u64 = 100;
pub(super) const MIN_HTLC_RELAY_HOLDING_CELL_MILLIS: u64 = 100;

/// For events which result in both a RevokeAndACK and a CommitmentUpdate, by default they should
/// be sent in the order they appear in the return value, however sometimes the order needs to be
Expand Down Expand Up @@ -2446,6 +2446,18 @@ where
self.send_payment_along_path(path, payment_params, payment_hash, payment_secret, total_value, cur_height, payment_id, keysend_preimage, session_priv))
}

/// Similar to [`ChannelManager::send_payment`], but will automatically find a route based on
/// `route_params` and retry failed payment paths based on `retry_strategy`.
pub fn send_payment_with_retry(&self, payment_hash: PaymentHash, payment_secret: &Option<PaymentSecret>, payment_id: PaymentId, route_params: RouteParameters, retry_strategy: Retry) -> Result<(), PaymentSendFailure> {
let best_block_height = self.best_block.read().unwrap().height();
self.pending_outbound_payments
.send_payment(payment_hash, payment_secret, payment_id, retry_strategy, route_params,
&self.router, self.list_usable_channels(), self.compute_inflight_htlcs(),
&self.entropy_source, &self.node_signer, best_block_height,
|path, payment_params, payment_hash, payment_secret, total_value, cur_height, payment_id, keysend_preimage, session_priv|
self.send_payment_along_path(path, payment_params, payment_hash, payment_secret, total_value, cur_height, payment_id, keysend_preimage, session_priv))
}

#[cfg(test)]
fn test_send_payment_internal(&self, route: &Route, payment_hash: PaymentHash, payment_secret: &Option<PaymentSecret>, keysend_preimage: Option<PaymentPreimage>, payment_id: PaymentId, recv_value_msat: Option<u64>, onion_session_privs: Vec<[u8; 32]>) -> Result<(), PaymentSendFailure> {
let best_block_height = self.best_block.read().unwrap().height();
Expand All @@ -2457,7 +2469,7 @@ where
#[cfg(test)]
pub(crate) fn test_add_new_pending_payment(&self, payment_hash: PaymentHash, payment_secret: Option<PaymentSecret>, payment_id: PaymentId, route: &Route) -> Result<Vec<[u8; 32]>, PaymentSendFailure> {
let best_block_height = self.best_block.read().unwrap().height();
self.pending_outbound_payments.test_add_new_pending_payment(payment_hash, payment_secret, payment_id, route, &self.entropy_source, best_block_height)
self.pending_outbound_payments.test_add_new_pending_payment(payment_hash, payment_secret, payment_id, route, Retry::Attempts(0), &self.entropy_source, best_block_height)
}


Expand Down Expand Up @@ -3250,6 +3262,12 @@ where
}
}

let best_block_height = self.best_block.read().unwrap().height();
self.pending_outbound_payments.check_retry_payments(&self.router, || self.list_usable_channels(),
|| self.compute_inflight_htlcs(), &self.entropy_source, &self.node_signer, best_block_height, &self.logger,
|path, payment_params, payment_hash, payment_secret, total_value, cur_height, payment_id, keysend_preimage, session_priv|
self.send_payment_along_path(path, payment_params, payment_hash, payment_secret, total_value, cur_height, payment_id, keysend_preimage, session_priv));

for (htlc_source, payment_hash, failure_reason, destination) in failed_forwards.drain(..) {
self.fail_htlc_backwards_internal(&htlc_source, &payment_hash, &failure_reason, destination);
}
Expand Down Expand Up @@ -7278,6 +7296,9 @@ where
hash_map::Entry::Vacant(entry) => {
let path_fee = path.get_path_fees();
entry.insert(PendingOutboundPayment::Retryable {
retry_strategy: Retry::Attempts(0),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have some reasonable default? Retry::Attempts(0) would mean no retries. In what situations would we hit this case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We hit this while deserializing ChannelManager if a monitor has some htlcs that the ChannelManager forgot about due to being behind the monitor on shutdown, IIRC. Idea is that we never want to retry payments after restart, because the payments are probably stale anyway

Copy link
Contributor

@jkczyz jkczyz Jan 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we chose not to retry after restart in InvoicePayer because we didn't want to have something else for the user to serialize. But now that the retry logic is in ChannelManager, that rational is no longer applicable. We can still use the no-retry behavior for older serializations, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's a consideration for follow-up, but it does seem like payments would be stale after restart? And even if they weren't, it seems like the sanest default would be to not retry 🤔 I think the invoice payer differed a bit since it has a static retry strategy config, which we don't have here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's a consideration for follow-up, but it does seem like payments would be stale after restart? And even if they weren't, it seems like the sanest default would be to not retry 🤔

Restart might just be failing over to a new server, FWIW. Or maybe for mobile switching back and forth between apps?

I think we should at least decided what we'd like to happen to avoid unnecessary churn in the serialization format.

I think the invoice payer differed a bit since it has a static retry strategy config, which we don't have here

It was mostly because we didn't want to serialize it. If we serialized, the static strategy would have been serialized along with it, I'd imagine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, not gonna retry on startup due to some race conditions with abandon_payment (I'm forgetting the details..)

attempts: PaymentAttempts::new(),
route_params: None,
session_privs: [session_priv_bytes].iter().map(|a| *a).collect(),
payment_hash: htlc.payment_hash,
payment_secret,
Expand Down
Loading