Add Multi-RFQ Send #1613

GeorgeTsagk · 2025-06-23T12:22:42Z

Description

Allows payments to use multiple quotes across multiple peers in order to pay an invoice. Previously we had to define a specific peer to negotiate a quote with, with this PR this is no longer required (but still supported) as Tapd will automatically scan our peers and establish quotes with all valid ones for the asset/amount of this payment.

The signature of ProduceHtlcExtraData had to be changed, as it's not possible to distinguish which of the quotes in the rfqmsg.Htlc should be used. We now provide the pubkey of the peer this HTLC is being sent to, in order to help Tapd extract the corresponding quote and calculate the correct amount of asset units.

Closes #1358

Depends on: lightningnetwork/lnd#9980

guggero

Looks pretty good! Have a couple of questions and suggestions, nothing major though.

rfqmsg/records.go

guggero · 2025-06-25T07:57:23Z

rfqmsg/records.go

+	if err != nil {
+		return err
+	}
+	list := make([]ID, num)


We allocate memory from a number we received over the wire. Need to check and error out the length before to make sure we limit the number of bytes we allocate.

Good point, so far the total length of available RFQ IDs is open ended.

We should introduce a static limit (which would also limit the max number of quotes we acquire) and enforce it everywhere.

guggero · 2025-06-25T08:15:58Z

tapchannel/aux_traffic_shaper.go

@@ -456,16 +457,43 @@ func (s *AuxTrafficShaper) ProduceHtlcExtraData(totalAmount lnwire.MilliSatoshi,
 		return totalAmount, htlcCustomRecords, nil
 	}

-	if htlc.RfqID.ValOpt().IsNone() {
-		return 0, nil, fmt.Errorf("no RFQ ID present in HTLC blob")
+	if htlc.AvailableRfqIDs.IsNone() {


Same here re backward compatibility. And also just to make sure we don't miss any edge case. After all, we explicitly set the singular htlc.RfqID in this function, so are we a 100% sure there is no code path where either the ProduceHtlcExtraData or PaymentBandwidth would be called with htlc.RfqID set?
IMO we should check both and only error out if none of them (or both) are set at the same time, with a comment that we'd only really expect the list to be set in the future.

For the PaymentBandwidth and ProduceHtlcExtraData the htlc.RfqID is replaced by htlc.AvailableRfqIDs

The only use of htlc.RfqID is when ProduceHtlcExtraData locks into a specific RFQ, which will be used to produce the actual HTLC that will be sent out to our peer

From that point forward everything is the same (we only use the htlc.RfqID to route HTLCs / accept HTLCs to invoices)

Same here re using an existing quote in the SendPayment RPC... Don't we have that covered by an itest?

Yeap, we have it covered, and the itest was failing! so +1 for our coverage, but I realize I never noticed the itest failing

So the reason it failed is because we were setting the existing RPC user provided RFQ as the single RfqID, instead it had to change to instead set it as the only available RFQ ID in the new field

rpcserver.go

coveralls · 2025-06-26T11:34:00Z

Pull Request Test Coverage Report for Build 16071985844

Details

70 of 313 (22.36%) changed or added relevant lines in 6 files are covered.
10 unchanged lines in 4 files lost coverage.
Overall coverage increased (+17.1%) to 56.386%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
server.go	0	4	0.0%
fn/send.go	0	6	0.0%
rfqmsg/records.go	64	81	79.01%
tapchannel/aux_traffic_shaper.go	0	77	0.0%
rpcserver.go	3	142	2.11%

Files with Coverage Reduction	New Missed Lines	%
rpcserver.go	1	61.44%
asset/group_key.go	2	72.15%
commitment/tap.go	2	84.77%
tapchannel/aux_traffic_shaper.go	5	6.4%

Totals
Change from base Build 16059643780:	17.1%
Covered Lines:	57092
Relevant Lines:	101252

💛 - Coveralls

rfqmsg/records.go

ZZiigguurraatt · 2025-06-27T16:12:43Z

Please update

taproot-assets/taprpc/tapchannelrpc/tapchannel.proto

Lines 125 to 130 in df8ce52

    
           // The node identity public key of the peer to ask for a quote for sending 
        
           // out the assets and converting them to satoshis. This must be specified if 
        
           // there are multiple channels with the given asset ID. 
        
           bytes peer_pubkey = 3;

to reflect this new capability. Also, I think we need to be more specific to include group_key in addition to asset_id in there.

ZZiigguurraatt · 2025-06-27T16:15:25Z

What is this expected to do with an invoice with no amount specified?

ZZiigguurraatt · 2025-06-28T02:18:54Z

rfq_id is of type bytes

taproot-assets/taprpc/tapchannelrpc/tapchannel.proto

Lines 139 to 144 in df8ce52

    
           // The rfq id to use for this payment. If the user sets this value then the 
        
           // payment will immediately be dispatched, skipping the rfq negotiation 
        
           // phase, and using the following rfq id instead. 
        
           bytes rfq_id = 5;

but with multi-rfq send, I would expect the ability to provide an array. Not sure that you want to fix that in this PR, but maybe it should be a separate issue? I did not make this comment for multi-rfq receive because of #1442 .

guggero

Did another pass, getting very close here too.

guggero · 2025-07-01T11:31:09Z

tapchannel/aux_traffic_shaper.go

-			"outgoing channel")
+	// If the HTLC doesn't have any available RFQ IDs, it's incomplete, and
+	// we cannot determine the bandwidth.
+	if htlc.AvailableRfqIDs.IsNone() {


So you're 100% certain for all the code paths that lead into paymentBandwidth there isn't a single one where potentially we only have the old RFQ ID set? What about this for example:

taproot-assets/rpcserver.go

Line 7674 in 2e7ae0b

nil, fn.Some(quote.ID), fn.None[[]rfqmsg.ID](),

guggero · 2025-07-01T11:35:51Z

tapchannel/aux_traffic_shaper.go

-	default:
-		return 0, fmt.Errorf("no accepted quote found for RFQ ID "+
-			"%x (SCID %d)", rfqID[:], rfqID.Scid())
+		if bandwidth > 0 {


I think "Given we establish 1 quote per peer, only one of the RFQ IDs in the array should produce some bandwidth." would probably be a good comment to add above this statement.

tapchannel/aux_traffic_shaper.go

guggero · 2025-07-01T11:41:23Z

tapchannel/aux_traffic_shaper.go

@@ -456,16 +457,43 @@ func (s *AuxTrafficShaper) ProduceHtlcExtraData(totalAmount lnwire.MilliSatoshi,
 		return totalAmount, htlcCustomRecords, nil
 	}

-	if htlc.RfqID.ValOpt().IsNone() {
-		return 0, nil, fmt.Errorf("no RFQ ID present in HTLC blob")
+	if htlc.AvailableRfqIDs.IsNone() {


Same here re using an existing quote in the SendPayment RPC... Don't we have that covered by an itest?

We add a new field to rfqmsg.Htlc which expresses the available quotes that may be used to send out this HTLC. This is done to allow LND to store an array of RFQ IDs to use later via the AuxTrafficShaper interface in order to query asset bandwidth and produce the correct asset related records for outgoing HTLCs.

This commit performs a small refactor to the paymentBandwidth helper. Since we now have multiple candidate RFQ IDs, we extract the main logic of calculating the bandwidth into a helper, and call it once for each of the available RFQ IDs.

When LND queries the PaymentBandwidth there's no way to signal back which RFQ ID ended up being used for that bandwidth calculation. We rely on the assumption that one quote is established per peer within the scope of a payment. This way, the AuxTrafficShaper methods spot the quote that it needs to use by matching the peer of the quote with the peer that LND is going to send this HTLC to.

ffranr

Added comments around robustness within the context of multi peer RFQ queries.

rpcserver.go

ffranr · 2025-07-03T11:25:35Z

rpcserver.go

+		// For each peer that satisfies our query above, we'll try and
+		// establish an RFQ quote for the asset specifier and max amount
+		// of this payment.
+		for peer := range chanMap {
+			quote, err := r.acquireSellOrder(
+				ctx, &rpcSpecifier, paymentMaxAmt, expiry,
+				&peer,
+			)


I think the code would be better if this for-loop happened in a separate function.

I think acquireSellOrder is blocking, am I right? If so then the first peer could block until expiry (which i think is the invoice expiry) and then we don't have a chance to query any other peer.

We might need to call r.acquireSellOrder for each peer synchronously here. And the quote response timeout should be much less than the invoice expiry.

Changed the quote acquisition to be async, also used a shorter timeout for the RFQ negotiation

rpcserver.go

ZZiigguurraatt · 2025-07-03T13:59:41Z

Getting a vague error with litcli. Not sure why. Has anyone else tested mutli-RFQ send with litcli or only in itest?

24165ac30ed6:/$ litcli ln payinvoice --pay_req lnbcrt4440n1p5xdzkcpp5rk3tglvgl3nc8krwxv28qx4h6cvys33ta4ayhmzjcw97hs8hyj7qdqqcqzzsxqyz5vqsp5968vj7g9ngxv639jk4p9lu9mexcwlqp7r992xaj2645n0x8ay4ys9qxpqysgq46z9q5tgnq0t0e50vzsxwf7aq6cmwrajqyy0f79fe2s6jl3r40m3kxdcw8cd2q56eu6hvjdfkt8amyg0xznncejkn8mq4gp97xdsgngq6k3tgy --asset_id 12ef620e67294ac18f218e7bec769299d3eb33a3915aa3122fb1baafc14d0e53
Payment hash: 1da2b47d88fc6783d86e3314701ab7d61848462bed7a4bec52c38bebc0f724bc
Description: 
Amount (in satoshis): 444
Fee limit (in satoshis): 444
Destination: 02f57d125bb94141add3a82b9209858989000e70849005e9e031fdd6dbb388d6dd
Confirm payment (yes/no): yes
Got quote for 45065 asset units at 9 msat/unit from peer 02d63eec2b2dd001241e40a93d2b90c693f2efd8f011015accc0f73da0b22ad51e with SCID 17596861635263353475
[litcli] unexpected nil result
24165ac30ed6:/$

The RPC method now uses all of the introduced features. Instead of acquiring just one quote we now extract that logic into a helper and call it once for each valid peer. We then encode the array of available RFQ IDs into the first hop records and hand it over to LND.

GeorgeTsagk self-assigned this Jun 23, 2025

This was referenced Jun 23, 2025

(temp)build: bump lnd & tapd lightninglabs/loop#968

Open

Multi rfq send itest lightninglabs/lightning-terminal#1097

Open

levmi added enhancement New feature or request payment-channel rfq tap-channels P1 labels Jun 23, 2025

levmi added this to Taproot-Assets Project Board Jun 23, 2025

github-project-automation bot moved this to 🆕 New in Taproot-Assets Project Board Jun 23, 2025

levmi moved this from 🆕 New to 🏗 In progress in Taproot-Assets Project Board Jun 23, 2025

levmi requested review from guggero and ffranr June 23, 2025 15:36

guggero reviewed Jun 25, 2025

View reviewed changes

GeorgeTsagk force-pushed the multi-rfq-send branch from 2bb2369 to df8ce52 Compare June 26, 2025 11:10

GeorgeTsagk requested a review from guggero June 26, 2025 14:30

ffranr reviewed Jun 26, 2025

View reviewed changes

rfqmsg/records.go Show resolved Hide resolved

rfqmsg/records.go Outdated Show resolved Hide resolved

rfqmsg/records.go Show resolved Hide resolved

ZZiigguurraatt mentioned this pull request Jun 28, 2025

[feature]: SendPayment: allow settingmax_shard_size_asset #1625

Open

GeorgeTsagk force-pushed the multi-rfq-send branch from df8ce52 to aa55b42 Compare June 30, 2025 09:47

GeorgeTsagk requested a review from ffranr June 30, 2025 11:15

guggero reviewed Jul 1, 2025

View reviewed changes

ZZiigguurraatt mentioned this pull request Jul 1, 2025

[feature]: AddInvoice: automatically skip dead end wallet nodes from RFQ process #1576

Open

GeorgeTsagk added 3 commits July 2, 2025 12:26

GeorgeTsagk force-pushed the multi-rfq-send branch from aa55b42 to 019b54a Compare July 2, 2025 10:31

GeorgeTsagk requested a review from guggero July 2, 2025 10:55

ffranr reviewed Jul 3, 2025

View reviewed changes

GeorgeTsagk added 4 commits July 4, 2025 12:39

fn: add SendOrDone

662e60f

taprpc: update SendPayment docs

69e3d2f

taprpc: deprecate EncodeCustomRecords rpc

d90193a

GeorgeTsagk force-pushed the multi-rfq-send branch from 019b54a to d90193a Compare July 4, 2025 10:44

GeorgeTsagk requested a review from ffranr July 4, 2025 12:58

Add Multi-RFQ Send #1613

Are you sure you want to change the base?

Add Multi-RFQ Send #1613

Conversation

GeorgeTsagk commented Jun 23, 2025

Description

Uh oh!

guggero left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coveralls commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 16071985844

Details

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ZZiigguurraatt commented Jun 27, 2025

Uh oh!

ZZiigguurraatt commented Jun 27, 2025

Uh oh!

ZZiigguurraatt commented Jun 28, 2025

Uh oh!

guggero left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ffranr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ZZiigguurraatt commented Jul 3, 2025

Uh oh!

Uh oh!

coveralls commented Jun 26, 2025 •

edited

Loading