Announcing LND 0.18 beta: Smarter Sweeps, Inbound Fees, and Improved Scaling! 🧹 📥📈
We are excited to announce the release of LND 0.18 beta! This version is brimming with improvements spanning multiple functional areas. Some of the release highlights include:
- Improved UTXO Sweeping process, to ensure timely confirmations in highly variable on-chain fee environments. This is achieved by incorporating a deadline-aware sweeping methodology which provides users with increased control over the urgency of transactions and the fees they are willing to pay.
- SQL Schema store for invoices, which allows users setting up new nodes to choose a relational database like SQLite or Postgres for storing invoice data. This enables better scaling and performant node operations with the growth of invoice data on the node.
- Inbound fees, a much requested feature for advanced routing node operations, will provide an ability to apply discounts to route through specific channels of a node.
- Route Blinding, a new protocol for improving payment receiver privacy, is now a supported feature on LND, providing the ability to send and forward payments to blinded routes.
- Fee estimation for payments, now available as a feature via the
EstimateRouteFee
RPC, will provide reliable fee estimates for invoices.
We are grateful for the ongoing support from external contributors on all fronts from development to review to testing. A total of 55 developers contributed to this release. For the complete list of changes included in 0.18, take a look at the full release notes.
Sweeper Overhaul 🧹
In the last year, we’ve seen a very volatile fee environment on the Bitcoin network. We’ve seen fees rise rapidly, impacting on-chain operations like co-op and force channel closures for Lightning nodes. It’s critically important for on-chain actions to function efficiently in a wide variety of fee environments, from sustained high fees to high variability.
The sweeper is an LND sub-system that handles funds sent from the dispute resolution contracts (e.g. a force closed channel) to LND’s internal wallet. It ensures timely settlement of the transactions generated from the events like channel closes and channel force-closes. Prior to 0.18, the LND sweeper process was indiscriminate and applied the same sweeping criteria for any pending output regardless of the deadline constraint (e.g. a pending to_local
output from a channel closure is not as time-sensitive to sweep as the output from a pending HTLC on the closed channel). Additionally, there was no flexibility on the fee expense applied to sweep these outputs which can make sweeping difficult when the on-chain fee rate rises.
This release significantly improves this process of sweeping the UTXO. The new process is more granular, considering both the urgency of transactions and budget constraints. This ensures timely confirmations, even when the network fees are high. Additionally, users now have control over defining the urgency of sweeps and the budget for on-chain expenses.
Users can specify a fee budget using the parameter budget
with the bumpfee
RPC. This budget sets the ceiling on the amount which can be spent as fee for sweeping.
Users can also get their transactions swept immediately by setting the immediate
flag with the bumpfee
RPC. The sweeper handles the creation, publishing and fee bumping of the sweeping transactions automatically. Every time a new block comes in, unless the sweeping transaction is confirmed, an RBF(Replace-by-Fee) is attempted. To override this automated process, users can use the bumpfee
RPC to specify a customized fee rate, budget, deadline and whether the sweep should happen immediately. This RPC will also be useful when users want to perform a Child-Pays-For-Parent (CPFP) type of fee bumping for pending transactions with a low fee rate.
Also, upon startup LND will now perform an immediate sweep of the time-sensitive outputs. This will ensure that any pending urgent transactions are processed promptly after a software update or restart.The detailed changes for sweeper functionality and various configuration options can be found in our Sweeper user guide.
SQL Schema Store for Invoices 🧾
With growing Lightning adoption and usage, the data storage per node is increasing. Along with this growth comes the need for data analysis and reporting, which requires the ability to execute complex queries on the node’s database.
The current database default for LND is bbolt, which is an embedded key-value pair database (kvdb). Such databases are designed with simplicity in mind, focusing on key/value storage, where each data entry consists of a unique identifier (the key) and an associated piece of data (the value). This design choice inherently limits its queryability, making it less suitable for applications requiring complex queries and data relationships. The solution to this problem requires relational databases like SQLite, Postgres etc. which can scale well with the data growth.
We have embarked on an initiative to make SQLite the default database for LND, as well as making the option of Postgres DB available for users. The execution of this initiative requires converting the major data stores of LND like invoice, payments, and forwarding history from kvdb to relational DB, which is SQL native. The conversion includes defining data tables and indices for the data stores, as well as rewriting all data services into SQL queries. This release includes the first major step in that direction, with the Invoice DB.
With this release, users setting up new nodes now have the ability to choose a relational database like SQLite or Postgres for storing the Invoice data. For SQLite, users can set the optional parameters db.backend=sqlite
and db.use-native-sql=true
under the [db]
section of the config file to utilize the Invoice db.
Gradually, over subsequent releases, more data stores like payments and forwarding history will also be enabled with this option. A migration option will also be provided to move the kvdb data to SQL native databases, for existing nodes. These changes will go a long way in improving LND’s ability to scale effectively along with the data growth on nodes.
Inbound Fees 📥
Inbound channel fees can allow node operators to more efficiently signal where liquidity is scarce. In 0.18, when setting an inbound fee, node operators can provide a discount on outbound fees, but they cannot go below zero.This can improve capital allocation in the Lightning Network and allow channels to be utilized more efficiently.
Until now, fee policies could only be set for outgoing channels, however forwarding events also involve an incoming channel. Setting an inbound fee on the incoming channel acts as a discount for senders, or a negative fee, which reduces the cost of forwarding on the outgoing channel. The application of this discount is capped at zero for a forwarding event (i.e. the sum total of the discount on an incoming channel and the fee on the outgoing channel cannot become negative). This implies that inbound fees are only active if an outbound fee is encountered during a forwarding event. Keeping the overall fees positive avoids losses for the node operator and prevents routing nodes from attracting infinite traffic. In the first stage of this roll-out, by default, inbound fees cannot be set to positive values, so node operators leveraging the feature will only be offering discounts. This is to ensure backwards compatibility, as senders don't need to upgrade in order to be compatible.
The adoption of inbound fees can be tracked via normal gossip messages, where extension fields were added. Inbound fees therefore provide a means for node operators to express their preferences more freely. For example, node operators use inbound fees to encourage backward-balancing flows for channels which experience consistent outbound payment flow. In turn, this will signal to pathfinding that inbound liquidity is available over the discounted channels, leading to better routing performance. You can read more about inbound fees in our Inbound fee user guide.
Route Blinding 🛤️
Route Blinding is a feature which improves the privacy of payment receivers on Lightning Network. The concept involves enhancing the privacy of the payment path by generating a “blinded” route for the sender, whereby pubkeys in the route are changed. This technique allows the receiver to provide a route to the sender without revealing their public key or the exact payment path. The sender knows only the nodes in the payment path up to an ‘introduction point’.
With this release, we now have support to send and forward payments to blinded routes. Mission control, the LND subsystem responsible for tracking payment attempts and informing routing decisions, has also been enhanced to support blinded routes and manage associated pathfinding errors.
Route Blinding is an active area of development on LND, with additional features like ability to receive payments on blinded routes expected in future releases.
Fee Estimation for Payments 🩺
With this release we are providing a feature to reliably estimate fees for potential payments. The EstimateRouteFee
RPC has been updated and now optionally accepts an invoice and a timeout parameter which returns a flag indicating if the destination is reachable or not and the estimated fee for the route. The RPC can also be used to estimate fee with a pubkey and an amount parameter to get the same result.
Scale with LND 📈
As Bitcoin enters its fourth halving epoch and Lightning adoption scales, LND is prepared to deliver reliable performance. This update delivers significant enhancements to core functionality, improving performance under high transaction fees and increasing overall scalability. Advanced routing management tools address the demands of a growing network, while privacy features are substantially strengthened. This release also resolves a large number of bugs, ensuring greater stability and reliability.
Further improvements to architecture, data management, and contract resolution are expected in the future releases. We remain dedicated to advancing the Lightning Network with innovative new features and enhancements.
We'd like to thank the entire community for their feedback on and contributions to this release, and we look forward to hearing your thoughts. You can find us on the LND Slack, reach out to us on Twitter, contribute a PR, or subscribe to our newsletter.