Announcing lnd 0.12-beta: Improving the LND Developer Experience

Justin O'Brien
January 28, 2021

We’re pleased to announce lnd 0.12-beta, our first release of 2021! One of our major goals this year is to make it easier for developers to build on Lightning, and to that end, we recently shipped new lnd developer docs. We’re also committed to making the Lightning Network more reliable, robust, and secure, and our latest release does just that: lnd v0.12 includes anchor channels for improved reliability, faster startup times for improved robustness, additional support for Partially Signed Bitcoin Transactions (PSBTs) for improved security, and a new default Autopilot heuristic for improved resiliency of the network.

This release wouldn’t have been possible without the tremendous support of the community. In this post, we’ve included a list of the highlights below, and you can read the full details in the release notes.

Introducing Anchor Channels

With the recent rise in the price of bitcoin, we may see a future of increasing or volatile on-chain fees. Anchor output-based channels take away the up-front guesswork of determining what the proper on-chain fees will be, as they allow a node to dynamically increase the fee of a pending commitment transaction using Child Pays for Parent (CPFP). The latest version of lnd now allows users to opt into using the latest version of anchor channels between peers that support it. Anchor channels, which have been available to advanced users since lnd v0.10, are a safer and more reliable channel type as they allow for fee bumping the commitment transaction in the event a channel is force closed (e.g. because the other party has become unresponsive).

Each time the balance is updated within a payment channel, a new commitment transaction is signed by both parties. This transaction is only broadcast if one party decides to close the channel unilaterally. However, in cases where a channel may have been open for a long period of time, fee estimation initially may result in the underpayment or overpayment of transaction fees.

The introduction of anchor channels not only increases reliability and introduces fee savings but also helps increase channel capacities by reducing the required commitment fee reserve. Note that in order to take advantage of this new, safer and potentially lower cost type of Lighting channel, new channels will need to be opened.

In the near future, we hope to deploy a new extension to the Lightning protocol spec that allows channels to be dynamically updated on the fly without any additional on-chain costs. Such a decoupled, desynchronized update mechanism for the Lightning Network will allow lnd to automatically update users to the latest channel format.

Safer Defaults for Payments & New Channels

The Lightning Network is highly extensible with support for rolling out features in both a forwards and backwards compatible manner. As new features are introduced, nodes determine whether or not to support a feature by specifying whether the feature is required or optional. By flipping a previously optional feature to required, we can safely upgrade the functionality and security of the entire network. This new version of lnd changes payment addresses and static remote key channels features from optional to required.

Payment addresses were introduced with the Multi-Path Payments (MPP) protocol feature. In the context of MPP, they allow a receiver to correlate disparate payment shards as part of a single payment. They also improve end-to-end security of payments by having the sender include a random secret that’s only known to the receiver in the final payload.

Channels that use a static key for the commitment output of the remote party make disaster recovery of channels easier; in the event a channel is closed by a remote party, the funds will go directly to the wallet of the other party.

The immediate implication of these changes is that lnd may refuse to open the legacy channel type with older nodes and reject payments that don’t include the end-to-end payment secret, as both are less secure. As the leading implementation, lnd is committed to ensuring our users adopt safe, modernized defaults to promote network safety.

Improved Operational Security & Reliability

In this new version of lnd, we’ve made strides towards making lnd more reliable by improving the robustness of our initial solution for replicated data storage for lnd: etcd. The new etcd database backend now passes all existing integration and unit tests that our default database, boltdb, does. With this new milestone under our belt, we aim to make the new backend system more production ready, targeting lnd 0.13 as its ready-for-production debut.

In addition, we’ve extended the Channel Acceptor API to allow users to be more selective about the set of channels they accept, as well as bind cooperative closure addresses to pre-configured cold addresses in order to improve operational security.

To improve the security of our credentials-based authorization system, we’ve added the ability to revoke or rotate existing macaroons generated by the macaroon bakery. If a node operator detects that any of its custom generated macaroons have been compromised, they can now automatically refresh credentials with a single command.

Finally, we’ve made lnd more aware of its critical dependencies (disk access, chain access, etc.) by adding a series of new “health checks” that will continually run to ensure that lnd is able to carry out its duties safely. If any of these health checks fail, then lnd will begin a graceful shutdown, either to be manually intervened upon by a user, or automatically restarted by a hypervisor if the location/name of one of its resources has changed.

Improved Loading Times

We’ve been working hard to make it easier than ever for developers to get up and running with lnd. A big step in this direction has been to minimize the resource requirements of lnd. This not only helps ensure that lnd can be run on resource constrained devices (such as Raspberry Pis) but also reduces the cost of running an lnd node making the Lightning Network not only more accessible, but also more resilient.

Speeding up Graph Sync & Node Startup Times

In an effort to make onboarding onto the network as frictionless as possible we’ve made improvements to graph sync resulting in a 3x speed increase based on initial benchmarks. This was accomplished by batching all insertion operations related to the channel graph. These improvements may be especially apparent on mobile and other constrained platforms.

As the network matures, users of lnd are pushing the limits of the daemon, uncovering areas that are ripe for optimization to keep up with increased user and payment growth. Users of lnd that run larger nodes with hundreds of channels may have noticed that over time, their node has been taking longer to start up. With this latest version of lnd, we’ve optimized the startup time for larger nodes by batching more of the sanity and consistency checks run on channels at startup to ensure lnd is able to properly manage its set of pending and active channels. Those running larger nodes should see a significant reduction in disk and CPU utilization when standing up their nodes.

Minimizing Resource Consumption

To help minimize resource consumption we’ve added the ability to opt-in to automatic database compaction for the channel database. A bloated channel database not only takes up a significant amount of disk space, but also slows startup time. With automatic database compaction, we not only free up disk space, but also defragment and validate the integrity of the data each time compaction occurs.

Networking bandwidth is a scarce resource to be preserved when participating in decentralized peer-to-peer networks, such as the Lighting Network. As the network has grown over the past few years, the network bandwidth burden for well connected nodes has increased disproportionately. In this new version of lnd, we’ve implemented a series of optimizations to throttle the rate of gossip activity flowing into and out of lnd, resulting in dramatically lower CPU, disk, and network utilization for nodes across the network. These optimizations will serve to greatly reduce the idle system utilization of a given lnd node, particularly on more resource constrained devices such as Raspberry Pis.

Towards a More Resilient Network

We’re continually looking for ways to improve the resiliency of the Lightning Network. Autopilot, which automates the process of onboarding onto the network, previously used a preferential attachment mechanism as the default heuristic for connecting to the network.

In the latest release of lnd we’ve upgraded the default heuristic for Autopilot, now optimizing for the betweenness centrality of the node. This means lnd will attempt to connect to the nodes that frequently appear in the shortest paths within the Lightning Network instead of connecting to nodes with the most channels.

New Generalized PSBT Capabilities Towards Cold Key lnd

In prior versions of lnd, we added support for Partially Signed Bitcoin Transactions (PSBTs) to make it possible to fund channels directly from an air-gapped hardware wallet or other form of segregated signer. In this version, we’ve further augmented lnd’s PSBT capabilities to allow users to craft and sign/finalize arbitrary PSBTs funded by the internal lnd wallet. Combined with lnd’s set of custom concurrent safe coin selection APIs, users can now use lnd as the basis of more advanced multi-party signing protocols that utilize PSBTs.

In the near future, we aim to further build upon this new base by extending lnd’s internal wallet (btcwallet) with the capabilities to import arbitrary public keys and extended public keys (xpubs). Once finalized, this will allow lnd to serve as a watch-only wallet, and also take on more of the roles in a PSBT-based workflow by being able to fully populate a PSBT instance, export to an external system for signing, and finally finalize the PSBT by producing a valid, fully signed transaction. Once complete, these capabilities will allow lnd to function in a “cold key” manner, serving to minimize, and potentially eliminate, the amount of funds that are explicitly hot within an active instance. As we begin to onboard more exchanges and existing Bitcoin companies seeking to execute larger payment flows within the network, we’re committed to ensuring that lnd continues to exceed the security requirements of our users.

Those interested in the details can take a look at this example of Partially Signed Bitcoin Transactions being used with lnd and bitcoind.

Fully Verifiable & Deterministic Automated Builds Featuring Docker

As recent events have shown the critical importance of the software supply chain, we’ve made it easier for any third-party to reproduce and validate our deterministic builds. With our new and improved build system, users on both Linux and MacOS are able to verify every single binary created by our build system for proper reproducibility. In addition to this, we’ve worked to remove direct reliance on roasbeef’s GPG signature in the build system. As even the compressed archives of our build system are reproducible, any developer that is active within the project can now upload their own independent signature, serving to build upon prior signatures and “bless” the sanctity of a new lnd release. In order to make this process easier, we’ve added new verification scripts that can be run by an end user with only Docker installed.

We are now also automatically building and pushing Docker images of all our releases to Docker Hub, thanks to the power of GitHub Workflows. All published releases will now automatically be published to our Docker Hub account as valid images. We’ve also augmented our uploaded images by allowing them to optionally verify the hash of the binary generated during the build process to match the “official” hash generated by the set of lnd developers. This serves as defense in depth to ensure that users building their own version of lnd produce the exact same binary as all other Lightning users and developers around the world.

Improved Developer Documentation

We recently published a new set of developer docs to make getting started with lnd easier than ever. These documents include tutorials for integrating Lightning, guides for running LiT, best practices, and more. Stay tuned for more resources coming soon.

Conclusion

We’ve introduced a number of improvements in 0.12 to make it easier than ever for developers to get up and running with lnd and we are committed to continuing this trend. Developments such as anchor channels are a big step towards a safer and more reliable network.

We anticipate that 2021 will be a momentous year for developers building on Lightning, so we're especially excited to be starting it off right with this release. If you haven’t already, please join the lnd Slack, reach out to us on Twitter, or subscribe to our newsletter!

About the authorJustin O'Brien

Justin holds an MSc in Analytical Finance from St. Andrews. Before joining Lightning Labs he worked as a product manager at Coinbase. Justin made his first foray into the industry with 21 Inc. in 2015. Prior to 21 he launched an internal course while at Google to teach software engineers the Bitcoin protocol.

Justin holds an MSc in Analytical Finance from St. Andrews. Before joining Lightning Labs he worked as a product manager at Coinbase. Justin made his first foray into the industry with 21 Inc. in 2015. Prior to 21 he launched an internal course while at Google to teach software engineers the Bitcoin protocol.