Litecoin's latest upgrade is a masterpiece in the making. Over the past several years, the lead litecoin developer on MWEB has provided monthly updates on the Litecoin MWEB upgrade and progression. The dedication to such a monumental undertaking shouldn't be taken lightly. This was a grassroots funded project from the litecoin community alongside matching funding from Charlie Lee the creator of Litecoin and Litecoin Foundation Director. We are approaching the final progress update as the Litecoin MWEB update is ready for activation.
The litecoin community and people of the world will be able to use best money on the planet after the upgrade.
Below are the monthly updates from David Burkett, the lead Litecoin MWEB developer. If you want to catch up on the technical development and inter-working of MWEB start reading:
December 2019
Announcement
Hi everyone!
I’m David Burkett, the developer of Grin++. I’ve been working with ecurrencyhodler, Charlie, and others for several months now to design a Mimblewimble extension block so that LTC users can have confidential transactions and more fungible money.
I spoke a bit about our progress at the LTC Conference last month, but for those who are unaware, we’ve published 2 new LTC improvement proposals (LIPs):
LIP-0002: lips/lip-0002.mediawiki at master · litecoin-project/lips · GitHub 178 This describes extension blocks, and how they can be implemented and activated in LTC.
LIP-0003: lips/lip-0003.mediawiki at master · litecoin-project/lips · GitHub 150 This is all about Mimblewimble on LTC.
After the conference, much has happened. The first big announcement is that the LTC Foundation has graciously decided to fund my efforts to implement the MW extension block, and to continue working on Grin++! More details about that will be announced shortly, but I want to thank you all for the opportunity.
Since that was announced, great progress has been made. On the Grin++ side, I’ve implemented payments via TOR. Mimblewimble requires interacting with the receiver to build a transaction collaboratively, so by using TOR for that interaction, we’re able to use familiar-looking addresses, a simpler set-up, and much better privacy.
Original proposal to use TOR: https://github.com/mimblewimble/grin-rfcs/pull/24 34
Grin++ Implementation: Transacting via TOR hidden services by DavidBurkett · Pull Request #89 · GrinPlusPlus/GrinPlusPlus · GitHub 34
Also, I’ve performed the first ever (AFAICT) pre-broadcast Mimblewimble CoinJoin. For those familiar with Mimblewimble, you may know that at the time transactions are included in a block, they are already joined with all other transactions in that block. The Achilles heel of mimblewimble privacy though, has always been that transactions are broadcast before they’ve had a chance to be joined with other transactions. That means nodes monitoring the network can see the original input-to-output links of most transactions. Sending a transaction directly to a CoinJoin server before broadcasting is one of many different techniques we can use to combat that.
Original coinjoin proposal: Yo Dawg, I heard you like CoinJoins - Grin 9
Block with first coinjoined transactions: GrinScan - Block 456,188 16
On the LTC side, I started by familiarizing myself with the existing code by completing the LTC Dev Training Session. This meant starting with the latest bitcoin source, and making all modifications necessary to get it to sync.
Dev Training Session: Litecoin Developer Training Session 1 91
My BTC fork which now fully syncs the LTC blockchain: GitHub - DavidBurkett/bitcoin: Bitcoin Core integration/staging tree 39
After that, I got to work re-designing the Grin++ codebase in a way that will allow LTC to reuse Grin++ with minimal modifications. The design is beyond the scope of this update, but I’ll formally write-up the design sometime over the next few weeks and link it here.
Going forward, I will be providing monthly progress updates to keep everyone in the loop on what progress has been made. They’ll likely be much more succinct than this one, but I thought it would be good to take a few extra minutes with this first update to let everyone know who I am, where we’re at, and set some expectations for the future.
Thanks for reading, and thanks again for all of your support!
December 2019
December Progress:
After months of planning, development of the mimblewimble extension block has officially started! My efforts this month focused mostly around restructuring the core logic that will be shared between Grin++ and LTC. This involves all of the logging, serialization, crypto, error handling, and common data structures (headers/blocks/txs).
Most of this was already written for Grin++, but since some parts were hurriedly coded on a tight time schedule, I’m using this opportunity to go over every line with a fine-tooth comb. As part of this, I’m standardizing patterns that were originally followed inconsistently, backfilling missing tests, and adding better documentation.
The LTC priorities for January will be:
Determine build method - Since the code will now be spread across a few repos, a slightly more complex build process is necessary. I’m still figuring out the best way to handle this, but am currently leaning toward a simplified version of libbitcoin’s build system (https://github.com/libbitcoin/libbitcoin-build 26)
Define all LTC models - Determine the exact fields and serialization format of the headers & kernels (including signatures)
Move the database implementations over to libmw-core, cleanup to match new standards, and add full tests.
I also will be working on a faster sync mechanism for Grin next month, which LTC will directly benefit from.
For those technical-minded readers who want to follow along as changes are made, feel free to “watch” the following repos:
Contribute to GrinPlusPlus/libmw-core development by creating an account on GitHub.
Contribute to GrinPlusPlus/libmw-ltc development by creating an account on GitHub.
January 2020
January Progress:
The biggest news from January is that I’ve found a way to support non-interactive transactions in Mimblewimble 42! The biggest difficulty with using MW is the need for sender and receiver to communicate, which requires receiver to be online when sending. My proposal, along with an updated version of the write-up that will be released soon, eliminates that need. This removes a major UX hurdle, limits long-term maintenance requirements, and supports receiving via cold-storage, making hardware wallets easier to support.
On the development side, the build process has been determined for libmw, and local builds are working for libmw-ltc (checkout libmw-core and libmw-ltc to the same parent directory, and you should be able to build libmw-ltc). I’ll setup CI/CD in the next month or so, but the important part is multi-repo local builds are working.
I’ve also built out a robust Database framework with transactional capabilities to support atomic updates across multiple tables, and have implemented the block database querying and updates, which is coin-agnostic, and has been partially tested using LTC-specific header and block models.
The security audit results are back from Grin++, so I’ve applied all fixes to Grin++ and libmw, and will be awaiting final review from the auditors. The audit turned out to be a humbling lesson in just how complicated C++ really is. I’ve learned enormously as part of the process, and the Grin++ & libmw codebases are significantly better as a result. A huge thanks again to the contributors in the Grin, Beam, and LTC communities who made that audit possible.
On the Grin++ side, we’ve had a successful planned hardfork, our pre-hardfork syncing issues have been resolved, and Grin++ 0.7.5 is available now which has turned out to be the most stable release yet. For the first time since Grin++'s creation, the support channels are finally quiet, which means more time to focus on actual development. As long as things remain quiet, and I don’t have to focus on putting out fires, I expect an even faster development pace over the coming months.
The priority for February will be to implement the consensus rules for the LTC EB, including all validation and a full suite of tests. This is the most important part of the code, so it will be time consuming to make sure all of the details are correct, and the code has full test coverage. Once that’s complete, I will work on the API for the extension block, so we can start integrating libmw into the existing LTC codebase.
I will also focus on getting the new one-sided tx proposals thoroughly reviewed, and if no major security concerns are found, I will create an LIP for community feedback.
February 2020
Happy Leap Day!
February Progress:
The non-interactive transaction proposal has gone through a few design iterations this month. I believe we’ve finally got a design that resolves the issues found with the first write-up, and I’ve turned that into a LIP (https://github.com/litecoin-project/lips/pull/13 59). I suspect it will still take some time before the LIP has had sufficient review to be considered safe enough to accept, but I’ll continue to keep one-sided transactions in mind as I develop.
Aside from the LIP, I’ve made good progress on the validation rules, but there’s still a lot more to do. I’ve made some modifications to the original kernel design to support the ability to soft-fork in new features in the future. I’ve also started to build out the merkle mountain ranges (MMRs), which are a data structure we use to commit to kernels & outputs. Once the MMR logic is finished, I should be able to get back to the block validation logic. I didn’t get to work on the Node API like I had hoped, but that’s something I also expect to get to next month.
I’ve so far been very hesitant to give exact dates on when things should be finished, because writing blockchain software is difficult, time-consuming, and unpredictable at times. I didn’t want artificial deadlines to force us to rush through parts of the code and introduce defects or security vulnerabilities. Having said that, I think it’s finally time to commit to the first major event…
End of Summer: MW Testnet Launch - This will include all block & tx validation rules, basic p2p messaging, transaction pool, syncing, and the ability to mine blocks. This will NOT include a usable GUI wallet for casual users to test it out. Transactions will likely need to be created manually at first, or via a cli or automated tool.
March 2020
Hi everyone! It’s that time again.
March Progress:
I’ve finished up the Merkle Mountain Range (MMR) logic, and wrote some good tests around that. When originally writing Grin++, this was one of the most difficult pieces to code, and ended up being one of the messiest parts of the logic. It consistently led to many frustrations and headaches for myself. Fortunately, I learned from those mistakes, and was able to design a much cleaner implementation this time.
That was the final piece I needed in place to write the validation rules. The first pass at header and block validation is now complete, but is so far only minimally tested.
The next few months will be all about integrating with the existing litecoin codebase, and lots and lots of testing. Don’t be alarmed if the updates for April and May are shorter and less detailed, as this will be one of the most tedious and least exciting parts of the project. It’s also the most crucial part though, so lots of time will be dedicated to making sure everything is integrated correctly.
Thanks again for your continued support. Stay safe, stay home, and don’t forget to wash your hands. Happy Quarantine!
April 2020
April Progress:
I’ve built out a functional testing framework that builds valid headers, blocks, and transactions. I’ve now got some (mostly) complete end-to-end block validation tests.
I also began integrating with the litecoin codebase, starting with the ConnectBlock logic (See: https://en.bitcoin.it/wiki/Bitcoin_Core_0.11_(ch_6):_The_Blockchain#Connecting_a_block 21). This is the part where blocks are validated to ensure valid UTXOs are being spent, no double spends, etc, and is also responsible for actually adding the blocks to the chain.
I haven’t created the repo for this portion yet, but will try to get that out on github sometime in the next few weeks, once I’ve had the chance to clean the code up a bit.
There’s a few different directions I could go once I finish testing the ConnectBlock logic, but I’m still deciding what makes sense to tackle next. All I can give at this time is a high level plan for May, which involves continued integration with the LTC codebase, and lots more testing.
It’s been a few months since I gave an update on the Grin++ side, so for those interested how that’s going, there’s been some big news recently. David Tavarez (https://github.com/davidtavarez 17) has completely taken over the UI development, so I can focus solely on the C++ node & wallet logic. He actually rewrote the entire UI (https://github.com/GrinPlusPlus/GrinPlusPlusUI/tree/1.0 23) and it’s much nicer than it was before. Things are so stable now that we actually have a release candidate for v1.0.0, what we’re calling the first non-beta version of Grin++. That release has been tested with the help of several Grin community members, and will be pushed to all Grin++ users this weekend.
Happy Friday Everyone!
May 2020
May Progress:
I’ve finished implementing the AcceptBlock and ConnectBlock integration logic, meaning that when a mimblewimble block is received, it will be fully validated, added to the chain, and the UTXO set will be updated. This ended up requiring a lot more rework than I hoped, since the approach I originally took on the mimblewimble side for adding, validating, and removing blocks didn’t line up well with the existing code’s approach to handling the UTXO set using “views” and “caches” (See “Data Access Layer and Caching” here: https://en.bitcoin.it/wiki/Bitcoin_Core_0.11_(ch_2): 18Data_Storage#The_UTXO_set.28chainstate_leveldb.29).
The logical next step was to work on DisconnectBlock, which will remove a block from the chain in the case of chain reorgs. However, I chose instead to start working on the mempool, with the hopes that we can soon start mining Mimblewimble blocks. Being able to mine valid mw blocks will allow us to test the end-to-end flow of including mw transactions in a block, validating those blocks, removing the transactions from the mempool, adding the blocks to the chain, updating the UTXO set, and flushing everything to disk. There’s a lot that could go wrong there, so the sooner we can get everything working together, the smoother the rest of development should go.
This is all subject to change, but for now, here’s the plan between now and the end of Summer testnet launch:
June: Wrap up mempool logic, and start mining valid chains with mw extension blocks, making sure all pieces work well together.
July: Initial block download. The way initial block download is handled in mimblewimble is quite a bit different than how litecoin does it. In LTC, every single block from the genesis block is downloaded and validated in order. With mimblewimble, we don’t need the whole history, but instead only parts of each transaction, and all of the unspent outputs. I’ll need to find a way to make those two very different syncing approaches work smoothly together.
August: DisconnectBlock logic (for chain reorgs).
September: Testnet activation logic and launch of the first mimblewimble testnet. Then start fixing everything I broke
June 2020
June Progress:
The existing LTC mempool logic ended up being quite a bit more complex than I predicted, so we’ll need to revisit this area after the testnet launches. For now though, a very minimal implementation supporting MW transactions has been written.
In addition to the mempool changes, code was written to support mining extension blocks, although there are a few edge cases left to handle, and much more testing is needed.
I’ve had concerns about the way we were storing mimblewimble block data in a separate database. It was originally designed this way to be a clean separation from the existing code, in order to facilitate merging future bitcoin commits. Having separate databases is generally a bad idea though, because we lose the ability to make atomic updates, which means the 2 databases could become out of sync. This can lead to a whole host of problems, potentially even ones that are exploitable by remote attackers.
Because of these concerns, I decided to spend some time modifying the code for serializing and deserializing MW blocks & transactions to disk. I was able to take advantage of the groundwork that was laid down as part of the segwit enhancement, to cleanly support serializing the additional data without making major changes to the existing block storage format. As a result, upgraded nodes can successfully save extension block data to disk the same place they’ve always saved blocks, without having to introduce an additional database. A side effect of these changes is that it was relatively straightforward to add support for sharing mimblewimble transactions over the p2p network, which is the first step toward July’s goal of handling MW data as part of the Initial Block Download.
The high-level plan for the rest of the summer remains the same:
July: Initial Block Download
August: Chain reorg logic
September: Activation logic and testnet launch
July 2020
July Progress:
The focus this month was on the Initial Block Download. Before I can detail the progress made, I need to give some background info for those not intimately familiar with mimblewimble.
The biggest innovation behind mimblewimble is that, in order to verify the chain, you just need to know all of the unspent coins/outputs, and a small part of each transaction called the “Kernel.” These 2 things together are called the “chain state.”
In bitcoin/litecoin, each block header uses a merkle tree to commit to only the transactions in that block. Since we don’t want to require everyone to download all old mimblewimble blocks, or to know about all old, spent outputs, we use a different structure to commit to the transactions. Each mimblewimble header commits to the root of 2 different Merkle Mountain Ranges(MMRs). One represents all historical kernels up to that block, and the other represents all historical outputs/coins. Merkle Mountain Ranges are a different sort of tree that supports “pruning”, which means we can verify the root of the structure without knowing all of its members (called leaves). For an in-depth look at how this works, I recommend reading https://github.com/mimblewimble/grin/blob/master/doc/mmr.md 18
Since old blocks don’t need to be downloaded, Initial State Download (ISD) is a more accurate term for the initial sync process that I worked on this month. To facilitate the ISD I ended up defining the following P2P messages:
GETMWSTATE - This simply consists of the hash of a header sometime in the recent past. We’ll call this the “horizon” header. A new node, after syncing most of the canonical (non-mimblewimble) blocks in the chain, sends this message to a single peer, requesting the mimblewimble chain state at the time of that block.
MWSTATE - The response to a GETMWSTATE message. This contains the entire chain state, which is everything you need to validate the extension chain up to the point of the horizon. The chain state consists of:
kernels - All of the kernels in the chain, in order.
utxos - All of the outputs that had not yet been spent at the time of the horizon, including their associated rangeproofs, in order.
unspent_bitmap - A compact structure that indicates the MMR leaf indices of the provided utxos.
parent_hashes - Some additional metadata that allows you to verify the MMR root without knowing all of the past, spent outputs.
Quick sidenote - This design relies on one peer providing quite a bit of data at once. After launch, it would be ideal to parallelize the sync process using something like https://github.com/jaspervdm/grin-rfcs/blob/pibd/text/0000-parallel_ibd.md 3 or https://github.com/BeamMW/beam/wiki/UTXO-set,-horizons-and-cut-through 5
All of the logic for serializing and deserializing those messages, as well as choosing when to send them has been implemented. I also implemented the logic for verifying the kernel signatures, and the kernel MMR roots.
Unfortunately, I was short on time this month due to a Grin hardfork that ended up requiring an enormous amount of effort to support in Grin++. As a result, I was unable to finish up the logic for validating and processing MWSTATE messages - in particular, the output roots are not yet verified, and I still need to validate that the kernels match up with the pegins and pegouts for each block. Grin++ is now in maintenance mode for the remainder of the year, so this should be the last time it interferes with litecoin progress.
Updated Plan:
The plan was to spend August focusing on reorg logic, but I’m going to delay that until September. Instead, I’m just going to focus on wrapping up the MWSTATE message processing, and focus on cleaning up the code, adding more tests, and maybe even some documentation if I’m feeling really wild. When developing projects this large, things rarely go exactly as planned, so the design changes over time. Having a chance like this to go back through the code and clean things up, re-evaluate decisions made, etc. will really pay off in terms of code quality, hopefully allowing for a smooth first testnet launch at the end of September.
August 2020
I apologize in advance for the much-briefer-than-usual update. I’ve been busy with tying up loose ends, which are important & necessary, but not very exciting to write about.
August Progress:
As planned, I was able to wrap up the Initial State Download, in particular the validation of the MWSTATE messages. There are still a number of ways where malicious peers can use these new messages to DoS nodes. I’m not yet guarding against all of those attacks, but there will be plenty of time to focus on that after the testnet is released.
I also took advantage of the opportunity to go back and clean up some code & strengthen some of the more fragile logic, particularly around management of the UTXO set (the collection of unspent coins).
Remaining work:
Everything is on track for a testnet release at the end of this month. I’m still figuring out a few final details, but I’ll share more information later this month about what the testnet will look like, and how everyone can get involved. I’m hoping to have a basic CLI or JSON-RPC option for advanced users to experiment with while I transition to focusing on adding non-interactive transaction support.
Sidenote:
For those interested, I recently gave a talk about future directions in Mimblewimble which highlighted the weaknesses of vanilla Mimblewimble, and what we can do/are doing to try to solve those. Here’s a recording of the talk: https://www.youtube.com/watch?v=x5vJzp1ehaI&feature=youtu.be 56
September 2020
September Progress:
Testnet is here!!!
So far, there are only a few nodes connected and mining, and there haven’t been enough blocks to activate mimblewimble yet (we’re using BIP9-style activation[1]). As more peers connect and start to mine, I’m hoping we can get the chain moving fast enough to activate in a day or so, but time will tell.
I’m still roughing in very minimal cli wallet support, but hopefully we’ll have a simple way to create mimblewimble transactions by the time it activates. For technical users who aren’t afraid of CLIs, and are willing to build the code themselves, I encourage you to join in on the fun by going to https://t.me/MWEB_Testnet 16.
The new home for the code is at https://github.com/ltc-mweb 17. For those of you unaware, MWEB is now the new official acronym for “Mimblewimble Extension Block.” It’s so super official, that we’ve got an awesome logo and everything! Check it out: https://twitter.com/SatoshiLite/status/1311376460158955521 37.
Going Forward:
Now that the MWEB testnet is running, I’m going to focus on ways to make it easy for non-technical litecoin users to start testing it out as well. This means improving the automated builds, better documentation, and starting to build out wallet support.
Right now, there are a number of areas in the code that are fragile or lack the necessary validation around edge cases, so I’ll also be taking some time this month to harden the code, and start validating any remaining consensus rules we missed. Once I’m confident everything is working as designed, I’ll start looking for ways to break the testnet, to make sure we find and resolve any security or stability weaknesses.
Next month, I’ll share a detailed plan of all of the remaining work necessary to get MWEB merged to the main repo, so that miners and node operators can start signaling for activation sometime in 2021!
October 2020
October Progress:
The testnet has been running for a month now and has highlighted the areas where more work is needed. Several bugs were found as a result, which means the early testnet launch was highly valuable.
After sorting out the most critical bugs (thanks to the help of a couple of awesome contributors[1]), I switched to adding full wallet support. I was able to finish this up a few days ago[2], and I’m planning to roll those changes out to all the testers tomorrow evening.
I’ve also changed how MW blocks are built[3] to avoid a lot of downfalls in the existing approach.
Remaining Work:
November '20 - Wallet testing and start implementing non-interactive transactions. Most of this month will be about fixing testnet issues and trying to improve the one-sided txs proposal[4].
December '20 - Finish coding one-sided txs. Include MWEB wallet functionality into the GUI.
January '21 - Launch MWEB testnet 2 with one-sided tx support and include non-technical users. Open a pull request to merge in the code to the main repo!
As always, this is subject to change (blame it on my ADHD). But by the current looks of things, we should be done coding sometime in January.
P.S. I just want to say a huge thank you to the many people who have helped out with the testnet in various ways!
[1] https://github.com/ltc-mweb/libmw/graphs/contributors 25 [2] https://github.com/ltc-mweb/libmw/pull/20 12 [3] https://github.com/ltc-mweb/litecoin/pull/5 4 [4] https://github.com/litecoin-project/lips/pull/13 5
November 2020
Hi Everyone!
It has been a long journey, but we’re getting close to the finish line. 2021 is shaping up to be a great year for LTC
November Progress:
The testnet is running smoothly, with only a few bugs remaining to track down.
I’ve updated the one-sided txs proposal[1] to guard against Wagner’s attacks[2], use stealth addresses, include payment proofs, and guard against replay attacks. It’s still in desperate need of reviewers, but the current form of the proposal is likely to be in the final form (barring no new attacks are found).
After finishing the proposal, I got started on the implementation of one-sided transactions[3]. The new fields have all been added and existing tests fixed.
Up Next:
Now that the fields have all been added to the input and output structures, I will add the new validation logic for checking signatures and performing the new ownership check (See validation rules here 5).
I will then add in wallet support for the new one-sided transactions, add MWEB functionality in the GUI, and prepare for a second testnet that will also include non-technical users this time.
[1] https://github.com/DavidBurkett/lips/blob/master/lip-0004.mediawiki 25 [2] https://joinmarket.me/blog/blog/avoiding-wagnerian-tragedies/ 8 [3] https://github.com/ltc-mweb/libmw/tree/one_sided 11
December 2020
Hi All!
December Progress:
As some of you may have already seen, the first implementation of non-interactive txs 10 is finally ready for testing! I’ve also updated LIP-0004 8 to match the code.
Community member Hector Chu worked hard to get the MWEB functionality added to the GUI, and it turned out great! Thanks also to @ecurrencyhodler for helping with the design.
I’ve also started documenting all consensus rules 9 to make it easier for those testing, reviewing, or auditing the code.
Up Next:
I’ll need to take ~1 week to prep Grin++ for Grin’s final planned hardfork. Once I get the new version released, I’ll get builds of the new MWEB code ready and launch the new community-wide testnet so everybody, regardless of technical abilities, can test out the MWEB and provide feedback. Watch for more info about that sometime mid-January.
I still don’t have an exact date ready yet for when the code will be finished, but we’re getting very close. We have a lot of automated tests to backfill still, there’s still a few outstanding questions about max weight for the EBs & peg-in/peg-out maturity, and there’s lots of small cleanup tasks remaining. I’m still expecting to have the code finished sometime this quarter (Q1 2021) though, so it won’t be long.
Happy New Year Everyone!
January 2021
January Progress:
We were able to get Grin++ ready in time for the hardfork, which went smoothly. That was the final planned fork, so Grin++ is back in maintenance mode, and should no longer be a distraction for the remainder of this project
The bad news - I was not able to get the automated windows build ready, so I did not kick off the community-wide testnet this month like I hoped.
The good news - The delay was mostly a consequence of a new proposal by Tevador (lead developer of RandomX) called MingleJingle[1]. MingleJingle is another design for providing non-interactive txs in Mimblewimble, and by far the most detailed one yet. Tevador was able to use his extensive experience with Monero stealth addresses to make a number of improvements on our design (LIP-0004). The benefits of his design for sub-addresses[2] and output encoding[3] were immediately obvious, so I decided to switch gears for a bit and use what I learned from MingleJingle to improve MWEB addressing and output structure[4].
The big advantage of this new addressing scheme is that we no longer have to “grind” through different possible stealth addresses when restoring from a wallet seed. With my far-inferior design, the wallet would have to scan the MWEB utxo set and attempt to restore using the private keys once for each stealth address. So if you generated 10 different stealth addresses in the past, it would scan the UTXO set 10 times, which could take several hours after a decade of MWEB usage. While stealth addresses allow reuse without leaking information about your transactions, if you wanted to maintain several identities with a wallet (lame example: multiple pseudonyms on a site like litecointalk.io 8 that each advertise a donation address), then each identity would still need to generate its own stealth address, resulting in these long scan times. But with Monero’s subaddresses (and Tevador’s improved version), we only have to scan each UTXO once, regardless of how many stealth addresses you’ve generated in the past
Some more good news - After finishing the new addressing improvements, I took another pass at the UI and transaction history display and found a number of ways to make it easier to follow where funds are moving. One change I made was to add a new column for MWEB amounts in the transaction history, so peg-ins & peg-outs look a little more intuitive:
The code is now ready for the new testnet, which we can launch as soon as we’ve get a working windows build. If anyone has experience setting up automated builds of bitcoin or litecoin for distribution on windows (preferably using Github Actions, but any CI would be fine), I could really use your help! Come chat with me at Telegram: Contact @MWEB_Testnet 20
The great news - There are still a ton of ways we can (and will) continue to improve the MWEB code, in particular around syncing, but most of what remains is not required for initial launch. So I think I’m finally ready to commit to a code-complete* timeline…
I will be submitting the code for review on March 15th.
This will contain all of the consensus, P2P, and wallet code necessary for supporting MWEB. The only thing it will be missing is the activation code, which I will add after the reviews and audits are all finished.
* Code-complete means ready for developer reviews & formal audits. It will not be ready for activation until after all reviews & audits are finished.
[1] minglejingle.md · GitHub 22 [2] Subaddress - Monero Documentation 6 [3] minglejingle.md · GitHub 4 [4] New output encoding by DavidBurkett · Pull Request #44 · ltc-mweb/libmw · GitHub 9
Happy Groundhog Day everyone! Although COVID killed the in-person celebrations here in Punxsutawney, you can still watch this crucial event unfold live from the comfort of your home: https://www.groundhog.org/ 14
February 2021
I’m very much in crunch mode now, and haven’t slept in days, so please forgive my incoherent rambling
February Progress:
The expectation was that I would have a community testnet launched, but I have failed yet again Sorry about that!
I was all prepared to launch it, but wanted to “quickly” (HA!) add the logic for automatically pegging-in when sending from a LTC output to an MWEB address (and vise-versa) so users who weren’t real familiar with the intimate details of pegging-in & out could still send and receive MWEB transactions. But the more I dug into it, the more I realized our wallet design just didn’t mesh well with the existing LTC wallet at all.
It started with coin selection. If you’ll recall, LTC works by spending coins you’ve received before and creating new coins of equal or lesser value for another user’s address (and change back to yourself). The process for selecting which coins to spend is incredibly complex, since poor choice of coins can lead to paying excess fees or serious privacy leaks. In fact, it’s so complex that bitcoin developer Murch wrote his 67-page master’s thesis about that single topic[1].
Up to this point, we hadn’t needed to make many changes to the coin selection logic, since we were either sending LTC coins - in the case of a typical LTC transaction or when pegging-in (sending LTC coins to an MWEB address) - or we were sending MWEB coins - in the case of an MWEB-to-MWEB transaction or when pegging-out (sending MWEB coins to a LTC address). This was relatively straightforward to implement, since it meant simply applying Murch’s algorithm to the available LTC coins (for LTC-to-LTC or when pegging-in) or applying Murch’s algorithm to the available MWEB coins (for MWEB-to-MWEB or when pegging-out). There was no “mixed” mode where some LTC coins were sent with some MWEB coins.
That all changed when I wanted to add the ability to automatically peg-in/out without requiring the user’s knowledge of how that works. Now, we could get ourselves into all kinds of weird scenarios. If a user decides to send to an MWEB address, the wallet first checks if they have enough MWEB coins to just send them in a normal MWEB-to-MWEB transaction. If they don’t have enough, then we check if they have enough LTC coins so we can just create a peg-in transaction (LTC-to-MWEB). But if they have 5 LTC coins and 7 MWEB coins, and they want to send 10 coins to an MWEB address, then we have to create a mixed MWEB transaction which spends some from the LTC side and some from the MWEB side, creating a sort of partial peg-in tx.
And it gets even worse…
If we change the scenario to wanting to send 10 coins to a LTC address, but they only have 5 LTC coins and 7 MWEB coins, we actually can’t just peg-out (MWEB-to-LTC) 5 of those MWEB coins and combine them with the 5 LTC coins. Due to the structure of the transactions, we actually have to peg-in those 5 LTC coins, combine them with the 7 MWEB coins, and then peg-out 10 of those. Doing this all in one transaction and deciding the best coins to use - all while trying to minimize changes to the existing LTC codebase - has kept me up for weeks. So much for “quickly” adding that in
So if anyone is looking for a master’s thesis topic, consider going with An Evaluation of Coin Selection Strategies for Mimblewimble Extension Blocks
Moving on, I finally figured out the coin selection logic, and started doing some manual testing of the new code. Immediately, I started finding all kinds of weaknesses with the wallet code that simply weren’t a problem when we didn’t have mixed/partial MWEB transactions. So I had to rethink the entire design of the wallet (fortunately, the much-more-critical consensus code remains in great shape).
To make a long story short, I rewrote about 80% of the MWEB wallet code to support mixed-coin MWEB transactions. While I tossed around the idea of launching the testnet in parallel without this new functionality, the wallet rewrite more than burned through all of the buffer I had left in the timeline. The time I would’ve spent supporting the testnet certainly would’ve delayed the March 15th code-complete deadline.
But the good news is we’re (mostly) back on track again! I guess I’ll just kick the can a little bit further down the road, and shoot for launching a community testnet sometime after the code is completed this month. There’s still a ton of work left to do before the code review can be submitted, but I remain confident & committed to meeting the March 15th goal.
TL;DR:
Testnet deprioritized again (yikes!)
Coin selection kicked my ass
Still shooting for code-complete on March 15th
March 2021
March Progress:
The first code review[1] was submitted on March 15th covering the libmw project! This is where the bulk of the MWEB logic resides, and is completely independent from the existing LTC code in order to make it easier to take the latest fixes and enhancements from bitcoin.
MWEB is a massive enhancement, much larger than any previous upgrade to the LTC network, including SegWit. Not surprisingly, there is a ton of code to review. libmw alone is more than 15k lines of code, and that’s not including the “glue” that ties it into the existing LTC code. Reviewing all of this, and getting it professionally audited is going to take us well into the summer, so please be patient with us all as we painstakingly pore over each line to make sure it is safe and bug free.
The consequences of introducing a bug in consensus code can be quite severe, sometimes even catastrophic. Grin recently suffered the worst kind of attack due to a consensus bug[2]. One of the 2 node implementations used a cache improperly, which led to an invalid transaction being included in a block. This transaction massively inflated the supply for a short period of time. Luckily, Grin++ did not have the bug, so the issue was caught and fixed right away. Forks of Grin were not as lucky…
While I’m certain MWEB will not be vulnerable to any major attacks, the Grin experience was a bit unnerving for me personally. I want to make sure we’re doing everything we can to eliminate possible sources of bugs. The biggest such source is rushing through code, which is something I’ve done a lot of this past 2 months
As a result, I will no longer be committing to exact deadlines. I obviously value transparency a lot, and want to keep everyone as informed as possible, so I will continue to provide estimates (now, in an easy-to-follow timeline at https://wenmweb.com 128). But these will be no more than estimates, subject to change at any time. This project is too important to rush.
April Tasks:
This month, I will be working on cleaning up LIP-0002 and LIP-0003 based on feedback we received from one of our auditors. I will also be addressing comments on the libmw code review, and continuing to add more tests and documentation to make life easier for reviewers, auditors, and future maintainers. I’ll also be reaching out to more auditors, bitcoin developers, and mimblewimble experts to solicit their review of the design and code.
These next few updates will mostly just involve the same few things over and over: documentation, testing, review, and general housekeeping tasks. But I’ll take advantage of the briefer updates by describing the various perks of MWEB, future directions we can take it, cool projects we can build on top of it, and general reasons why bitcoiners should and will be jealous of Litecoin
[1] https://github.com/litecoin-project/litecoin/pull/703 32 [2] grin-security/CVE-2021-XXXX.md at 940c9447614237fc2ec6da9e1bb31a5eed7d2573 · mimblewimble/grin-security · GitHub 11
April 2021
April Progress:
Documentation
I’ve re-written LIP-0002 based on feedback received. The revised version is available for review here 19. The LIP-0003 re-write is still in progress.
I’ve also improved some of our code documentation, and will continue to do so over the next few months.
Coding & Testing
Hector Chu 17 has helped a lot these past 2 weeks with testing & coding. He tested the major flows and quickly identified & helped fix a number of bugs. He has also helped with adding functional tests to cover some of the untested & under-tested consensus rules.
I’ve been working through the litecoin side of the code, and doing my own pre-review of the code before submitting it for review. I was able to clean up a lot of the validation and mempool logic to harden it against attacks, as well as fix some bugs in the wallet code.
Reviews & Audits
Hector has finished his review of the libmw code[1]. I’ve addressed a number of the issues he found, and will continue to work through that list over the next few days.
Once the code for Litecoin’s v0.21 release has been reviewed, I’ll merge all of the MWEB code into that branch and submit the 2nd and 3rd code reviews (2nd=validation, mempool, & mining code; 3rd=wallet code).
I’ve also reached out to Quarkslab with the hope that they’ll have capacity to audit the design & code.
Instead of going over May’s tasks, which are going to be more of the same, I promised I’d discuss something fun about MWEB instead.
One thing that was proposed to Grin earlier this year by John Tromp was a “CoinSwap” service. If you’ve got a decent understanding of the math behind mimblewimble, then I recommend reading his proposal here 17.
This CoinSwap proposal aims to improve transaction unlinkability, much like a CoinJoin service does. But existing designs for MW CoinJoin services have a few important limitations. One is they currently require trusting the CJ server operator to not leak your transactions (there may be ways to eliminate that trust, but none have been found yet). Another is that using CoinJoin services have more annoying UX, since you must wait for the CJ server to aggregate enough transactions before the large CJ transaction is finally broadcasted and confirmed on chain.
CoinSwaps merely swap a single output for a different one of the same value (minus a very small fee). There’s no need to link any of your inputs together, risking revealing your identity to the CJ server operator. And you don’t have to trust any single server to preserve your privacy, since that trust can easily be spread over a large number of CoinSwap servers, with the requirement that only one of them needs to be honest in order to protect the privacy of all of the participants.
And this can all happen behind the scenes without any user interaction! When your wallet identifies new coins that were sent to you, it can automatically send those coins to the CoinSwap service to be trustlessly replaced with a different coin, so that once they’re mixed, even the person who sent you the coins won’t even be able to tell when or if you’ve spent them. The same can be done automatically for your change outputs when you send coins, or can just periodically be done by your wallet to ensure you’ve got clean coins.
The design appears to work as is, but I’m optimistic it can be improved even further to allow dynamic selection of coinswap operators. In its current form, the coinswap server list must be static. But if we allow the ability to easily add coinswap operators, you could always just add an additional one if there’s ever any fear that the existing servers have been compromised.
It’s worth keeping an eye on CoinSwaps. They help eliiminate Mimblewimble’s greatest weakeness (transaction linkability), and can easily be integrated into MWEB wallets, which means this is an excellent opportunity for someone to build a profitable business, while improving LTC’s fungibility at the same time.
May 2021
May Progress:
Coding & Testing
We’ve made the following significant improvements to the code over this past month:
Unified the build process to support existing release strategies. We originally intended on sharing the libmw codebase with Grin++, so the new code used a different technology (cmake) for managing builds than the existing litecoin code. That plan turned out to be unworkable due to significant differences in the Grin and MWEB protocols, so there was no longer a reason to continue using 2 different build management solutions. We’ve stopped using cmake entirely now, and just include the libmw code as part of the existing (automake) build. We’ve also downgraded from C++ 17 to C++ 14 to support gitian builds, which is how we generate verifiable releases.
Removed superfluous interfaces and boilerplate code. This also stemmed from the fact that we no longer need to share logic with Grin++. We had a rigid interface layer that prevented us from calling the existing litecoin code from within the new libmw library, which resulted in the need to duplicate serialization, logging, and other infrastructure code within libmw. Since litecoin is now the only consumer of libmw, we were able to eliminate this artificial separation, resulting in less code (and hopefully fewer bugs) overall.
Working automated builds for multiple platforms. The automated builds are working again for linux, and now also a windows build is being generated with each code commit. This has allowed for quicker feedback from changes, and opened the door for non-developers to help out with testing without needing to build the code themselves.
Reviews & Audits
We’re still struggling to get developers to review the libmw code. The code has changed quite a bit since the PR was first submitted though, so I will likely be closing it and creating a number of new, smaller, and hopefully more digestible PRs over the coming weeks.
I met with Quarkslab yesterday to discuss what we need audited, and I will continue to work with them over the next week or two while they gather what they need to do their initial assessment & estimate for the audit.
We remain on track for activation around the end of the year, barring no major surprises from the auditors
June 2021
June Progress:
Documentation
I’ve spent some time documenting all of the code changes to assist our auditors. For those interested, this is the current list of technical documents describing the MWEB changes (several are only partially complete):
LIP-0002 11 - This describes the process for adding extension blocks to LTC, including describing how pegging-in, pegging-out, and integrating transactions work
LIP-0003 14 - This is our original design for Mimblewimble extension blocks. It’s a bit outdated, but should provide a high level understanding of how it works.
LIP-0004 5 - This describes our approach to supporting one-sided txs, instead of relying on interactive transactions like traditional Mimblewimble
LIP-0005 2 - I’m still filling in missing data structures here, but this documents the P2P protocol changes and describes how MWEB transactions and blocks are serialized
Consensus Rules 5 - Not a comprehensive list, but describes the most important consensus rules
Kernels 2 - Describes how kernels are serialized, the various features that are supported (e.g. lock height), and how new features can be soft forked in later on.
Data storage 2 - Describes the new database tables and data files that were added
Stealth addresses 10 - How stealth addresses are generated, how we support subaddresses, how addresses are serialized, etc.
Pruned Sync 1 - Describes how pruned sync will be supported in future releases
Coding & Testing
I performed my own review of all of the node logic to look for ways to improve security and performance, which resulted in a number of improvements to the code & design:
Kernel MMR is now per block, instead of a perpetually growing MMR. It was determined that we don’t gain much by having a cumulative kernel MMR, so switching to a per-block MMR means less time spent hashing, and a lot less disk space needed to store the MMR.
Switched from sha256 to the much-faster blake3 for all MWEB hashes.
New stealth address format that’s more consistent with previous address types, along with having better error detection.
More compact serialization formats for all MWEB data structures which will result in less disk space usage, less data transferred between peers, and therefore slightly higher throughput.
Better test coverage
Audits and Reviews
I’ve officially handed off the final code changes and documentation to Quarkslab, so we should have a more detailed timeline from them any day now.
I’ve created a new code review 22 with the most critical changes to the litecoin consensus code. It’s much smaller and more focused than the libmw review from a few months ago, so hopefully we will get more reviewers and faster feedback from other devs.
I’ve made slight changes to the estimated timeline on wenmweb.com 94, but overall, we’re still working toward MWEB activation at the end of the year.
July 2021
I’ve shifted away from writing new code so the MWEB codebase can remain stable while the auditors perform their review. I’ve been focusing instead on rebasing our changes from the LTC 0.18 branch to the LTC 0.21 branch.
For those unfamiliar with git terminology, what this means is we started with the code from the LTC 0.18 release (the latest at the time), and have been coding MWEB on top of that. However, LTC 0.18 is no longer the most recent code, since the release for LTC 0.21 is nearly ready. So I’m merging the MWEB code we wrote onto the LTC 0.21 codebase so that we still have all of the latest features (like taproot) in our MWEB release. This is a manual process that can be very tedious and time consuming, since code changes that were made in LTC 0.21 could interfere with the changes we made to support MWEB. These conflicts must be carefully addressed individually, which has been taking up much of my time.
Fortunately, I’ve finally got all of the node changes merged and building successfully. I’m working on fixing some failing tests, and then I will start doing the same thing with the wallet code. Once that is all finished and reviewed, we should be able to make a pre-release to start testing how the code works on mainnet. In particular, we’ll focus on making sure it communicates fine with older nodes, validates and propagates transactions & blocks, does not prematurely enforce MWEB rules before activation, and is able to successfully mine valid blocks.
Reviews & Audits
Members of the team developing Tari 22 were kind enough to spend some time to give a high-level review of the design & code. They provided us with their findings here 26 and here 18. In addition to finding a few potential issues, they’ve also thoroughly documented their understanding of the changes, which can serve as a useful resource for others trying to learn about MWEB.
Also, we received the official audit proposal from Quarkslab, which we’ve accepted[*], so they will be officially starting their audit in just a few days. This is expected to wrap up by mid-October, when we will be provided a report of all findings. Once we address the issues found, we’ll be able to create an official release (release number TBD).
[*] Thorough auditors like Quarkslab can make all the difference in the success of a project. Unsurprisingly, this means they can be quite pricey. By “we’ve accepted”, what I really mean is Charlie has, since he generously chose to pay for this audit. We are very fortunate to have him around
I had to take some time off this month for personal reasons, so my update was a little lighter than we’ve been used to. Things will be back to normal next month. Thanks for your understanding.
August 2021
August Progress:
Coding
Last month I merged the node code on top of the LTC 0.21 release codebase (see July update), so this month began with me merging the wallet code. It turns out that a lot changed in the wallet between 0.18 and 0.21, so I had to spend a few weeks redesigning how we calculate and manage MWEB addresses and keys. The end result was even cleaner than we had with 0.18 though, so this turned out to be a good thing, despite requiring the extra effort
For the code monkeys among you, the majority of the changes were in: scriptpubkeyman.cpp 6, mweb_wallet.cpp 8, and Keychain.cpp 4.
Aside from that, I had a lot of bugs and tests to fix as a result of the merge, which I was able to do just in time for the auditors at Quarkslab to include in their review. On that note…
Reviews & Audits
I’m working very closely with one of the Quarkslab auditors now. She has been going over the code and is in the initial stages of testing. We were able to get her synced against mainnet, and she also requested I create a testnet for them, which they’re now connected to and experimenting on.
Releases
While we’re now merged on top of 0.21 release code, we actually haven’t even released 0.21 yet . I was asked to take over managing this release, so I spent the past week getting that release code ready. Assuming I can herd the other devs together (which is a lot like herding cats), then I expect we’ll be able to have 0.21 released by my next update. This will include, among many other things, support for Taproot .
Hopefully we can get 0.21 adopted quickly, and miners signaling for Taproot, because the following month, I’ll be cutting the release for MWEB (which I’m calling 0.22-ish until someone gives me an official release number). The release date for 0.22-ish will be highly dependent on the results of the audit, which won’t be completely finished until mid-October. So while we’ll remain optimistic for an October 31st release for now, write the date down very lightly in pencil, not pen
Notable Mentions
Ryan Wright is an absolute baller. While I was searching for a quiet street corner to live (see: background 57), he swooped in with a 100 LTC donation 23 just before I listed my house on the market
MWEB is now fully funded!
All additional donations collected will continue to fund me for future development projects. There’s still so much more I want to do: Dandelion for P2P-level privacy, CoinSwap support, syncing improvements, Atomic Swaps, watch-only addresses, MWEB support for hardware wallets & light clients, and much, much more. So still chip in 28 whenever you can
September 2021
September Progress:
v0.21 Release
I was hoping to have the v0.21 release out already, but I’m waiting on one last developer to review. A couple of us have already run through a test build to make sure our environments are setup correctly, so once everyone has signed off on the code, we should be able to get a release candidate built and signed fairly quickly.
The most notable change in v0.21 is the inclusion of Taproot support. The Taproot logic is the same as in bitcoin, but activation will be done differently. We chose to trial the mechanism we’re planning to use for MWEB activation, which is bip8 24 with lockinontimeout set.
Soft fork activation can be hard to follow at times, because there’s a few different ways it can be done (BIP8, BIP9, UASF, etc.), and activation takes place through a number of steps or “states”, that aren’t usually explained well for non-technical users. I want to make sure everyone can follow what’s happening, so we’ll walk through the process for activating taproot.
Taproot Activation
Each block has a version field, which miners can use to “vote” for soft forks. Miners will be using a small part of the version field to signal for Taproot activation.
Every 8,064 blocks, a new “window” is started. At the end of each window, nodes tally up all of the blocks that signal for a feature, and if the total meets the defined threshold, the feature “locks in” for activation in the following window. In our case, the threshold is defined as 6,048 blocks or 75% of the blocks in the window.
So here’s how this will look for Taproot:
Taproot will initially be in the DEFINED state, which just means it’s a known feature but can’t be voted on yet.
At block 2,153,088 (early- to mid-November), the feature will switch to a STARTED state, meaning upgraded miners can start signaling/voting for activation of taproot. After 8,064 blocks (the first window), nodes will add up the number of blocks that signaled for Taproot activation.
The process repeats until one of two conditions is met:
A window occurs with at least 6,048 (75%) of the blocks signaling for Taproot:
At the end of this window, Taproot switches to the LOCKED_IN state. It stays LOCKED_IN for the next full window (8,064 blocks), allowing everyone time to upgrade.
After being LOCKED_IN for one full window, Taproot switches to ACTIVE. Nodes begin enforcing Taproot consensus rules for all blocks
If the threshold is NOT met by block 2,362,752 (Nov 2022), we rely on the lockinontimeout option I mentioned earlier:
Taproot switches to LOCKED_IN, despite not meeting the threshold. Miners must start signaling for Taproot. Any block that doesn’t signal for Taproot will be ignored by the nodes on the network.
After being LOCKED_IN for one full window, Taproot switches to ACTIVE. Nodes begin enforcing Taproot consensus rules for all blocks
I hope this is easy enough to understand, but if any of it is unclear, the full BIP8 spec is available here 24.
Audit
I met with the Quarkslab auditors Wednesday for a mid-audit check-in. They’re wrapping up their static analysis of the code, and have found very few issues so far, which is very encouraging.
We also discussed priorities for the remainder of the audit, to make sure the most important pieces are thoroughly covered.
For the next few weeks, one auditor will be focusing on manually testing, trying to make sure it works as expected, and more importantly, trying to see if they can break it.
The other auditor is knowledgeable on cryptography, so will be focusing heavily on the one-sided tx design (LIP-0004) to make sure we didn’t miss any attacks that could compromise key integrity, lead to tx malleability (i.e. allowing someone to modify a transaction that they aren’t the creator of), or any other number of security issues.
I expect us to have the results of the audit in just a few weeks
October 2021
October Progress:
Audit
Quarkslab has finished their audit of the code!
I’ll be meeting with them Friday to discuss their findings. After that, they’ll work on releasing the audit report in a blog post, which I look forward to sharing with you all.
Findings
Since you’ll be able to read the full report once they share their blog post, I won’t dive too deeply into the findings here. But at a quick glance:
There was one critical issue found that resulted from a mistake while merging the MWEB code & v0.21.1 code together. So when copying the changes into the latest release code, I missed a small, but crucial line of validation code that could’ve been exploited by a malicious attacker to cause serious disruptions to the chain
This tells us…
We could really benefit from better functional test coverage around our validation logic to make sure we would catch similar issues ourselves in future releases.
We should think about adding some processes we can follow to minimize the possibility of this happening. That could mean documenting all changes, or having 2 people perform the merge separately then comparing results, or a change to how we approach the code reviews.
The audit was a really good idea (thanks Quarkslab!)
There were also some smaller findings, and some great suggestions for how we could improve the quality and safety of the code. Overall, they were impressed with the code quality, which was exciting to hear
v0.21.1 (Taproot) Release
The release process 16 we inherited from bitcoin can be quite painful. It uses gitian 5 to build repeatable and deterministic binaries from the source code. This means that multiple people can all build the code on different machines (and even different operating systems) and still get the same exact release binaries. We can then all compare the results and then sign the release, certifying that we all agree that the published release is safe & accurate.
There’s a lot of magic involved to make this work, which leads to a time-consuming & often frustrating experience (especially for n00bs like me). So I really dragged my feet on this one . I finally forced myself to push through this a few days ago, and after fighting with some outdated scripts, was able to build all of the binaries successfully. I’ll finish signing these tomorrow and hand them off for the other developers to repeat the build & verify results.
MWEB Testnet
After lots of promises and then take-backs, I’ve finally decided to release a binary that allows non-technical users to try out the MWEB testnet. I only have the windows release available right now, but I’ll work on getting binaries for Mac OS X on Friday. Linux users can build their own, because I’m tired
Link: MWEB Testnet Release 56
Here’s my gpg key 11 if you’d like to verify the binaries first (you should). I’ll add instructions on how to do that on the release page when I have some time.
There’s no installer, because I didn’t want anyone accidentally replacing their actual litecoin wallet, so to use it:
Download (and verify) the zip file
Extract the litecoin-63fe928e4e8a folder
Find and run litecoin-qt.exe from inside the bin folder
This will default to using the MWEB testnet, which you can tell by the off-colored logo and the [mwebtest] in the title bar. These use mwebtest coins, not actual litecoin coins. So pleeease don’t try to use it with real money.
You’ll either have to mine a block to get mwebtest coins (you can CPU mine a block in no time), or find someone to give you some. If anyone is willing to setup a faucet, I’ve got a ton of coins you can have
Also, if someone feels like writing a guide for how to create stealth addresses, send to and receive from them, and all of the fun stuff that goes along with it, you’d be my new favorite person.
Remaining Schedule
You’re pretty much back to just waiting on me again while I finish applying audit suggestions and then pushing through the tedious process of merging, coordinating final reviews, writing release notes, and finally kicking off the beloved gitian builds. I don’t know exactly how long that will take, but rumor has it that it increases by a full day for every person that asks me
What a long journey this has been
P.S. https://wenmweb.com 237 is up to date.
November 2021
November Progress:
Security Vulnerability
As shared on Twitter yesterday:
Kurt 10, a long-time GRIN community member, contacted Charlie and I to inform us of a vulnerability in the design for non-interactive transactions. While the attack is difficult to perform in practice, it does allow for theft of funds if the conditions line up just right.
This attack is rather technical, and difficult to understand without first learning all of the crypto behind MWEB. Very informally, it works like this:
Alice sends 2 coins to Bob:
coin 1 = 10 LTC
coin 2 = 20 LTC
Bob creates 2 transactions, 1 to Charlie, and another back to Alice, and sends them at roughly the same time:
tx1 = spend coin 1 to send 8 LTCs to Alice (8 LTC Alice, 2 LTC Change)
tx2 = spend coin 2 to send 15 LTCs to Charlie (15 LTC Charlie, 5 LTC Change)
Alice changes tx1 to spend coin 2 instead, keeping the additional 10 LTCs for herself:
tx3 = spend coin 2 to send 18 LTCs to Alice and 2 LTC back to Bob as Change
tx1 & tx2 dropped and replaced with tx3
There are a number of reasons why this attack would fail in practice nearly every time. But the consequences if it did succeed would be very serious, so it was obvious this was something we had to prevent.
We are very grateful for Kurt taking the time to study MWEB’s design, and for reaching out to share this attack with us. Due to the importance of the finding, Charlie generously donated his own money to pay Kurt a well-deserved 0.15 BTC bounty 19.
The Fix
Considering the proximity to the planned release date, panic started to set in. Fortunately, I realized there’s a relatively straightforward fix for the attack that consists of introducing a new public key in each input that prevents reuse of input signatures.
At the same time we were working through the details of the attack & fixes, I was put in contact with some top-notch cryptographers who offered to do a security audit of our design, which they were considering to use as a starting point for another project they were working on.
The need for a more formally documented design became evident, so I spent the next few weeks rewriting LIP-0004 7 into a more complete and formally specified design, making minor tweaks along the way to harden it where I could. Clearly, I should’ve done this from the beginning, because we’ve had nearly as many reviewers of LIP-0004 in this past month as we have for the previous 1.5 years While I would’ve loved to have all of these eyes on the design long ago, I’m thrilled about all of the feedback I’ve received.
Unfortunately, some changes do need to be made to the code to now match the new design, which means a few more more weeks of dev work. Fortunately, nearly all of the changes will be in the libmw subproject, which is highly modularized and heavily tested. This is great news, since it means the changes should be easier to make, test, and most importantly, review. This review can be carefully performed by the other LTC developers, so I don’t believe it’s necessary to send the changes back to the auditors. This will have an impact on release date, but the delay should be minimal.
Release Process
I mentioned last month that the release build process was time-consuming, and the scripts were outdated, so I spent some time cleaning all of the old scripts up, and creating a simpler, more automated build process. The build scripts and verification keys are going to be maintained in a separate repo going forward. Right now, the new ltc-release-builder 18 is just under my personal github account, but if it works out well for the MWEB release, we’ll get that moved to litecoin’s github org 17.
Timeline updates
I’ve chosen to push the release to January to ensure we have enough time to fix the vulnerability found. Hopefully that will be the last time . wenmweb.com 184 is once again up-to-date.
v0.21.1 any day now™ for real this time™
December 2021
December Progress:
Taproot Release
The first release candidate of v0.21 with taproot support (v0.21.1rc1) is now available for download 36.
I’ve been testing it out, and everything has been working fine for me, so the official release is being built and signed now. Feel free to test out rc1, or wait a few days for the official release (v0.21.1), which will be made available for download at https://litecoin.org 8 after it’s been signed by the devs.
Please report any issues you find with the release using github 12.
Security Fixes
As reported last month, a security vulnerability was found, which required changes to the non-interactive transaction design, and therefore changes to the consensus code. These changes have all been made, tested, and documented.
Final Reviews
Next week, I’ll be meeting with the other devs daily so we can go through the code together with a fine-tooth comb. If no critical issues are found, that should be the final MWEB code review.
MWEB Release
If the review is successful, which I’m optimistic it will be, I’ll create the first release candidate this month
January 2022
January Progress:
We did it, guys
MWEB Release
The first MWEB release candidate is now available at https://dev.litecoin.org/ 24. A “release candidate” (RC) is like a pre-release. It’s fully functional - miners can signal for activation, users can generate MWEB addresses, and (once activated) send to MWEB addresses - but it hasn’t been thoroughly tested for bugs yet. The more people who try out the RC and report any issues they find, the better off the final release will be. But please plan to upgrade to the full v0.21.2 release once it’s available.
Verifying the binaries
It’s recommended that you verify the release binaries, to ensure they have not been tampered with. It may be too difficult for some non-technical users, but for those capable, it’s worth trying before running the downloaded file. To do this:
Download & install gpg and sha256sum
Download SHA256SUMS.asc 2 to the same directory as the downloaded release binary.
Then, from your terminal (Mac/Linux) or command prompt (Windows):
Change directory to the location you downloaded the release binary and SHA256SUMS.asc to: cd <insert path here>
Download my pubkey: gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys D35621D53A1CC6A3456758D03620E9D387E55666
Verify the signature in SHA256SUMS.asc: gpg --verify SHA256SUMS.asc The response should include something like: Good signature from "David Burkett <davidburkett38@gmail.com>", telling you that SHA256SUMS.asc has been signed by me, and has not been tampered with.
Now that we know SHA256SUMS.asc is valid, we need to verify the release binary you downloaded matches the one inside of SHA256SUMS.asc. As long as the release you downloaded is in the same directory as SHA256SUMs.asc, you can just run: sha256sum --ignore-missing --check SHA256SUMS.asc You should see your release binary listed, followed by an OK. So, if you downloaded the windows version, you’d see: litecoin-0.21.2rc1-win64-setup.exe: OK
What’s Next?
I’ll spend the next month watching for bug reports, and fixing any issues found. Unfortunately due to the complexity of the change, bugs will exist. So far, this includes an issue that prevents some long-lived wallets from generating MWEB addresses, which I’m actively working to fix. Once we’re satisfied with the testing & fixes, we’ll likely release another release candidate (0.21.2rc2) and go from there.
Meanwhile, miners running v0.21.2rc1 can start signaling for activation at block 221760 (~Feb 25th). @losh11 is working on a website to monitor signaling and activation status, which I’ll share here once it’s available.
February 2022
February Progress:
I’m even later than normal this month. Sorry about that!
Testnet Activation
After a slightly bumpy start, MWEB has successfully activated on testnet, and the chain is progressing forward. Thank you to Loshan and Charlie for driving this effort, and to the LTC community for really stepping up in a big way to help test and debug issues.
Speaking of which, there were a number of issues identified, mostly around:
1. Encrypted and non-HD wallets
Certain wallet setups were not able to generate stealth addresses with rc1, and during some flows would actually crash. Fortunately, those issues were found and reported right away, so we were able to get those fixed quickly.
2. Mining APIs
The code for solo mining was written and tested long ago, but of course, almost nobody solo mines these days. Mining is done through mining pools, which use a slightly different method for getting new mining jobs, and submitting new solutions. I didn’t have access to a mining pool at the time to test, so the stratum changes just went unimplemented. At least until last week.
Loshan and I worked together to code and test the mining API changes needed for pool mining, which he describes here 17. If you know any pool operators, please point them there so they can be ready for mainnet activation.
3. Wallet Transaction List
There were a few bugs around transaction building that seem to be fixed now, but most of the wallet issues are around how we display transactions. Between Mimblewimble’s pruning and coinjoining, and the way “pegging out” works with extension blocks, displaying the wallet history is one of the most complex changes we’ve had to make for MWEB.
Pegging out actually splits sends into 2 transactions, which can make for a very confusing history if we’re not careful. And I wasn’t very careful… Immediately after testnet activation, people were confused by the numbers of transactions and the amounts they were seeing. Fees displayed incorrectly, credits and debits were unreliable, and some really large or whacky numbers would appear at times. These were all just display issues, not actual problems with the amounts sent/received, but it didn’t make for a great user experience at all.
I’ve been doing my best to fix each of these issue as they’re identified, and it’s finally starting to act a little more predictably. We’ll continue to test and fix these issues until we’ve got a release candidate that we’re 100% comfortable with. Once that’s the case, we’ll create the full v0.21.2 release and start (gently) badgering node operators to upgrade and miners to start signaling.
March 2022
March Progress:
Another month of testing, another month of bugs.
Fixes:
1. Wallet transaction List
It took several iterations of fixes, but we seem to finally be displaying the transaction history reliably. As part of these fixes, the code was moved to a location where we could more easily write automated tests for it. Having repeatable tests should ensure that this continues to function properly in v0.21.2, and in all future releases. You can find most of the related changes here 7.
2. Miner Crashes
When the mining logic was written, it was intentionally made overly simplistic at first, allowing only spends of confirmed outputs. It would not allow you to spend outputs in the same block they were created. The plan was to add support for this after launch sometime.
However, one of the testers was able to create a batch of transactions that caused mining nodes to crash when they tried to include those transactions in a block. After some investigation, we found the underlying problem, and realized that adding full support for including spends of unconfirmed outputs would resolve the issues. So that’s what was done here 3, along with some redundant checks to prevent similar failures from occurring.
3. Subtract Fee From Amount
We’ve had a large handful of issues related to the creation of transactions using the “subtract fee from amount” functionality. After playing whack-a-mole with these bugs for a few weeks, I finally decided it was time for a minor overhaul of the spend code. This was done here, here 6, and here. From what I can tell, it is working well now, but we’ll know more once testers get their hands on the rc6 release.
Release Status
I was hoping to be able to share rc6 with everyone today, but a few problems were reported last minute that needed to be fixed. I believe we’re finally ready to tag release candidate 6 and start building now though, so you should see that sometime this week. All known critical issues are believed to be fixed, so unless new ones are found during testing, rc6 could be our final release candidate. With any luck, I may be able to share the official v0.21.2 release with everyone by next month’s update
Activation Status
I haven’t had much time to keep up with signaling, but I have been working with most of the mining pools, and I expect them all to be signaling very soon (some already are). @coblee is tracking the major pools and their statuses here 60. You can also keep an eye on block signaling here 41. Just a reminder, MWEB activates once 75% of blocks signal for activation within an 8,064 block window. We are currently in MWEB activation window 3, but we will not meet the activation threshold this window. The next one starts in about 2.5 days, at block 2,241,792.
P.S. This is my ~30th update now, and with the various other comments in here, it’s starting to get difficult just to scroll through to reference old posts. This will probably be my last update here, but I will continue on in a fresh new thread for next month’s update.
April 2022
I’m starting this new topic for monthly MWEB updates which will continue on from the previous one 2.
April 2022 Progress:
Testing
Testing of the final RCs went really well. All major workflows seem to be working smoothly.
The majority of mining pools have reported to us that they’ve successfully mined MWEB blocks in testnet, and are ready for activation on mainnet.
Activation
MWEB officially locked in 1 at height 2,257,920!
Starting at block 2,265,984 (~May 19th), MWEB will be activated on mainnet, and everyone who has updated to v0.21.2 will be able to start sending and receiving LTC using the MWEB.
Final v0.21.2 Release
The final release will be available later this evening at https://litecoin.org/ 8. Anyone interested in using the MWEB, and especially those who installed one of the earlier release candidates, should upgrade to the official v0.21.2 before MWEB activates. If you wait until after MWEB activation to upgrade, you’ll unfortunately be required to resync the blockchain from scratch.
While the consensus logic has been solid for a while now, the wallet has undergone drastic changes these past few months as we worked to resolve issues found during testing. The major workflows all seem to be working well now, and I expect most people to be able to use MWEB without issue, but this was an enormous change, so don’t hesitate to report any bugs or behavior that seems off to you.
Comments