← All calls
ARBORIST#49

DAGSync non-linear sync design

The DAGSync design for fast sync was shared, and it was corrected that Zelcash/Flux, not Firo, had been confusing the network.

Agenda summary 9

  1. Zcashd update - wallet overhaul and ZIP 317 adoption

    Electric Coin Company · Kris, Daira

    • The team merged changes that remove duplicate functionality in the zcashd wallet and extend the transaction construction features related to unified addresses.
    • For fast light wallet sync, the new note commitment tree structure was integrated into the zcash client backend, and work is under way to have zcashd provide the metadata needed to spend existing funds immediately.
    • Rust crates were updated across the board, from the ff crate up to librustzcash and zcashd. The ZIP 317 implementation for the zcashd wallet and miner side is in its final stage.
    • Because the wallet ecosystem has not yet adopted ZIP 317 fee calculation, it is undecided whether enforcement will go into 5.5.0 (to be released within a few weeks). Daira said the block construction algorithm is designed to tolerate some wallets not paying the higher fee right away.
    • Kris said they would help introduce ZIP 317 as a reusable module of librustzcash and the zcash client backend.
    • The halo2 proof crate fix for 32-bit non-determinism and the incremental Merkle tree crate fix will be included in orchard 0.4.0.
  2. Zebra update and the Flux node correction

    Zcash Foundation · Arya, Dodger, Str4d, Daira

    • The focus is on triaging and fixing audit issues. The team merged a security fix that limits the number of unconfirmed chains Zebra tracks and a fix for a rare concurrency bug in getblocktemplate.
    • The team fixed an issue where the newly activated mempool ignored transactions, added a log file output option and a confirmations field in the getrawtransaction response, and made Zebra check that a block is in all chains before downloading it.
    • Firo nodes were removed from Zebra's address book, and releases now build a Docker Hub binary with testnet mining enabled. A terminal progress bar PR and a draft PR making zebrad refuse to run when the release is too old are open.
    • Dodger said there was a correction that the chain sharing the network magic discussed last week is Zelcash/Flux, not Firo.
    • Daira fixed the title and description of issue 6505, and Str4d said the seeder excludes these nodes from advertising.
    • After finishing the audit and issuing a stable release, the team will push to expand mining. It appears no miners use Zebra yet, and to provide implementation redundancy it plans to work with ECC to spread miners across both.
  3. DAGSync progress

    Electric Coin Company · Kris, Str4d, Adi

    • Kris said they are designing the upper-level API changes to the zcash client backend and the resulting changes needed in the lightwalletd API.
    • In zcashd, collection of Orchard-side metadata is in progress, while on the Sapling side the work of replacing zcashd's Sapling code from C++ with a Rust library must come first.
    • The gRPC interface change is needed on both sides, including Zebra's lightwalletd backend. It can be used with the existing lightwalletd, but only some of the benefits of the new sync approach are obtained.
    • Str4d said the biggest change is the shift from linear sync to non-linear asynchronous sync. Without the new API, it can fall back internally to linear sync.
  4. DAGSync - wallet impact and timeline

    Electric Coin Company · Kris, Str4d, Adi

    • It fetches recent blocks first so notes can be spent immediately, and it trial-decrypts first the blocks most likely to contain notes.
    • When Adi asked about the increase in calls, Kris said it amounts to one extra call for an intermediate note commitment tree root per wallet session. Str4d said the number of requests will rise but total bandwidth will fall.
    • Kris said they would like to release a sketch of the new API for wallet teams to look at around next week. Nate said they published a blog post with a timeline a week or two ago and will update frequently via the forum going forward.
  5. Foundation and ECC collaboration - lightwalletd maintenance and sharing contacts

    Dodger, Kris, Str4d, Nate, Adi

    • On a community call attended by new Foundation directors Marta Belcher and J.W. Verret, there was a suggestion to find ways to help address performance problems.
    • Dodger said they would help with code review or brainstorming, and Kris and Str4d said the Zebra team's async experience would be useful on the lightwalletd API side.
    • Kris pointed out that lightwalletd has no dedicated maintainer. Larry Ruane, who was in charge, moved to Bitcoin work and only provides support, so ECC and the Foundation need to take it on together or find a third party.
    • Dodger said they would set up a time to discuss this.
  6. Sharing contacts and the full-node wallet idea

    Dodger, Kris, Str4d, Nate, Adi

    • Adi said the Foundation can help spread word of ZIP 317 fees to various communities such as Trezor, Ledger and miners, and Daira said it is meaningless unless enforced.
    • Dodger suggested sharing the ecosystem contacts ECC has built up with the Foundation.
    • Nate mentioned the problem of consolidated inquiries piling up at ECC, as with the new block explorer 3xpl.com, and said it would be good to have an entry point that guides people to the various organizations and their roles.
    • Kris and Str4d sketched a path to build a full-node wallet by extending the zcash client backend and moving away from the C++ zcashd wallet. Dodger said the full-node wallet strategy will be discussed on the core developer call.
  7. ZSA update - new ZIP PR and circuit changes

    QEDIT · Vivek, Daira

    • The team moved from PR 649 to PR 680 in the ZIPs repository. Comments on 649 were resolved or recorded, and people were asked to leave new comments on 680.
    • The transfer and issuance ZIPs (226, 227) are being tidied up to reflect discussions in meetings with ECC and the Foundation and to fit the ZIP 0 format.
    • The QEDIT fork has several circuit-change PRs for the halo2 and orchard crates. They add variable-base scalar multiplication and change note and value commitments, and will be submitted to the official repositories after review and integration.
    • Daira said the ZSA ZIPs are also covered in the ZIP sync, but there is a separate meeting for spec editing and issue resolution.
  8. Nighthawk app update

    Nighthawk Apps · Adi

    • The block explorer gained a feature that decodes the coinbase hex to show miner messages, and the type display and fees for v5 transactions were fixed.
    • The light wallet service has begun building globally distributed infrastructure in South America, Europe and Asia, in addition to the existing North America.
    • The Nighthawk wallet resumed development in line with its 2023 roadmap.
  9. Open discussion - collaboration proposals and front-end developer participation

    Kris, Daira, Dodger, Nate, Adi

    • Kris proposed that if Foundation engineers took part in the Rust implementation of the transparent script interpreter, the circular dependency between the two codebases could be removed.
    • Daira said it must be compatible with the C++ code down to its bugs, and that retiring T-addresses will be a multi-step process.
    • Dodger said ECC's Sasha is building a Rust seeder with Zebra network and proposed a joint effort with Foundation engineers.
    • Daira is preparing a presentation on things that can be removed from the protocol to simplify it, and Dodger said they would host it as a community call.
    • Nate said protocol changes such as ZIP 317, ZSA and FROST affect UX, so they want front-end and wallet developers to take part more in this call.
    • Adi suggested having an app developer invitation corner on the Arborist call, and Dan volunteered to reach out to other ecosystem developers.

This summary is based on the original meeting notes; check them for exact details.