← All calls
ARBORIST#32

zcashd 5.1.0 performance and ZSA design

zcashd 5.1.0 cut block verification time by about 80%, and the ZSA design and FROST ZIP work were shared.

Agenda summary 7

  1. Zebra update - performance problems and the FROST ZIP

    Zcash Foundation · Conrado

    • The team is addressing performance problems caused by the increase in shielded transactions. There is one Halo 2 proof per Orchard transaction, but verification was placed in the wrong spot in a loop and ran per action, and that bug was fixed.
    • Sapling proofs and signatures had a batch verifier, but only one ran at a time so it could not use multiple cores. It was changed to run several batch verifiers at once, and the structure was reworked to process blocks and batch items in parallel.
    • Sync previously stalled but now succeeds. However, by the latest figures full sync grew from 6 hours to 10 hours, and further issues are being investigated. Because of GitHub Actions' 6-hour limit, CI was split into per-segment jobs.
    • A ZIP is being written for using the FROST threshold signature protocol in Zcash. FROST is based on ordinary Schnorr signatures, so it has to be adjusted to support the re-randomized signatures Zcash needs.
    • The first version of the ZIP contains everything needed, with reference fixes and added background explanation remaining. The goal is to finish before Zcon.
  2. Zcashd 5.1.0 release contents

    Electric Coin Company · Steven, Str4d

    • 5.1.0 sped up Sapling and Orchard block verification, cutting verification time by about 80%, and the worst-case verification time for past blocks was also cut by about 80%.
    • An option was added to use V4 transactions by default for transactions with no Orchard component. It is for legacy wallets that do not yet support V5.
    • getblocktemplate contains only already-verified transactions, so it skips proof and signature checks and got faster. Orchard action information was added to getrawtransaction and decoderawtransaction.
    • z_gettreestate performance and wallet rescan performance for post-NU5 blocks improved, and the time the wallet holds blocks was reduced for RPC throughput.
    • A deprecation approach was introduced for phasing out features such as RPCs in stages. Str4d added that it was implemented in 5.0.0 and applies to all zcashd features, such as disabling Sprout, not just RPCs.
  3. Additional zcashd performance PRs - witness tracking and the verification cache

    Electric Coin Company · Steven, Str4d

    • The bottlenecks are block sync and IBD costs, wallet scanning and so on. The Orchard crate has a batch verifier that handles multithreading internally, whereas Zebra has the opposite structure, putting single-threaded batches on top of multiple cores.
    • PR 6069 addresses the problem of tracking note witnesses. The approach of continuously updating all notes for rollbacks was a serious bottleneck in large wallets with many spent notes.
    • Using the consensus assumption that the network does not roll back more than 100 blocks, it stops witness tracking after enough time has passed. They also saw that making trial decryption in wallet scanning multi-threaded is possible.
    • PR 6073 is a cache so that Sapling and Orchard bundles already verified in the mempool do not get proof-checked again in blocks. On top of the 80% improvement in 5.1.0, it cuts another 80%, so steady-state block verification returns to the expected level.
    • The worst case, such as IBD where every transaction is being seen for the first time, does not improve. Instead, reduced verification time is expected to speed up block propagation and lower the orphan block rate.
  4. 0xPARC Halo 2 study group

    0xPARC · Ying Tong

    • The Halo 2 study group held in New York over four weeks ended last week. Ying Tong, Daira and Str4d helped with PLONKish and Halo 2 concepts and library onboarding through Discord and sessions.
    • Halo 2-based projects introduced were Orbis's tiny RAM for a Cardano zk rollup, the VM and DSL of Darkfi, which is building private DAOs and PoS, and the Ethereum Foundation's zkEVM.
    • More than 20 participants built 10 projects. They included zkWordle, a Halo 2 Wasm that runs in the browser, and an MiMC hash in Halo 2.
    • As follow-ups, discussions continue on PLONKish DSL collaboration and development of Halo 2 library features such as dynamic lookup tables.
  5. ZSA development status

    QEDIT · Daniel Benarroch, Aurélien Nicolas

    • PRs for two ZSA issues (including issuance) were posted the day before, and since the issuance side has a little work left, the team asked people to comment on the issue.
    • Progress was made on the classical computation of note commitments and value commitments that support both ZEC notes and ZSA notes, test vectors, and zcashd integration.
    • The goal is to stay fully backward compatible with ZEC nodes and replace the Orchard bundle with a ZSA bundle without a turnstile, and this is the biggest challenge.
    • In ZSA, producing two outputs by splitting a note also requires a valid input, so the existing dummy note becomes a will-not-spend note and the client logic also changes.
    • For circuit performance, an input word must be added to the fixed initialization of the Sinsemilla gadget. Str4d asked to keep the fixed-domain API as well, and Aurélien agreed.
    • The value commitment changes from fixed-base multiplication of a 64-bit scalar to variable-base, which is already implemented in Halo 2. They proposed holding a session to recheck ZSA security in two weeks or at Zcon3.
  6. Roadmap after ZSA - programmable protocols

    QEDIT · Aurélien Nicolas, Daira, Str4d

    • Aurélien thought features after ZSA, such as trustless asset exchange, bridges and multisig, are better implemented on a programmable protocol than in the core protocol.
    • Daira answered that one of the motivations for the move to Halo was that recursive proofs open up later programmability.
    • Str4d said bridges leave the Orchard pool anyway, so they can be handled with a separate circuit, and Daira thought that if transaction aggregation is supported, indistinguishability comes naturally.
  7. Wallet performance and light wallet discussion

    Dodger, Str4d, Steven

    • Str4d said light wallets need a full node behind them, so fixing zcashd performance problems was the top priority, with two PRs in progress and one wallet improvement for large users (mining pools, exchanges) remaining.
    • The next priority is sync performance of the light client based on the Rust zcash_client_backend crate and of the mobile SDK, which will take longer because it requires architectural changes.
    • Daira mentioned that there is also the option of raising costs through a fee policy change, and Steven assessed that 5.1.0's effect on light wallet servers was minimal.
    • On a wallet preference question in the forum, RJ Ricasata asked whether the wallet market will converge on one, and Deirdre, Adi and Daira answered that a variety of wallets benefits security and decentralization.
    • Steven said they use Nighthawk and ECC's internal demo wallet and that both eventually sync, and said Aditya is dealing with Zecwallet Lite.

This summary is based on the original meeting notes; check them for exact details.