Detection keys and the recursion priority debate
After ECC said it will focus on payment scalability for a year, a debate broke out over the priority of Halo 2 recursion.
Agenda summary 8
Proposal to host the Arborist call
Electric Coin Company · Nate
- A proposal was made for community members to take turns hosting the Arborist call. It could take the form of handling the calendar and Zoom.
- ECC wants to share responsibility across organizations to increase stakeholder participation. Collaboration could later extend to setting agendas and lining up presenters, and people who are interested were asked to let the team know.
Zebra rc.2 and FROST status
Zcash Foundation · Teor, Deirdre
- A week ago the team released the third release candidate, rc.2. It implemented the ZIP 317 default fee used for mempool transaction eviction and fixed intermittent crashes and memory errors when the RPC server shuts down.
- Mining pool support is being developed behind a Rust feature. The core getblocktemplate and submitblock are done and the RPCs for stats and status are being filled in, and it can now build block templates similar to zcashd's.
- The draft of the next release applies ZIP 317 to block template selection, so when the mempool is full it refuses or delays mining large transactions with low fees. The condition for an official release is an audit of all production code.
- Deirdre said the IRTF's FROST last call has ended and version 12, with the feedback incorporated, will become the actual RFC. Conrado implemented all parameter sets including ed448 and is working on the re-randomized variant for Zcash.
- The team plans to publish the entire FROST signing library on crates.io and update it to match RFC changes.
Zebra wallet support and network protocol fixes
Zcash Foundation · Teor, Deirdre, Nate, Adi
- On the point that the Zecwallet full node depends on zcashd wallet RPCs, Teor answered that there are no plans to put wallet functionality into Zebra. Wallets use lightwalletd, and other JSON RPC can simply be called directly.
- The reason is reluctance to keep spending keys in the same process as the consensus node implementation. Deirdre said there is a design for a separate wallet binary that registers viewing keys and scans, but it has not been pursued yet.
- So that Zebra handles the version message and connection setup of the Zcash network protocol more accurately, a fix will go into the next release. It may be related to the problem of nodes not appearing in block explorers or the DNS seeder.
- It currently supports submitrawtransaction, and with getblocktemplate enabled it can also submit externally mined blocks. Str4d said a custom wallet can build transactions with Rust crates and submit them to Zebra.
Research direction - shielded wallet UX and payment scalability
Electric Coin Company · Nate, Pacu, Str4d, Adi
- Nate said the team has stopped PoS and interoperability research and is focusing on improving shielded mobile wallet performance under heavy output load. It plans to keep its focus on payment scalability and UX improvements for about a year afterward.
- As ideas, replacing or supplementing lightwalletd was raised, along with an off-chain auxiliary notification service in which the sender gives the recipient a payment notification or a hint about which blocks to scan.
- Adi asked about blockchain size growth and pruning. Nate said wallet UX is the top priority and that the idea of asking miners to reduce block size was discussed. Str4d said they welcome a PR improving zcashd's partial pruning.
- Nate said an ECC mobile wallet launch is hard in the first half because it was pushed back by SDK improvements, and the second half is also hard to be sure of. Pacu said the ECC wallet is behind the SDK in priority.
Liberated payments, detection keys and out-of-band notification discussion
Dodger, Nate, Str4d, Deirdre, Kris
- Dodger mentioned detection keys and the OMR that Deirdre is researching as ways to get away from a mobile wallet structure that must download every shielded transaction. Detection keys require a protocol change.
- Nate said they will start this research after Q1 2023. First, speeding up scanning with new lightwalletd methods will take a few months, and they said each approach's trade-offs should be compared side by side.
- Str4d said the options have grown since 2019 and that adoption requires a network upgrade, so it would be easier to review if the community made mock-up proposals.
- The upcoming lightwalletd change annotates compact blocks with the tree position of the first note and adds the ability to look up 2^16 subtree roots. Kris submitted a draft PR.
- Deirdre pointed out that for both detection keys and OMR, the computational load is linear in the number of registered keys. Str4d said the solution is to make the load proportional to the size of one's private transaction graph, and UX work like NFC or QR hints helps.
Halo 2 recursion and the priority debate
Nate, Deirdre, Str4d
- Nate said recursive proofs help lower validating node costs but it is less certain that they directly help payment scalability, and that payments are the priority now.
- Deirdre raised the issue that NU5 was pushed on the grounds that Halo 2 scales Zcash through recursion, yet recursion is being pushed down the priority list.
- Nate said that before the sandblasting they had argued for a PoS transition and saw recursion as lowering validator costs, but afterward they changed priorities because of the payment scaling problem.
- Str4d summarized that scalability is used to mean two mixed things: wallets working well at the current capacity, and increasing chain capacity. They said recursion is still needed as a long-term component of the latter. Work on the recursion implementation continues.
Core update - zcashd 5.3.x and DAGSync
Electric Coin Company · Str4d
- 5.3.1 is a release that reduced high memory use during IBD, and the community's headers-first fix was backported. However, it turned out that at the first flush after all headers are downloaded, memory temporarily doubles to about 10GB.
- Headers are large because of Equihash solutions, so cache size tracking was off. 5.3.2 temporarily reverted the headers-first fix, and the plan is to fix the cache problem in 5.4.0 in mid-to-late January and get both.
- Greg is unifying transaction building in the zcashd wallet. It is the foundation of a propose-commit RPC that, when the user states the amount and fee, returns a proposal containing change outputs and privacy implications, and the mobile SDK is going in the same direction.
- For DAGSync, after finishing the tree structure work, the new tree is applied to the mobile SDK, zcashd and lightwalletd. Then known notes are made spendable first and the existing linear sync continues running.
- ZIP 317 implementation began targeting zcashd 5.4.0 but has lower priority than DAGSync. Early next year the team will hold a Halo 2 developer call and is envisioning an overhaul, in the manner of Rust nightly, beta and stable, to make outside contributions easier.
Learning resources for new developers
Ian, Deirdre, Str4d, Pacu
- Deirdre said to read the ZIPs in the Zcash spec and to follow current code changes through GitHub PRs for zcashd and Zebra. Str4d said to lurk in the Discord channels and use the Halo 2 book as an introduction to the cryptography.
- Str4d said PR review comments are a good way to learn how security and performance are protected, and that they want to put more review guides in the zcash book.
- Pacu said that as long as PoW remains, build the basics with Mastering Bitcoin and then move on to the Zcash documentation and the light client protocol.
- Str4d said developer experience is also a usability issue, so people should tell the issue tracker or Discord about difficulties. Half of the reported problems, such as the ELI15 Halo 2 rulebook, were fixed quickly.
This summary is based on the original meeting notes; check them for exact details.