← All calls
ARBORIST#106

Debate over per-organization Zebra forks

A proposal to develop post-NU7 upgrades in per-organization forks sparked debate.

Agenda summary 15

  1. Zebra update: the 2.4.0 DB bug and 2.4.1

    Zcash Foundation · Alfredo

    • Zebra 2.4.0 contained a database format upgrade designed to be less disruptive for node operators.
    • A bug was found through an RPC-related ticket rather than the test framework; the root cause was found and fixed in 2.4.1, and the 2.4.0 release was withdrawn.
    • Recovery instructions were provided for DBs corrupted by 2.4.0, and whether to issue 2.4.2 or 2.4.1 suffices will be checked that day.
    • There were bottlenecks and CI problems in the release process, which the team wants to fix faster, and to improve so the test framework catches such bugs in advance.
  2. Zebra update: crate cleanup and the NU6.1 PR

    Zcash Foundation · Alfredo

    • The zebra-scan crate and the related zebra-grpc crate, which had been added when the zcashd retirement stack was unclear and were no longer used, were removed.
    • Coinbase transactions are now built with the transaction builder in zcash primitives, and the team wants to replace duplicate code inside Zebra with zcash primitives too.
    • Missing regtest features are being worked on to support testing the RPC framework.
    • The NU6.1 implementation PR is under review, and PRs are also open in several repositories of the stack such as librustzcash.
    • External contributor PRs are being merged and tickets opened by people answered.
  3. ECC update: Z|ECC Summit and roadmap

    ECC · Daira Emma

    • At the Z|ECC Summit and hacker house, the core team explained to wallet teams Tachyon, the new scalability architecture researched by Sean Bowe, and discussed changes to the payment flow.
    • The new architecture puts more emphasis on payment requests than on broadcast transfers through the blockchain.
    • The ECC Q3 roadmap review is finished and a blog post will follow soon.
    • The core team plans the NU6.1 mainnet activation window for October or November (probably October), around the end of the current lockbox disbursement stream.
    • NU7 has a lot to do and will take shape in the schedule; work on FROST key derivation and the Orchard quantum-resistance spec is in progress and an audit is wanted.
  4. ECC update: P2SH support, compact blocks, Zashi Tor option

    ECC · Daira Emma

    • There was a lot of work on P2SH multisig (transparent multisig) support, which is needed for both zcashd retirement and NU6.1.
    • Because NU6.1 lockbox disbursements go to a P2SH multisig address, zcashd supports it, but Zallet and Keystone support is needed and a Keystone firmware update is required.
    • The team plans to propose putting transparent information into compact blocks to improve privacy.
    • The zcash_script work is essential for P2SH support; a large number of librustzcash PRs were reviewed, and a new key, qsk, useful for hardware wallets, was added to the quantum-resistance spec.
    • Because using Tor is dangerous in some countries, Zashi's Tor opt-in and opt-out are being made more robust.
  5. ECC update Q&A: lockbox address and compact block changes

    ECC, Zcash Foundation · Alfredo, Daira Emma, Pacu, Zooko

    • Alfredo asked whether the node side needs support for a new address type, since the lockbox address list is empty in the NU6.1 PRs for Zebra and zcashd.
    • Daira Emma answered that spending from a P2SH address is an ordinary transaction needing no separate consensus rule, and they believe that PR contains all the consensus changes for the one-time disbursement.
    • When Pacu asked how far the spec for putting transparent information into the compact block format had progressed, they said it is not yet concrete, is close to defining a protocol format, and will be discussed at the light wallet meeting.
    • When Zooko asked where the NSM spec is tracked, the answer was that it is the most recent PR in zcash/zips.
    • On the Orchard review, they said the review of the Halo circuit changes is next.
  6. Zingo Labs: Zaino update

    Zingo Labs · Za, Daira Emma

    • Zaino's internal chain-state representation was heavily refactored to have non-finalized and finalized state as first-class objects.
    • The aim is to stay compatible when consensus moves to Crosslink, be future-proof, and provide an ergonomic interface.
    • The optimistic goal for shipping a Crosslink-compatible release and putting the Zaino light wallet proxy into production is two weeks.
    • Arlo did a lot of work on adding TX outputs to compact blocks, and it was agreed to discuss it at the light client working group meeting to avoid duplication.
  7. zcashd retirement: progress visibility tool and the Zallet alpha

    ECC, Zcash Foundation · Pacu, Daira Emma, Nate

    • Pacu wants to build a visibility tool showing the progress of the retirement work at a glance without being tied to dates, and plans to ask each team lead which tools they use.
    • There was a view that teams that must make quarterly plans find it hard to grasp progress.
    • Nate said such a visibility tool is needed across all organizations and that the Shielded Labs roadmap page is a good example.
    • Daira Emma said a lot of recent Zallet work has smoothed many of the rough edges that got in the way of using it as a wallet, and the alpha is coming soon.
  8. Zcash Shielded Assets (ZSA) update

    QEDIT · Vivek, Daira Emma

    • An encoding change that makes issuance authorization signature versioning easier, and a naming cleanup that reduces ambiguity, were reflected in the ZIP; the ZIP editor is finishing up, so the design will be frozen again soon.
    • The same changes were reflected in the Python reference implementation, and since upstream moved ahead, a new PR will be opened instead of the old test vector PR.
    • The design changes are being propagated up the stack in the order Orchard, librustzcash, Zebra, and in Orchard the circuit and external code are being simplified to reduce divergence from upstream.
    • librustzcash has mostly resolved the ZSA impact of the PCZT addition, and the Halo 2 PR awaits ECC's review.
    • The Zebra upstream PR was reviewed by Arya and incorporated internally, and the approach to upstreaming is being discussed with ZF.
    • The Zcash transaction tool improved generalization and CI automation, and Daira Emma corrected that what changed this time is the issuance key scheme, not NSM.
  9. NSM update: librustzcash PR and test vectors

    Shielded Labs, ECC · Mariusz, Daira Emma, Jason McGee

    • Mariusz said there has not been much progress since last time, and asked about ECC creating a new PR, 1879, on top of the librustzcash PR.
    • Daira Emma said it is opened as a separate PR because pushing across organizations is not possible, that it adds commits to Mariusz's PR, and that work can be based on this PR going forward.
    • ECC accepts the remaining review suggestions as they are, so Mariusz has nothing more to do.
    • The Zcash test vector PR needs workflow-run approval, which Daira Emma will give; giving merge rights to Mark of Shielded Labs and Arya of ZF was overlooked, and they said they would take care of it.
    • To merge ZIP 233, Mariusz must rebase the Zebra PR, and they agreed to discuss this after the call.
  10. Crosslink update: on-chain signature design and Milestone 3

    Shielded Labs, ECC · Nate

    • The team presented the current status at the ECC Summit and received feedback.
    • There is a timing-offset problem in that a BFT block signature cannot be included in the block it signs and must go in a later block, and Daira Emma has already laid out a design in the TFL document.
    • The main motive for putting signatures on chain is to reward highly available finalizers more, and the team is also discussing approaches that do not, weighing scalability and security trade-offs.
    • A test vector framework is being built that replays PoW and BFT blocks in different orders.
    • Milestone 3 changes PoW blocks to point to BFT blocks and implements the finality logic, with no proof-of-stake layer and no transaction format changes, and the design goal is the full PoS design.
    • Milestone 4 implements the entire PoS design, including delegation and finalizer candidate registration, and includes transaction format changes, so it is large.
  11. Crosslink formal verification tools discussion

    ECC, Shielded Labs · Daira Emma, Nate, Zooko, Alfredo

    • In the TFL book repository, Daira Emma formalized in Lean the PoW chain and BFT chain assumptions needed for the Crosslink 2 security proof; the protocol itself is not yet done.
    • Whether signatures are on the chain does not directly affect the assumptions, and depends on where the BFT protocol abstraction boundary is drawn.
    • Zooko reported that Informal Systems, which built Malachite, had reached out, and that an early attempt to formalize part of Crosslink in Quint is on the Shielded Labs GitHub.
    • Daira Emma said Quint is a front end for TLA+, that TLA+ focuses on model checking, and that their pencil proof is fine with high-level properties and does not need detailed messages.
    • Alfredo said TLA+ is strong at modeling distributed and concurrent behavior.
    • Nate said they would hold a meeting to discuss all this at once, and proposed that Alfredo, Daira Emma, the Informal team and Str4d attend.
  12. Open discussion: proposed network upgrade process after NU7

    Shielded Labs, ECC, Zcash Foundation · Jason McGee, Alfredo, Daira Emma

    • Josh's forum post proposed that after NU7 each organization maintain a Zebra fork and be responsible for developing and integrating the changes it proposes, and Jason McGee asked for opinions.
    • Alfredo said they think the current process loads the burden onto particular organizations, but that the commitment to proceed with the existing repositories and people through zcashd retirement and NU7 comes first.
    • Daira Emma explained that each organization maintains patches and rebases from upstream, and that the upstream used by exchanges and mining pools remains a single one.
    • They said a fork means a code fork, the proposed-feature versions are for running on testnet, and they are not compatible with mainnet.
  13. Open discussion: forks and CI cost

    Shielded Labs, Zcash Foundation, ECC · Nate, Daira Emma, Alfredo, Arya

    • Nate said that Shielded Labs tried to replicate the entire Zebra CI but found it hard to set up and costly, so it uses a simple CI and wants to get the full CI through a draft PR at each milestone.
    • Daira Emma said ECC bears the PR CI cost of long-lived branches, but Zebra is different.
    • Alfredo said external contributors cannot run the full CI, and that it would be possible if they were given permission to create branches inside the ZF organization, and noted that CI takes days because of chain syncing.
    • Arya said CI is being made simple and cheap, and that the aim is for forks to be able to run the same CI easily in the future.
  14. Open discussion: per-fork releases and process clarity

    Zcash Foundation, ECC · Arya, Daira Emma, Conrado, Zooko

    • Arya said forks for developing large features are uncontroversial, but producing releases from each fork is costly and hard to take on unless there is a competing implementation.
    • Daira Emma said Josh's post has no requirement for official releases, that whether to release is up to the fork's maintainer, and Daira Emma and Arya agreed that a single upstream central repository would be efficient.
    • Conrado said people understand the proposal differently, that Kris's forum reply that each publishes its own node release is confusing, and asked for it to be stated what is being solved.
    • Daira Emma answered that it takes too long for protocol changes to get into an upgrade and there is a tendency to bundle changes into one upgrade, and some should have been separated.
    • Zooko pointed out that NSM, the simplest patch, took about 24 months, and Daira Emma likened it to NU7 being blocked so NU6.1 had to be squeezed in.
  15. Open discussion: decentralization and upgrade coordination debate

    ECC, Zcash Foundation, Shielded Labs · Pacu, Daira Emma, Alfredo, Nate, Natalie, Arya

    • Pacu said they understand ECC stepping back from the role of upgrade leader, but worried that without a vision for how to coordinate later NUs, things could get slower, and suggested surveying other protocols' examples.
    • Daira Emma said the benefit of decentralization is not conceptual but reducing the risk of legal and denial-of-service attacks on a single organization, and that other organizations taking over ECC's role is not beneficial in the long run.
    • Alfredo said to first finish the already-promised zcashd retirement and ZSA, and Daira Emma answered that they are not proposing to apply this to NU7, but that it should be discussed now rather than later.
    • Arya said the risk of depending on a single organization can be addressed by other organizations getting familiar with Zebra, and Daira Emma said maintaining a fork is practically the only way to learn the code.
    • Nate noted that fork has two meanings, a source fork and a chain split, and Natalie worried that competing forks are unhealthy in the long run.
    • Daira Emma said they would talk with Josh to clear up misunderstandings, and that if people want forks not to be used for anything but testing, that can simply be stated.

This summary is based on the original meeting notes; check them for exact details.