Skip to content

Latest commit

 

History

History
138 lines (94 loc) · 9.02 KB

fip-0059.md

File metadata and controls

138 lines (94 loc) · 9.02 KB
fip title author discussions-to status type category created
0059
Synthetic PoRep
@Kubuxu @Luca @Rosario Gennaro @Nicola @Irene @nemo
Final
Technical
Core
10 feb 2023

FIP-0059: Synthetic PoRep

Simple Summary

This proposal presents a new PoRep protocol (Synthetic PoRep) that reduces the size of the temporary data stored between PreCommit and ProveCommit (150 epochs) from ~400GiB to ~25GiB, with no impact on security.

Abstract

Synthetic PoRep achieves reduction in used up space by reducing the set of challenges that might be chosen during the interactive Commit step from all possible challenges to some predetermined number that is feasible to precompute.

  • A Storage Provider can complete the challenge generation and vanilla proof computation before performing PreCommit on-chain thus removing layers data before the sector is pre-committed on-chain;
  • The GPU cost for SNARK generation during Commit is not significantly increased.

Motivation

The current interactive PoRep protocol (PreCommit + ProveCommit) requires to seal and keep a buffer of 12 layers (11 SDR layers + 1 data layer) between PreCommit and ProveCommit. This is the cost that we are targeting to reduce with this FIP.

Specification

Protocol Overview

Differences between currently deployed PoRep and Synthetic PoRep are limited to challenge generation and additional capabilities for the Storage Provider.

  1. Starting point
    1. We assume there is a sector S for which the Storage Provider completed PreCommit1 and PreCommit2 computations.
    2. This sector is not listed on-chain.
    3. The Storage Provider possesses knowledge or CommR and CommD of that sector (acquired in PreCommit2 step) and the layers needed to generate CommR
  2. Storage Provider generates “Synthetic” challenges from CommR
    1. Based on the CommR, the Storage Provider generates a list of N_synchallenges
    2. The Storage Provider computes responses for all the N_syn challenges, which take the form of N_syn vanilla proofs and saves them for future use.
  3. Storage Provider can remove layers data
    1. As the Storage Provider knows responses to all possible challenges that will be asked in the interactive step, he can remove the layers data which is needed to respond to challenges.
  4. Storage Provider publishes “PreCommitsSector”
    1. Using the same flow as today Storage Provider submits the sector for PreCommit.
    2. This establishes when in the future the randomness for interactive response will be known (PreCommitChallengeDelay)
  5. Storage Provider generates and publishes ProveCommitSectors proof
    1. Storage Provider waits PreCommitChallengeDelay (150 epochs).
    2. The randomness revealed at PreCommitEpoch+PreCommitChallengeDelay selects N_verified challenges to be verified on-chain from the N_syn challenges generate in step 2.
    3. Storage Provider takes the N_verified vanilla proofs which were generated earlier corresponding to selected challenges and computes SNARK proofs of these challenges.
    4. Storage Provider publishes the ProveCommit either in individual or aggregated form.
  6. Chain verifies proof
    1. Using interactive randomness as a seed chain generates N_verified challenges by selecting N_verified indices out of N_syn and computing them.
    2. Generated challenges are fed into proof verification.

Actor changes

  • Add two new proof types to the list of proof types that can be used when pre-committing a new sector
    • RegisteredSealProof_SynthStackedDrg32GiBV1
    • RegisteredSealProof_SynthStackedDrg64GiBV1
  • The allowable delay of the new proof types is the same as the StackedDRG proof types V1.1;
  • No changes in PreCommitSector and ProveCommitSector methods;
  • ProveCommitSector passes the new proof type to the proof verification syscall;

Proof changes

  • Add two new proof types to the list of proof types that can be used when pre-committing a new sector

    • RegisteredSealProof::FeatureSynthStackedDrgWindow32GiBV1_2
    • RegisteredSealProof::FeatureSynthStackedDrgWindow64GiBV1_2

  • Related constants

    • N_syn set to $2^{18}$;
    • N_verified set to 176 (same as current *N_porep_challenges) ;

  • New challenge generation functions.

    Note that we have determined using ChaCha20 in the place of Sha256 is preferred.

  • Proof construction and verification can use the same functions as today.

Synthetic PoRep, while likely enabled by default after it's available, is an optional feature that can be opt-out for those interested. The previously available proofs types can be used to continue the PoRep behaviour.

Rationale

Synthetic PoRep is a PoRep optimization which has basically no downside with respect to the status. Indeed, it would allow for more than 90% storage cost savings between PreCommit and ProveCommit. Additionally, we would have

  • No impact on the current on-chain flow,
  • No need of new Trusted Setup,
  • No proving overhead on the StorageProvider side,
  • No impact on PoRep security.

Note that in the design of Synthetic PoRep we considered the case of increasing the number of PoRep challenges (N_verified) in order to be able to have higher storage saving thanks to a reduced number of synthetic challenges (N_syn smaller). However since this requires a new trusted setup and more proving, we decided to go with the proposed parameters that do not change the number of PoRep challenges and still allows significant saving.

An alternative design we considered to reduce the storage between PreCommit and ProveCommit is Non-Interactive PoRep (see for example discussion #603 ). NI-PoRep allows to get a 100% storage cost saving and has other benefits (for example, removing the need of PreCommitDeposit) but has a ~8.5x proving overhead that we consider prohibitive for now. Note that implementing Synthetic PoRep now does not prevent us from implementing NI-PoRep in the future, once we have solved/reduced the proving overhead problem.

Backwards Compatibility

Synthetic PoRep would become a new proof type with the same on-chain flow as current PoRep.

Test Cases

Will be included with implementation to be presented.

Security Considerations

In the current PoRep protocol if 3.9% or more of nodes in a layer are wrongly encoded by the SP, then the ProveCommit step, where 176 random nodes are checked, will fail with large probability (larger than $1-2^{10}$). In the new protocol, the SP first samples a set of N_syn positions and then proceeds to sample the 176 random nodes from there. In order to be able to keep the same security as before, we need that the distribution of errors in the synthetic challenge set is as close as possible to the original distribution of errors in the layer. However the adversary can try different sets to get one where the fraction of wrongly encoded nodes is smaller than 3.9%. We consider an adversary that wants to pass with probability $2^{-9}$, who then needs a set of synthetic challenges where at least a fraction $q$ are goods (correctly encoded) with $q$ = 0.9651 ($q^{176} ≤ 2^{-9)}$). We can show that if N_syn is large enough, then this is not possible in practice. More in details: the probability that the number of correctly encoded nodes in the sythetic set is larger or equal to $q$N_syn is given by the binomial probability:

$$ P = \sum_{i=0}^{qN_{syn}} \binom{N_{syn}}{i}p^i (1-p)^{N_{syn}-i} \text{ with } p ≤ 0.961 $$

and with N_syn = $2^{18}$, we have that $P < 2^{-92}$.

Incentive Considerations

This proposal does not affect the current incentive system of the Filecoin network.

Product Considerations

This proposal reduces the hardware usage for the PoRep and therefore represents a cost saving opportunity for Storage Providers.

Moreover, Synthetic PoRep can be also beneficial in terms of sealing throughput. Today SPs need to have ~500 GiB SSD for sealing a sector. After PC1 and PC2 this storage capacity is mostly filled with the 11 layers of SDR which need to stay there for 150 epochs, before being proved at ProveCommit. With Synthetic PoRep, only a small buffer of less than 25GiB need to be kept around until ProveCommit. This means that with less than 5% more SSD storage available, SPs can start sealing a new sector right after completing PC1 and PC2 of the old sector, without need to wait ProveCommit to be over. Note that, assuming PC1 takes almost 3h and we have 150 epochs between PreCommit and ProveCommit, this result in a possible 25% additional sealing throughput.

Implementations

See the tracking issue filecoin-project/lotus#10984.

Copyright Waiver

Copyright and related rights waived via CC0.