Skip to content

Commit 4d0da83

Browse files
jennijujujennijuju
authored
Merge pull request #72 from filecoin-project/provecommitaggregate-ipp
Add ProveCommitSectorAggregated method to reduce on-chain congestion
2 parents 4ca907d + 78581dd commit 4d0da83

File tree

2 files changed

+171
-0
lines changed

2 files changed

+171
-0
lines changed

‎FIPS/fip-0013.md

+170
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
---
2+
fip: "0013"
3+
title: Add ProveCommitSectorAggregated method to reduce on-chain congestion
4+
author: Nicola (@nicola), Add others
5+
discussions-to: https://github.com/filecoin-project/FIPs/issues/50
6+
status: Draft
7+
type: Technical
8+
category: Core
9+
created: 2021-17-02
10+
spec-sections:
11+
- section-systems.filecoin_mining.sector.lifecycle
12+
13+
---
14+
15+
## Simple Summary
16+
17+
<!--"If you can't explain it simply, you don't understand it well enough." Provide a simplified and layman-accessible explanation of the FIP.-->
18+
Add a method for a miner to submit several sector prove commit messages in a single one.
19+
20+
## Abstract
21+
22+
<!--A short (~200 word) description of the technical issue being addressed.-->
23+
On-chain proofs scale linearly with network growth. This leads to (1) blockchain being at capacity most of the time leading to high base fee, (2) chain capacity is currently limiting network growth.
24+
25+
The miner `ProveCommitSector` method only supports committing a single sector at a time.
26+
It is both frequently executed and expensive.
27+
This proposal adds a `ProveCommitSectorAggregated` method to amortize some of the costs across multiple sectors, removes some redundant but costly checks and drastically reduces per-sector proof size and verification times taking advantage of a novel cryptography result.
28+
29+
30+
## Change Motivation
31+
32+
<!--The motivation is critical for FIPs that want to change the Filecoin protocol. It should clearly explain why the existing protocol specification is inadequate to address the problem that the FIP solves. FIP submissions without sufficient motivation may be rejected outright.-->
33+
34+
The miner `ProveCommitSector` method only supports committing a single sector at a time.
35+
It's one of the two highest frequency methods observed on the chain at present (the other being `PreCommitSector`).
36+
High-growth miners commit sectors at rates exceeding 1 per epoch.
37+
It's also a relatively expensive method, with multiple internal sends and state loads and stores.
38+
39+
Aggregated proof verification allows for more sector commitments to be proven in less time which will reduce processing time and therefore gas cost per prove commit.
40+
41+
In addition to this primary optimization, there are several secondary opportunities for improved processing time and gas cost related to batch processing many prove commits at once:
42+
- Using a bitfield to specify sector numbers in the `ProveCommitSector` parameters could reduce message size
43+
- PreCommits loading overhead can be done once per batch
44+
- Power actor claim validation can be done once per batch
45+
- Market actor `ComputeDataCommitment` calls can be batched
46+
47+
Additionally the `ProveCommitSectorAggregated` method can do away with the temporary storage and cron-batching currently used to verify individual prove commits. This opens further cost reduction opportunities:
48+
- PreCommit info can be loaded once per prove commit rather than once for recording, and again for batch verifying in call back
49+
- Sectors proven through `ProveCommitSectorAggregated` will not need to be stored and read from the power actors `ProofValidationBatch` map
50+
51+
If miner operators implemented a relatively short aggregation period (a day), the `ProveCommitAggregated` method has the potential to reduce gas costs for:
52+
53+
- State operations: some of the costs listed above can be amortized across multiple sectors
54+
- Proof verification: the gas used for proofs can scale sub-linearly with the growth of the network using a novel proof aggregation scheme.
55+
56+
## Specification
57+
58+
<!--The technical specification should describe the syntax and semantics of any new feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Filecoin implementations. -->
59+
60+
### Actor changes
61+
62+
Add a new method `ProveCommitSectorAggregated` which supports a miner prove-committing a number of sectors all at once.
63+
The parameters for this method are a list of prove-commit infos:
64+
65+
```
66+
type ProveCommitSectorAggregatedParams {
67+
SectorsNumbers bitfield.BitField
68+
}
69+
```
70+
71+
Semantics will be similar to those of `ProveCommitSector` with the following proposed changes:
72+
73+
- Sectornumber bitfield in place of a single abi.SectorNumber in parameters
74+
- Aggregate proof in place of single porep proof in parameters
75+
- MaxProveCommitSize parameter will change
76+
- Minimum and maximum number of sectors proven will be enforced
77+
- PreCommitInfos read in batch
78+
- SealVerifyInfos constructed in batch
79+
- Market actor ComputeDataCommittment method changed to compute batches of commDs
80+
- Gas cost for verification will be updated and now computed as a function of the number of sectors aggregated
81+
- No storing proof info in power actor for batched verification at the end of the epoch.
82+
- `ProveCommitSectorAggregated` will call into a new runtime syscall `AggregateVerifySeals` in place of power actor `BatchVerifySeals` call.
83+
- ConfirmSectorProofsValid logic will be copied over to the second half of `ProveCommitSectorAggregated`.
84+
85+
86+
#### Failure handling
87+
88+
- If any predicate on the parameters fails, the call aborts (no change is persisted).
89+
- If the miner has insufficient balance for all prove-commit pledge, the call aborts.
90+
91+
#### Scale and limits
92+
93+
The number of sectors that may be pre-committed in a single aggregation starts from 50 (TODO) and a maximum of 819 (TODO)
94+
95+
### Proof scheme changes
96+
97+
Protocol Labs research teams in collaboration with external researchers have worked on an improvement of the Inner Product Pairing result from [Bunz et al.](https://eprint.iacr.org/2019/1177.pdf).
98+
99+
In high level, the idea is the following: given some Groth16 proofs, one can generate a single proof that these were correctly aggregated.
100+
101+
Note the this type of aggregation sits on top of the existing SNARKs that we do. In other words, there is no need for a new trusted setup.
102+
103+
A more detailed technical report on the new constructions can be found here (TODO).
104+
105+
#### Proofs API
106+
107+
TODO
108+
109+
#### Trusted Setup change
110+
111+
## Design Rationale
112+
113+
The existing `ProveCommitSector` method will not become redundant, since aggregation of smaller batches may not be efficient in terms of gas cost (proofs too big or too expensive to verify).
114+
The method is left intact to support smooth operation through the upgrade period.
115+
116+
### Failure handling
117+
118+
Aborting on any precondition failure is chosen for simplicity.
119+
There is no good reason for submitting an invalid pre-commitment, so this should never happen for correctly-functioning miners.
120+
Aborting on failure will provide a clear indication that something is wrong, which might be overlooked by an operator otherwise.
121+
122+
An alternative could be to allow sectors in the aggregation to succeed or fail independently.
123+
In this case, the method should return a value indicating which sectors succeeded and which failed.
124+
This would complicate both the actor and node implementations somewhat, though not unduly.
125+
126+
### Scale and limits
127+
128+
The bound on aggregation size is not intended to actually constrain a miner's behaviour, but limit the impact of potentially mistaken or malicious behaviour.
129+
A miner may submit multiple batches in a single epoch to grow faster.
130+
131+
132+
## Backwards Compatibility
133+
134+
This proposal introduces a new exported miner actor method, and thus changes the exported method API.
135+
While addition of a method may seem logically backwards compatible, it is difficult to retain the precise behaviour of an invocation to the (unallocated) method number before the method existed.
136+
Thus, such changes must be delivered through a major version upgrade to the actors.
137+
138+
This proposal retains the existing non-batch `ProveCommitSector` method, so mining operations need not change workflows due to this proposal (but _should_ in order to enjoy the reduced gas costs).
139+
140+
## Test Cases
141+
142+
Test cases will accompany implementation.
143+
144+
## Security Considerations
145+
146+
All significant implementation changes carry risk.
147+
148+
The cryptography used is novel and is audited by TODO.
149+
150+
The trusted setup used is the Filecoin Powers of Tau and the ZCash Powers of Tau (TODO: add links)
151+
152+
## Incentive Considerations
153+
154+
This proposal amortizes per-sector costs for high-growth miners, providing an economy of scale. This same economy cannot be enjoyed by miners growing more slowly.
155+
156+
This may present a minor incentive against splitting a single physical operation into many miner actors (aka Sybils).
157+
158+
## Product Considerations
159+
160+
This proposal reduces the aggregate cost of committing new sectors to the Filecoin network.
161+
162+
This will reduce miner costs overall, as well as reduce contention for chain transaction bandwidth that can crowd out other messages.
163+
164+
## Implementation
165+
166+
Implementation to follow discussion and acceptance of this proposal.
167+
168+
## Copyright
169+
170+
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -58,3 +58,4 @@ This improvement protocol helps achieve that objective for all members of the Fi
5858
|[0010](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0010.md) | Off-Chain Window PoSt Verification |@Stebalien, @anorth |Accepted |
5959
|[0011](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0011.md) | Remove reward auction from reporting consensus faults |@Kubuxu |Draft |
6060
|[0012](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0012.md) | DataCap Top up for FIL+ Client Addresses |@dshoy, @jnthnvctr, @zx |Draft |
61+
|[0013](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0013.md) | Add ProveCommitSectorAggregated method to reduce on-chain congestion | @nicola, Add others |Draft |

0 commit comments

Comments
 (0)