Skip to content

Commit cc54a1e

Browse files
codyguntoncodygunton
authored
feat: Client IVC API (#10217)
- Establish API in purely virtual class - This is just a first pass. I will continue to work on this before showing dev rel and others to get buy-in. - Implement some API functions for ClientIVC: prove, verify, prove_and_verify - Support for constructing CIVC proof for input a single circuit - This is interpreted as a "compiletime stack" - Produces ECCVM and Translator proofs from dummy/empty data; future optimization could avoid. - Add `one_circuit` to CIVC to encode whether the MH part of the CIVC proof should be a hiding circuit (which takes a folding proof) or a proof for the single circuit. - Run almost all ACIR tests against ClientIVC - Previously only ran MegaHonk tests, which are not totally meaningful. - Four are skipped because they fail. These failures are expected to be superficial (see AztecProtocol/barretenberg#1164 and the references to it in the PR's new code). - fold_and_verify and mega honk flows go away in bb, but remain until bb.js alignment. - Delete large log file that should not be track (accounts for big negative diff).
1 parent 9106102 commit cc54a1e

31 files changed

+554
-2947
lines changed

‎.github/workflows/ci.yml

+5-5
Original file line numberDiff line numberDiff line change
@@ -444,7 +444,7 @@ jobs:
444444
timeout-minutes: 40
445445
run: earthly-ci --no-output ./+barretenberg-acir-tests-bb-ultra-honk
446446

447-
bb-acir-tests-bb-mega-honk:
447+
bb-acir-tests-bb-client-ivc:
448448
needs: [noir-build-acir-tests, build, configure]
449449
runs-on: ${{ needs.configure.outputs.username }}-x86
450450
if: needs.configure.outputs.barretenberg == 'true' || needs.configure.outputs.noir == 'true'
@@ -453,11 +453,11 @@ jobs:
453453
with: { ref: "${{ env.GIT_COMMIT }}" }
454454
- uses: ./.github/ci-setup-action
455455
with:
456-
concurrency_key: barretenberg-acir-tests-bb-mega-honk-x86
457-
- name: "BB Native Acir Tests (Megahonk)"
456+
concurrency_key: barretenberg-acir-tests-bb-client-ivc-x86
457+
- name: "BB Native Acir Tests (ClientIVC)"
458458
working-directory: ./barretenberg/
459459
timeout-minutes: 40
460-
run: earthly-ci --no-output ./+barretenberg-acir-tests-bb-mega-honk
460+
run: earthly-ci --no-output ./+barretenberg-acir-tests-bb-client-ivc
461461

462462
bb-acir-tests-sol:
463463
needs: [noir-build-acir-tests, build, configure]
@@ -969,7 +969,7 @@ jobs:
969969
- bb-acir-tests-bb
970970
- bb-acir-tests-bb-ultra-plonk
971971
- bb-acir-tests-bb-ultra-honk
972-
- bb-acir-tests-bb-mega-honk
972+
- bb-acir-tests-bb-client-ivc
973973
- bb-acir-tests-sol
974974
- bb-acir-tests-sol-honk
975975
- bb-acir-tests-bb-js

‎barretenberg/Earthfile

+7-9
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,9 @@ barretenberg-acir-tests-bb:
2727
ENV VERBOSE=1
2828

2929
# Fold and verify an ACIR program stack using ClientIvc
30-
RUN FLOW=fold_and_verify_program ./run_acir_tests.sh fold_basic
30+
RUN INPUT_TYPE=compiletime_stack FLOW=prove_and_verify_client_ivc ./run_acir_tests.sh fold_basic
3131
# Fold and verify an ACIR program stack using ClientIvc, then natively verify the ClientIVC proof.
32-
RUN FLOW=prove_then_verify_client_ivc ./run_acir_tests.sh fold_basic
32+
RUN INPUT_TYPE=compiletime_stack FLOW=prove_then_verify_client_ivc ./run_acir_tests.sh fold_basic
3333
# Fold and verify an ACIR program stack using ClientIvc, recursively verify as part of the Tube circuit and produce and verify a Honk proof
3434
RUN FLOW=prove_then_verify_tube ./run_acir_tests.sh fold_basic
3535
# Run 1_mul through native bb build, all_cmds flow, to test all cli args.
@@ -79,7 +79,7 @@ barretenberg-acir-tests-bb-ultra-honk:
7979
# Construct and verify a UltraHonk proof for a single program that recursively verifies a Honk proof
8080
RUN FLOW=prove_and_verify_ultra_honk ./run_acir_tests.sh verify_honk_proof
8181

82-
barretenberg-acir-tests-bb-mega-honk:
82+
barretenberg-acir-tests-bb-client-ivc:
8383
FROM ../build-images/+from-registry
8484

8585
COPY ./cpp/+preset-clang-assert/bin/bb /usr/src/barretenberg/cpp/build/bin/bb
@@ -92,12 +92,10 @@ barretenberg-acir-tests-bb-mega-honk:
9292
ENV TEST_SRC /usr/src/acir_artifacts
9393
ENV VERBOSE=1
9494

95-
# Construct and separately verify a MegaHonk proof for all acir programs
96-
RUN FLOW=prove_then_verify_mega_honk ./run_acir_tests.sh
97-
# Construct and verify a MegaHonk proof for a single arbitrary program
98-
RUN FLOW=prove_and_verify_mega_honk ./run_acir_tests.sh 6_array
99-
# Construct and verify a MegaHonk proof for all ACIR programs using the new witness stack workflow
100-
RUN FLOW=prove_and_verify_mega_honk_program ./run_acir_tests.sh
95+
# Construct and verify a ClientIVC proof for a single arbitrary program
96+
RUN FLOW=prove_and_verify_client_ivc ./run_acir_tests.sh 6_array
97+
# Construct and separately verify a ClientIVC proof for all acir programs
98+
RUN FLOW=prove_then_verify_client_ivc CLIENT_IVC_SKIPS=true ./run_acir_tests.sh
10199

102100
barretenberg-acir-tests-sol:
103101
FROM ../build-images/+from-registry

‎barretenberg/acir_tests/flows/fold_and_verify_program.sh

+2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
#!/bin/sh
22
set -eu
33

4+
# this flow is deprecated. currently it is bb.js only. for bb is is replaced by:
5+
# prove_and_verify --scheme client_ivc --input-type compiletime_stack
46
VFLAG=${VERBOSE:+-v}
57

68
$BIN fold_and_verify_program $VFLAG -c $CRS_PATH -b ./target/program.json

‎barretenberg/acir_tests/flows/prove_and_verify_client_ivc.sh

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/sh
2+
set -eu
3+
4+
VFLAG=${VERBOSE:+-v}
5+
INFLAG=${INPUT_TYPE=compiletime_stack}
6+
7+
FLAGS="$CRS_PATH -b ./target/program.json $VFLAG --scheme client_ivc -c --input_type $INFLAG"
8+
9+
$BIN prove_and_verify $FLAGS

‎barretenberg/acir_tests/flows/prove_and_verify_mega_honk.sh

+5
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,9 @@ set -eu
33

44
VFLAG=${VERBOSE:+-v}
55

6+
# this flow is deprecated. currently it is bb.js only. for bb is is replaced by:
7+
# prove_and_verify --scheme client_ivc --input-type compiletime_stack
8+
# NB: In general, it is not meaningful to produce a MegaHonk proof an its own since
9+
# the MegaHonk proof does not attest to the correctness of every possible kind
10+
# of gate that could appear in a Mega execution trace.
611
$BIN prove_and_verify_mega_honk $VFLAG -c $CRS_PATH -b ./target/program.json

‎barretenberg/acir_tests/flows/prove_and_verify_mega_honk_program.sh

+5
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,9 @@ set -eu
33

44
VFLAG=${VERBOSE:+-v}
55

6+
# this flow is deprecated. currently it is bb.js only. for bb is is replaced by:
7+
# prove_and_verify --scheme client_ivc --input-type compiletime_stack
8+
# NB: In general, it is not meaningful to produce a MegaHonk proof an its own since
9+
# the MegaHonk proof does not attest to the correctness of every possible kind
10+
# of gate that could appear in a Mega execution trace.
611
$BIN prove_and_verify_mega_honk_program $VFLAG -c $CRS_PATH -b ./target/program.json

‎barretenberg/acir_tests/flows/prove_then_verify_client_ivc.sh

+5-3
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@ set -eu
33

44
VFLAG=${VERBOSE:+-v}
55
BFLAG="-b ./target/program.json"
6-
FLAGS="-c $CRS_PATH $VFLAG"
6+
INFLAG=${INPUT_TYPE=compiletime_stack}
77

8-
$BIN client_ivc_prove_output_all $FLAGS $BFLAG
9-
$BIN verify_client_ivc $FLAGS
8+
FLAGS="--scheme client_ivc -c $CRS_PATH $VFLAG"
9+
10+
$BIN prove $FLAGS $BFLAG --input_type $INFLAG
11+
$BIN verify $FLAGS

‎barretenberg/acir_tests/flows/prove_then_verify_tube.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ mkdir -p ./proofs
55

66
VFLAG=${VERBOSE:+-v}
77

8-
$BIN client_ivc_prove_output_all $VFLAG -c $CRS_PATH -b ./target/program.json
8+
$BIN prove --scheme client_ivc --input_type compiletime_stack $VFLAG -c $CRS_PATH -b ./target/program.json
99
$BIN prove_tube -k vk -p proof -c $CRS_PATH $VFLAG
1010
$BIN verify_tube -k vk -p proof -c $CRS_PATH $VFLAG
1111

‎barretenberg/acir_tests/flows/prove_tube.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,5 +5,5 @@ VFLAG=${VERBOSE:+-v}
55
BFLAG="-b ./target/program.json"
66
FLAGS="-c $CRS_PATH $VFLAG"
77

8-
$BIN client_ivc_prove_output_all $VFLAG -c $CRS_PATH -b ./target/program.json
8+
$BIN prove --scheme client_ivc --input_type compiletime_stack $VFLAG -c $CRS_PATH -b ./target/program.json
99
$BIN prove_tube -k vk -p proof $FLAGS

‎barretenberg/acir_tests/run_acir_tests.sh

+16
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ trap handle_sigchild SIGCHLD
1414
BIN=${BIN:-../cpp/build/bin/bb}
1515
FLOW=${FLOW:-prove_and_verify}
1616
HONK=${HONK:-false}
17+
CLIENT_IVC_SKIPS=${CLIENT_IVC_SKIPS:-false}
1718
CRS_PATH=~/.bb-crs
1819
BRANCH=master
1920
VERBOSE=${VERBOSE:-}
@@ -57,6 +58,21 @@ if [ "$HONK" = true ]; then
5758
SKIP_ARRAY+=(single_verify_proof double_verify_proof double_verify_nested_proof)
5859
fi
5960

61+
if [ "$CLIENT_IVC_SKIPS" = true ]; then
62+
# At least for now, skip folding tests that fail when run against ClientIVC.
63+
# This is not a regression--folding was not being properly tested.
64+
# TODO(https://github.com/AztecProtocol/barretenberg/issues/1164): Resolve this
65+
# The reason for failure is that compile-time folding, as initially conceived, is
66+
# only supported by ClientIVC through hacks. ClientIVC in Aztec is ultimately to be
67+
# used through runtime folding, since the kernels that are needed are detected and
68+
# constructed at runtime in Aztec's typescript proving interface. ClientIVC appends
69+
# folding verifiers and does databus and Goblin merge work depending on its inputs,
70+
# detecting which circuits are Aztec kernels. These tests may simple fail for trivial
71+
# reasons, e.g. because the number of circuits in the stack is odd.
72+
SKIP_ARRAY+=(fold_basic_nested_call fold_fibonacci fold_numeric_generic_poseidon ram_blowup_regression)
73+
fi
74+
75+
6076
function test() {
6177
cd $1
6278

‎barretenberg/cpp/docs/src/sumcheck-outline.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -195,9 +195,9 @@ Observe that \f$ G \f$ has several important properties
195195
- The coefficients of \f$ G \f$ are independent and uniformly distributed.
196196
- Evaluations of \f$ G \f$ at \f$ \vec \ell \in \{0,1\}^d\f$ and related Sumcheck Round Univariates are efficiently computable.
197197

198-
The first two properties imply that the evaluations of Sumcheck Round Univariates for \f$G\f$ are independent and uniformly distributed. We call them Libra Round Univarites.
198+
The first two properties imply that the evaluations of Sumcheck Round Univariates for \f$G\f$ are independent and uniformly distributed. We call them Libra Round Univariates.
199199

200-
Consider Round Univariates for \f$ \tilde{F} + \texttt{libra_challenge}\cdot G\f$ which are the sums of the Sumcheck Round Univariates for \f$ \tilde{F} \f$ and Libra Round Univarites multiplied by the challenge.
200+
Consider Round Univariates for \f$ \tilde{F} + \texttt{libra_challenge}\cdot G\f$ which are the sums of the Sumcheck Round Univariates for \f$ \tilde{F} \f$ and Libra Round Univariates multiplied by the challenge.
201201
The fact that the degrees of Libra Round Univariates are big enough (i.e. \f$ \tilde{D}\geq D \f$) and that their evaluations are random imply that the evaluations \f$ \tilde{S}^i(0),\ldots,\tilde{S}^i(\tilde D)\f$ defined in [Compute Round Univariates](#ComputeRoundUnivariates) are now masked by the evaluations of Libra Round Univariates. These evaluations are described explicitly [below](#LibraRoundUnivariates).
202202

203203
### Example {#LibraPolynomialExample}

‎barretenberg/cpp/src/barretenberg/bb/acir_format_getters.hpp

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#pragma once
2+
#include "barretenberg/bb/config.hpp"
3+
#include "barretenberg/bb/file_io.hpp"
4+
#include "barretenberg/bb/get_bytecode.hpp"
5+
#include "barretenberg/dsl/acir_format/acir_format.hpp"
6+
#include "barretenberg/dsl/acir_format/acir_to_constraint_buf.hpp"
7+
8+
namespace bb {
9+
10+
acir_format::WitnessVector get_witness(std::string const& witness_path)
11+
{
12+
auto witness_data = get_bytecode(witness_path);
13+
return acir_format::witness_buf_to_witness_data(witness_data);
14+
}
15+
16+
acir_format::AcirFormat get_constraint_system(std::string const& bytecode_path, bool honk_recursion)
17+
{
18+
auto bytecode = get_bytecode(bytecode_path);
19+
return acir_format::circuit_buf_to_acir_format(bytecode, honk_recursion);
20+
}
21+
22+
acir_format::WitnessVectorStack get_witness_stack(std::string const& witness_path)
23+
{
24+
auto witness_data = get_bytecode(witness_path);
25+
return acir_format::witness_buf_to_witness_stack(witness_data);
26+
}
27+
28+
std::vector<acir_format::AcirFormat> get_constraint_systems(std::string const& bytecode_path, bool honk_recursion)
29+
{
30+
auto bytecode = get_bytecode(bytecode_path);
31+
return acir_format::program_buf_to_acir_format(bytecode, honk_recursion);
32+
}
33+
34+
} // namespace bb

‎barretenberg/cpp/src/barretenberg/bb/api.hpp

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
#pragma once
2+
#include <filesystem>
3+
4+
namespace bb {
5+
6+
class API {
7+
public:
8+
struct Flags {
9+
std::optional<std::string> output_type; // bytes, fields, bytes_and_fields, fields_msgpack
10+
std::optional<std::string> input_type; // compiletime_stack, runtime_stack
11+
};
12+
13+
virtual void prove(const Flags& flags,
14+
const std::filesystem::path& bytecode_path,
15+
const std::filesystem::path& witness_path,
16+
const std::filesystem::path& output_dir) = 0;
17+
18+
virtual bool verify(const Flags& flags,
19+
const std::filesystem::path& proof_path,
20+
const std::filesystem::path& vk_path) = 0;
21+
22+
virtual bool prove_and_verify(const Flags& flags,
23+
const std::filesystem::path& bytecode_path,
24+
const std::filesystem::path& witness_path) = 0;
25+
26+
virtual void gates(const Flags& flags,
27+
const std::filesystem::path& bytecode_path,
28+
const std::filesystem::path& witness_path) = 0;
29+
30+
virtual void contract(const Flags& flags,
31+
const std::filesystem::path& output_path,
32+
const std::filesystem::path& vk_path) = 0;
33+
34+
virtual void to_fields(const Flags& flags,
35+
const std::filesystem::path& proof_path,
36+
const std::filesystem::path& vk_path,
37+
const std::filesystem::path& output_path) = 0;
38+
};
39+
} // namespace bb

0 commit comments

Comments
 (0)