chore: Move barretenberg to top of repo. Make circuits build off barretenberg build. (#2221)

This moves barretenberg out of circuits project, to the top level of the
repo, and cleans up a lot of build-system stuff.

## BB and Circuits
* Circuits expects barretenberg to still be where it was before (root of
its repo), but it has a symlink now to the top level.
* Docker scoping is handled by ingesting barretenberg from prior build,
which outputs both the source code, and libraries needed for circuits
(libbarretenberg, libenv, libwasi).
* Circuits not longer builds barretenberg itself.
* barretenberg now does a formatting check over it's code as part of
build. `./format.sh check`
* Remove some refs to openmp in containers as we're phasing it out....
* Remove some cmake around installing barretenberg, as this isn't needed
anymore.

## Build System
* Remove hacks around e2e running locally. Use now official
`cond_run_script` script.
* Remove superfluous scripts and normalise around the
`cond_spot_run_script` and `remote_runner` script.
* `cond_spot_run_build` and `cond_spot_run_test` both implemented in
terms of above.
* Now test runs are handled by `remote_runner` they perform a full
checkout of code and fit in the general pattern of the build system. No
more hacky arbitrary script folder to remote copies.
* Added `yarn-project-base` as dep in `build-manifest.json` to all TS
projects so rebuilds are correctly computed.
* Improved performance of `dependencies` command in `query_manifest` and
also `rebuildPatterns`.
* `query_manifest` outputs absolute paths by default, meaning can remove
various assumptions in other scripts as to current working directory.

Author: charlielye

.circleci/config.yml

@@ -5,8 +5,8 @@ on:
     branches:
       - master
     paths:
-      - 'circuits/cpp/barretenberg/**'
-      - '!circuits/cpp/barretenberg/.gitrepo'
+      - "barretenberg/**"
+      - "!barretenberg/.gitrepo"
 
 jobs:
   build:
@@ -23,7 +23,7 @@ jobs:
         run: |
           # we push using git subrepo (https://github.com/ingydotnet/git-subrepo)
           # with some logic to recover from squashed parent commits
-          SUBREPO_PATH=circuits/cpp/barretenberg
+          SUBREPO_PATH=barretenberg
           # identify ourselves, needed to commit
           git config --global user.name AztecBot
           git config --global user.email tech@aztecprotocol.com

.github/workflows/mirror_barretenberg_repo.yml

@@ -49,21 +49,21 @@ jobs:
 
       - name: Compile Barretenberg
         run: |
-          cd circuits/cpp/barretenberg/cpp
+          cd barretenberg/cpp
 
           cmake --preset default -DCMAKE_BUILD_TYPE=RelWithAssert
           cmake --build --preset default --target bb
 
       - name: Tar and GZip bb Binary (Ubuntu)
-        working-directory: circuits/cpp/barretenberg/cpp/build/bin
+        working-directory: barretenberg/cpp/build/bin
         run: tar -cvzf barretenberg-x86_64-linux-gnu.tar.gz bb
 
       - name: Upload artifacts
         uses: actions/upload-artifact@v2
         with:
           name: release-linux
           path: |
-            ./circuits/cpp/barretenberg/cpp/build/bin/barretenberg-x86_64-linux-gnu.tar.gz
+            ./barretenberg/cpp/build/bin/barretenberg-x86_64-linux-gnu.tar.gz
 
   build-wasm-ts:
     name: Build WASM and deploy to TS
@@ -100,21 +100,21 @@ jobs:
           sudo apt -y update && sudo apt -y install yarn
       - name: Install WASI-SDK
         run: |
-          cd circuits/cpp/barretenberg/cpp
+          cd barretenberg/cpp
 
           ./scripts/install-wasi-sdk.sh
 
       - name: Compile Typescript
         run: |
-          cd circuits/cpp/barretenberg/ts
+          cd barretenberg/ts
           yarn install && yarn && yarn build
 
       - name: Tar and GZip barretenberg.wasm
-        working-directory: circuits/cpp/barretenberg/cpp/build-wasm/bin
+        working-directory: barretenberg/cpp/build-wasm/bin
         run: tar -cvzf barretenberg.wasm.tar.gz barretenberg.wasm
 
       - name: Tar and GZip acvm_backend.wasm
-        working-directory: circuits/cpp/barretenberg/cpp/build-wasm/bin
+        working-directory: barretenberg/cpp/build-wasm/bin
         run: tar -cvzf acvm_backend.wasm.tar.gz acvm_backend.wasm
 
       - name: Setup Node.js
@@ -126,7 +126,7 @@ jobs:
       - name: Deploy Typescript to NPM
         if: github.event.inputs.tag != 'nightly' && github.event.inputs.tag != '' # Do not deploy to npm if it is a nightly build
         run: |
-          cd circuits/cpp/barretenberg/ts
+          cd barretenberg/ts
           yarn deploy
         env:
           NODE_AUTH_TOKEN: ${{secrets.NPM_TOKEN}}
@@ -136,8 +136,8 @@ jobs:
         with:
           name: release-wasm
           path: |
-            ./circuits/cpp/barretenberg/cpp/build-wasm/bin/barretenberg.wasm.tar.gz
-            ./circuits/cpp/barretenberg/cpp/build-wasm/bin/acvm_backend.wasm.tar.gz
+            ./barretenberg/cpp/build-wasm/bin/barretenberg.wasm.tar.gz
+            ./barretenberg/cpp/build-wasm/bin/acvm_backend.wasm.tar.gz
 
   build-mac:
     name: Build on Mac (${{ matrix.target }})
@@ -163,20 +163,20 @@ jobs:
 
       - name: Compile Barretenberg (x86_64)
         if: matrix.target == 'x86_64-apple-darwin'
-        working-directory: circuits/cpp/barretenberg/cpp
+        working-directory: barretenberg/cpp
         run: |
           cmake --preset default -DCMAKE_BUILD_TYPE=RelWithAssert
           cmake --build --preset default --target bb
 
       - name: Compile Barretenberg (ARM)
         if: matrix.target == 'aarch64-apple-darwin'
-        working-directory: circuits/cpp/barretenberg/cpp
+        working-directory: barretenberg/cpp
         run: |
           cmake --toolchain ./cmake/toolchains/aarch64-darwin.cmake --preset default -DCMAKE_BUILD_TYPE=RelWithAssert
           cmake --build --preset default --target bb
 
       - name: Package barretenberg artifact
-        working-directory: circuits/cpp/barretenberg/cpp/build/bin
+        working-directory: barretenberg/cpp/build/bin
         run: |
           mkdir dist
           cp ./bb ./dist/bb
@@ -186,7 +186,7 @@ jobs:
         uses: actions/upload-artifact@v3
         with:
           name: barretenberg-${{ matrix.target }}
-          path: ./circuits/cpp/barretenberg/cpp/build/bin/barretenberg-${{ matrix.target }}.tar.gz
+          path: ./barretenberg/cpp/build/bin/barretenberg-${{ matrix.target }}.tar.gz
           retention-days: 3
 
   release:

.github/workflows/publish-bb.yml

@@ -1,18 +1,18 @@
 [submodule "legacy-barretenberg-build-system"]
-	path = circuits/cpp/barretenberg/build-system
+	path = barretenberg/build-system
 	url = https://github.com/AztecProtocol/build-system
 [submodule "l1-contracts/lib/openzeppelin-contracts"]
 	path = l1-contracts/lib/openzeppelin-contracts
 	url = https://github.com/openzeppelin/openzeppelin-contracts
 [submodule "l1-contracts/lib/forge-std"]
 	path = l1-contracts/lib/forge-std
 	url = https://github.com/foundry-rs/forge-std
-[submodule "circuits/cpp/barretenberg/sol/lib/forge-std"]
-	path = circuits/cpp/barretenberg/sol/lib/forge-std
+[submodule "barretenberg/sol/lib/forge-std"]
+	path = barretenberg/sol/lib/forge-std
 	url = https://github.com/foundry-rs/forge-std
-[submodule "circuits/cpp/barretenberg/sol/lib/solidity-stringutils"]
-	path = circuits/cpp/barretenberg/sol/lib/solidity-stringutils
+[submodule "barretenberg/sol/lib/solidity-stringutils"]
+	path = barretenberg/sol/lib/solidity-stringutils
 	url = https://github.com/Arachnid/solidity-stringutils
-[submodule "circuits/cpp/barretenberg/sol/lib/openzeppelin-contracts"]
-	path = circuits/cpp/barretenberg/sol/lib/openzeppelin-contracts
+[submodule "barretenberg/sol/lib/openzeppelin-contracts"]
+	path = barretenberg/sol/lib/openzeppelin-contracts
 	url = https://github.com/OpenZeppelin/openzeppelin-contracts

.gitmodules

@@ -1,2 +1,2 @@
-/circuits/cpp/barretenberg/ts
-/circuits/cpp/barretenberg/foundation
+/barretenberg/ts
+/barretenberg/foundation

.ignore

@@ -1,5 +1,5 @@
 {
   ".": "0.7.0",
-  "circuits/cpp/barretenberg": "0.7.0",
-  "circuits/cpp/barretenberg/ts": "0.7.0"
+  "barretenberg": "0.7.0",
+  "barretenberg/ts": "0.7.0"
 }

.release-please-manifest.json

@@ -28,7 +28,7 @@ All packages need to be included in the [build manifest](`build_manifest.json`),
 
 ## Debugging
 
-Logging goes through the [`info` and `debug`](circuits/cpp/barretenberg/cpp/src/barretenberg/common/log.hpp) functions in C++, and through the [DebugLogger](yarn-project/foundation/src/log/debug.ts) module in Typescript. To see the log output, set a `DEBUG` environment variable to the name of the module you want to debug, to `aztec:*`, or to `*` to see all logs.
+Logging goes through the [`info` and `debug`](barretenberg/cpp/src/barretenberg/common/log.hpp) functions in C++, and through the [DebugLogger](yarn-project/foundation/src/log/debug.ts) module in Typescript. To see the log output, set a `DEBUG` environment variable to the name of the module you want to debug, to `aztec:*`, or to `*` to see all logs.
 
 ## Releases