Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: upgarde all docker images to use rocky linux 8 (#9587) #9598

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/licenserc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ header:
- '.github/'
- 'docs/'
- 'tests/testsdata/'
- 'release-centos7-llvm/dockerfiles/'
- 'release-linux-llvm/dockerfiles/'
- '**/.gitignore'
- '**/*.md'
- '**/*.json'
Expand Down
9 changes: 4 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ callgrind.out.*
# ignore build files
CMakeCache.txt
CMakeFiles
Makefile
cmake_install.cmake
CTestTestfile.cmake
*.a
Expand Down Expand Up @@ -78,8 +77,8 @@ debian/*.substvars
*.pb.h

# TiFlash built and test files
release-centos7/tiflash
release-centos7/build-release
release-rocky8/tiflash
release-rocky8/build-release
release-darwin/tiflash
release-darwin/build-release

Expand All @@ -91,5 +90,5 @@ tests/docker/log
CoverageReport

# installation and sysroot path
release-centos7-llvm/tiflash
release-centos7-llvm/build-release
release-linux-llvm/tiflash
release-linux-llvm/build-release
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,9 @@ sudo pacman -S clang lld libc++ libc++abi compiler-rt openmp lcov cmake ninja cu
</details>

<details>
<summary><b>CentOS 7</b></summary>
<summary><b>Rocky Linux 8</b></summary>

Please refer to [release-centos7-llvm/env/prepare-sysroot.sh](./release-centos7-llvm/env/prepare-sysroot.sh)
Please refer to [release-linux-llvm/env/prepare-sysroot.sh](./release-linux-llvm/env/prepare-sysroot.sh)

</details>

Expand Down Expand Up @@ -329,10 +329,10 @@ More usages are available via `./dbms/bench_dbms --help`.

## Generate LLVM Coverage Report

To build coverage report, run the script under `release-centos7-llvm`
To build coverage report, run the script under `release-linux-llvm`

```shell
cd release-centos7-llvm
cd release-linux-llvm
./gen_coverage.sh
# Or run with filter:
# FILTER='*DMFile*:*DeltaMerge*:*Segment*' ./gen_coverage.sh
Expand All @@ -354,7 +354,7 @@ Before submitting a pull request, please resolve clang-tidy errors and use [form
```shell
# In the TiFlash repository root:
merge_base=$(git merge-base upstream/master HEAD)
python3 release-centos7-llvm/scripts/run-clang-tidy.py -p cmake-build-debug -j 20 --files `git diff $merge_base --name-only`
python3 release-linux-llvm/scripts/run-clang-tidy.py -p cmake-build-debug -j 20 --files `git diff $merge_base --name-only`
# if there are too much errors, you can try to run the script again with `-fix`
python3 format-diff.py --diff_from $merge_base
```
Expand Down
58 changes: 0 additions & 58 deletions release-centos7-llvm/Makefile

This file was deleted.

65 changes: 0 additions & 65 deletions release-centos7-llvm/dockerfiles/misc/bake_llvm_base_amd64.sh

This file was deleted.

30 changes: 0 additions & 30 deletions release-centos7-llvm/dockerfiles/misc/install_git.sh

This file was deleted.

26 changes: 0 additions & 26 deletions release-centos7-llvm/dockerfiles/misc/install_go.sh

This file was deleted.

34 changes: 0 additions & 34 deletions release-centos7-llvm/dockerfiles/tiflash-llvm-base-amd64

This file was deleted.

46 changes: 46 additions & 0 deletions release-linux-llvm/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Copyright 2022 PingCAP, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

DOCKER ?= docker
SUFFIX ?= ""

# Base docker files with prerequisites build tools
image_tiflash_llvm_base:
$(DOCKER) build dockerfiles -f dockerfiles/Dockerfile-tiflash-llvm-base -t hub.pingcap.net/tiflash/tiflash-llvm-base:rocky8$(SUFFIX)

# Build tiflash binaries under directory "tiflash"
build_tiflash_release:
$(DOCKER) run --rm -v $(realpath ..):/build/tics hub.pingcap.net/tiflash/tiflash-llvm-base:rocky8$(SUFFIX) /build/tics/release-linux-llvm/scripts/build-release.sh

# Package tiflash docker image using the binaries under directory "tiflash"
image_tiflash_release:
$(DOCKER) build -f dockerfiles/Dockerfile-tiflash-rocky8 -t hub.pingcap.net/tiflash/tiflash-server-rocky8 .

# Add build_tiflash_debug target to enable FailPoints. Since outputs are the same as release version, no new package targets added.
build_tiflash_debug:
$(DOCKER) run --rm -v $(realpath ..):/build/tics hub.pingcap.net/tiflash/tiflash-llvm-base:rocky8$(SUFFIX) /build/tics/release-linux-llvm/scripts/build-debug.sh

build_tiflash_tsan:
$(DOCKER) run --rm -v $(realpath ..):/build/tics hub.pingcap.net/tiflash/tiflash-llvm-base:rocky8$(SUFFIX) /build/tics/release-linux-llvm/scripts/build-tsan.sh

build_tiflash_asan:
$(DOCKER) run --rm -v $(realpath ..):/build/tics hub.pingcap.net/tiflash/tiflash-llvm-base:rocky8$(SUFFIX) /build/tics/release-linux-llvm/scripts/build-asan.sh

# Base docker with prerequisites running CI
image_tiflash_ci_base:
$(DOCKER) build -f dockerfiles/Dockerfile-tiflash-ci-base -t hub.pingcap.net/tiflash/tiflash-ci-base .

# Package tiflash CI docker image using the binaries under directory "tiflash"
image_tiflash_ci:
$(DOCKER) build -f dockerfiles/Dockerfile-tiflash-ci -t hub.pingcap.net/tiflash/tiflash-ci-rocky8 .
25 changes: 8 additions & 17 deletions release-centos7-llvm/README.md → release-linux-llvm/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# Docker Builder for centos 7
# Docker Builder for Rocky Linux 8

Scripts to build TiFlash binaries under centos 7
Scripts to build TiFlash binaries under Rocky Linux 8

Usage:

```bash
> make build_tiflash_release_amd64
# Or aarch64
#> make build_tiflash_release_aarch64
> make build_tiflash_release
# The binaries is built under directory `tiflash`, check its information
> ./tiflash/tiflash version
```
Expand All @@ -16,17 +14,13 @@ Usage:

Update the script

- `dockerfiles/misc/bake_llvm_base_amd64.sh`
- `dockerfiles/misc/bake_llvm_base_aarch64.sh`
- `dockerfiles/misc/bake_llvm_base.sh`

```bash
# Build the base docker with suffix.
> SUFFIX=-llvm-17.0.6 make image_tiflash_llvm_base_amd64
> SUFFIX=-llvm-17.0.6 make image_tiflash_llvm_base
# Use the new base docker to build TiFlash binary
> SUFFIX=-llvm-17.0.6 make build_tiflash_release_amd64
# Or aarch64
#> SUFFIX=-llvm-17.0.6 make image_tiflash_llvm_base_aarch64
#> SUFFIX=-llvm-17.0.6 make build_tiflash_release_aarch64
> SUFFIX=-llvm-17.0.6 make build_tiflash_release
# The binaries is built under directory `tiflash`, check its information
> ./tiflash/tiflash version
TiFlash
Expand All @@ -40,7 +34,7 @@ Raft Proxy

# Push the new base docker to the dockerhub, usually it is "hub.pingcap.net"
> docker login -u <your-username> -p <your-password> <dockerhub-url>
> docker image push <dockerhub-url>/tiflash/tiflash-llvm-base:amd64-llvm-17.0.6
> docker image push <dockerhub-url>/tiflash/tiflash-llvm-base:rocky8-llvm-17.0.6
```

At last, change the suffix in `${REPO_DIR}/.toolchain.yml` and check the built target in the CI.
Expand All @@ -50,10 +44,7 @@ How to get into a build container:

```
SUFFIX=-llvm-17.0.6
# x86_64
docker run -it --rm -v /your/path/to/tiflash:/build/tics hub.pingcap.net/tiflash/tiflash-llvm-base:amd64${SUFFIX} /bin/bash
# arm
docker run -it --rm -v /your/path/to/tiflash:/build/tics hub.pingcap.net/tiflash/tiflash-llvm-base:aarch64${SUFFIX} /bin/bash
docker run -it --rm -v /your/path/to/tiflash:/build/tics hub.pingcap.net/tiflash/tiflash-llvm-base:rocky8${SUFFIX} /bin/bash
```

## Local toolchain prerequisites
Expand Down
Loading