Add docker based package builder and switch to system compiler #51

xinyazhang · 2024-10-29T21:50:47Z

Major Changes:

dockerfile/build.sh script that builds AOTriton from offical AlmaLinux 8 docker image.
Deprecate hipcc since GCC >= 9 / Clang >= 10 is capable to compile the C++ part
- hipcc is not compatible with SCL. scl enable gcc-toolset-13 "/opt/rocm/bin/hipcc -v" is supposed to show "Found candidate GCC installation" but nothing is displayed, and eventually this triggers Missing cmath/libcxx in ROCm 5.3.0 ROCm#1843 during compiling.

Minor Changes

In pyaotriton, register hipError_t locally to avoid "hipError_t is already registered" bug.

jithunnair-amd · 2024-10-30T15:34:39Z

dockerfile/manylinux_2_28.Dockerfile

+    python -m pip install cmake ninja wheel
+RUN mkdir /root/build
+
+ARG amdgpu_installer="https://repo.radeon.com/amdgpu-install/6.2.2/el/8.10/amdgpu-install-6.2.60202-1.el8.noarch.rpm"


This is a maintenance headache, we need to make this more generic for this to be generally usable for any ROCm release

Discussed this offline, this is the default value. The intention is for any CI job or users to provide a link to the desired amdgpu installer for their desired ROCm version.
We could use logic similar to this file:
https://raw.githubusercontent.com/wiki/ROCmSoftwarePlatform/pytorch/files/install_kdb_files_for_pytorch_wheels.sh
to keep the default argument always up-to-date.

jithunnair-amd · 2024-10-30T17:15:21Z

dockerfile/input/install_aotriton.sh

+cd /root/build
+scl enable gcc-toolset-13 "cd aotriton/third_party/triton/python; python setup.py develop --user"
+scl enable gcc-toolset-13 "mkdir -p aotriton/build; cd aotriton/build; cmake .. -DCMAKE_PREFIX_PATH=/opt/rocm -DPYTHON_EXECUTABLE=/usr/bin/python3.11 -DCMAKE_INSTALL_PREFIX=installed_dir/aotriton -DCMAKE_BUILD_TYPE=Release -DAOTRITON_GPU_BUILD_TIMEOUT=0 \"-DTARGET_GPUS=${AOTRITON_TARGET_GPUS}\" -DAOTRITON_NO_PYTHON=ON -G Ninja && ninja install"
+rocmver=$(scl enable gcc-toolset-13 "cpp -I/opt/rocm/include /input/print_rocm_version.h"|tail -n 1|sed 's/ //g')


@naromero77amd This is another interesting and even more compact way of getting the ROCM_VERSION* defines. Something to consider if we want to make our LoadHIP.cmake logic even more compact.

jithunnair-amd · 2024-10-30T17:43:28Z

v2src/CMakeLists.txt

 if(AOTRITON_BUILD_FOR_TUNING)
  target_compile_definitions(aotriton_v2 PRIVATE -DAOTRITON_BUILD_FOR_TUNING=1)
 else(AOTRITON_BUILD_FOR_TUNING)
  target_compile_definitions(aotriton_v2 PRIVATE -DAOTRITON_BUILD_FOR_TUNING=0)
 endif(AOTRITON_BUILD_FOR_TUNING)
-target_link_libraries(aotriton_v2 lzma_interface)
+target_link_libraries(aotriton_v2 PRIVATE lzma_interface)
+target_link_libraries(aotriton_v2 PUBLIC hip::host hip::amdhip64)


We need to introduce an explicit dependency on the HIP targets because we are not using the hipcc compiler as the default compiler anymore: https://github.com/ROCm/aotriton/pull/51/files#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20aL35

jithunnair-amd · 2024-10-30T17:47:33Z

CMakeLists.txt

@@ -31,9 +33,6 @@ if(NOT AOTRITON_NO_PYTHON)
  add_subdirectory(third_party/pybind11)
 endif()

-# Must be after pybind11
-set(CMAKE_CXX_COMPILER hipcc)


We do not need to use hipcc since we are not compiling any kernels, so a host compiler should be good enough.

jithunnair-amd · 2024-10-30T18:18:20Z

v2src/CMakeLists.txt

@@ -168,13 +168,14 @@ target_include_directories(aotriton_v2 PUBLIC ${CMAKE_CURRENT_LIST_DIR}/../inclu
 target_include_directories(aotriton_v2 PUBLIC ${CMAKE_BINARY_DIR}/include)  # for <build>/include/aotriton/config.h. Code should use <aotriton/config.h>
 target_include_directories(aotriton_v2 PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
 target_include_directories(aotriton_v2 PRIVATE ${CMAKE_CURRENT_LIST_DIR}/../third_party/incbin)
-target_compile_options(aotriton_v2 PRIVATE -fPIC --no-offload-arch=all)


Removed since we're not using hipcc anymore

jithunnair-amd · 2024-10-30T18:19:40Z

v2src/flash/attn_bwd.cc

    .cu_seqlens_q = &cu_seqlens_q,
    .cu_seqlens_k = &cu_seqlens_k,
    .num_seqlens = num_seqlens,
    .max_seqlen_q = max_seqlen_q,
    .max_seqlen_k = max_seqlen_k,
-    .head_dim = head_size,


Required because we're switching to gcc13, which is more strict about the order of struct fields

jithunnair-amd · 2024-10-30T18:24:55Z

bindings/hipruntime.cc

@@ -8,7 +8,7 @@ namespace py = pybind11;
 namespace pyaotriton {
 #define EV(name) value(#name, name)
  void def_hipruntime(py::module_& m) {
-    py::enum_<hipError_t>(m, "hipError_t")
+    py::enum_<hipError_t>(m, "hipError_t", py::module_local())


If this is not due to the compiler switch or any other changes in this PR, then it would be better to separate this into its own PR with reasoning.

xinyazhang · 2024-10-30T18:26:47Z

Dockerfile tested locallly and can build 0.7.1b
Compiler switch tested locally and can run test pytest ../test/test_backward.py -k 1.2 -v -x

jithunnair-amd · 2024-10-30T18:38:30Z

@ethanwee1 Can you please validate this PR by trying to build aotriton 0.7b and 0.7.1b using the Dockerfile in this PR? Please make sure that it generates a tarball with the following files:

$ tar tvf output/aotriton-0.7.1b-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz 
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/lib/
-rw-r--r-- root/root 382452768 2024-10-29 09:11 aotriton/lib/libaotriton_v2.so
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/aotriton/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/aotriton/_internal/
-rw-r--r-- root/root      1490 2024-10-29 08:48 aotriton/include/aotriton/_internal/triton_kernel.h
-rw-r--r-- root/root       592 2024-10-29 08:48 aotriton/include/aotriton/_internal/util.h
-rw-r--r-- root/root       566 2024-10-29 08:48 aotriton/include/aotriton/cpp_tune.h
-rw-r--r-- root/root       422 2024-10-29 08:48 aotriton/include/aotriton/dtypes.h
-rw-r--r-- root/root      5435 2024-10-29 08:48 aotriton/include/aotriton/flash.h
-rw-r--r-- root/root       695 2024-10-29 08:48 aotriton/include/aotriton/runtime.h
-rw-r--r-- root/root      3316 2024-10-29 08:48 aotriton/include/aotriton/util.h

ethanwee1 · 2024-10-31T20:28:26Z

@ethanwee1 Can you please validate this PR by trying to build aotriton 0.7b and 0.7.1b using the Dockerfile in this PR? Please make sure that it generates a tarball with the following files:

$ tar tvf output/aotriton-0.7.1b-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz 
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/lib/
-rw-r--r-- root/root 382452768 2024-10-29 09:11 aotriton/lib/libaotriton_v2.so
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/aotriton/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/aotriton/_internal/
-rw-r--r-- root/root      1490 2024-10-29 08:48 aotriton/include/aotriton/_internal/triton_kernel.h
-rw-r--r-- root/root       592 2024-10-29 08:48 aotriton/include/aotriton/_internal/util.h
-rw-r--r-- root/root       566 2024-10-29 08:48 aotriton/include/aotriton/cpp_tune.h
-rw-r--r-- root/root       422 2024-10-29 08:48 aotriton/include/aotriton/dtypes.h
-rw-r--r-- root/root      5435 2024-10-29 08:48 aotriton/include/aotriton/flash.h
-rw-r--r-- root/root       695 2024-10-29 08:48 aotriton/include/aotriton/runtime.h
-rw-r--r-- root/root      3316 2024-10-29 08:48 aotriton/include/aotriton/util.h

Validated 0.7.1b and 0.7b after following build.sh

aotriton-0.7.1b-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz 
aotriton-0.7b-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz

jithunnair-amd · 2024-11-07T21:49:34Z

@ethanwee1 Can you please validate this PR by trying to build aotriton 0.7b and 0.7.1b using the Dockerfile in this PR? Please make sure that it generates a tarball with the following files:

$ tar tvf output/aotriton-0.7.1b-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz 
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/lib/
-rw-r--r-- root/root 382452768 2024-10-29 09:11 aotriton/lib/libaotriton_v2.so
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/aotriton/
drwxr-xr-x root/root         0 2024-10-29 09:11 aotriton/include/aotriton/_internal/
-rw-r--r-- root/root      1490 2024-10-29 08:48 aotriton/include/aotriton/_internal/triton_kernel.h
-rw-r--r-- root/root       592 2024-10-29 08:48 aotriton/include/aotriton/_internal/util.h
-rw-r--r-- root/root       566 2024-10-29 08:48 aotriton/include/aotriton/cpp_tune.h
-rw-r--r-- root/root       422 2024-10-29 08:48 aotriton/include/aotriton/dtypes.h
-rw-r--r-- root/root      5435 2024-10-29 08:48 aotriton/include/aotriton/flash.h
-rw-r--r-- root/root       695 2024-10-29 08:48 aotriton/include/aotriton/runtime.h
-rw-r--r-- root/root      3316 2024-10-29 08:48 aotriton/include/aotriton/util.h

Validated 0.7.1b and 0.7b after following build.sh

aotriton-0.7.1b-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz 
aotriton-0.7b-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz

@xinyazhang Taking this out of Draft mode as it is ready to merge (even if queued) based on Ethan's validation

xinyazhang · 2024-11-07T22:19:29Z

@jithunnair-amd nope, queued PR should not be taken out of draft because it's based on previous work (for this PR specifically its #50)

ethanwee1 · 2024-11-13T19:20:54Z

Tested with these commands to build
aotriton-e278d4a853170c7a9063cfe847419414cb7b62b6-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz

Commands:

git clone https://github.com/ROCm/aotriton.git
cd aotriton/
git checkout xinyazhang/manylinux_2_28-dockerfile
cd dockerfile/
export AMDGPU_INSTALLER=https://repo.radeon.com/amdgpu-install/6.2.4/el/8.10/amdgpu-install-6.2.60204-1.el8.noarch.rpm
mkdir -p output
TRITON_LLVM_HASH="b5cc222d" bash build.sh input tmpfs output e278d4a853170c7a9063cfe847419414cb7b62b6 "MI300X;MI200" 2>&1 | tee buildlog2.log
tar tvf output/*.tar*

Output:
Size: 107MB
aotriton-e278d4a853170c7a9063cfe847419414cb7b62b6-manylinux_2_28_x86_64-rocm6.2-shared.txt
buildlog2.log

Use AlmaLinux 8 as base image.

However due to their more limit support of designated initializer the order of fields have to be adjusted.

This is more general since -b only supports tags and branches.

jithunnair-amd · 2024-11-13T19:34:10Z

Tested with these commands to build aotriton-e278d4a853170c7a9063cfe847419414cb7b62b6-manylinux_2_28_x86_64-rocm6.2-shared.tar.gz

Commands:
git clone https://github.com/ROCm/aotriton.git
cd aotriton/
git checkout xinyazhang/manylinux_2_28-dockerfile
cd dockerfile/
export AMDGPU_INSTALLER=https://repo.radeon.com/amdgpu-install/6.2.4/el/8.10/amdgpu-install-6.2.60204-1.el8.noarch.rpm
mkdir -p output
TRITON_LLVM_HASH="b5cc222d" bash build.sh input tmpfs output e278d4a853170c7a9063cfe847419414cb7b62b6 "MI300X;MI200" 2>&1 | tee buildlog2.log
tar tvf output/*.tar*
Output: aotriton-e278d4a853170c7a9063cfe847419414cb7b62b6-manylinux_2_28_x86_64-rocm6.2-shared.txt buildlog2.log

Some notable lines in the log:

-rw-r--r-- root/root 13064056 2024-11-13 18:01 aotriton/lib/libaotriton_v2.so: libaotriton_v2.so reduced to 13MB
aks2 files generated for each GFX arch eg. aotriton/lib/aotriton.images/amd-gfx90a/flash/attn_fwd/FONLY__^bf16@16,False,128,False,False,False,0___MI200.aks2

xinyazhang · 2024-11-13T19:39:43Z

13 MiB is also significant considering it's functionality. I believe most of the size comes from the generated dispatching code.
I've added this into the keepbook, but no concrete plan to implement it.

xinyazhang requested review from jithunnair-amd and pruthvistony October 29, 2024 21:50

jithunnair-amd reviewed Oct 30, 2024

View reviewed changes

xinyazhang force-pushed the xinyazhang/manylinux_2_28-dockerfile branch from 529395c to 2147593 Compare October 30, 2024 18:13

jithunnair-amd reviewed Oct 30, 2024

View reviewed changes

xinyazhang changed the title ~~[Queued PR] Add docker based package builder~~ [Queued PR] Add docker based package builder and switch to system compiler Oct 30, 2024

jithunnair-amd reviewed Oct 30, 2024

View reviewed changes

jithunnair-amd approved these changes Oct 30, 2024

View reviewed changes

xinyazhang force-pushed the xinyazhang/storagev2 branch from 43f5f33 to 1ff178e Compare October 30, 2024 18:47

xinyazhang force-pushed the xinyazhang/manylinux_2_28-dockerfile branch from 2147593 to 7ca74d9 Compare October 30, 2024 18:48

xinyazhang force-pushed the xinyazhang/storagev2 branch from 1ff178e to 801842e Compare October 30, 2024 23:13

xinyazhang force-pushed the xinyazhang/manylinux_2_28-dockerfile branch from 7ca74d9 to 63fe1cc Compare October 30, 2024 23:13

xinyazhang force-pushed the xinyazhang/storagev2 branch from 801842e to 93299bd Compare October 30, 2024 23:27

xinyazhang force-pushed the xinyazhang/manylinux_2_28-dockerfile branch from 63fe1cc to ca18036 Compare October 30, 2024 23:27

xinyazhang force-pushed the xinyazhang/storagev2 branch from 93299bd to 0dc5705 Compare November 7, 2024 18:08

xinyazhang force-pushed the xinyazhang/manylinux_2_28-dockerfile branch from ca18036 to 4357504 Compare November 7, 2024 20:24

jithunnair-amd marked this pull request as ready for review November 7, 2024 21:48

xinyazhang marked this pull request as draft November 7, 2024 22:18

xinyazhang force-pushed the xinyazhang/manylinux_2_28-dockerfile branch 3 times, most recently from 3d075ee to e278d4a Compare November 13, 2024 17:35

jithunnair-amd mentioned this pull request Nov 13, 2024

Kernel Storage V2 #50

Merged

xinyazhang changed the base branch from xinyazhang/storagev2 to main November 13, 2024 19:29

xinyazhang added 9 commits November 13, 2024 19:29

Dockerfile and script to build glibc_2_28/manylinux_2_28 libaotriton.

cb08a50

Use AlmaLinux 8 as base image.

dockerfile: Add documentations.

4c86606

dockerfile/build.sh: guess TRITON_HASH from know tags

b9a9581

Update the method to support different ROCM.

b8c34af

Deprecate hipcc since GCC >= 9 / Clang >= 10 should work.

5c13134

However due to their more limit support of designated initializer the order of fields have to be adjusted.

binding: register hipError_t locally.

a6104d1

TRITON_HASH -> TRITON_LLVM_HASH

23b4b86

Document the flexibility of input argument of build.sh

de39717

Replace git clone -b with git clone+checkout+submodule update

8338d4c

This is more general since -b only supports tags and branches.

xinyazhang force-pushed the xinyazhang/manylinux_2_28-dockerfile branch from e278d4a to 8338d4c Compare November 13, 2024 19:30

Fix missing packages when installing Triton in non venv

e24a539

xinyazhang changed the title ~~[Queued PR] Add docker based package builder and switch to system compiler~~ Add docker based package builder and switch to system compiler Nov 13, 2024

xinyazhang marked this pull request as ready for review November 13, 2024 19:50

xinyazhang merged commit 3fe6ca6 into main Nov 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docker based package builder and switch to system compiler #51

Add docker based package builder and switch to system compiler #51

xinyazhang commented Oct 29, 2024 •

edited

Loading

jithunnair-amd Oct 30, 2024

jithunnair-amd Oct 30, 2024

jithunnair-amd Oct 30, 2024

jithunnair-amd Oct 30, 2024 •

edited

Loading

jithunnair-amd Oct 30, 2024

jithunnair-amd Oct 30, 2024

jithunnair-amd Oct 30, 2024 •

edited

Loading

jithunnair-amd Oct 30, 2024

xinyazhang commented Oct 30, 2024

jithunnair-amd commented Oct 30, 2024

ethanwee1 commented Oct 31, 2024 •

edited

Loading

jithunnair-amd commented Nov 7, 2024

xinyazhang commented Nov 7, 2024

ethanwee1 commented Nov 13, 2024 •

edited

Loading

jithunnair-amd commented Nov 13, 2024 •

edited

Loading

xinyazhang commented Nov 13, 2024

Add docker based package builder and switch to system compiler #51

Add docker based package builder and switch to system compiler #51

Conversation

xinyazhang commented Oct 29, 2024 • edited Loading

Major Changes:

Minor Changes

jithunnair-amd Oct 30, 2024

Choose a reason for hiding this comment

jithunnair-amd Oct 30, 2024

Choose a reason for hiding this comment

jithunnair-amd Oct 30, 2024

Choose a reason for hiding this comment

jithunnair-amd Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

jithunnair-amd Oct 30, 2024

Choose a reason for hiding this comment

jithunnair-amd Oct 30, 2024

Choose a reason for hiding this comment

jithunnair-amd Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

jithunnair-amd Oct 30, 2024

Choose a reason for hiding this comment

xinyazhang commented Oct 30, 2024

jithunnair-amd commented Oct 30, 2024

ethanwee1 commented Oct 31, 2024 • edited Loading

jithunnair-amd commented Nov 7, 2024

xinyazhang commented Nov 7, 2024

ethanwee1 commented Nov 13, 2024 • edited Loading

jithunnair-amd commented Nov 13, 2024 • edited Loading

xinyazhang commented Nov 13, 2024

xinyazhang commented Oct 29, 2024 •

edited

Loading

jithunnair-amd Oct 30, 2024 •

edited

Loading

jithunnair-amd Oct 30, 2024 •

edited

Loading

ethanwee1 commented Oct 31, 2024 •

edited

Loading

ethanwee1 commented Nov 13, 2024 •

edited

Loading

jithunnair-amd commented Nov 13, 2024 •

edited

Loading