ghidra: use the first release of our fork #5331

mpollmeier · 2025-02-27T10:57:01Z

@DavidBakerEffendi would you mind double checking that your change has an effect? It is included in the ghidra jar (I checked Ghidra/Processors/ARM/data/languages/ARM.sinc), so from my side it looks good.

DavidBakerEffendi

Can confirm that the changes are in the JAR, just checking more tests. I see the macOS test is failing here

DavidBakerEffendi · 2025-02-27T11:45:42Z

I currently only see ./_Root/Ghidra/Features/Decompiler/os/linux_x86_64/decompile.

https://mvnrepository.com/artifact/io.joern/ghidra/11.3.1_PUBLIC_20250219-0 has

./_Root/Ghidra/Features/Decompiler/os/win_x86_64/decompile.exe
./_Root/Ghidra/Features/Decompiler/os/linux_x86_64/decompile
./_Root/Ghidra/Features/Decompiler/os/mac_arm_64/decompile
./_Root/Ghidra/Features/Decompiler/os/mac_x86_64/decompile

mpollmeier · 2025-02-27T12:33:50Z

Interesting, I guess it makes sense to have a side by side comparison...

diff -r --brief ghidra-11.3.1_PUBLIC_20250219-0-extracted ghidra-11.4_c3d8b6a0b3-202502271115-extracted | grep ^Only | grep -v '\.class'|sort
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/help/topics/FrontEndPlugin/images: UsersPanelMini.png
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Debug/Debugger-agent-dbgeng: os
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/DemanglerGnu/os: mac_arm_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/DemanglerGnu/os: mac_x86_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/DemanglerGnu/os: win_x86_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Features/Decompiler/os: mac_arm_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Features/Decompiler/os: mac_x86_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Features/Decompiler/os: win_x86_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Features/FileFormats/os: mac_arm_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Features/FileFormats/os: mac_x86_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Features/FileFormats/os: win_x86_64
Only in ghidra-11.3.1_PUBLIC_20250219-0-extracted/_Root/Ghidra/Features/PDB: os
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/ghidra/app/plugin/core/debug: taint
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/ghidra/feature/vt/gui/wizard: add
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/ghidra/feature/vt/gui/wizard: session
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/ghidra/framework/main: wizard
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/help/topics/FrontEndPlugin/images: ViewProjectAccessPanel.png
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug/Debugger-agent-dbgeng/data/debugger-launchers: svrcx-dbgeng.bat
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug/Debugger-agent-dbgeng/data/support: svrcx-dbgeng.py
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug: Debugger-agent-drgn
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug/Debugger-agent-lldb/data/debugger-launchers: android-lldb.bat
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug/Debugger-agent-lldb/data/debugger-launchers: kernel-lldb.bat
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug/Debugger-agent-lldb/data/debugger-launchers: remote-lldb.bat
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug/Debugger-agent-lldb/data/debugger-launchers: ssh-lldb.bat
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Debug/Debugger-agent-lldb/data/debugger-launchers: ssh-lldb.sh
Only in ghidra-11.4_c3d8b6a0b3-202502271115-extracted/_Root/Ghidra/Features/DecompilerDependent/ghidra_scripts: ExportSourceSetScript.java

DavidBakerEffendi · 2025-02-27T12:51:19Z

Seems gradle buildGhidra will always build for the OS it's running on. We may need to use a multi-stage docker builder e.g.,

# Stage 1: Linux Build
FROM ubuntu:22.04 AS linux-builder
RUN apt update && apt install -y openjdk-17-jdk gradle git cmake build-essential
WORKDIR /ghidra
COPY . .
RUN gradle assembleAll

# Stage 2: Windows Build
FROM ubuntu:22.04 AS windows-builder
RUN apt update && apt install -y openjdk-17-jdk gradle git cmake mingw-w64
WORKDIR /ghidra
COPY . .
RUN CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ gradle assembleAll

# =======================
# Final Stage: Merge Artifacts
# =======================
FROM alpine:latest
WORKDIR /final_build
COPY --from=linux-builder /ghidra/Build/Ghidra /final_build/linux/
COPY --from=windows-builder /ghidra/Build/Ghidra /final_build/windows/
# mount `/final_build` to the host and assemble the macos one there + compress

For simplicity, the CI/CD runner could be macOS, so that we don't have to worry about more cross-compilation, then call it a day.

Needs docker buildx and can be built like docker buildx build --platform linux/amd64,windows/amd64

mpollmeier · 2025-02-27T17:07:54Z

I'd like to avoid using docker inside github actions, which is already containerised. I'm toying around with plain github actions and different jobs in a workflow.
We might get the entire release up on github actions afterall, not relying on me running it locally, that'd be sweet!

DavidBakerEffendi · 2025-02-27T20:36:00Z

Ah yes, runs can emit artifacts, and each run can possibly run on a different OS. Sounds good!

https://github.com/joernio/ghidra/actions/runs/13589776349

mpollmeier · 2025-02-28T15:17:53Z

🎉 first release completely off github actions, including native libraries built from scratch for macos/win
https://github.com/joernio/ghidra/actions/runs/13589776349

I updated this branch, can you please test again?

DavidBakerEffendi · 2025-02-28T17:51:52Z

The windows and macos binaries dont seem to be placed in the correct location for the JAR building to package it with the final JAR. In particular, they seem to be outside the build directory and in the source directory when fetched from the other build processes?

mpollmeier · 2025-02-28T18:08:47Z

oh, ok, sounds like some fine tuning still required, but overall the process feels good
will address monday

mpollmeier · 2025-03-04T14:22:46Z

@DavidBakerEffendi can you please check again? the native libs were lost along the ghidra build process - i adapted it and created a new release, this time they're in the jar, see e.g.

unzip -l io.joern.ghidra-11.4_e4db88418c-202503041155.jar|grep Decompiler/os
        0  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/
        0  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/linux_x86_64/
        0  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/mac_arm_64/
        0  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/win_x86_64/
  1116768  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/linux_x86_64/sleigh
  3872624  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/linux_x86_64/decompile
   960296  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/mac_arm_64/sleigh
  3273896  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/mac_arm_64/decompile
  3345920  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/win_x86_64/decompile.exe
  1248768  2025-03-04 12:05   _Root/Ghidra/Features/Decompiler/os/win_x86_64/sleigh.exe

DavidBakerEffendi · 2025-03-04T14:48:27Z

Thanks! Will check - hopefully the runners from the other tests will also pass confirming it works

DavidBakerEffendi · 2025-03-04T14:56:01Z

@mpollmeier Only difference compared to v11.3 is that mac_x86_64 is not there, but I think that is fine. The runners look happy, I restarted test-scripts to see if it will pass this time.

DavidBakerEffendi

Looks great, I'm happy

mpollmeier · 2025-03-04T16:35:29Z

🎉

mpollmeier · 2025-03-04T16:40:38Z

ok, so apparently macos-latest is now on apple's m1 silicon. we can add another os to the matrix for mac_x86_64, looks like macos-13 is the only one left.
will give it a try tomorrow

mpollmeier requested a review from DavidBakerEffendi February 27, 2025 10:57

DavidBakerEffendi reviewed Feb 27, 2025

View reviewed changes

mpollmeier added 2 commits February 28, 2025 16:15

ghidra: use our fork https://github.com/joernio/ghidra

d9171fb

use first release made from github actions runner

524354b

https://github.com/joernio/ghidra/actions/runs/13589776349

mpollmeier force-pushed the michael/ghidra/0-our-fork branch from da4f560 to 524354b Compare February 28, 2025 15:16

next try, this time with mac and win binaries

cb9d9af

DavidBakerEffendi approved these changes Mar 4, 2025

View reviewed changes

mpollmeier merged commit 6385dbb into master Mar 4, 2025
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ghidra: use the first release of our fork #5331

ghidra: use the first release of our fork #5331

mpollmeier commented Feb 27, 2025 •

edited

Loading

DavidBakerEffendi left a comment

DavidBakerEffendi commented Feb 27, 2025 •

edited

Loading

mpollmeier commented Feb 27, 2025

DavidBakerEffendi commented Feb 27, 2025 •

edited

Loading

mpollmeier commented Feb 27, 2025

DavidBakerEffendi commented Feb 27, 2025

mpollmeier commented Feb 28, 2025

DavidBakerEffendi commented Feb 28, 2025

mpollmeier commented Feb 28, 2025

mpollmeier commented Mar 4, 2025

DavidBakerEffendi commented Mar 4, 2025

DavidBakerEffendi commented Mar 4, 2025 •

edited

Loading

DavidBakerEffendi left a comment

mpollmeier commented Mar 4, 2025

mpollmeier commented Mar 4, 2025

ghidra: use the first release of our fork #5331

ghidra: use the first release of our fork #5331

Conversation

mpollmeier commented Feb 27, 2025 • edited Loading

DavidBakerEffendi left a comment

Choose a reason for hiding this comment

DavidBakerEffendi commented Feb 27, 2025 • edited Loading

mpollmeier commented Feb 27, 2025

DavidBakerEffendi commented Feb 27, 2025 • edited Loading

mpollmeier commented Feb 27, 2025

DavidBakerEffendi commented Feb 27, 2025

mpollmeier commented Feb 28, 2025

DavidBakerEffendi commented Feb 28, 2025

mpollmeier commented Feb 28, 2025

mpollmeier commented Mar 4, 2025

DavidBakerEffendi commented Mar 4, 2025

DavidBakerEffendi commented Mar 4, 2025 • edited Loading

DavidBakerEffendi left a comment

Choose a reason for hiding this comment

mpollmeier commented Mar 4, 2025

mpollmeier commented Mar 4, 2025

mpollmeier commented Feb 27, 2025 •

edited

Loading

DavidBakerEffendi commented Feb 27, 2025 •

edited

Loading

DavidBakerEffendi commented Feb 27, 2025 •

edited

Loading

DavidBakerEffendi commented Mar 4, 2025 •

edited

Loading