C#: Fallback to CoreCLR/MonoVM hosting APIs when hostfxr/NativeAOT fails

[raulsntos]

Custom builds to test this PR can be downloaded from #88803 (comment)

---

Some platforms don't support hostfxr but we can use the coreclr/monosgen library directly to initialize the runtime.

Android exports now use the android runtime identifier instead of linux-bionic, this removes the restrictions we previously had:

• Adds support for all Android architectures (arm32, arm64, x32, and x64), previously only the 64-bit architectures were supported.
• Loads System.Security.Cryptography.Native.Android (the .NET library that binds to the Android OS crypto functions).
  • Fixes C# Android SSL Crash using System.Net.Http.HttpClient #84559.

[raulsntos]

I don't like including the .jar like this. This .jar comes from the .NET runtime pack, but we don't have it at the time of building the export template and adding a .jar when exporting the project doesn't seem to be possible without forcing the user to enable Use Gradle Build.

[m4gr3d]

In addition to the comment about using flavor to guard the dependency, the jar file also needs to be within the app directory otherwise this will break editor gradle builds.

[raulsntos]

For gradle builds we probably want to get the jar file from the built project. But for now I can just move the jar to the app directory, should I put it in any specific subdirectory?

[m4gr3d]

it would typically go into the libs directory, but the contents of that directory get deleted during a clean build, so let's create a new monoLibs directory and include the jar there.

For gradle builds we probably want to get the jar file from the built project

How would that work? Does the user project include the jar file?

[raulsntos]

How would that work? Does the user project include the jar file?

When the C# project is built, during the restore step it will retrieve all the dependencies packages. For android builds, this means it will retrieve the Mono runtime package (e.g.: Microsoft.NETCore.App.Runtime.Mono.android-arm for android-arm32). This package contains the jar file for the runtime your project targets and we can retrieve it from your NuGet packages.

[m4gr3d]

We use the assemble gradle task for building and it's structured like this: assemble[flavors...][buildtypes...].
So by default if you only specify the assemble task, it builds all the flavors and build types; if you specify the build type (e.g: assembleDebug), then it builds all the flavors; and so on.. So if you don't change anything, the mono flavor will be included as part of the build, and if you only want to build the mono flavor, you can add it as a prefix to the assemble task.

After you configure the flavor, you can use the Gradle tool window (or the following command) to see the tasks that are being added.

And I guess I have to add a standard flavor for the non-mono version?

Yes you would need to add a standard flavor as an alternative to the mono flavor.

Do I have to duplicate every copy{Target}BinaryToBin task for the mono flavor?

As currently setup, you would need to duplicate the copy tasks. Let me take a look and see if I can clean that setup.

[m4gr3d]

Let me take a look and see if I can clean that setup.

@raulsntos I've cleaned the gradle build setup in #91271. Feel free to comment on the PR if you have questions on the update.

[raulsntos]

Looks great, is something like raulsntos@ad0f8fe what you had in mind? Or should the generateBuildTasks function take an extra edition parameter to only include a single edition build task? If so, should I create a new generateGodotTemplatesMono task for this or can the generateGodotTemplates task use parameters?

[m4gr3d]

Let's go with the second approach and have generateBuildTasks take an extra edition parameter and add a generateGodotMonoTemplates task that invokes it.

That matches our current build setup, where we have separate sections for standard and mono builds. And it allows users to only build the edition they need.

cc @akien-mga as this'll affect the android build scripts.

[raulsntos]

Made a PR in the godot-build-scripts repo:

• .NET: Build Android templates with 'mono' edition godot-build-scripts#94

[m4gr3d]

@raulsntos We'll need to create a separate mono flavor for the app module that includes the mono dependencies.

The addition of that flavor means we'll be able to guard mono specific logic in the java code by checking for the flavor:

if (BuildConfig.FLAVOR.equals("mono")) {
    // load mono library
}

This will also result in the creation of additional build templates specifically for the mono builds. @akien-mga we'll need to update the build tools logic for the mono build templates.

[Zamir7]

Could you explain what this means and how long to wait for corrections, and then review?

[raulsntos]

@Zamir7 As explained in the PR description, this adds support for 32-bit Android architectures and Android OS APIs (mainly crypto, see the linked issue) to C# projects.

This PR is marked as a draft, which means it's still a work-in-progress. It is tentatively planned for 4.4 (as you can see in the assigned milestone).

[m4gr3d]

The code looks good!

I haven't been able to test, so additional testing and validation would be needed, but that should be the easy part :)

[m4gr3d]

There's one more set of changes that'll be needed to support gradle builds.

export_plugin uses the assemble command for the gradle builds, so that logic will need to be updated to use assembleStandard on standard builds and assembleMono on mono builds.

The crypto jar file will also need to be copied to the build directory after it's downloaded. Alternatively if you know the path where it'll be downloaded, then app/build.gradlecan be updated to look in that path in addition to the path it's checking now.

[raulsntos]

The crypto jar file will also need to be copied to the build directory after it's downloaded. Alternatively if you know the path where it'll be downloaded, then app/build.gradle can be updated to look in that path in addition to the path it's checking now.

It should be copied because this file is retrieved when building the .NET project and it's stored in a temporary directory that will be deleted after the export. We can copy it as part of the ExportPlugin implementation, but I'd need a way to get the path to the gradle build directory as well as a way to detect if the export is part of a gradle build.

It looks like I can check if it's a gradle build with get_option("gradle_build/use_gradle_build"), and then use add_shared_object to copy the .jar file. It copies it to the res://android/libs/{BUILD_TYPE}/{ARCH} directory which I assume is enough for it to be included in the gradle build.

[raulsntos]

They are not! Should I reference it?

@felipejfc If the workaround mentioned in the issue you linked doesn't work, you could try doing that. Ideally you shouldn't have to do anything manually.

I want to mention that the Windows Single File export feature, which involves embedding script files, is currently encountering a similar issue.

@Delsin-Yu Yeah, this is an existing issue. It's reported in #96299.

[felipejfc]

@raulsntos Tested in a Motorola ARMv7 device with Adreno GPU

1 - Build crashed with Mobile rendered (because of vulkan I assume)

godot E  USER ERROR: .NET: Failed to get GodotPlugins initialization function pointer
      E     at: initialize_coreclr_and_godot_plugins (modules/mono/mono_gd/gd_mono.cpp:485)
      E  USER ERROR: .NET: Failed to load hostfxr
      E     at: initialize (modules/mono/mono_gd/gd_mono.cpp:549)
      E  USER ERROR: BUG: Unreferenced static string to 0: ShaderCompilation
      E     at: unref (core/string/string_name.cpp:127)
libc  F  FORTIFY: pthread_mutex_lock called on a destroyed mutex (0x81bb9e74)`

2 - When built with OpenGL the game worked fine!

Idk if the crash has to do with godot or with vulkan driver incompatibility but I would guess the later

[raulsntos]

I would not expect to see the Failed to load hostfxr error, it sounds like there may be a libhostfxr.so in the cache again¹.

Footnotes

1. https://github.com/godotengine/godot/pull/88803#issuecomment-2325401156 ↩

[raulsntos]

Rebased and made new custom builds to test, the cached data should no longer be an issue since #96301 was merged:

🏁 Windows	🍎 macOS	🐧 Linux
Editor x86_64	Editor universal	Editor x86_64
Editor x86_32		Editor x86_32

And the Android templates you'll need to use to export:

🤖 Android
Templates

Note

Remember to push the provided nupkgs to a local NuGet source so .NET can find the custom packages, see the documentation for more details.

[scgm0]

I’m curious, after the PR is merged, can I still use this change #86791 and export it to Android via aot?

[raulsntos]

NativeAOT should not be affected by this PR. If your csproj contains <PublishAot>true</PublishAot>, then it won't use the Mono runtime, the libmonosgen-2.0.so library won't be in the exported project so the changes in this PR will not take place.

Keep in mind NativeAOT for Android is still experimental upstream, and will likely require more changes to make it work properly but that's outside of the scope of this PR which only focuses on enabling the Mono runtime.

• NativeAOT status for Android dotnet/runtime#106748.

You may also try to enable MonoAOT with <RunAOTCompilation>true</RunAOTCompilation> but I haven't tested it and will likely not work.

[felipejfc]

Any chance we backport this to 4.3? Using native binding for handling http calls in android is a hassle

[raulsntos]

I don't think so, this is a pretty big change to C# Android exports that builds on top of a number of other changes to the Android platform that I don't expect to see backported to 4.3 either.

Issues with crypto functions not being available in 4.3 can be worked around by avoiding crypto APIs from the BCL or thirdparty C# libraries, and instead relying on the crypto APIs provided by Godot (i.e.: Use Godot.HttpClient instead of System.Net.Http.HttpClient).

Alternatively, you can include the required native libraries (libssl.so and libcrypto.so) in the exported APK and that should work too but I reckon it's even more of a hassle to do this.

[akien-mga]

Needs rebase after #96967.

[akien-mga]

Thanks! Amazing work 🎉

[tudor07]

I may be wrong but will this help in anyway with exporting C# using NativeAOT to Android? Relevant issue: dotnet/runtime#106748

[raulsntos]

@tudor07 No, NativeAOT is a different runtime so it's completely unrelated to this PR. See also: #88803 (comment)

The only way to export C# using NativeAOT to Android is using linux-bionic. The linux-bionic runtime is a Linux runtime using the Android C library, so it's basically Android but without the JNI. This means the Android bindings are not available, so some APIs (such as SSL) will crash the game. Also, only the x64 and arm64 architectures are supported.

As you can read in the issue you linked (dotnet/runtime#106748) there's interest in adding NativeAOT support to the android runtime identifier, but there are a number of challenges that would need to be resolved first. This is an upstream issue.

[Xevion]

Not so fun fact: MessagePack-CSharp relies on System.Security.Cryptography as well, so out the door that goes. MessagePack does not have a clear replacement here, it requires C# intrinsics that can't be replicated easily.

• I was okay with dropping the HttpClient because, while it hurts to not have more control, at least the networking still exists, even if simpler (e.g. I can't control DNS resolution for things like DNS over HTTPS).
• Tested the v4.4 bleeding edge build above, can confirm that HttpClient and MessagePack work just fine with it.

This looks like great work, and I'm very happy to see such recent progress instead of an abandoned 2yo thread telling me I'm out of luck. Just kinda upset that I only discovered this now.

Which brings me to what what I think is a reasonable question: What APIs are not supported or should people avoid requiring with Android, iOS, Web etc.?

• Besides just continuously testing on Android as I added dependencies, were there any warning indicators in the documentation that could told me I was using a dependency that would not function on Android?
• Written documentation would be fantastic so C# developers could understand what mess they may or may not be plunging into.

It's written everywhere: C# on Android is experimental - but what does that mean? What's stable, what isn't? Is Android the only one like this, what about OSX, iOS, etc. etc.

[raulsntos]

What APIs are not supported or should people avoid requiring with Android, iOS, Web etc.?

• For Android, I don't think there should be any more unsupported APIs (starting from Godot 4.4).
• For iOS we are using NativeAOT, so you may run into issues with trimming. Keep in mind NativeAOT for iOS with .NET 8 is experimental as per Microsoft's own documentation.
• And Web is not supported yet. See: Readd support for web platform exports when using the C# (.NET) version of the engine #70796

Besides just continuously testing on Android as I added dependencies, were there any warning indicators in the documentation that could told me I was using a dependency that would not function on Android?

As long as the platform support is marked experimental, there could be a lot of things that don't work. We document the known limitations in the C# platform support documentation but it's not an exhaustive list. We won't know what bugs exist unless users report them.

You may also want to keep an eye on the blog for progress updates. For example, the SSL issue was mentioned in the Current state of C# platform support in Godot 4.2 article which was meant to be a thorough explanation of the current platform support.

It's written everywhere: C# on Android is experimental - but what does that mean? What's stable, what isn't? Is Android the only one like this, what about OSX, iOS, etc. etc.

iOS is also marked experimental. This is documented in the C# platform support documentation.

Users interested in exporting to these platforms will test them and report the bugs they find. We'll eventually remove the experimental label once we're confident that the platform support is reliable enough and has no critical bugs.

[Delsin-Yu]

Now that .Net 9 is out and the Apple platforms have graduated from the NAOT experimental list, the NAOT for Android seems to be the last commonly used platform that's marked experimental.