Godot 4.0 does not find .NET native DLLs that aren't in the directory the editor is installed in

[godotdot]

Godot version

v4.0.beta1.mono.official [4ba934b]

System information

Windows 10

Issue description

In Godot 4, I have to copy native dlls into the install directory of the Godot Editor, or else the dlls will not be found when running in the Editor.
In Godot 3.5, this is not necessary because the dlls can be found if they are in the output directory, when running in the Editor.

I would expect the Editor to find native dlls in the output directory of the project and not require them to be copied to the Editor's install directory.

Steps to reproduce

If you run the minimal reproduction project's Example.tscn in the Editor, it should fail to find the xlua dll.

If you edit Example.cs by commenting out the LuaEnv instantiation and set "UseSteam" to true, it will fail to find the steam api dll.

It will, however, copy the native dlls to Debug under mono.
If you copy the native dlls to the same directory the Godot 4 Editor is installed, it will find the dlls when running in the Editor.
It will still have errors, but those are unrelated to this issue.

Unrelated, but I'll mention it here because I'm not sure if this is a Godot issue or something else:
In Godot 3.5, Steamworks.SteamClient.Init() causes the overlay to appear on launch and be usable with shift+tab.
In Godot 4.0, this does not work.
There is an error reported relating to input, but that also happens in 3.5.

Minimal reproduction project

minimal_game_godot_project.zip

[Calinou]

Is this GDExtension DLLs or .NET native DLLs? You were referring to xlua at some point and SteamWorks in another. I believe the former is a .NET native DLL while the latter is a GDExtension.

[godotdot]

Is this GDExtension DLLs or .NET native DLLs? You were referring to xlua at some point and SteamWorks in another. I believe the former is a .NET native DLL while the latter is a GDExtension.

I believe they are both .NET native DLLs.

Facepunch.SteamWorks is not a GDExtension, I'm guessing you may be confusing it with GodotSteam?

[RedworkDE]

Basically the issue is that the native libraries are not correctly declared as native dependencies and thus their location is not considered for lookup.

The proper fix would be to declare them as native dependencies, but unfortunately documentation on this are non-existent (https://github.com/NuGet/docs.microsoft.com-nuget/issues/2070) but AFAIK this basically boils down to creating a nuget package that contains them at the runtimes/<rid>/native/<library> package path, which should cause them to be copied to the project output and for the correct version for the current runtime to be selected automatically.

As a workaround, simply copying the libraries to the output and adding a call to NativeLibrary.Load(Path.Join(AppContext.BaseDirectory, "xlua.dll")); (for the version for the current platform of all required libraries) before the libraries are required also works.

I looked into doing something on the godot side of things, but I was not able to come up with a good solution that works with exported projects.

[jolexxa]

I cannot get Steamworks.NET to work in Godot 4 on macOS (Apple silicon), presumably because of this issue. It throws a FileLoadException that I suspect is caused by an incorrect resolution of the native Steamworks dynamic libraries.

If it helps, you can check out the original Steam Godot 3.x project that did work with Steamworks.NET and the Steamworks native libraries: chickensoft-games/GameTemplate#1

Finally, here is my broken reproduction sample (broken on macOS, at least) for Godot 4 that fails to run (despite a nearly identical setup) https://github.com/chickensoft-games/SteamGameProject. It may have other issues, too, but I can't fix those until I get Godot not to crash when it runs.

[RedworkDE]

@definitelyokay Please see the comment above yours for a workaround.

[jolexxa]

@RedworkDE I'm not sure how to try what you suggested, because the program crashes simply by linking to the Steamworks.NET.dll, and adding that code snippet you suggested (but changing the dll name to the native dependency needed) doesn't seem to make a difference. Open to input on how to go about this properly.

[RedworkDE]

1. There is no error when simply adding a reference to the managed library (As seen in your MRP which has no issues, as it never executes anything steam related)
2. The error occurs when the native library is first used by the managed library.
3. Thus as mentioned the native library must be loaded from a full path before the managed library is used for the first time. This is probably somewhere in your initialization code, but in general this cannot be said more precisely.
4. (One caveat that does not apply here, is that in some rare cases it may be necessary to defer the first use of the managed library behind a method with MethodImpl(NoInlining) to ensure the managed library isn't loaded to early.)

[jolexxa]

1. There is no error when simply adding a reference to the managed library (As seen in your MRP which has no issues, as it never executes anything steam related)

On macOS, I was encountering an error simply by linking to Steamworks.NET.dll, even without any code that contained a using Steamworks;. I want to say that at some point yesterday it was running successfully after binding to Steamworks.NET.dll (but still without the using Steamworks; anywhere), but it seems to have stopped altogether in the last few runs :/ I will try a dotnet clean and verify I'm using the correct dylibs and whatnot just to double-check and get back to you on this later today, but after a few runs I wasn't able to get it to work at all as of last night. But who knows, I may have accidentally changed something while troubleshooting.

2. The error occurs when the native library is first used by the managed library.
3. Thus as mentioned the native library must be loaded from a full path before the managed library is used for the first time. This is probably somewhere in your initialization code, but in general this cannot be said more precisely.

Okay, that confirms what I suspected. Thank you for explaining so clearly.

4. (One caveat that does not apply here, is that in some rare cases it may be necessary to defer the first use of the managed library behind a method with MethodImpl(NoInlining) to ensure the managed library isn't loaded to early.)

That's a neat trick 👀

[jolexxa]

@RedworkDE just as a follow-up since I had some time tonight, I pushed another branch that shows the issue. It actually isn't failing on the native libsteam_api.dylib. Rather, it seems to be failing whenever we try to load the managed Steamworks.NET.dll (despite it being in the .godot/mono/temp/bin directory).

https://github.com/chickensoft-games/SteamGameProject/tree/fix/steam

I have no idea why the managed assembly has trouble loading.

If you comment out all the contents of SteamManager.cs and remove the var steamworksDotNetAssembly = Assembly.Load(steamworksDotNet); in Main.cs, it runs perfectly fine. So essentially, any code that has using Steamworks or any code that forces the Steamworks.NET.dll to actually be referenced causes an exception.

[RedworkDE]

Not sure what you are trying to do there, but

		NativeLibrary.Load(Path.Join(AppContext.BaseDirectory, "steam_api64.dll"));
		new SteamManager(440).Initialize();

in Game._Ready has loaded things correctly for me.

[jolexxa]

Not sure what you are trying to do there, but

		NativeLibrary.Load(Path.Join(AppContext.BaseDirectory, "steam_api64.dll"));

		new SteamManager(440).Initialize();

in Game._Ready has loaded things correctly for me.

Were you running on Apple Silicon? The issue seems specific to macs. I had thrown some code in that demonstrated that the error was coming from trying to load the managed assembly, not the native one.

[Fruitsalad]

For anyone else struggling to link a native DLL, Redwork's suggestion of using just NativeLibrary.Load(path) didn't work for me, but the following did:

public override void _Ready() {
	var assembly = Assembly.GetExecutingAssembly();
	NativeLibrary.SetDllImportResolver(assembly, DllImportResolver);
	
	Bindings.hello_world();
}

private static IntPtr DllImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath) {
	// Replace "your_library" with the native library name
	if (libraryName == "your_library") {
		// Replace `libyour_library.so` with the DLL/SO's filename
		var path = Path.Join(AppContext.BaseDirectory, "libyour_library.so");
		return NativeLibrary.Load(path);
	}
	return IntPtr.Zero;
}

Along with some <CopyToOutputDirectory> tag in the .csproj file so that the .dll/.so file that we're trying to load gets copied to AppContext.BaseDirectory.

It's very hacky of course but at least it works. I'm pretty inept with Dotnet so I couldn't figure out the proper way to do it with NuGet packages.

[Theome]

Has someone figured out a way to get this work on Apple Silicon? I'm running into the same issue when trying to add Steamworks.NET.dll to my Godot 4.2 project on an M2 Mac mini.

The workarounds mentioned here don't work for me. As far as I understand this isn't too surprising, because NativeLibrary is for native code, correct? The .dll we're trying to load is managed code. (I'm new to the C# world, so please correct me if I'm wrong).

If I try to load Steamworks.NET.dll with NativeLibrary methods, the exception I get says:

ERROR: System.DllNotFoundException: Unable to load shared library '[...]Steamworks.NET.dll' or one of its dependencies. [...] (not a mach-o file)

I can load and other native libraries (.dylibs on macOS) without problems.

If I try to load Steamworks.NET.dll with Assembly.LoadFrom(), I'm getting this exception, without further details:

ERROR: System.IO.FileLoadException: Could not load file or assembly 'Steamworks.NET, Version=20.2.0.0, Culture=neutral, PublicKeyToken=null'.

[hhyyrylainen]

To refer to C# libraries, you need to add reference to the library file in your .csproj file (for example: <Reference Include="Steamworks.NET"><HintPath>third_party\linux\Steamworks.NET.dll</HintPath></Reference>). Then once the project is packaged with Godot, you need to manually (or preferably with a script) copy the steamworks DLL file to the exported folder.

I have not tested on a mac (yet) but it should work as long as the file is copied to the right place so that the C# runtime can manage to load it as a dependency for your game's DLL file. One slight caveat is that you might need a custom plist that allows JIT compiled code and turns off some other security stuff that may prevent the DLL from being loaded.

[jolexxa]

@Theome I never figured out how to get Steamworks.NET.dll working on macOS w/ Apple Silicon (and I spent quite a bit of time with it). Hopefully, someone will and demonstrate how in a template project.

Until then, I have been pointing folks to https://github.com/LauraWebdev/GodotSteam_CSharpBindings.

[Theome]

After doing some more tests it looks to me as if the problem is not how the .dll is referenced, but how it is built:

I created a new C# project and added all .cs files from Steamworks.NET to it. This is the new Steamworks.NET.csproj file:

<Project Sdk="Godot.NET.Sdk/4.2.2">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <PropertyGroup>
    <DefineConstants>$(DefineConstants);STEAMWORKS_LIN_OSX</DefineConstants>
  </PropertyGroup>
</Project>

When I reference this project as a dependency in my main project like this:

<ProjectReference Include="path/to/Steamworks.NET.csproj" />

The created library Steamworks.NET.dll is created and loaded correctly.

If I move that .dll to my project folder, remove the ProjectReference from above and then add a reference like @hhyyrylainen has suggested above

<Reference Include="Steamworks.NET">
  <HintPath>path/to/my/Steamworks.NET.dll</HintPath>
</Reference>

This also works fine.

What doesn't work is if I do any of these steps:

• Download the Steamworks.NET project from Github, use that to build a .dll and add it to the project
• Download the standalone Steamworks.NET release from Github, add that .dll to the project
• Add the package as a dependency to my project with dotnet add package Steamworks.NET --version 20.2.0

In all cases, I get the same exception with no further details:

ERROR: System.IO.FileLoadException: Could not load file or assembly 'Steamworks.NET, Version=20.2.0.0, Culture=neutral, PublicKeyToken=null'. Could not find or load a specific file. (0x80131621)
File name: 'Steamworks.NET, Version=20.2.0.0, Culture=neutral, PublicKeyToken=null'

So, my current workaround is to recreate all my dependencies as new C#-Projects with Sdk set to "Godot.NET.Sdk/4.2.2".

@definitelyokay It doesn't look like this problem is specific to Steamworks.NET, I run into the same problems with other unrelated C# libraries.

Edit: I just noticed that my issue is not the same as the original question (Godot doesn't find .dlls) - I should probably open a new ticket for this.

[jolexxa]

So, my current workaround is to recreate all my dependencies as new C#-Projects with Sdk set to "Godot.NET.Sdk/4.2.2".

Wow, really neat that you got it working — that's a brilliant insight, but an incredibly high-friction workaround.

[maridany1999]

Yeah, in the version 4.3 Stable, Godot cannot find any DLLs once I hit play (even though there are no errors during compiling).

Edit: It worked on previous versions including 4.3 Beta.

[hhyyrylainen]

I found out when porting my game to Godot 4 that the default library search paths for C# are all gone. So now the engine doesn't really find anything except system-wide installed libraries. The good new is that the new C# version has really good feature to manually control where to load libraries from using this: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.nativelibrary.setdllimportresolver?view=net-8.0 (there's some more documentation on the usage here: https://learn.microsoft.com/en-us/dotnet/standard/native-interop/native-library-loading). So the end effect is that it requires a bit more setup in Godot 4 but overall the system is much much better. Here's an example of how I use this in my game: https://github.com/Revolutionary-Games/Thrive/blob/69c8c31a950461bc4d45602246b9c8f6a1962a5e/src/native/interop/NativeInterop.cs#L445

[chandlerpl]

For the people that are having an issue specifically on MacOS with Apple Silicon, I have been testing with Steamworks.NET and believe part of the issue is that the projects are not being exported for arm64-based devices.
I have updated Steamworks.NET to support arm64 MacOS and have had success with Godot loading the DLL after.

I will be continuing testing soon and will be PRing in my fix for Steamworks once it is stable, my existing working fork can be found here: Steamworks.NET

[jolexxa]

@chandlerpl awesome, thank you SO much!!