Add external texture support (GLES3)

[dsnopek]

This is a continuation of the work started on PR #83519

To avoid a naming conflict, this builds on PR #96928

It's marked as a DRAFT because PR #96928 needs to be merged first, and I'd like to figure out how to add Vulkan support (right now this is just GLES3).

UPDATE: I started working on adding Vulkan support, however, I discovered that it depends on Android API level 26 or higher. Presently, we have an API minimum of 21, which would prevent this code from compiling. So, for now, I've split Vulkan support off into PR #97163 to finish some time later.

Here is a demo project that uses this functionality to show the camera feed on Android.

[fire]

I don't know where to ask this. Do external textures exist on PC? Like video buffers.

[dsnopek]

Do external textures exist on PC?

In my research on this, I saw a reference to OES_EGL_image_external working on desktop Linux, but (1) I don't know what APIs you can use it to interact with and (2) I imagine it would require using EGL and real GLES3 (not "GLES over GL" which we usually use on desktop).

I didn't see anything about external textures working on desktop with Vulkan and the necessary extension has "ANDROID" in the name (VK_ANDROID_external_memory_android_hardware_buffer).

[dsnopek]

I started working on adding Vulkan support, however, I discovered that it depends on Android API level 26 or higher. Presently, we have an API minimum of 21, which would prevent this code from compiling. So, for now, I've split Vulkan support off into PR #97163 to finish some time later.

I feel very good about the GLES3 support in this PR here, though! Once PR #96928 (which this PR depends on) is merged, then I'll take this one out of draft.

[BraveEvidence]

I'm a native Android developer working on creating Android plugins for Godot. For a simple use case, let's say I want to embed an EditText from native Android into a Godot scene. Will this PR assist me in achieving that? Specifically, would I need to use a surface texture as the parent view and add the EditText as a child view, allowing it to be displayed within the Godot scene? Additionally, will the rest of the Godot scene elements (3D/2D nodes) still function as expected alongside the native Android EditText?

[akien-mga]

Needs rebase to solve merge conflict.

[dsnopek]

@akien-mga:

Needs rebase to solve merge conflict.

Rebased!

@BraveEvidence:

For a simple use case, let's say I want to embed an EditText from native Android into a Godot scene. Will this PR assist me in achieving that? Specifically, would I need to use a surface texture as the parent view and add the EditText as a child view, allowing it to be displayed within the Godot scene?

Yes!

Take a look at the demo project linked in the PR description. It uses an Android SurfaceTexture to render the camera preview onto a Godot ColorRect, but you could certainly modify it to render an EditText from Android instead. However, it doesn't do anything to forward input, so if you wanted the EditText widget to be functional, you'd need to implement your own solution to forward input to it.

You could also use this to show a WebView, or play a video, or really render almost anything that could be displayed on the Android side.

[BraveEvidence]

@dsnopek Thanks for the reply. I did check the sample project attached. You're opening the camera preview on a button click.

1. I was expecting the sample project to demo more of a custom node in Godot which can help me render a native Android widget in an existing Godot scene along with Godot's 2d/3d node.
2. What do you mean by I need to implement your own solution to forward input to it for the EditText question which I asked. If I try to embed Exoplayer(Android library to play video) will I need to implement my solution again to forward play, pause etc as those functionality come out off the box from Exoplayer
3. On the Godot side for the sample project you have camera.gdshader file, is that really necessary, sorry I don't understand shaders.

[dsnopek]

1. I was expecting the sample project to demo more of a custom node in Godot which can help me render a native Android widget in an existing Godot scene along with Godot's 2d/3d node.

That's effectively what this does, just not with a new node, it does it with a new texture resource, ExternalTexture (which is the same way this feature works in Godot 3). The node I'm using in the demo is a ColorRect, and a very simple shader to draw the texture onto the ColorRect.

2. What do you mean by I need to implement your own solution to forward input to it for the EditText question which I asked. If I try to embed Exoplayer(Android library to play video) will I need to implement my solution again to forward play, pause etc as those functionality come out off the box from Exoplayer

Yes, in the context of adding a video player, you'd need to make your own controls for play, pause, etc in Godot, because all you're getting from Android is the display. Input still goes to Godot. If you wanted to show Android UI, you'd need to write some code to forward the input from Godot to Android so that the Android UI could receive touches.

3. On the Godot side for the sample project you have camera.gdshader file, is that really necessary, sorry I don't understand shaders.

I think it is, at least in the context of a camera, because the video feed from the camera is YUV, and the shader triggers some magic to convert it to RGB. I haven't tested it, but maybe you could use a TextureRect to show an ExternalTexture with normal RGB data? But if not, the shader I wrote is pretty universal - you could just copy it and it should work for any ExternalTexture regardless of the source.

[akien-mga]

Thanks!

[AThousandShips]

This broke building on MSVC, like this case:

• Store GLES over GL status in the OS to avoid direct renderer includes in the editor. #96525

See:

• Update rendering driver name on fallbacks. Fix rendering driver/method in the editor system info. #95887 (comment)

The problem is once again platform_gl.h being included where it shouldn't be

As a temporary workaround I'm using:

diff --git a/scu_builders.py b/scu_builders.py
index fc5461196f..ed7b729df3 100644
--- a/scu_builders.py
+++ b/scu_builders.py
@@ -337,7 +337,7 @@ def generate_scu_files(max_includes_per_scu):
     process_folder(["scene/animation"])
     process_folder(["scene/gui"])
     process_folder(["scene/main"])
-    process_folder(["scene/resources"])
+    process_folder(["scene/resources"], ["external_texture"])
     process_folder(["scene/resources/2d"])
     process_folder(["scene/resources/3d"])

[m4gr3d]

@BraveEvidence you can take a look at the Gast library which did something similar to what you are describing for Godot 3.x.

https://github.com/m4gr3d/GAST

Now that this functionality is available in Godot 4.x, it would be possible to update the Gast library to also support Godot 4.x.

[BraveEvidence]

@m4gr3d Thanks, I was kind of expecting something like this. Will you please upgrade the GAST plugin to Godot 4 as I don't have expertise for this and include a Simple Native Android sample, currently all the samples looks like for XR.

[m4gr3d]

@m4gr3d Thanks, I was kind of expecting something like this. Will you please upgrade the GAST plugin to Godot 4 as I don't have expertise for this and include a Simple Native Android sample, currently all the samples looks like for XR.

@BraveEvidence the project includes examples for uses with regular Android projects, see https://github.com/m4gr3d/GAST/blob/master/core/src/plugins/java/org/godotengine/plugin/gast/plugins/video/README.md#gast-video-plugin

In terms of update, this is unlikely to happen until next year, which is when Godot 4.4 (in which these changes will go) will be released.