Add support for game window embedding on macOS

[jrapf32]

Describe the project you are working on

I'm working on a game that needs object selection, and it's getting tiresome having to switch to the editor, hit the 2d button in the game tab, and then switch back.

Describe the problem or limitation you are having in your project

Currently, "Game embedding is not available on your OS." People use Macs to make games, which could be a problem.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Adding it would help.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

It should work like all other platforms.

If this enhancement will not be used often, can it be worked around with a few lines of script?

If this is not implemented, people can still use Cmd+Tab to switch between windows, but it will be inconvenient.

Is there a reason why this should be core and not an add-on in the asset library?

This functionality is built into the editor and can't be changed by an addon.

[tetrapod00]

The need for this is known, and there are some ideas on how to accomplish it. See comments in the original PR implementing the feature: godotengine/godot#99010 (comment) and godotengine/godot#99010 (comment).

[ettiSurreal]

Since the proposal text is vague and it gets many views based on the reactions, here's a more thorough explanation.

The reason embedding doesn't work on MacOS, isn't captured by screen-capturing applications, is disabled in single window mode, and often runs into window-related edge cases (like window-changing hotkeys from the system), is because it was implemented by having the game window be the parent of the editor and making it borderless. So in practice this means that it's simply a borderless window glued to the editor.

Potentially all of this could be solved by re-implementing embedding by directly getting the Vulkan/Metal frame buffer from the game process (or a hidden window if that's not possible?), and displaying it over a TextureRect(?) in the Game workspace. Someone will also have to figure out how to account for Input and Audio, the latter should also preferably get emitted by the editor window so recording it would also capture the audio.
Due to obvious benefits this should be the "default" way embedding should work where possible, with the sub-window approach being a fallback when it's not supported. This also means that if possible the implementation should be kept the least vendor specific.
Some pointers I've seen that someone could use as a starting point:
godotengine/godot#99010 (comment)
godotengine/godot#99010 (comment)

(this is all just information I collected from people smarter than me, please correct this if I got anything wrong)

[Nordup]

I've got a working macOS game embedding on my fork
https://github.com/thegatesbrowser/godot/blob/9fe18c2a83ca47f3ca777ce3e7d1f83f05fbdf43/drivers/vulkan/rendering_device_driver_vulkan.cpp#L1437

It uses IOSurface with Vulkan VK_EXT_metal_objects extension
Main process exports IOSurface and child process imports it and copies it's swapchain into that shared texture

I use some ipc library to handle interaction and sending games input

I can work on porting the code to godot with the help of other contributors
I'm waiting for this feature anyway so I don't have to deal with maintaining it and solving merge conflicts :)

[nubels]

I've got a working macOS game embedding on my fork https://github.com/thegatesbrowser/godot/blob/9fe18c2a83ca47f3ca777ce3e7d1f83f05fbdf43/drivers/vulkan/rendering_device_driver_vulkan.cpp#L1437

It uses IOSurface with Vulkan VK_EXT_metal_objects extension Main process exports IOSurface and child process imports it and copies it's swapchain into that shared texture

I use some ipc library to handle interaction and sending games input

I can work on porting the code to godot with the help of other contributors I'm waiting for this feature anyway so I don't have to deal with maintaining it and solving merge conflicts :)

I wanted to try your implementation of game embedding so I compiled Godot from the master branch of your fork but for me, it's Godot 4.3 and there's no embedding. I don't know if I missed something

[Nordup]

I wanted to try your implementation of game embedding so I compiled Godot from the master branch of your fork but for me, it's Godot 4.3 and there's no embedding. I don't know if I missed something

Yeah, sorry for not being clear
My implementation doesn't work in the editor

I'm using it from within the application I'm developing
To launch child processes and display them in the main app

[nubels]

I wanted to try your implementation of game embedding so I compiled Godot from the master branch of your fork but for me, it's Godot 4.3 and there's no embedding. I don't know if I missed something

Yeah, sorry for not being clear My implementation doesn't work in the editor

I'm using it from within the application I'm developing To launch child processes and display them in the main app

Ah okay, I see. :)

On another note, since you mentioned the help of other contributors, maybe you can get in touch with @stuartcarnie and @nevyn? They mentioned they also want to look into game embedding on mac. :)

godotengine/godot#99010 (comment)