Add external camera feed and occlusion based on a depthmap #95187
Conversation
This version of occlusion makes use of the depth buffer initialisation. No blending of the occlusion edges is done.
AThousandShips
changed the title
|
Please open a proposal to track the support and details of this feature |
There was a problem hiding this comment.
Please do not include your own build scripts in your PR
|
@AThousandShips yes, this draft is only for communication purposes with the XR team |
| @@ -748,6 +753,27 @@ void TextureStorage::texture_free(RID p_texture) { | |||
| texture_owner.free(p_texture); | |||
| } | |||
|
|
|||
| RID TextureStorage::texture_set_external(RID p_texture, int p_width, int p_height) { | |||
There was a problem hiding this comment.
We've discussed this before, but if we're going to call this "external" (which I think makes sense because GL_TEXTURE_EXTERNAL_OES) we need to rename TextureStorage::texture_create_external() to something else.
There was a problem hiding this comment.
agreed but still open to suggestions :)
There was a problem hiding this comment.
@dsnopek I'm going to open a proposal and clean the code so that it can be reviewed. What would be a good replacement for texture_set_external ? Something like texture_reallocate ?
|
Alright, had a quick review of the code during our ARCore meeting. In general lines this is a nice re-implementation of the original ARCore functionality from Godot 3's defunct ARCore PR. The only thing I think we need to change is how occlusion is handled, these are a lot of changes that are very specific to how ARCore handles this, while we have a similar need to solve this for OpenXR when pulling in the depth buffer from the depth sensor but there we're leaving it to the XR compositor to overlay things. I'd like to change this to remove the current occlusion implementation and just focus on re-introducing the external texture support and camera feed for the background. Then deal with occlusion in a more holistic way that also covers the needs for OpenXR. This will involve copying the depth data into the depth buffer before the opaque pass (is also more efficient), where the depth buffer data can come either from ARCore, OpenXR, or any other source, and then applying the camera image later as a copy possibly with an alpha blend (ergo we'll have the background set to transparent, where we haven't rendered geometry due to the prepopulated depth buffer, we'll be underlying the camera image. |
|
cc @clayjohn we'll need to discuss in more detail during one of the render meetings to finalise the approach for preloading a depth buffer. |
As this is my first proposal I hope I did it right, tell me if this is ok: |
This version of occlusion makes use of the depth buffer initialisation. No blending of the occlusion edges is done.