Metal: Multiview support #98803

stuartcarnie · 2024-11-03T22:18:48Z

This PR adds multi-view support for the Metal rendering device driver.

Testing

GodotStereoDemo

A screenshot of XR enabled and rendering multiple views, thanks to @BastiaanOlij

Clearing attachments

It is not easy to test clearing a subset of an attachment when multi-view is enabled in Godot outside of XR, so I validated it by creating a multi-view frame buffer and clearing it and then validating the output in the Metal Debugger.

The following screenshot has the following properties:

A 1024x1024 frame buffer with a view count of 2.
Clear rect of { x=10, y=10, w=50, h=50 }
Clear colour of Color.AQUAMARINE
Clear depth of 0.5

In this screenshot, we can see the following:

Two views of a multi-view frame buffer, ❶ and ❷
The colour texture, ❸ and ❹, is cleared to aquamarine at the rectangle 10, 10, 60, 60
The depth texture, ❺ and ❻, is cleared to 0.5 at the rectangle 10, 10, 60, 60

The GDScript to set up the multi-view frame buffer:

var framebuffer2: RID
var clearColors2 := PackedColorArray([Color.AQUAMARINE])
var framebuf_texture
var d_framebuf_texture

func create_multiview():
	
	rd = RenderingServer.get_rendering_device()
	
	
	#  <<  --  FRAMEBUFFER  --  >>
	
	var attachments = []
	
	#  <<  -- COLOR  -- >>
	
	var tex_format := RDTextureFormat.new()
	tex_format.texture_type = RenderingDevice.TEXTURE_TYPE_2D_ARRAY
	tex_format.height = 1024
	tex_format.width = 1024
	tex_format.array_layers = 2
	tex_format.format = RenderingDevice.DATA_FORMAT_R8G8B8A8_UNORM
	tex_format.usage_bits = RenderingDevice.TEXTURE_USAGE_COLOR_ATTACHMENT_BIT | RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT  

	var tex_view := RDTextureView.new()
	framebuf_texture = rd.texture_create(tex_format, tex_view)
	print("framebuffe texture : ", rd.texture_is_valid(framebuf_texture))
	
	var af := RDAttachmentFormat.new()
	af.set_format(tex_format.format)
	af.set_samples(RenderingDevice.TEXTURE_SAMPLES_1)
	af.usage_flags = tex_format.usage_bits
	attachments.push_back(af)
	
	#  WORKS, but is actually slower than with texture_get_data()
	
	#t2drd = Texture2DRD.new()
	#t2drd.texture_rd_rid = RID()
	#t2drd.texture_rd_rid = framebuf_texture	
	
	#  <<  --  DEPTH  --  >>
	
	var dtex_format := RDTextureFormat.new()
	var dtex_view := RDTextureView.new()
	dtex_format.texture_type = RenderingDevice.TEXTURE_TYPE_2D_ARRAY
	dtex_format.height = 1024
	dtex_format.width = 1024
	dtex_format.array_layers = 2
	dtex_format.format = RenderingDevice.DATA_FORMAT_D16_UNORM
	dtex_format.usage_bits = RenderingDevice.TEXTURE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT
	d_framebuf_texture = rd.texture_create(dtex_format, dtex_view)
	print("depth buffer texture : ", rd.texture_is_valid(d_framebuf_texture))
	
	
	var daf := RDAttachmentFormat.new()
	daf.set_format(dtex_format.format)
	daf.set_samples(RenderingDevice.TEXTURE_SAMPLES_1)
	daf.usage_flags = dtex_format.usage_bits
	attachments.push_back(daf)
	
	# ---
	
	var framebuf_format := rd.framebuffer_format_create(attachments, 2)
	framebuffer2 = rd.framebuffer_create([framebuf_texture, d_framebuf_texture], framebuf_format, 2)
	print("framebuffer : ", rd.framebuffer_is_valid(framebuffer2))

and some code to clear it:

func draw_test():
	var dlist := rd.draw_list_begin(framebuffer2,
		RenderingDevice.INITIAL_ACTION_CLEAR, RenderingDevice.FINAL_ACTION_READ,
		RenderingDevice.INITIAL_ACTION_CLEAR, RenderingDevice.FINAL_ACTION_READ,
		clearColors2, 0.5, 3, Rect2(10, 10, 50, 50))
	rd.draw_list_end()

stuartcarnie · 2024-11-03T22:19:16Z

@BastiaanOlij and @RevoluPowered might be interested in this PR

BastiaanOlij · 2024-11-03T23:59:16Z

Amazing stuart! I'm going to see how far I get with implementing XR_KHR_metal_enable ontop of this, that should be a great test for it.

dsnopek · 2024-11-04T17:02:12Z

Thanks!

I can't review the actual rendering code (it's beyond me) but I tested this with a simple XR project that I threw together using MobileVRInterface and it worked great in my testing on an M2 Mac mini!

However, with verbose output turned on, I'm getting this debug message spammed:

And in the console, it's spamming blank lines (probably one per frame?)

So, there's something wrong with that vformat(), but we probably shouldn't be printing something every frame anyway.

stuartcarnie · 2024-11-04T19:06:14Z

Thanks @dsnopek it's now removed!

AThousandShips · 2024-11-05T10:56:42Z