Viewport.push_input's second parameter: in_local_coords not working

[jinyangcruise]

Godot version

v4.0.beta17.official.c40020513

System information

Windows 10, Forward+, NVIDIA GeForce GTX 860M

Issue description

I use a SubViewport as the Sprite3D's texture to render a 2D-scene in this 3D scene.

In order to trigger mouse event into the 2D scene, I use this script:

extends Node3D

func _input(event: InputEvent):
	$SubViewport.push_input(event, false) 

The second parameter of push_input behaves like having no effect whether it's trueor false.

this function's information in GODOT API DOC:
https://docs.godotengine.org/en/latest/classes/class_viewport.html#class-viewport-method-push-input

Steps to reproduce

1. open minimal reproduction project below
2. run this project and check if you can click the buttons (by moving some left-offsets)
3. modify $SubViewport.push_input(event, false) second parameter to true or false
4. re-run and check

Minimal reproduction project

test for viewport.zip

[jinyangcruise]

when running this project, also see an error (not sure whether related to this bug or anther bug):

E 0:00:01:0341   setup_local_to_scene: Condition "proxy.is_valid()" is true.
  <C++ Source>   scene/main/viewport.cpp:88 @ setup_local_to_scene()

and when I put the SubViewport node below the Sprite3D node, more errors occur:

[Zireael07]

Why close?

[jinyangcruise]

Why close?

my bad, reopened

[Sauermann]

setup_local_to_scene is an unrelated issue.

The difference between $SubViewport.push_input(event, false) and $SubViewport.push_input(event, true) is $SubViewport.get_final_transform(). Since this is in your case the identity-transform, the second parameter doesn't change anything. So push_input works as expected.

In func _input(event: InputEvent):, the event is in the coordinate system of the root-viewport and not the viewports embedder, which is the Sprite3D-Texture.

As far as I can tell this is a problem with the coordinate systems. And you are missing the transformation from the mouse-click within the 3D-world to the 2D local coordinates of the Texture.

In order to make $SubViewport.push_input(event, false) work, you would need to adjust the events position, so that it is in the local coordinate system of the Sprite3D.
I haven't done this myself, by my approach would be

1. Create an Area3D + CollisionShape3D combination at the same location/size as the Sprite3D and override their _input_event function.
   This allows you to get an event of the mouse-click at the position of the Sprite3D
2. Transform the event-position so that it aligns with the Texture-coordinates and use that event in the push_input function.

Maybe we could improve the push_input documentation and give a few examples, what the embedder is.

[jinyangcruise]

I see. Thank you very much.

Do you have any plan to make an out-of-the-box method to deal with such cases?

And I'm curious about the exception of embedded Windows and SubViewports in InputEvent process.
In this artical: https://docs.godotengine.org/en/latest/tutorials/inputs/inputevent.html

When sending events to its child and descencand nodes, the viewport will do so, as depicted in the following graphic, in a reverse depth-first order, starting with the node at the bottom of the scene tree, and ending at the root node. Excluded from this process are embedded Windows and SubViewports.

and

Since Viewports don't send events to other SubViewports, one of the following methods has to be used:
- Use a SubViewportContainer, which automatically sends events to its child SubViewports during 
Node._input() and Node._unhandled_input().
- Implement event propagation based on the indivitual requirements.

btw, I notice there isn't a 3D version SubViewportContainer.

Any problem prevents Viewports to send events to its SubViewports? (complex, no need and no time for this at the moment, maybe?)

I ask this because it's kind of counter-intuitive and it might break the simplicity and coherence of Godot's node-based design concept.

[Sauermann]

It was a design decision, that SubViewports by default don't receive events, because there are use-cases for Viewports, which don't need events.
In order to send events to SubViewports it is necessary, to provide a coordinate system-conversion (for mouse-clicks and other events with positions). SubViewportContainer does exactly this for the 2D-use-case (SubViewports don't have a position).

Embedded Windows only receive input events, if they are focused, so there is a different mechanism for them and the above mentioned restriction is actually a necessity. (See step 2 in the cited InputEvent tutorial)

I believe, that a 3D-variant of SubViewportContainer could be an interesting idea.
Feel free to create a proposal for this on https://github.com/godotengine/godot-proposals

[Sauermann]

I have created a small project file that shows the general idea of how to implement event propagation for 3D.

SubViewportEventPropagationTutorial.zip

Advanced features like Drag and Drop would probably need more work.

This function does the heavy lifting of the coordinate-transforms.

func _on_input_event(_camera: Camera3D, event: InputEvent, pos: Vector3, _normal: Vector3, _shape_idx: int):
	# Position of the event in Sprite3D local coordinates.
	var texture_3d_position = s.get_global_transform().affine_inverse() * pos
	if !is_zero_approx(texture_3d_position.z):
		# Discard event because event didn't happen on the side of the Sprite3D.
		return
	# Position of the event relative to the texture.
	var texture_position: Vector2 = Vector2(texture_3d_position.x, -texture_3d_position.y) / s.pixel_size - s.get_item_rect().position
	# Send mouse event.
	if event is InputEventMouse:
		var e: InputEvent = event.duplicate()
		e.set_position(texture_position)
		e.set_global_position(texture_position)
		v.push_input(e)

[jinyangcruise]

Thx again! A lot to learn from your turorial.
I will consider to create a proposal for a 3D-variant of SubViewportContainer after doing some research on the current 2D SubViewportContainer : )

[Sauermann]

I just found, that there is already a proposal for a similar use-case: godotengine/godot-proposals#4093

[jinyangcruise]

hope there wil be more progresses made to break totally the wall of 2D world and 3D world : P