MeshInstance's Create Outline Mesh option should weld vertices to avoid visible gaps

[QbieShay]

Describe the project you are working on

Stylized 3D game

Describe the problem or limitation you are having in your project

It is very common in 3D games to achieve sharp edges by using edge splitting (the same is done on the default cube for Godot)
The issue with this, is that auto-generating outline meshes from Godot is currently not functional on anything that has sharp edges (see godotengine/godot#55385)

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Imo, it would be useful to:

1. Add an API to weld vertices
2. Automatically weld overlapping vertices when generating outline mesh

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Change the "generate outline mesh" function in engine

If this enhancement will not be used often, can it be worked around with a few lines of script?

It can be done with an addon, but it would duplicate the base functionality which doesn't work for all meshes

Is there a reason why this should be core and not an add-on in the asset library?

See above

[Calinou]

Add an API to weld vertices

I believe this is something meshoptimizer can do (and is already used when generating shadow meshes).

The downside is that welding vertices (and making them all smooth) will break shaded shell/outline rendering, but most use cases of shell/outline rendering involve unshaded materials anyway. If you want shaded shell/outline rendering to work, you need to generate additional vertices at the locations of the split edges with custom normals (weighted face normals). This way, the whole outline mesh remains smooth, but it can have corners that look sharp. Outline generation will still work correctly too, assuming that the custom normals are used as expected.

This also applies at the source mesh level. If you apply this weighted normal trick on the source mesh, Godot's current outline generation should no longer result in visible gaps.

meshoptimizer can't automate the creation of new vertices with weighted face normals, so this is probably best left to custom code or manual artist work (see above).

[QbieShay]

I believe that by far the most common usecases for shells is unshaded outlines.

Perhaps a solution that could work is to have a combo of a shader+mesh generation that leaves the original mesh unchanged, by adding welded normals to one of the userdata channels? This can be done at import time