Change Shadow Rendering Pipeline

[mrjustaguy]

Describe the project you are working on

Open World Game

Describe the problem or limitation you are having in your project

Having to tweak settings around a lot to balance out Shadow Acne and Peter Panning

Describe the feature / enhancement and how it helps to overcome the problem or limitation

By Default Render Shadows with Reverse Face culling, and ignore all Biasing (so as if bias and normal bias were both at 0, for all shadows), instead, Shrink the Shadow Meshes by some user defined value (Default of 0.0025 seems to do the trick really well)

Pros:

1. It is a "Just Works" Solution, with way fewer edge cases compared to the Current normal Face Culling with biasing, so User is generally care-free.
2. Shadow Acne is a thing of the Past, as it is moved to the area within the mesh between the Original Mesh and the Shadow Mesh (which is typically never visible)
3. Peter Panning is also a thing of the Past * (a light leak might appear where two objects intersect, however that is why the shadow mesh is shrunk to hide that)
4. Allows for Significant reductions in Shadow Mesh Polycount for essentially the same shadow quality, without introducing Self Shadowing (aka Shadow Acne) allowing the user to Improve Shadow Performance Significantly
5. All Shadow Types Benefit (Directional Shadows (moderate benefits), Omni Shadows (huge benefits), Spot Shadows (small benefits))

Cons:

1. Shadows are slightly Uniformly shrunk, by a constant value, This is only clearly observable in a direct side-by-side comparison
2. Some meshes might exhibit holes, due to the shrinking, because of some faces not being connected properly. This is solved in Blender with Weld, but the exact values needed might vary from mesh to mesh (Default of 0.001 seems to do the trick well), as going to aggressive with the merge distance can incorrectly merge stuff (thus losing geometry detail), however probably not a big problem when only dealing with Shadows

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Have Two Shadow Rendering Modes:

1. Enhanced/Automatic/... Find appropriate name (this method)
2. Classic (current)

Using this Method Do the following:

1. Flip all Shadow Mesh Triangles (so Reverse Face Culling On)
2. Shrink all Shadow Meshes by Shrink Value (default 0.0025 probably) This setting should be on a Per Surface Basis
3. Set All Shadow Bias and Normal Bias values to 0 - Normal Bias is actually still useful for really low Res Shadows, so a Value of 1 still makes sense, but nothing higher
4. Optionally Reduce Shadow Mesh LOD (like Generate shadow meshes from a lower LOD level than the source mesh #2600 (comment) for example)

If this enhancement will not be used often, can it be worked around with a few lines of script?

Yes, you can set Lights to Reverse Face Cull and Shrink them a little in Blender or using only Base/Standard Material 3D with separate Meshes for Shadows and Normal Rendering.

Is there a reason why this should be core and not an add-on in the asset library?

It turns the Shadow Rendering (which is core) into "It just Works"

NOTES

Flip Mesh Triangles can be Done by Changing Cull Mode to Front in Base/Standard Material 3D
Shrink can be done using Base/Standard Material 3D Grow

[mrjustaguy]

Some Comparisons:
Omni-Spot With Proposal (Left to Right) - Bias 0, Reversed Face Culling, Shrunk by 0.003 - 3k Shadow Triangles

Omni-Spot Without Proposal (Left to Right) - Default Bias G4 A7, Normal Face Culling, No Shrinking - 30k Shadow Triangles

Directional Light with Proposal - Bias 0, Reversed Face Culling, Shrunk by 0.003 - 3k Shadow Triangles

Directional Light Without Proposal - Default bias G4 A7, Normal Face Culling, No Shrinking - 30k Shadow Triangles

30k Triangle Shadow Mesh

3k Triangle Shadow Mesh

[mrjustaguy]

I have found a rather, unusual benefit of this approach...
It seems due to Face Culling, this approach can in some cases result in significant increases in shadow rendering performance, without cutting triangle counts down.
Here's a Test Project, 16k Shadow Map on an 80k triangle terrain, 4 splits -
Shadows.zip
In G4 A15 On a GTX 1050 Ti when the shadow casting terrain (2nd mesh instance) is set to front face culling Shadow rendering takes 0.17ms. When changing to back face culling, the Shadow rendering takes 1.18ms

it seems this approach doubly improves shadow rendering performance, both because you can significantly simplify meshes without causing shadow quality degradation (aside from a very slight shrinking of the shadows), and Shadow Casters such as Terrains face cull most of their triangles

[mrjustaguy]

Closing as users can already do this, and would be a fairly big compatibility breakage that's probably not worth it for most users, especially if/when newer shadowing techniques become available

[ODtian]

Will this approach improve quality of self-shadow (mean it by object casting shadows on it self) and shadows that casted by close distance object?

[mrjustaguy]

if you mean deal with shadow acne, yes, though you will have to shrink the shadow geometry a little to avoid light leaking (similar to peter panning but with it acting as more of a halo around shadow casting geometry)