render depth map

[cdfan0627]

Thank you for your great work.
How can I use your code to render the depth map?

[half-potato]

Does it need to be differentiated through? It's possible to add but it is not trivial.
The expected termination depth would need to be added to splinestate here:
https://github.com/google/ever/blob/fe6811328991ab03edcff4ef0462292e81be5c3f/splinetracers/fast_ellipsoid_splinetracer/slang/spline-machine.slang#L32
and here:
https://github.com/google/ever/blob/fe6811328991ab03edcff4ef0462292e81be5c3f/splinetracers/fast_ellipsoid_splinetracer/structs.h#L53

Then, in the spline machine file, the update and inverse_update_dual functions would need to updated to track how the expected termination depth would change.
To extract the result, I would access the states attribute on extras['saved'].states. This will contain the full state. You would need to do something similar to how the distortion loss is calculated, concatenating another dimension to the output:
https://github.com/google/ever/blob/fe6811328991ab03edcff4ef0462292e81be5c3f/splinetracers/fast_ellipsoid_splinetracer.py#L81
At this point, the tensor for the loss wrt the output will be a larger tensor, and we need to capture the derivative from the depth here:
https://github.com/google/ever/blob/fe6811328991ab03edcff4ef0462292e81be5c3f/splinetracers/fast_ellipsoid_splinetracer/slang/backwards_kernel.slang#L301
Add the expected termination to the extract_color function:
https://github.com/google/ever/blob/fe6811328991ab03edcff4ef0462292e81be5c3f/splinetracers/fast_ellipsoid_splinetracer/slang/spline-machine.slang#L143
https://github.com/google/ever/blob/fe6811328991ab03edcff4ef0462292e81be5c3f/splinetracers/fast_ellipsoid_splinetracer/slang/spline-machine.slang#L147

I think this would work. A little involved, as you can see.

[cdfan0627]

Can I render a depth map in gaussian_renderer/ init.py like GaussianShader? If it's possible to render the depth map in init.py, it should be automatically differentiated.

[half-potato]

Yes. Interesting approach.