EXR/HDR file import is very slow if using VRAM compression

[WebFreak001]

Godot version

v4.1.1.stable.arch_linux

System information

Godot v4.1.1.stable unknown - Arch Linux #1 SMP PREEMPT_DYNAMIC Wed, 02 Aug 2023 10:12:55 +0000 - Vulkan (Forward+) - integrated Intel(R) HD Graphics 620 (KBL GT2) () - Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (4 Threads)

Issue description

Sometimes when having an OpenEXR file in the project, the import takes multiple minutes for a 4k x 4k texture on my machine. This occurred when exporting the image from Blender, with half-float color depth.

Sometimes it imported much faster, in these cases I exported the EXR file with full-float color depth instead of half-float color depth. I'm not sure if that's really the root reason here though. I haven't tried the importing on this machine of the full-float file, only on my much faster PC, which was able to load the full float one quite quickly and also needed like 20s to load the half-float one, but I haven't done precise measurements. The time difference might not actually be that high.

Steps to reproduce

I used a kind of big blender scene for my rendered EXR file, it didn't work for me with a simple default cube render exported from Blender. You can try out the exr files from this folder (they are half-float exported and take up to multiple minutes to import): https://github.com/NanahiraCommunity/Birthday23/tree/da501510cc34ef9ca79650e6c2e6dd5fec0dd95d/shared/sky

The .blend file is also in there, if you want to render a full-float exr file. The resulting exr file is larger than 100 MiB, so I can't upload it to my github project. It takes about 2h to render on CPU (AMD Ryzen 7 5700X) in full size though.

See minimal reproduction project below for something to just try out.

Minimal reproduction project

slow_exr_repro.zip

when opening this project, it takes unreasonably long until the import is done. When moving the file into an already open godot project, the import also seems to be much faster for some reason. The .import file might also affect something here, though I'm not sure.

[Calinou]

• Related to Importing a Radiance HDR image for use in a PanoramaSkyMaterial freezes the editor #61948.

Can you reproduce this if you change the compress mode for the EXR to Lossless in the Import dock? I think VRAM compression can't represent data with the same precision as full-float EXR anyway.

[oxben]

I observe a similar behavior with an EXR file I use for the sky in my project in Godot 4.0.3.
Sometimes it takes 30 seconds to reimport it, and sometimes it seems almost instantaneous. The texture resolution is (only) 2048×1024.
Where is this Import dock to set the compress mode to lossless ?

[Calinou]

Where is this Import dock to set the compress mode to lossless ?

Select the EXR image in the FileSystem dock, go to the Import dock and change the compression mode there. Make sure to click Reimport afterwards.

[WebFreak001]

changing to lossless does indeed make it almost instant. Note that half-float is the slow one, I think full-float is much faster, but I need to render it again to actually check for sure. Will come back with the file here once that's done.

[clayjohn]

I think the reason full floats are fast is because they don't get compressed. For some reason our BPTC compression code is only used with half float formats. If full floats are used, then the compression is skipped:

godot/modules/cvtt/image_compress_cvtt.cpp

Lines 152 to 156 in f7bc653

	bool is_hdr = (p_image->get_format() >= Image::FORMAT_RH) && (p_image->get_format() <= Image::FORMAT_RGBE9995);
	if (!is_ldr && !is_hdr) {
	return; // Not a usable source format
	}

However, our import code sends the .exr image to be compressed regardless of if its full float or half float:

godot/editor/import/resource_importer_texture.cpp

Lines 608 to 609 in f7bc653

	const bool is_hdr = (image->get_format() >= Image::FORMAT_RF && image->get_format() <= Image::FORMAT_RGBE9995);
	const bool can_s3tc_bptc = ResourceImporterTextureSettings::should_import_s3tc_bptc();

godot/editor/import/resource_importer_texture.cpp

Lines 654 to 656 in f7bc653

	if (high_quality || is_hdr) {
	image_compress_mode = Image::COMPRESS_BPTC;
	image_compress_format = "bptc";

[WebFreak001]

here are full float exr files if you need some to test performance:

• sky1_full.exr (151 MiB)
• sky1_nofx_full.exr (133 MiB)

I used the VDB files from https://jangafx.com/software/embergen/download/free-vdb-animations/ for the clouds, which are CC0 - I also license my derivative work as CC0, so feel free to use sky1.exr, sky1_nofx.exr, sky1_full.exr and sky1_nofx_full.exr however you want.