slint

[vigilanz]

I'd like to draw something that has to be rendered into a window of the Slint toolkit (Rust version of course).
Slint can fill an image from an OpenGL texture (https://releases.slint.dev/1.1.0/docs/rust/slint/struct.image#method.from_borrowed_gl_2d_rgba_texture)
and they also have an OpenGL example at https://github.com/slint-ui/slint/tree/master/examples/opengl_texture,
but I wasn't able yet to find out how to combine this with three-d.
Did anyone ever combine these 2 things or can give a hint?

[asny]

You need to do some native OpenGL code yourself. You can create a framebuffer with a texture in OpenGL (e.g. using the glow crate) and then use from_framebuffer to create a three-d RenderTarget that you can render into using the three-d functionality. You can create a three-d context using from_gl_context.

You can maybe also use into_framebuffer somehow.

[vigilanz]

Thanks. But I'm a bit confused, I thought it should somehow be possible to let three-d do everything and just send the resulting thing into slint at the right spot? At some point on an example on the slint side, I came across this line:

glow::Context::from_loader_function_cstr(|s| get_proc_address(s))

and I think I somehow saw "from_loader_function_cstr" on a three-d example as well. I have to dig again where exactly that was.

[vigilanz]

So what you describe, if I understand correctly, is basically the other way round, creating the context externally and feeding this into three-d?

[asny]

Well, it's really up to you. It's probably easiest to use three-d as normal and then use the low-level Context provided by three-d to create a framebuffer and attached texture and then use from_framebuffer to create a three-d render target. Then use three-d to render into that render target and finally transfer the result from the texture to slint. But it's just a suggestion.

[vigilanz]

OK yeah that sounds reasonable, I will give it a try, thanks.

[vigilanz]

So I thanks @asny, I figured this out now and the three-d scene ends in a slint Image component on the screen. I write a shortened version down here in case someone else needs this later. Please add suggestions if something is bad or wrong in the code below. I assume a width/height of 100 throughout the example. I hope I did not make mistakes when shortening this, but you should get the idea.

// On the slint side
component MainWindow inherits Window {
    in property <image> texture <=> image.source; // so you can call .set_texture() with an image on the window instance

    image := Image {
        preferred-width: 100px;
        preferred-height: 100px;
        min-width: 100px;
        min-height: 100px;        
    }
}

// On the code side
   // The p_context is a three_d::Context, so the high-level one
   pub fn render(p_context: &Context) -> slint::Image {
        unsafe { 
            let width = 100;
            let height = 100;

            // Setup texture
            let native_texture = p_context.create_texture().unwrap();
            p_context.bind_texture(TEXTURE_2D, Some(native_texture));
            p_context.tex_image_2d(TEXTURE_2D, 0, RGBA as i32, width, height, 0, RGBA, UNSIGNED_BYTE, None);
            p_context.tex_parameter_i32(TEXTURE_2D, TEXTURE_MIN_FILTER, LINEAR as i32);
            p_context.tex_parameter_i32(TEXTURE_2D, TEXTURE_MAG_FILTER, LINEAR as i32);
            
            // Setup framebuffer and attach texture
            let framebuffer = p_context.create_framebuffer().unwrap();
            p_context.bind_framebuffer(FRAMEBUFFER, Some(framebuffer));
            p_context.framebuffer_texture_2d(FRAMEBUFFER, COLOR_ATTACHMENT0, TEXTURE_2D, Some(native_texture), 0);

            // Create a rendertarget which will draw into the framebuffer
            // and thus into the texture
            let rendertarget = RenderTarget::from_framebuffer(p_context, width as u32, height as u32, framebuffer);       

            // Rendering as normal in the three-d examples
            rendertarget
                .clear(ClearState::color_and_depth(0.0, 0.0, 0.0, 1.0, 1.0))
                .render( /* your render code here */);

            // This will extract a slint::Image which can be used as a source of the Image component by slint
            slint::BorrowedOpenGLTextureBuilder::new_gl_2d_rgba_texture(native_texture.0, (width as u32, height as u32).into()).build()
        }
    }

// Now, in the main function of the program where the window is created
    // Create main window and the UI interface
    let main_window = MainWindow::new().unwrap();

    // Make a clone
    let win_clone = main_window.as_weak();
    main_window.window().set_rendering_notifier(move |state, graphics_api| {
        match state {
            // RenderingSetup is the place where the GL context needs to be created
            slint::RenderingState::RenderingSetup => {
                match graphics_api {
                    slint::GraphicsAPI::NativeOpenGL { get_proc_address } => unsafe { 
                        // This is a low-level OpenGL context, NOT the p_context that
                        // the render() function wants to have.
                        let gl_context = three_d::context::Context::from_loader_function_cstr(|s| get_proc_address(s));
        
                        // Convert this to a three-d::Context
                        let tmp = Arc::new(gl_context);
                        let context = Some(three_d::Context::from_gl_context(tmp).expect("Cannot convert OpenGL context"));

                        // TODO: Store this somewhere
                        store_somehow(context);
                    }
                    _ => { panic!("Wrong graphics API, need OpenGL"); }
                }
            },

            // This is when the actual rendering happens
            slint::RenderingState::BeforeRendering => {
                // TODO: Get the stored three_d::Context somehow
                let context = load_somehow();

                // Do the actual rendering, returning a slint::Image
                let tmp_image = render(&context);

                // Set the texture property of the slint window
                win_clone.upgrade().unwrap().set_texture(tmp_image);
                
                // Force an immediate redraw
                win_clone.upgrade().unwrap().window().request_redraw();
            }

            _ => {}
        }

    }).expect("Unable to set rendering notifier");
    
    // Run the event loop
    main_window.run().unwrap();

[asny]

Nice 💪 Thanks for sharing @vigilanz 🙏 The code looks good, at least to me. Maybe it's a bit expensive to create a new texture and frame buffer each frame, it could be reused as long as the window size is the same, but it might not matter much.

[bryce-happel-walton]

I've implemented this with a combinitation of the three-d triangle example, and the result is that the window just flashes black and the slint UI is very broken.

The rectangular region is the actual slint image where the texture is being written, however it is in the wrong place. That is the only consistency.

[bryce-happel-walton]

I found this to be a bug with skia. Everything works normally with femtovg