Metal backend: optional sampler-aware user textures

[ssh4net]

Version/Branch of Dear ImGui:

Version 1.92, Branch: docking

Back-ends:

Metal

Compiler, OS:

MacOS

Details:

One thing that came up while integrating Dear ImGui's Metal backend in OpenImageIO imiv is that the current backend assumes a user texture is just an MTLTexture* sampled with a backend-defined linear sampler. That works for the usual UI case, but it becomes awkward in image tools where filtering mode is part of the view state. In imiv, the same image may be shown in two places with different expectations: the main view may follow a linear or nearest interpolation toggle, while the close-up view wants nearest sampling so individual pixels stay readable. Right now that means carrying a local fork of imgui_impl_metal, because the Metal shader hardcodes linear sampling.

What would help here is a small opt-in path that lets the Metal backend accept a user texture ID built from a (texture, sampler) pair, while leaving the current raw-MTLTexture* path alone. Existing code would keep working exactly as it does now. Plain MTLTexture* IDs would still use the backend default sampler, and only callers that need it would opt into a sampler-aware wrapper. A possible API shape is:

IMGUI_IMPL_API ImTextureID ImGui_ImplMetal_CreateUserTextureID(id<MTLTexture> texture,
                                                               id<MTLSamplerState> sampler_state);
IMGUI_IMPL_API void ImGui_ImplMetal_DestroyUserTextureID(ImTextureID tex_id);

On the render side, the backend would accept either a plain texture or a small backend-owned wrapper, bind the attached sampler when present, fall back to the default sampler otherwise, and change the Metal shader to take sampler [[sampler(0)]] instead of baking linear filtering into the shader source. That is a small change, but it removes a real fork point for Metal applications that care about how images are sampled.

This also seems like a better fit than draw callbacks. Callbacks can change sampler state, but the need here is not really custom rendering. It is just normal image presentation where different widgets need different filtering in the same frame. In imiv, the requirement is simple: the main image follows the interpolation setting, and the close-up view stays nearest. Having that travel with the texture ID is simpler to use and easier to reason about than pushing ordinary image widgets through callback-driven render-state changes.

I have a working workaround in https://github.com/ssh4net/OpenImageIO/blob/_imgui/src/imiv/external/imgui_impl_metal_imiv.mm
but maybe there are better way for this.

[ocornut]

Using Draw Callback would be a better fit because we can tend toward providing a backend agnostic solution for it.

Some backends already provide (among other things) a way to alter this via the ImGui_ImplXXXX_RenderState struct but that's pretty advanced.

The general plan to provide ImDrawCallback_SetSamplerLinear ImDrawCallback_SetSamplerNearest symbols to do exactly just that (90%+ of the requests for changing draw state are for linear vs nearest) and it would work for all backends.

I actually have the code for most backends for not for Metal.

Can you help reworking the Metal backend so that:

• The sampler is not hard-coded in the fragment shader but a parameter to it.
• The backend creates both linear and nearest sampler.
• (If the above doesn't align with Metal align, then: the backend creates both linear and nearest shader?)
• Add functions to change sampler (which may more or less easy to use depending on their signature).

That above I can potentially merge today to get ahead, even if it's not the 100% solution.

My WIP patch for e.g DX12 does:

                if (pcmd->UserCallback == ImDrawCallback_ResetRenderState)
                    ImGui_ImplDX12_SetupRenderState(draw_data, command_list, fr);
                else if (pcmd->UserCallback == ImDrawCallback_SetSamplerLinear)
                    ImGui_ImplDX12_SetupSamplerLinear(command_list);
                else if (pcmd->UserCallback == ImDrawCallback_SetSamplerNearest)
                    ImGui_ImplDX12_SetupSamplerNearest(command_list);

imgui.h

#define ImDrawCallback_ResetRenderState     (ImDrawCallback)(-8)
#define ImDrawCallback_SetSamplerLinear     (ImDrawCallback)(-16)   // FIXME-WIP: May move to function pointers stored in ImGuiPlatformIO
#define ImDrawCallback_SetSamplerNearest    (ImDrawCallback)(-24)   // FIXME-WIP: May move to function pointers stored in ImGuiPlatformIO

The only thing I am not sure if using defines vs function pointers stored in io struct.
The later feels nicer but would requires us to probably redesign the callback signature. While the symbolic define can be made to work today as they are processed directly in the render loop where we have access to all the data.

[ssh4net]

@ocornut Yes, OpenGL and Vulkan can do this without digging inside a backend code.

If you will be fine, if this will be a Codex GPT5.4 ExtraHigh I can help with MacOS for sure. Will try to be strict with LLM as much as possible to follow your coding style.

[ocornut]

Attached is a WIP patch (that doesn't include the Vulkan stuff which needed more work) for reference.

imgui-15508d2-WIP add ImDrawCallback_SetSamplerLinear, ImDrawCallback_SetSamplerNearest (DX11, DX12, GL3, SDLGPU).patch

I don't care how it's coded as long as it is rigorously careful and respectful of my attention.
If you know Metal well (I don't) honestly this seems easy.
Even if I can't settle on adding the define today, if I can merge 90% of it (which I did already for most others backends apart from Vulkan) then it becomes a <10 lines patch for you.

[ocornut]

I'm currently working on a better standard for this across backends, so best to wait until I'm done (soon).