ImDrawList::AddPolyline() misses pixels

[memononen]

Version/Branch of Dear ImGui:

1.92.7

Back-ends:

n/a

Compiler, OS:

win11, msvc

Full config/build information:

No response

Details:

When drawing a line using ImDrawList::AddPolyline(), from pixel center to pixel center, one of the end pixels may not be filled.

This is due to AddPolyline() not drawing line caps. A butt cap would fix the issue.

Butt caps are simple to implement, we just need to extend the first and last point along the line direction.

void ImDrawList::AddPolyline(const ImVec2* points, const int points_count, ImU32 col, ImDrawFlags flags, float thickness)
{
    ...

        if (use_texture || !thick_line)
        {

            const float cap_size = thickness * 0.5f;

            // If line is not closed, the first and last points need to be generated differently as there are no normals to blend
            if (!closed)
            {
                const ImVec2 dir0(-temp_normals[0].y, temp_normals[0].x); // Normal to direction
                temp_points[0] = points[0] - dir0 * cap_size + temp_normals[0] * half_draw_size;
                temp_points[1] = points[0] - dir0 * cap_size - temp_normals[0] * half_draw_size;
                // The other end point cannot be done here, since it will be overridden by the loop below!
            }

            // The loop...

            if (!closed)
            {
                const ImVec2 dir1(-temp_normals[points_count-1].y, temp_normals[points_count-1].x);  // Normal to direction
                temp_points[(points_count-1)*2+0] = points[points_count-1] + dir1 * cap_size + temp_normals[points_count-1] * half_draw_size;
                temp_points[(points_count-1)*2+1] = points[points_count-1] + dir1 * cap_size - temp_normals[points_count-1] * half_draw_size;
            }
        }
    ...
}

The end cap could be made anti-aliased by adding extra division that is faded to zero alpha. Or the whole endcap could be baked into texture in which case round caps could be supported.

Screenshots/Video:

The image tagged after is rendered with butt caps as per above explanation.

Minimal, Complete and Verifiable Example code:

No response

[ocornut]

Linking to #7972 which might or not handle that, tho I'm generally favorable to merging simple iterative fixes.
(FYI initially #7972 was aimed at solving #2964, then added a bunch of extra features, which was a bit overwhelming to me, so we reworked #7972 to first have a "minimum" version, there's currently a commit to remove feature from it, with the idea that we can incrementally review/merge things).

[ocornut]

For both this #9360 and #9359 or any changes of that nature, my general process would be to (1) evaluate the change in cost or complexity (here in both cases this is a non-issue) and (2) try to understand how it would affect things in existing code-bases.

For the later, any explicit test cases or a mini testbed of variations would be great.
For this sort of stuff I usually temporarily bind the changes into e.g. a keyboard modifier ImGui::GetIO().KeyShift or an easy toggle.

Hovering a window border, with current code and your change:

In Demo->Examples->Custom Rendering only one end seems altered?

imgui-c869f5a-WIP #9360 ImDrawListAddPolyline() misses pixels.patch

I'm not against breaking changes if going in the right direction, but I'd probably need a survey of what is changing so we can fix things or direct users into detecting issues and fixing them.

[memononen]

Looks like I was over confident about the solution :) I tried to treat this and #9359 as separate issues, but I think they are tangled together (might make sense to merge the issues or close the other one).

This is compound issue, I try to walk through my thinking:

Part of the problem is that due to the ends not being antialised, sometimes we might miss the pixel at end point due to the triangle rasterization rules.

The second issue is stemming from #9359, and is: what are the coordinates you pass to drawlist? This is separate issue from line placement but related.

For consistent results, the coordinates should be pixel coords starting from the upper left corner of the render target's first pixel (this is the standard). I think this is true for most of the cases in drawlist, except for lines, which are offset by a half pixel. My understanding is that it is done so that you don't get blurry 1px lines out of the box. And for that reason that method takes "pixels" as input, which breaks the consistency of the API.

Now combining this offset with the non-aliased ends draws the desired result, but it's kind of not-correct twice :)

I think if we just fixed the line-end and no AA, like I proposed earlier you will get too long lines.
If we fixed just the AA, there's likely some cases where you have too long lines.
Making the coords consistent and offsetting lines to the side should bunch of issues (but not the one in this issue)

I will test the following to see if my thinking works:

1. Line placement

• implement center/inside/outside (+legacy?) stroke placement
• remove half pixel offset from line API (and others if needed)
• implement offsetting based on line width
• see how things break :)
• This should get us to the state where we have consistent assumptions and coordinate system for the draw API, might miss pixels in some cases due to rasterization rules

2. Line AA ends

• Implement/port butt and square line AA ends
• this will add and extra set of vertices to the start and end of open line
• see how things break :)
• this should get rid of the rasterization rule based inconsistencies

Based on #7972 it was not clear what is currently in, but I will try to figure it out.

[edit] Looks like I mixed butt and square caps. Fixed in this post.

[memononen]

I'm up to speed with the #7972 and current state, is the simplified PR public, i'd be curious to have a look at it?

[ocornut]

It is the simplified PR as the last commit removes a bunch of features from it. I guess I’ll need to later rework PR history so that it ends up being a 1st mergable commit with minimum feature and the rest are additions.

[memononen]

Roger. Is that PR still in the works, or just waiting for merge?

I implemented the AddLine() and other stroke offset changes in this branch:
https://github.com/memononen/imgui

Comparison pictures here:
https://www.figma.com/design/c9QW0F06IrZsWBt8mPfXch/ImGui-lines?node-id=0-1&t=zlR4ONffvNqCSvdN-1

Any comments welcome (when your time permits).

My general thinking was try to take a look at the draw API and organize it to functions that have common behavior. At the lowest level we have the path API, and then some helpers for it, and stroked primitives and filled primitives.

The path API places the stroke at the center of the path, and for the primitives the default is inside the shape, but you have option to position the stroke outside or center.

I added horizontal and vertical line as primitives to add the specific sidedness behavior (I dont think it makes sense for the generic line). The hline and vline could be implemented as AddFilledRect() too, but I chose not to, since it does not support AA for non-rounded case.

It is possible to make the stroke placement generic too, by making the stroke expansion non-symmetric. That would allow AddTriangle() and AddQuad() to work the same as other prims. Not sure if it's worth it, though.

AddFillledRect() seems inconsistent on behavior since it does not support AA when corner radius is 0. I think that is the reason for #6971. I wonder if there could be a draw flag which would allow to pick the speedy version, while allowing AA too in general case? That flag should allow you to implement fast 9-slice (or 10 triangle) rounded rect too.

I used separate param for the stroke pos, instead of draw flags, since some of the draw functions dont have flags, and I was not sure your policy on adding new params in between.

While I was at it, I fixed bunch of cases which did not support wide lines, like tables, or cases where the thicker lines would be blurry (I sure missed some). There are some open questions, though, like if (and how) the table height should be adjusted by the outer border height so that it does not get clipped.

I will still need to do some more testing without AA.

It's very likely that the line cap AA will surface some issues. Most of those should be fixed by the above changes, since the geometry is now created so that it "surrounds" the pixel centers for the cases where we want crips lines, and there should be no half pixel things that rely on rasterization rules to break ties.

My concerns are covered with some version of my proposed change, and the new polyline rendering.

[ocornut]

Roger. Is that PR still in the works, or just waiting for merge?

Waiting for me to review and decide if there's no problem with it. If you feel you can review #7972 that'd be nice.

Thanks for your details, I'll try to review where I can.

[ocornut]

You can close #9359 and copy any useful info from it into here as well.

[memononen]

I'll review #7972.

Btw, do you have some procedure to test the rendering perf?

[ocornut]

The imgui_test_suite in the PERFS tab has a few basic stress tests and it outputs runs into a .csv file. There's also a Perf Tool in there to visualize the .csv data (I might scrap and rewrite it in a simpler way, it's too much code)

[memononen]

Thanks!

While looking at #7972 I was wondering about perf. And that led me thinking about the flag for AddFillledRect() that would indicate that we're rendering pixel aligned stuff, which could allow us to simplify the rendering. And more broadly, would it be possible to fix API anomality, and address the perf implication of the more accurate rendering.

I tested the truncated/snapped coordinate hint drawflag + using textures for rendering the corners.

I dont have perf numbers yet, but it does cut the triangle count almost in half when all rounding is turned on, and making those 9-slices is trivial compared to the outline feathering.

Tesselated rounding
11188 verts
10156 tris

Textured rounding
8816 vertis
6132 tris

Similar thing could be done for stroked rects too. I did not look into how to do dynamic atlas, but that might be a good idea for that as there are more dimensions for the inputs.

If nothing else, it could be a good plan B if the perf of the new polyline tessellation feels too high.

https://github.com/memononen/imgui/tree/textured_round_corners

[memononen]

TL;DR: After some more investigation and testing, I outlined below how the rendering could be improved. I wonder if the general idea all makes sense at all (it does to me :))? I think it could be implemented in 5-10 separate PRs. Technically I don't have any unknowns (that I know), but I'm not sure about some constraints, and the direction you want to go.

---

While reviewing #7972 I started to wonder why there's FringeWidth in ImDrawList (there are a lot of assumptions that drawlist coords are in pixels), and that sent me into the hi-dpi rabbit hole for a bit :)

I think #7972 is really good quality, but it's a lot of code, and the nested macros make it hard to reason about, and very hard to debug.

I've been testing a simpler variation of that code using textures for AA, and omitting some features for simplicity (some of which can be included later). In general, there are 3 variations of the polyline rendering: fringe-solid-fringe, fringe-fringe, solid. The textured versions can handle anything using the F-F and S variations. Using only vertex colors will require all 3 versions. Third option is to use F-S-F for general purpose, and F(S) for integer sized strokes, and for no-AA. Specializing snapped integer thickness (rounded) rects (which is what most of the UI uses) to use 9-slice like textures could allow a great perf improvement on top.

Each variation of the polyline is about 300-400 lines of code, the tessellation patterns make it tricky to share code (even if the main logic is the same). There's some options for more constrained systems, which I would like to understand better, as supporting them will add code complexity or may require to make other compromises.

I tried to outline the whole scope below. I hope that it makes it a bit easier to understand what I'm after. I guess my first question is that does it make sense at all? I tried to comb through the drawlist and dpi issues to get a general feeling if the issues, but I'm sure I'm missing some context that or constraints that only you have.

---

Goal
Robust, fast, pixel perfect rendering of UI widgets at at any scale.

1. Add hline and vline primitives
   • specific stroke placement behavior, below the line or right of line, which allows to render crips lines at any size
   • remove 0.5px offset in AddLine(), which is currently used to place lines at pixel (only works for 1px lines)
2. Make DrawList API consistent
   • coordinates are always treated as starting from upper-left corner of zero pixel (this is mostly true currently)
   • stroke position can be specified for shapes used commonly for UI (allows crips rendering of different outline thicknesses)
3. Remove 1px stroke assumptions
   • number of places in the widget and rendering code
4. Add snapping hint
   • since most widget coords are already snapped, we can skip the general AA in many cases and render using simpler primitives
   • this only works if the drawlist pixel scale is multiple of integer, fallback to general AA rendering
5. Make rendering more robust
   • Improve AA path rendering so that it handles wider variety of data and common degenerate cases
   • fix bunch of bug reports
   • caps for lines
   • handle pixel scale consistently
6. Improve rendering perf
   • snapped rectangles and filled rectangles can be rendered much faster using textures (9-slicing or similar)

High-DPI Scaling

The current setup allows pixel scaling both by scaling styles and scaling DrawList. Scaling the drawlist data has the issue that when you have fractional scaling, the rendering either gets blurry, or if we do snapping, the contents will wiggle when moving.

The simplest and most robust way to do hi-DPI scaling is to have one scaling factor for styles, and after that everything is in physical pixels. This allows robust pixel snapping during layout (as it is done currently), and crisp (and fast) rendering for the DrawList.

DrawList.FringeWidth can be supported but it will lead to blurry (but accurate with AA) rendering when the pixel scale (inverse of FringeWidth) is fractional. That is, crisp fractional scaling should scale the styles.