Add TextArea.transform

[elbaro]

This PR adds transform: Mat3 to TextArea so that a user can provide any transform.
There is no subpixel handling but looks clean on 90/180/270 rotations.

[grovesNL]

Thank you for looking into this! I think it would be super useful to have support for transformations directly in glyphon and this is a great start.

We should try to do something to handle the artifacts first though. Something I was thinking about is splitting the current case (simple 0 degree/horizontal text, clamping glyphs at edges) and transformed cases (arbitrary transformation, add padding/borders around glyph edges to avoid sampling neighboring glyphs). It would be really helpful if we could examples in other libraries to see they handle transformations, especially glyph padding.

Also we wouldn't want to take a dependency on glam, so we could think about some options there. We could add baked transformations if we want to limit it to 90 degree rotations for now, or use something like mint to accept some transformation matrix. As part of this we'd want to consider which transformations we should support in general.

Less important but we should also decide on the pivot point for rotations and how transformations relate to the different coordinates provided today.

[elbaro]

We can accept enums like this in the public API:

use mint::*;

enum Transform {
    RotateClockwise90{pub pivot: Point2},
    RotateCounterClockWise90{pub pivot: Point2},
    Rotate180{pub pivot: Point2},
    Translate{pub delta: Vector2},
    Scale{pub factor: u32},  // TODO: f32
    SimpleTransform (RowMatrix3) // private constructor. simple transforms.
    // Matrix(pub RowMatrix3), // TODO: complex transforms. address artifacts
}

impl Mul<&Transform> for &Transform {
  ..
  // the multiplication returns Transform::SimpleTransform
}

But without any linear algebra library, we will eventually need to implement a manual matrix multiplication in glyphon. Do you want that?

[grovesNL]

Right, I guess we'd need the result of the transformation CPU-side for culling and clipping (vs. applying it in the shader). If we need to do matrix multiplication we'd probably want some features to allow choosing between different crates (glam is probably the most common in gamedev, but a lot of people also use nalgebra and others).

Maybe we could avoid arbitrary transformations for now and start by supporting horizontal and vertical text only (0 degree rotation and 270 degree rotation clockwise). Scale and translations can already be handled through the current API. Would that be enough for your use case?

[jonmmease]

Hi, thanks for you work on this library @grovesNL and for this PR @elbaro.

My usecase for glyphon is in writing a wgpu renderer for a 2D charting library. Almost every chart includes a y-axis label rotated by 270 degrees clockwise. Text annotations can be rotated to arbitrary angles, so I'll eventually want to work out how to support this as well, but this is much less common and I would get a lot of immediate value out of the ability to orient text vertically.

FWIW, if it ends up being simpler to add a rotation angle to the existing scale and translation parameters in the API, rather than adding a general transform, that would fully satisfy my usecase.

[EggShark]

TAKE ALL THIS WITH A GRAIN OF SALT. I'M NOT A PROFESSIONAL DEVELOPER. I'M SELF TAUGHT AND HAVE ONLY BEEN DOING GRAPHICS PROGRAMMING FOR 2YEARS

tldr: I'm in full support of arbitrary transformations.

Personally, I'm in support of any type of transformation even 3D ones which would make use of a 4x4 matrix. My main reasoning behind this is it would give the end user significantly more control which I am for. I understand this would break the "2dness" of Glyphon, but not in a severe way in my opinion.

To my knowledge, Glyhpon is the only text-renderer that uses Wgpu still in active development. If someone is making a game or game-engine with Wgpu, Glyphon is an easy choice. However, if this user is making a 3d game it's hard to justify using Glyphon outside of UI or HUD elements. With a 4x4 matrix, Glyphon would still only render 2d text but the user could place the text anywhere in their 3d world. This could be used for text on a sign post or to have text in world space. Since these 3d games would presumably be using a 4x4 matrix for their camera, it would also be easy to apply the camera transformations to the text as well allowing for easy interoperability.

When thinking just about 2d space I can't really think of a reason not to implement arbitrary rotations (other than the development time). Why limit users to the cardinal directions, what if they want spinning text, wobbly text, etc. In my opinion the more control the user has the better so I'm in full support of arbitrary transformations.

AFAIK the current work around to still have arbitrary transformations in Glyphon is to render the text to a texture, then apply your transformations to said texture. The only benefit to this solution for me is more control of the draw order as I can draw other shapes than a texture, rather than drawing all the text in one go (I'm sure there are other ways around this. I'm not very experienced in graphics programming).

In terms of implementing transformations, I think just one option for a user to pass in a matrix is best, but from reading the previous messages I understand this could cause some artifacts. Another good option would be to ditch a public matrix and just have several different functions like:

struct Transformation {
    matrix: Matrix3x3 // or 4x4 from whatever lib you prefer
}

impl Transformation {
    pub fn rotate(pivot: f32, deg: f32) // math for that + artifact correction
    pub fn translate(scale: f32) // this is already in text area but who knows
    pub fn custom(matrix: Matrix3x3) // could leave this as an option for users but warn them about artifacting and let them decide
}```

[jonmmease]

Playing with arbitrary rotation, and wanted to share an observation. Here's the hello world example with a rotation hard-coded in the shader

    let angle = 3.14159 / 16.0;
    let rot = mat2x2(cos(angle), -sin(angle), sin(angle), cos(angle));
    let rot_pos = rot * vec2<f32>(pos);
    vert_output.position = vec4<f32>(
        2.0 * vec2<f32>(rot_pos) / vec2<f32>(params.screen_resolution) - 1.0,
        in_vert.depth,
        1.0,
    );

The text is rotated, but very jagged. If I change the sampler's FilterMode from FilterMode::Nearest to FilterMode::Linear, this is how it looks:

The text itself looks a lot better, but there are artifacts around the bounding boxes of characters. Does anyone have any ideas on what might be causing these artifacts? If we could eliminate the artifacts, I wonder if it would make sense to use the nearest sampler when text is horizontal or vertically aligned, and use the linear sampler in other cases.

Edit: Oh, these artifacts are probably linear interpolation against neighboring glyphs in the texture, and why @grovesNL mentioned "add padding/borders around glyph edges to avoid sampling neighboring glyphs" above.

[grovesNL]

Yeah this is a common issue when applying transformations to glyphs in a texture atlas like we're using here.

With linearly sampling, the artifacts are probably caused by interpolating the edges of two glyphs together. This could be improved by adding borders into the texture atlas, but borders adds extra overhead and usually needs to be tweaked based on the transformations and glyph size.

[jonmmease]

If I'm thinking about this correctly, for 2D rotation only, I think we'd only need a one-pixel buffer for width and height. Here's what my example above looks like with a 1 pixel buffer:

For my purposes in the short term, I think I'll create a branch that:

• Adds a rotation: f32 field to TextArea
• Switches to linear sampling (unconditionally)
• Adds 1 pixel of width/height buffer in the texture atlas (unconditionally)

Would you be interested in this as a PR to glyphon? Or would you prefer to wait to implement a more general solution.

---

Edit: here's the PR against my fork if you want to take a look: jonmmease#1

[grovesNL]

@jonmmease very cool, great job! I'm still not sure about the best path forward here (e.g., rotating and interpolating won't be as crisp as rasterizing the vector paths after rotating) so I'd probably wait to PR changes, but this is a great start. In general, having separate code paths (depending on the rotation) probably makes sense.

For what it's worth, you might get slightly better results in your branch if you can create a border around all edges instead, so e.g., a glyph at the very top of the texture atlas still has its top edge interpolated with a 0 alpha border instead of getting clamped to the top edge of the glyph. There are some other tricks people use to stop texture bleeding for sprites so all of those would still apply here too.

[elbaro]

@jonmmease I use glyphon for charts too!
The below is generated using the PR (no antialias).

This is one using old piet-gpu (no way to measure the text size, leading to clipped texts):

There are 2 hacks though:

• No API for good text size measurement: I referenced the measuring code in iced but it's still hacky.
• glyphon has a basic clipping logic, like min_x = max(text_area.min_x, 0.0). This does not work well with non-perpendicular transformations; I completely removed the clipping logic for my usage.

I am not knowledgable in good anti-aliasing algorithms, and probably will not continue the current PR. Feel free to take this over or create new PR.

The interpolation screenshots above still look blurry to me. I would rather support only 90/180/270 rotations until we have a better impl or implement a known vector-based GPU font rendering.