When do nodes have None position?

[mightyiam]

When a tree has been parsed from a string, will they always have positions?

Edit: I peeked at the construction of one node: Html and found that there is a case where position is None. But there are many nodes. I hope there's a straightforward answer to this. I'd also be able to pass on that option to my users, if it can be None in my usage, but I'd better be certain of it.

To be clear, my usage is:

1. parse markdown using to_mdast with default options.
2. Iterate over the children.

Can I expect all of these children to have Some positions?

[ChristianMurphy]

Welcome @mightyiam! 👋

Taking a step back, this feels a lot like an XY Problem.
What are you trying to do with positional information? What concerns do you have with None positional info?

When a tree has been parsed from a string, will they always have positions?

No, they will not.
There are referential constructs like: HTML, footnotes, and reference links which exist in multiple places or get synthesized and can have None positional information.
Also when plugin support is added (#32), plugins can add nodes which didn't exist in the original document, and can have None position.

[mightyiam]

What I do is upsert a range delineated by two comments. Therefore I need to know the range between those comments. markdown-rs seems to work great for this purpose, but its authors are unwilling to guarantee positions.

[ChristianMurphy]

This still feels like an XY Problem.

Why not modify the AST and serialize the AST back to HTML or Markdown?
It sound like you want to modify structure, that's what syntax trees are for, you can modify them safely without mucking with positions.

[mightyiam]

I appreciate the advice. Yes, that would have been great, but if I do that, I would end up reformatting the entire document, which must not occur.

[ChristianMurphy]

I would end up reformatting the entire document, which must not occur.

Then an Abstract Syntax Tree (AST) based project isn't suitable.
Abstract syntax trees focus on the abstract structure rather than syntax detail.
Positions in most (likely all) AST based libraries are designed for debugging and source mapping, not editing.

Most of the major rust based markdown parsers are AST based including:

• https://github.com/wooorm/markdown-rs
• https://github.com/kivikakk/comrak
• https://github.com/raphlinus/pulldown-cmark

What you are describing @mightyiam sounds more like a Concrete Syntax Tree (CST), which preserves all detail about the text so it can be exactly reconstructed.
There are very few libraries out there https://tree-sitter.github.io/tree-sitter is the only one I'm currently aware of that has a CST for Markdown along with Rust bindings.