Extending markdown-rs

[ShafSpecs]

It's not exactly a question, but is it possible to extend markdown-rs and are there alternatives to forking the repository and going down the rabbit hole of trying to figure out how to build yuour own new flavour, syntax, etc.?

I love markdown-rs and have been using for a while (not too long tho) and I've wanted for sometime to create a custom flavour of markdown I can share across multiple projects. Is there a possibility of seeing a situation where we can use markdown_rs and plug our own flavour in directly? I've also seen requests like this in a lot of forums where we have an extensible parser. If not, what would you be the steps you would recommend for building on top of markdown-rs?

[ChristianMurphy]

Welcome @ShafSpecs!
You may want to check out the ongoing discussion in #32

possible to extend markdown-rs

From the above thread

as far as I understand impossible in Rust. The extensibility needed to support extensions is not there in Rust. None of the other rust markdown parsers that I know support them either.

Since the time of writing, markdown-it has done some experimentation in the space.

it seems like https://github.com/rlidwka/markdown-it.rs/tree/master/src/plugins supports adding extensions to the language, as opposed to just modifying the AST.

I haven't personally had a chance to dig into this and see how feasible it is and how well it fits with markdown-rs's content model.
So take that with a grain of salt, and be aware it isn't currently supported.

alternatives to forking the repository and going down the rabbit hole of trying to figure out how to build your own new flavor, syntax, etc.?

With how the code works today.
Potentially, the recommendations from sqlparser-rs may apply here as well: https://github.com/sqlparser-rs/sqlparser-rs/blob/main/docs/custom_sql_parser.md

The concept is simply to write a new parser that delegates to the [markdown-rs] parser so that as much as possible of the core functionality can be re-used.

So not necessarily forking this repo, but creating a new repo and parser, and calling into the internals of markdown-rs for reuse of existing functionality.

[ShafSpecs]

Thanks Christian! Checking out markdown-it, it looks exactly like I want. I am still new to Rust but I should be able to get both libraries working together. Using markdown-rs internals but building mardown-it on top? Like I said, I'm still quite new but it would be interesting to see how far I can take it

[ChristianMurphy]

Taking a step back, if you're mostly look for something fast enough, and are more familiar with JS/TS.
You may want to consider using micromark, which does have a well defined extension system https://github.com/micromark/micromark#creating-a-micromark-extension

If you really want to dig in on the rust side, that's awesome.
Be aware that both markdown-rs and markdown-it.rs are alpha phase right now.
Feedback is definitely welcome on either, but the API and extension points are not stable just yet, they can and likely will change.

Using markdown-rs internals but building mardown-it on top?

I'm more getting at using one or the other, with the awareness that, either would need some additional engineering work to support your use case.
markdown-rs has fairly full featured commonmark, gfm, and mdx support already, but doesn't have an external extension system.
markdown-it.rs have an extension feature, but doesn't appear to fully cover gfm or mdx.

[ShafSpecs]

I think I am starting to see how I can pull extensibility all together into one. I checked out comrak which is basically a port of github cmark-gfm in rust. And markdown-it supports commonmark and plugins(custom syntaxes and nodes as well as behaviours). Both have AST support and I should be able to use markdown-rs to format it.

I could write gfm plugin for markdown-it using comrak as a template and generate AST, and use markdown-rs to generate html output. Like I said, I'm still new to rust and have no idea if it would work or not, but it's worth a shot imo

[ShafSpecs]

Not sure markdown-rs handles AST to HTML

[ChristianMurphy]

I think I am starting to see how I can pull extensibility all together into one. I checked out comrak which is basically a port of github cmark-gfm in rust. And markdown-it supports commonmark and plugins(custom syntaxes and nodes as well as behaviours). Both have AST support and I should be able to use markdown-rs to format it.

Merging many markdown parsers together is certainly an option.
It would likely be easier to work with a single one, and enhance it to meet your needs.
Some commentary on this in my other comment #52 (reply in thread)

Not sure markdown-rs handles AST to HTML

There is code to support this, it is not currently in markdown-rs, see: #27

[ShafSpecs]

It would likely be easier to work with a single one, and enhance it to meet your needs.

Definitely. A gfm plugin has been written for markdown-it, and that negates the need for comrak. markdown-it also supports a whole lot of extensibility and I am planning on using it as my base parser and build on top of it using markdown-rs to help

[digitalmoksha]

markdown-it also supports a whole lot of extensibility and I am planning on using it as my base parser and build on top of it using markdown-rs to help

I'm not sure I understand why you would need to use both markdown-it and markdown-rs. It seems like if you are going to use markdown-it, you can just build additional plugins as needed. Why the need to layer another parser on top of that?

If the changes you need can be done on an AST, then I think you could stick with markdown-rs

[ShafSpecs]

markdown-it also supports a whole lot of extensibility and I am planning on using it as my base parser and build on top of it using markdown-rs to help

I'm not sure I understand why you would need to use both markdown-it and markdown-rs. It seems like if you are going to use markdown-it, you can just build additional plugins as needed. Why the need to layer another parser on top of that?

If the changes you need can be done on an AST, then I think you could stick with markdown-rs

Yeah, figured that. I went with markdown-it and it fulfilled what I was trying to accomplish.