Use Seq.Syntax for message templating/property substitution?

[nblumhardt]

While Seq has always supported nested structures in properties, the more recent versions make heavier use of this via the @Resource object populated from OpenTelemetry resource attributes.

These kinds of events frequently carry important information nested under @Resource, for example @Resource.ApplicationName in the example presented at: datalust/seq-tickets#1937.

The app doesn't currently support substituting nested property paths into messages, nor including them as attachments in the comma-separated list - a.b is interpreted as a property with a dot in its name @Properties['a.b'].

While it would be possible to come up with a more elaborate syntax here, Seq.App.Mail.Smtp, Seq.App.Mail.Microsoft365, and Seq.App.HttpRequest now use a common Seq.Syntax package to deal with this.

Seq.Syntax provides templating along the lines of:

{@Level}: {@Message}!
--> Warning: Hello world!

The great things about Seq.Syntax are:

• Supports all of the Seq built-in properties consistently (@Level, @Message, @Exception, etc.)
• Handles Nested.Properties and @Properties['irregular-names'] using identical syntax to Seq itself
• Has a stack of built-in functions inherited also from Seq

... in fact, nearly 100% of Seq's expression language is covered, so Substring(), if, ... etc. all work.

We're also aiming for syntax highlighting and autocomplete in a future Seq version for app settings tagged with Syntax="template" or Syntax="expression", and on this basis we think it'll become commonplace to use with Seq apps (replacing the mix of Handlebars and custom syntaxes the app ecosystem uses today).

The downsides, and why there's a ? in the title and not an all-out pitch here are:

• Porting would cause some churn by requiring ISubscribeToAsync<LogEvent> instead of the ISubscribToAsync<LogEventData> implementation the app uses right now; bringing in a dependency on Serilog's LogEvent is not ideal, though it's also something we want to break upstream, so should hopefully only be a matter of time, and
• The syntax uses {braces} instead of [square brackets] for property substitutions - this would mean some busywork and potential breakage for consumers updating to the new (2.0.0) version, and
• In the IncludedProperties app setting, dotted property names like a.b today would need to be rewritten as @Properties['a.b'] to avoid them being interpreted differently after then change.

So definitely some drawbacks and support load.

The alternatives (besides preserving the status quo) I think are:

• Do something custom, here, to deal with @Resource and/or nested property names,
• Release a new version of the package under a different package name, so that migration is more explicitly opt-in

Neither seem great but both are reasonable I guess, if the drawbacks are too great.

See also #25 (comment)

[dazinator]

Pure speculation, from a total outsider (in terms of how seq works)
I was wondering if it would be desirable to provide this as a service, where a plugin could optionally enqueue a request (with event payload) to seq, to be templated according to seqs native template management, and receive the expanded result back for processing as something opaque.

The flow would be like: Receive Log Event data > submit request to Seq Templating Service > Receive Templated Result > Process the result making use of the template.

Does any of that make sense?

I guess the main point of this would be, rather than a plugin developer consuming templating capability as a library that it would use to produce / expand a template, it would instead be asking some seq service for an expanded result based on whatever trmplate is configured in seq and howver seq chooses to expand that payload based on that setup

Example:-

1. Email plugin receives log event data
2. Email plugin allows Subject and Body to be templated
3. Email plugin asks seq to expand "Subject"
4. Email plugin asks seq to expand "Body"
5. Inside seq two templates appear listed "Subject" and "Body"
6. User can edit and manage those inside seq according to native syntax
7. Plugin receives back results based on those templates being applied.
8. Plugin uses the results to set the email Subject and Body before sending.

[nblumhardt]

The mechanism might end up slightly different, but this is the kind of thing we've considered a few times and will definitely be thinking about for the next revision of the apps system. In the short term, building out consistent support for the same syntax across apps, via Seq.Syntax, is an incremental improvement in this direction :-)