pluggable event pipeline

[itaysk]

this is a concept I experimented with long time ago, I even wrote a POC but probably won't find the code now. I'm opening this to discuss and collect feedback and if needed I'll create an issue.

What
Tracee-eBPF has an event "pipeline" which orchestrates the steps in the way to collecting an event, for example: read, transform, prepare, and print. The idea is to make this pipeline pluggable and allowing anyone to load their own plugin into the pipeline.

Why
We will start to get more and more requests for adding more metadata on events. we would want to support every real use-case but on the other hand, there's a performance cost associated with it. With event processing plugins, (1) user can choose to disable unnecessary plugins to improve performance, and (2) plugins can be built by users, out of tree.

How

• My original POC rewrote the pipeline as a middleware pattern. The payload passed between handlers was the Event struct.
• The customizable pipeline starts after the core pipeline I described above (meaning after print stage).
• dynamic loading can be possible using go plugins.
• In addition it would require some configuration to specify which plugins to load and at what order.

Note that this discussion is for user-space enrichment. I'm not proposing ebpf pluggability here. the ebpf code would still need collect ALL the basic information.

Examples

• uid -> name, gid
• containerid -> container image

[yanivagman]

@itaysk this was one of the reasons why I created the pipeliine, back before we had tracee-rules.
I agree that we should add support for such plugins, especially for the case you mentioned of event enrichment.
Other use case is to change an argument (raw) data before printing it. For example, our argument printers are now part of the pipeline, but these can be moved to be a plugin that the user can choose to add or not.