Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auditbeat: system/process module backed by quark #42032

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
cb7c03f
auditbeat: system/process module backed by quark
haesbaert Nov 13, 2024
0ced9d8
update notice
haesbaert Dec 13, 2024
f17dc63
Make sure quark is linux only
haesbaert Dec 13, 2024
c79a0d0
please the linter
haesbaert Dec 13, 2024
e3b7332
Use kernel_tracing as config
haesbaert Jan 8, 2025
67a61ff
Make sure backend is valid, maybe we add another in the future, who k…
haesbaert Jan 8, 2025
a88d3f1
Add metrics, still unsure about the naming
haesbaert Jan 8, 2025
a069220
move maybeUpdateMetrics to the end
haesbaert Jan 9, 2025
63cf391
low level tests
haesbaert Jan 9, 2025
b7dfba0
simplify metric set
haesbaert Jan 9, 2025
42410cf
Add a test for the actual metric set, more to come
haesbaert Jan 9, 2025
0eb862d
Update x-pack/auditbeat/module/system/process/quark_provider_linux.go
haesbaert Jan 10, 2025
0548116
Apply suggestions from code review
haesbaert Jan 10, 2025
26a6c50
Don't forget to close hasher, spotted by nick
haesbaert Jan 10, 2025
04f76d0
Enable hashing on tests and improve comments
haesbaert Jan 10, 2025
d3285d4
Rename `proc` to `procfs` as the backend, like add_session_metadata does
haesbaert Jan 10, 2025
1d48fd3
Test both ebpf and kprobe backends on tests
haesbaert Jan 10, 2025
7c1a48f
Merge remote-tracking branch 'origin/main' into quark-process
haesbaert Jan 10, 2025
70db6bf
please the linter gods
haesbaert Jan 10, 2025
93038d2
Fix NewFromQuark() in _other
haesbaert Jan 10, 2025
be4d478
ignore block error on tests
haesbaert Jan 10, 2025
4681030
skip tests if not root
haesbaert Jan 13, 2025
a06274c
Update x-pack/auditbeat/module/system/process/quark_provider_linux.go
haesbaert Feb 13, 2025
6109c8b
Update x-pack/auditbeat/module/system/process/quark_provider_linux.go
haesbaert Feb 13, 2025
0e987ab
Update x-pack/auditbeat/module/system/process/quark_provider_linux_te…
haesbaert Feb 13, 2025
2bb1815
Lower processErr to warn
haesbaert Feb 14, 2025
ff23713
fix Errorf call
haesbaert Feb 14, 2025
d040234
Handle eventActionProcessError
haesbaert Feb 14, 2025
ae5a34f
tedu XXX
haesbaert Feb 14, 2025
2459dea
Merge remote-tracking branch 'origin/main' into quark-process
haesbaert Feb 14, 2025
554a5ce
Merge remote-tracking branch 'origin/main' into quark-process
haesbaert Feb 14, 2025
4e252b5
Merge branch 'main' into quark-process
haesbaert Feb 14, 2025
eab4067
mod tidy
haesbaert Feb 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions NOTICE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14216,11 +14216,11 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------------------
Dependency : github.com/elastic/go-quark
Version: v0.2.0
Version: v0.3.0
Licence type (autodetected): Apache-2.0
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/elastic/go-quark@v0.2.0/LICENSE.txt:
Contents of probable licence file $GOMODCACHE/github.com/elastic/go-quark@v0.3.0/LICENSE.txt:


Apache License
Expand Down Expand Up @@ -22667,11 +22667,11 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------------------
Dependency : github.com/stretchr/testify
Version: v1.9.0
Version: v1.10.0
Licence type (autodetected): MIT
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/stretchr/testify@v1.9.0/LICENSE:
Contents of probable licence file $GOMODCACHE/github.com/stretchr/testify@v1.10.0/LICENSE:

MIT License

Expand Down
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ require (
github.com/shopspring/decimal v1.3.1 // indirect
github.com/spf13/cobra v1.8.1
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.9.0
github.com/stretchr/testify v1.10.0
github.com/tsg/go-daemon v0.0.0-20200207173439-e704b93fd89b
github.com/ugorji/go/codec v1.1.8
github.com/vmware/govmomi v0.39.0
Expand Down Expand Up @@ -180,7 +180,7 @@ require (
github.com/elastic/elastic-agent-libs v0.17.4
github.com/elastic/elastic-agent-system-metrics v0.11.4
github.com/elastic/go-elasticsearch/v8 v8.14.0
github.com/elastic/go-quark v0.2.0
github.com/elastic/go-quark v0.3.0
github.com/elastic/go-sfdc v0.0.0-20241010131323-8e176480d727
github.com/elastic/mito v1.16.0
github.com/elastic/mock-es v0.0.0-20240712014503-e5b47ece0015
Expand Down
8 changes: 4 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -346,8 +346,8 @@ github.com/elastic/go-lumber v0.1.2-0.20220819171948-335fde24ea0f h1:TsPpU5EAwlt
github.com/elastic/go-lumber v0.1.2-0.20220819171948-335fde24ea0f/go.mod h1:HHaWnZamYKWsR9/eZNHqRHob8iQDKnchHmmskT/SKko=
github.com/elastic/go-perf v0.0.0-20241029065020-30bec95324b8 h1:FD01NjsTes0RxZVQ22ebNYJA4KDdInVnR9cn1hmaMwA=
github.com/elastic/go-perf v0.0.0-20241029065020-30bec95324b8/go.mod h1:Nt+pnRYvf0POC+7pXsrv8ubsEOSsaipJP0zlz1Ms1RM=
github.com/elastic/go-quark v0.2.0 h1:r2BL4NzvhESrrL/yA3AcHt8mwF7fvQDssBAUiOL1sdg=
github.com/elastic/go-quark v0.2.0/go.mod h1:/ngqgumD/Z5vnFZ4XPN2kCbxnEfG5/Uc+bRvOBabVVA=
github.com/elastic/go-quark v0.3.0 h1:d4vokx0psEJo+93fnhvWpTJMggPd9rfMJSleoLva4xA=
github.com/elastic/go-quark v0.3.0/go.mod h1:bO/XIGZBUJGxyiJ9FTsSYn9YlfOTRJnmOP+iBE2FyjA=
github.com/elastic/go-seccomp-bpf v1.5.0 h1:gJV+U1iP+YC70ySyGUUNk2YLJW5/IkEw4FZBJfW8ZZY=
github.com/elastic/go-seccomp-bpf v1.5.0/go.mod h1:umdhQ/3aybliBF2jjiZwS492I/TOKz+ZRvsLT3hVe1o=
github.com/elastic/go-sfdc v0.0.0-20241010131323-8e176480d727 h1:yuiN60oaQUz2PtNpNhDI2H6zrCdfiiptmNdwV5WUaKA=
Expand Down Expand Up @@ -848,8 +848,8 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tklauser/go-sysconf v0.3.12 h1:0QaGUFOdQaIVdPgfITYzaTegZvdCjmYO52cSFAEVmqU=
github.com/tklauser/go-sysconf v0.3.12/go.mod h1:Ho14jnntGE1fpdOqQEEaiKRpvIavV0hSfmBq8nJbHYI=
github.com/tklauser/numcpus v0.6.1 h1:ng9scYS7az0Bk4OZLvrNXNSAO2Pxr1XXRAPyjhIx+Fk=
Expand Down
12 changes: 11 additions & 1 deletion x-pack/auditbeat/module/system/process/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
package process

import (
"fmt"
"time"

"github.com/elastic/beats/v7/auditbeat/helper/hasher"
Expand All @@ -16,11 +17,19 @@ type Config struct {
ProcessStatePeriod time.Duration `config:"process.state.period"`

HasherConfig hasher.Config `config:"process.hash"`
Backend string `config:"process.backend"`
}

// Validate validates the config.
func (c *Config) Validate() error {
return c.HasherConfig.Validate()
if err := c.HasherConfig.Validate(); err != nil {
return err
}
if c.Backend != "kernel_tracing" && c.Backend != "proc" {
return fmt.Errorf("invalid process.backend '%s'", c.Backend)
}

return nil
}

func (c *Config) effectiveStatePeriod() time.Duration {
Expand All @@ -40,4 +49,5 @@ var defaultConfig = Config{
ScanRatePerSec: "50 MiB",
ScanRateBytesPerSec: 50 * 1024 * 1024,
},
Backend: "proc",
}
21 changes: 3 additions & 18 deletions x-pack/auditbeat/module/system/process/gosysinfo_provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -351,27 +351,12 @@ func putIfNotEmpty(mapstr *mapstr.M, key string, value string) {
}

func processMessage(process *Process, action eventAction) string {
if process.Error != nil {
return fmt.Sprintf("ERROR for PID %d: %v", process.Info.PID, process.Error)
}

var actionString string
switch action {
case eventActionProcessStarted:
actionString = "STARTED"
case eventActionProcessStopped:
actionString = "STOPPED"
case eventActionExistingProcess:
actionString = "is RUNNING"
}

var userString string
var username string
if process.User != nil {
userString = fmt.Sprintf(" by user %v", process.User.Username)
username = process.User.Username
}

return fmt.Sprintf("Process %v (PID: %d)%v %v",
process.Info.Name, process.Info.PID, userString, actionString)
return makeMessage(process.Info.PID, action, process.Info.Name, username, process.Error)
}

func convertToCacheable(processes []*Process) []cache.Cacheable {
Expand Down
45 changes: 45 additions & 0 deletions x-pack/auditbeat/module/system/process/process.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ package process
import (
"encoding/binary"
"fmt"
"runtime"
"time"

"github.com/elastic/beats/v7/auditbeat/ab"
Expand Down Expand Up @@ -36,6 +37,8 @@ const (
eventActionExistingProcess eventAction = iota
eventActionProcessStarted
eventActionProcessStopped
eventActionProcessRan
eventActionProcessChangedImage
eventActionProcessError
)

Expand All @@ -47,6 +50,10 @@ func (action eventAction) String() string {
return "process_started"
case eventActionProcessStopped:
return "process_stopped"
case eventActionProcessRan:
return "process_ran"
case eventActionProcessChangedImage:
return "process_changed_image"
case eventActionProcessError:
return "process_error"
default:
Expand All @@ -62,6 +69,8 @@ func (action eventAction) Type() string {
return "start"
case eventActionProcessStopped:
return "end"
case eventActionProcessChangedImage:
return "change"
case eventActionProcessError:
return "info"
default:
Expand Down Expand Up @@ -89,6 +98,14 @@ func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
return nil, fmt.Errorf("failed to unpack the %v/%v config: %w", system.ModuleName, metricsetName, err)
}

if runtime.GOOS == "linux" && ms.config.Backend == "kernel_tracing" {
if qm, err := NewFromQuark(base, ms); err == nil {
return qm, nil
} else {
ms.log.Errorf("can't use quark, falling back to sysinfo: %w", err)
}
}

return NewFromSysInfo(base, ms)
}

Expand All @@ -102,3 +119,31 @@ func entityID(hostID string, pid int, startTime time.Time) string {
binary.Write(h, binary.LittleEndian, int64(startTime.Nanosecond()))
return h.Sum()
}

func makeMessage(pid int, action eventAction, name string, username string, err error) string {
if err != nil {
return fmt.Sprintf("ERROR for PID %d: %v", pid, err)
}

var actionString string
switch action {
andrewkroh marked this conversation as resolved.
Show resolved Hide resolved
case eventActionProcessStarted:
actionString = "STARTED"
case eventActionProcessStopped:
actionString = "STOPPED"
case eventActionExistingProcess:
actionString = "is RUNNING"
case eventActionProcessRan:
actionString = "RAN"
case eventActionProcessChangedImage:
actionString = "CHANGED IMAGE"
}

var userString string
if len(username) > 0 {
userString = fmt.Sprintf(" by user %v", username)
}

return fmt.Sprintf("Process %v (PID: %d)%v %v",
name, pid, userString, actionString)
}
Loading