Inventory module reports unexpected stateless events after initial scan

[vikman90]

Description

The Inventory module reports its initial state (right after the scan following the creation of local.db) as create-type stateful messages. During this initial scan, no stateless messages should be reported, as there is no "change" in the inventory, only the initial population of data.

However, we have observed that the agent sends stateless messages right after the first scan.

Configuration

agent:
  thread_count: 4
  server_url: https://localhost:27000
  retry_interval: 30s
  verification_mode: none

events:
  batch_interval: 10s
  batch_size: 1MB

inventory:
  enabled: true
  interval: 1h
  scan_on_start: true
  hardware: true
  system: true
  networks: true
  packages: true
  ports: true
  ports_all: true
  processes: true
  hotfixes: true

Observed Behavior

After the initial scan, the following events are sent:

{"agent":{"groups":["validYaml"],"host":{"architecture":"x86_64","hostname":"rocky","ip":["172.28.54.98","fe80::215:5dff:fef2:3436"],"os":{"name":"Ubuntu","type":"Linux","version":"24.04.1 LTS (Noble Numbat)"}},"id":"f9629dbd-e18e-45fb-bd42-e6311f9f7e00","name":"dummy","type":"Endpoint","version":"5.0.0"}}

{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-14T12:22:26.614Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":16,"name":"11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz","speed":2496},"memory":{"free":29706760,"previous":{"free":29712724},"total":32749888,"used":{"percentage":10}}},"observer":{"serial_number":null}}

{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-14T12:22:29.128Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":16,"name":"11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz","speed":2496},"memory":{"free":29690596,"previous":{"free":29706760},"total":32749888,"used":{"percentage":10}}},"observer":{"serial_number":null}}

{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-14T12:22:30.297Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":16,"name":"11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz","speed":2496},"memory":{"free":29675180,"previous":{"free":29690596},"total":32749888,"used":{"percentage":10}}},"observer":{"serial_number":null}}

{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-14T12:22:32.472Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":16,"name":"11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz","speed":2496},"memory":{"free":29679072,"previous":{"free":29675180},"total":32749888,"used":{"percentage":10}}},"observer":{"serial_number":null}}

The hardware-updated events indicate memory changes, but since this is the first scan, these messages should not be generated.

Steps to Reproduce

1. Clean up the agent's data:

   rm -f /var/lib/wazuh-agent/{queue,command_store,local}.db
   

2. Start the mock server, listening on port 27001:

   ./mock-server --comms-port 27001
   

3. Start MITMweb, redirecting from port 27000 to 27001:

   mitmproxy/mitmweb --mode reverse:https://localhost:27001@27000 -k
   

Expected Behavior

• The first scan should only generate stateful create messages.
• No stateless change messages should be sent immediately after the first scan.

[nbertoldo]

I repeated the test following the same steps and what happens is that the mock-server constantly sends the command “set-group(CentralizedConfiguration)”, which causes the modules to reload and therefore when restarting the inventory a new scan is performed.

[2025-02-17 18:49:37.190] [wazuh-agent] [info] [INFO] [process_options_unix.cpp:24] [StartAgent] Starting wazuh-agent
[2025-02-17 18:49:37.193] [wazuh-agent] [warning] [WARN] [configuration_parser.hpp:252] [GetParsedConfigInRangeOrDefault] Requested setting is not found or out of range, default value used.
[2025-02-17 18:49:37.597] [wazuh-agent] [info] [INFO] [communicator.cpp:101] [SendAuthenticationRequest] Successfully authenticated with the manager.
[2025-02-17 18:49:37.598] [wazuh-agent] [info] [INFO] [inventory.cpp:20] [Start] Inventory module started.
[2025-02-17 18:49:37.598] [wazuh-agent] [info] [INFO] [logcollector.cpp:28] [Start] Logcollector module is disabled.
[2025-02-17 18:49:37.599] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:991] [SyncLoop] Module started.
[2025-02-17 18:49:37.599] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:974] [Scan] Starting evaluation.
[2025-02-17 18:49:39.384] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:986] [Scan] Evaluation finished.
[2025-02-17 18:49:40.657] [wazuh-agent] [info] [INFO] [command_handler_utils.cpp:79] [DispatchCommand] Dispatching command set-group(CentralizedConfiguration)
[2025-02-17 18:49:40.657] [wazuh-agent] [warning] [WARN] [centralized_configuration.cpp:69] [ExecuteCommand] Group name can not be an empty string.
[2025-02-17 18:49:40.660] [wazuh-agent] [info] [INFO] [command_handler.cpp:115] [CommandsProcessingTask] Done processing command: set-group(CentralizedConfiguration)
[2025-02-17 18:49:43.710] [wazuh-agent] [info] [INFO] [command_handler_utils.cpp:79] [DispatchCommand] Dispatching command set-group(CentralizedConfiguration)
[2025-02-17 18:49:43.758] [wazuh-agent] [info] [INFO] [agent.cpp:87] [ReloadModules] Reloading Modules
[2025-02-17 18:49:43.758] [wazuh-agent] [info] [INFO] [configuration_parser.cpp:128] [ReloadConfiguration] Reload configuration.
[2025-02-17 18:49:43.758] [wazuh-agent] [info] [INFO] [configuration_parser.cpp:139] [ReloadConfiguration] Reload configuration done.
[2025-02-17 18:49:43.758] [wazuh-agent] [info] [INFO] [inventory.cpp:73] [Stop] Inventory module stopping...
[2025-02-17 18:49:43.758] [wazuh-agent] [info] [INFO] [logcollector.cpp:101] [Stop] Logcollector module stopped.
[2025-02-17 18:49:43.758] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:974] [Scan] Starting evaluation.
[2025-02-17 18:49:43.758] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:986] [Scan] Evaluation finished.
[2025-02-17 18:49:43.759] [wazuh-agent] [info] [INFO] [inventory.cpp:40] [Start] Inventory module stopped.
[2025-02-17 18:49:43.759] [wazuh-agent] [info] [INFO] [inventory.cpp:20] [Start] Inventory module started.
[2025-02-17 18:49:43.761] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:991] [SyncLoop] Module started.
[2025-02-17 18:49:43.761] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:974] [Scan] Starting evaluation.
[2025-02-17 18:49:43.760] [wazuh-agent] [info] [INFO] [logcollector.cpp:28] [Start] Logcollector module is disabled.
[2025-02-17 18:49:43.769] [wazuh-agent] [info] [INFO] [agent.cpp:92] [ReloadModules] Modules reloaded
[2025-02-17 18:49:43.775] [wazuh-agent] [info] [INFO] [command_handler.cpp:115] [CommandsProcessingTask] Done processing command: set-group(CentralizedConfiguration)
[2025-02-17 18:49:43.934] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:986] [Scan] Evaluation finished.

Anyway, it can be seen that in the first scan no stateless events are generated.

First scan

[2025-02-17 18:49:37.599] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:974] [Scan] Starting evaluation.
[2025-02-17 18:49:39.384] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:986] [Scan] Evaluation finished.

Stateful events

First scan - "operation": "create"

{"collector":"hardware","id":"1d9f756eef489b27b5f09f85956d99146b159b56","module":"inventory","operation":"create"}
{"@timestamp":"2025-02-17T18:49:37.599Z","host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9428088,"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}

Next scans - "operation": "update"

{"collector":"hardware","id":"1d9f756eef489b27b5f09f85956d99146b159b56","module":"inventory","operation":"update"}
{"@timestamp":"2025-02-17T18:49:43.761Z","host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9418428,"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","id":"1d9f756eef489b27b5f09f85956d99146b159b56","module":"inventory","operation":"update"}
{"@timestamp":"2025-02-17T18:49:46.914Z","host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9415204,"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","id":"1d9f756eef489b27b5f09f85956d99146b159b56","module":"inventory","operation":"update"}
{"@timestamp":"2025-02-17T18:49:49.989Z","host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9405548,"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","id":"1d9f756eef489b27b5f09f85956d99146b159b56","module":"inventory","operation":"update"}
{"@timestamp":"2025-02-17T18:49:51.168Z","host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9406472,"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","id":"1d9f756eef489b27b5f09f85956d99146b159b56","module":"inventory","operation":"update"}
{"@timestamp":"2025-02-17T18:49:52.225Z","host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9404328,"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}

Stateless events

Stateless events are generated since the second scan.

{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-17T18:49:43.761Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9418428,"previous":{"free":9428088},"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-17T18:49:46.914Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9415204,"previous":{"free":9418428},"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-17T18:49:49.989Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9405548,"previous":{"free":9415204},"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-17T18:49:51.168Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9406472,"previous":{"free":9405548},"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}
{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-17T18:49:52.225Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":8,"name":"AMD Ryzen 7 5800X 8-Core Processor","speed":3800},"memory":{"free":9404328,"previous":{"free":9406472},"total":12247084,"used":{"percentage":24}}},"observer":{"serial_number":"0"}}

[cborla]

Thank you @nbertoldo for the evidence,
It's possible to see in the example logs that the agent events are of the update type

{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-14T12:22:26.614Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":16,"name":"11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz","speed":2496},"memory":{"free":29706760,"previous":{"free":29712724},"total":32749888,"used":{"percentage":10}}},"observer":{"serial_number":null}}

{"collector":"hardware","module":"inventory"}
{"event":{"action":"hardware-updated","category":["host"],"changed_fields":["host.memory.free"],"created":"2025-02-14T12:22:29.128Z","reason":"Hardware changed","type":["change"]},"host":{"cpu":{"cores":16,"name":"11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz","speed":2496},"memory":{"free":29690596,"previous":{"free":29706760},"total":32749888,"used":{"percentage":10}}},"observer":{"serial_number":null}}

They are not events of the create type, which would be the first scan. “action”: “hardware-updated”
Sending stateless events for changes makes sense due to the constant restart of the modules with the first scan set to true.