feat(agent): Adds label forwarding to log events #1027

mfulb · 2025-02-20T21:25:20Z

This PR adds the ability for labels to be forwarded with any log messages forwarded by the PHP agent.

newrelic-php-agent-bot · 2025-02-20T21:25:46Z

Test Suite	Status	Result
Multiverse	❌	0/8 passing
SOAK	❌	78/79 passing

codecov-commenter · 2025-02-24T22:02:50Z

Codecov Report

Attention: Patch coverage is 97.26027% with 2 lines in your changes missing coverage. Please review.

Project coverage is 77.98%. Comparing base (f02c410) to head (277d034).

Files with missing lines	Patch %	Lines
agent/php_txn.c	98.14%	1 Missing ⚠️
axiom/cmd_txndata_transmit.c	92.30%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1027      +/-   ##
==========================================
+ Coverage   77.93%   77.98%   +0.04%     
==========================================
  Files         198      198              
  Lines       27713    27784      +71     
==========================================
+ Hits        21599    21667      +68     
- Misses       6114     6117       +3

Flag	Coverage Δ
agent-for-php-7.2	`78.14% <97.26%> (+0.05%)`	⬆️
agent-for-php-7.3	`?`
agent-for-php-7.4	`77.87% <97.26%> (+0.05%)`	⬆️
agent-for-php-8.0	`77.26% <97.26%> (+0.05%)`	⬆️
agent-for-php-8.1	`77.76% <97.26%> (+0.05%)`	⬆️
agent-for-php-8.2	`77.37% <97.26%> (+0.05%)`	⬆️
agent-for-php-8.3	`77.37% <97.26%> (+0.05%)`	⬆️
agent-for-php-8.4	`77.39% <97.26%> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

agent/php_txn.c

agent/scripts/newrelic.ini.template

lavarou · 2025-02-26T14:23:25Z

agent/scripts/newrelic.ini.template

+; Scope  : per-directory
+; Default: ""
+; Info   : A list of labels to NOT add as attributes to logs which are forwarded
+;          to New Relic.  This list can be separated by commas.


Please update description to state that label matching is case insensitive and multiple list elements MUST be separated by commas.

Updated in commit 277d034.

lavarou · 2025-02-26T14:37:44Z

tests/integration/logging/monolog2/test_monolog_labels_forwarding_exclude_05.php

+      "common": {
+        "attributes": { }
+      },


For other reviewers - sending "common" object with empty "attributes" is an existing functionality.

lavarou · 2025-02-26T14:43:00Z

axiom/tests/.gitignore

@@ -95,7 +95,6 @@ test_segment
 test_segment_children
 test_segment_datastore
 test_segment_external
-test_segment_message


I think this has been removed by a mistake during rebase.

I had it in this PR, removed it, put it in #1026 and then rebased. Weird it didn't complain about a conflict - I'll look into it.

agent/php_txn.c

zsistla · 2025-02-27T14:53:56Z

Thanks for adding the new tests_monolog_label* tests.
I did a first look through and overall they look good.
A few questions/requests:

Could you add additional tests where:

Labels list is not set, but exclude log labels are
Log forwarding disabled, but exclude log labels are set (I.e., not default)
Exclude label list includes a label that matches multiple case sensitive labels; expected behavior is they are all removed.
Exclude label list has same entry more than once

Did you run the new integration/multiverse label tests with valgrind?
What is expected behavior if a label has a comma? Can you add a test case, so we can document the expected behavior that way?

The function nr_txn_begin() now accepts a nrobj_t *log_labels parameter which contains the logging labels to be associated with log events.

…og labels metric

…etric

- added PHP5 tests - these are not for PHP 5 but are intended to run on PHP5+!

… metric

…etric

…metric

…tric

Co-authored-by: Michal Nowacki <[email protected]>

zsistla · 2025-02-27T23:25:29Z

agent/php_txn.c

+    bool exclude = false;
+    nr_status_t rv;
+
+    exclude = nro_get_hash_boolean(exclude_labels_hash, lower_key, &rv);


exclude is set as a bool, but nro_get_hash_boolean returns an int, not a bool.

int nro_get_hash_boolean(const nrobj_t* obj, const char* key, nr_status_t* errp)

It returns -1 on failure/error cases and 0/1 to indicate a setting of false/true.

Consider saving some steps/checks and removing the need for a rv by taking advantage of the int return value.

It could then be simplified a bit, so that instead of:

bool exclude = false; nr_status_t rv; exclude = nro_get_hash_boolean(exclude_labels_hash, lower_key, &rv); if (NR_SUCCESS != rv) { exclude = false; } if (!exclude) { nro_set_hash_string(log_labels, key, value); } else { nrl_verbosedebug(NRL_TXN, "%s: Excluding label %s", __FUNCTION__, key); }

It could simplified to be:

if (nro_get_hash_boolean(exclude_labels_hash, lower_key, NULL) != 1) { nro_set_hash_string(log_labels, key, value); } else { nrl_verbosedebug(NRL_TXN, "%s: Excluding label %s", __FUNCTION__, NRSAFESTR(key)); }

zsistla · 2025-02-27T23:27:49Z

agent/php_txn.c

+    if (!exclude) {
+      nro_set_hash_string(log_labels, key, value);
+    } else {
+      nrl_verbosedebug(NRL_TXN, "%s: Excluding label %s", __FUNCTION__, key);


Key was not checked for NULL, this could generate a segfault. Check for NULL earlier, and/or wrap in NRSAFESTR.

zsistla · 2025-02-27T23:29:54Z

agent/php_txn.c

+   */
+  log_labels = nro_new(NR_OBJECT_HASH);
+  for (int i = 0; i < nro_getsize(label_keys); i++) {
+    const char* key = nro_get_array_string(label_keys, i + 1, NULL);


A NULL check for the key could help skip this round of the loop.

zsistla · 2025-02-27T23:31:36Z

agent/php_txn.c

+      exclude = false;
+    }
+    if (!exclude) {
+      nro_set_hash_string(log_labels, key, value);


Also would be a useless call if passing in a NULL key.

zsistla · 2025-02-27T23:46:41Z

agent/php_txn.c

+                     __FUNCTION__);
+    return NULL;
+  }
+


A check if log_forwarding_labels_exclude is an empty string (most common case) would allow an early exit from the function.

zsistla · 2025-02-28T00:14:04Z

daemon/internal/newrelic/log_events_test.go

+	events := NewLogEvents(10)
+	id := AgentRunID(`12345`)
+	log_data := []byte(`{"message":"test log event"}`)
+	label_data := []byte(`[{"label_type":"type1","label_value":"value1"},{"label_type":"type2","label_value":"value2"}]`)


Could you please add a test where label name has spaces?

zsistla · 2025-02-28T00:16:52Z

protocol/flatbuffers/protocol.fbs

@@ -118,7 +118,8 @@ table Transaction {
  sampling_priority:      double;  // added in the 8.2 PHP agent release
  span_events:            [Event];
  log_events:             [Event]; // added in the 10.1 PHP agent release
-  php_packages:           Event;   // added in the ??? PHP agent release
+  php_packages:           Event;   // added in the 10.17 PHP agent release
+  log_forwarding_labels:  Event;   // added in the ??? PHP agent release


I think you can go ahead and put 11.7 in there instead of ???

mfulb changed the base branch from main to dev February 20, 2025 21:25

mfulb requested review from bduranleau-nr and lavarou February 25, 2025 15:29

mfulb marked this pull request as ready for review February 25, 2025 15:29

zsistla reviewed Feb 25, 2025

View reviewed changes

agent/php_txn.c Show resolved Hide resolved

mfulb force-pushed the feat/apm_logging_labels branch from bc18e12 to df096d7 Compare February 25, 2025 18:06

lavarou reviewed Feb 26, 2025

View reviewed changes

agent/scripts/newrelic.ini.template Outdated Show resolved Hide resolved

lavarou reviewed Feb 26, 2025

View reviewed changes

zsistla reviewed Feb 27, 2025

View reviewed changes

agent/php_txn.c Show resolved Hide resolved

mfulb added 16 commits February 27, 2025 14:23

feat(agent): Adds INI values for labels for forwarded logs

012793d

feat(agent): Adds processing of log label from labels

6c84bb9

feat(agent): Label forwarding for APM logs

f38315d

feat(agent): Adds log forwarding labels supportability metric

78c8388

tests(logging): Adds Monolog3 tests for log forwarding labels

520169e

chore(tests): Removes unneeded spaces

9a3cd24

chore(daemon): Remove debugging statement

5983bc0

tests(daemon): Add simple tests for LogEvents

b8c415a

chore(daemon): Removes unneeded import

ed8cada

chore(daemon): Logs unmarshall error

6c0b65f

tests(daemon): Adds log forwarded label to test

0672ac0

fix(agent): Fixes log forwarded labels definition

5b23982

fix(agent): Fixes encoding of forwarded log labels to an Event

8186043

tests(agent): Adds test for encoding forwarded log labels

993403b

fix(agent): Updates function call for nr_txn_begin()

b5616d0

The function nr_txn_begin() now accepts a nrobj_t *log_labels parameter which contains the logging labels to be associated with log events.

fix(agent): Fixes decoding of forwarded log labels as an Event

bea9a98

mfulb added 20 commits February 27, 2025 14:23

tests(agent): Updates integration/basic tests to have log labels metric

c7fdf4a

tests(agent): Updates integration/distributed_tracing tests to have l…

f921ddb

…og labels metric

tests(agent): Updates integration/errors tests to have log labels metric

f73cfa0

tests(agent): Updates integration/external tests to have log labels m…

b5e7121

…etric

tests(agent): Updates integration/api tests to have log labels metric

83b2cbf

- added PHP5 tests - these are not for PHP 5 but are intended to run on PHP5+!

tests(agent): Updates integration/frameworks tests to have log labels…

1ed7d03

… metric

tests(agent): Updates integration/ini tests to have log labels metric

c6faa75

tests(agent): Updates integration/lang tests to have log labels metric

5ef7858

tests(agent): Updates integration/memcache tests to have log labels m…

a7595b0

…etric

tests(agent): Updates integration/memcached tests to have log labels …

da69a7c

…metric

tests(agent): Updates integration/mysql tests to have log labels metric

f02d0da

tests(agent): Updates integration/mysqli tests to have log labels metric

a3bad9b

tests(agent): Updates integration/pdo tests to have log labels metric

b97fa42

tests(agent): Updates integration/pgsql tests to have log labels metric

95f0d6c

tests(agent): Updates integration/predis tests to have log labels metric

1e8c326

tests(agent): Updates integration/queue tests to have log labels metric

3b40481

tests(agent): Updates integration/redis tests to have log labels metric

3167869

tests(agent): Updates integration/sqlite* tests to have log labels me…

2822bab

…tric

tests(agent): Adds another test for log forwarded labels using defaults

ee16060

fix(agent): Restores gitignore line for test_segment_message

c9bea37

mfulb force-pushed the feat/apm_logging_labels branch from df096d7 to c9bea37 Compare February 27, 2025 19:26

mfulb and others added 2 commits February 27, 2025 14:32

Update agent/scripts/newrelic.ini.template

192e04e

Co-authored-by: Michal Nowacki <[email protected]>

chore(agent): Improves INI description for log labels exclude directive

277d034

zsistla reviewed Feb 27, 2025

View reviewed changes

zsistla reviewed Feb 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(agent): Adds label forwarding to log events #1027

feat(agent): Adds label forwarding to log events #1027

mfulb commented Feb 20, 2025

newrelic-php-agent-bot commented Feb 20, 2025 •

edited

Loading

codecov-commenter commented Feb 24, 2025 •

edited

Loading

lavarou Feb 26, 2025

mfulb Feb 27, 2025

lavarou Feb 26, 2025

lavarou Feb 26, 2025

mfulb Feb 27, 2025

zsistla commented Feb 27, 2025 •

edited

Loading

zsistla Feb 27, 2025 •

edited

Loading

zsistla Feb 27, 2025 •

edited

Loading

zsistla Feb 27, 2025

zsistla Feb 27, 2025

zsistla Feb 27, 2025

zsistla Feb 28, 2025

zsistla Feb 28, 2025

feat(agent): Adds label forwarding to log events #1027

Are you sure you want to change the base?

feat(agent): Adds label forwarding to log events #1027

Conversation

mfulb commented Feb 20, 2025

newrelic-php-agent-bot commented Feb 20, 2025 • edited Loading

codecov-commenter commented Feb 24, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zsistla commented Feb 27, 2025 • edited Loading

zsistla Feb 27, 2025 • edited Loading

Choose a reason for hiding this comment

zsistla Feb 27, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

newrelic-php-agent-bot commented Feb 20, 2025 •

edited

Loading

codecov-commenter commented Feb 24, 2025 •

edited

Loading

zsistla commented Feb 27, 2025 •

edited

Loading

zsistla Feb 27, 2025 •

edited

Loading

zsistla Feb 27, 2025 •

edited

Loading