From 66dc97a6e7fb9c643525f32c6c6c491bbb382fbd Mon Sep 17 00:00:00 2001 From: Daniel Gomez Blanco Date: Thu, 16 Jan 2025 23:45:51 +0000 Subject: [PATCH 1/7] Add blog post on Go Compile-Time Instrumentation SIG --- .../2025/go-compile-time-instrumentation.md | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 content/en/blog/2025/go-compile-time-instrumentation.md diff --git a/content/en/blog/2025/go-compile-time-instrumentation.md b/content/en/blog/2025/go-compile-time-instrumentation.md new file mode 100644 index 000000000000..e4f20a479f66 --- /dev/null +++ b/content/en/blog/2025/go-compile-time-instrumentation.md @@ -0,0 +1,65 @@ +--- +title: Alibaba and Datadog Join Forces on Go Compile-Time Instrumentation +linkTitle: Go Compile-Time Instrumentation +date: 2025-01-17 +author: OpenTelemetry Governance Committee +--- + +Standards are only useful if they're widely adopted, and adoption is only +effective if the available tooling facilitates it. I imagine +[SI units](https://en.wikipedia.org/wiki/International_System_of_Units) would +not have been too popular when they were introduced if you had to build your own +scales to weigh things in Kilograms! + +If you use [OpenTelemetry in Go](https://opentelemetry.io/docs/languages/go/), +you'll be familiar with the challenges of configuring instrumentation libraries +to automatically generate telemetry from well-known open source components. Due +to the compiled nature of the language, you currently have two options (unless +you want to "build your own scales", or the OTel equivalent): + +- Use a separate binary that analyzes your Go process and attaches eBPF programs + to hooks in your application (see + https://github.com/open-telemetry/opentelemetry-go-instrumentation/). +- Manually configure instrumentation libraries in your code (see an + [example](https://opentelemetry.io/docs/languages/go/getting-started/#instrument-the-http-server) + to instrument `net/http`). + +For different reasons, it is possible that none of those options is viable, or +optimal, in your environment. However, things are about to change! + +## Industry collaboration at the heart of open standards + +Over the past few months, OpenTelemetry has received not one, but two donation +proposals from industry leaders to provide a solution to the problem described +above, and enable the use of zero-code, vendor-neutral, compile-time +instrumentation in Go applications. These are: + +- Alibaba's + [opentelemetry-go-auto-instrumentation](https://github.com/alibaba/opentelemetry-go-auto-instrumentation) + (see + [donation proposal](https://github.com/open-telemetry/community/issues/2344)). +- Datadog's [Orchestrion](https://github.com/datadog/orchestrion) (see + [donation proposal](https://github.com/open-telemetry/community/issues/2497)). + +We are really grateful to Alibaba and Datadog for these donation proposals. This +continues to demonstrate the convergence of the wider industry towards the +standards defined by OpenTelemetry. + +The most exciting part of this announcement is that it won't be Alibaba's or +Datadog's solution that "wins". In the true spirit of open source collaboration, +these two organisations have decided to join forces and commit the necessary +resources to bootstrap a new +[Go Compile-Time Instrumentation SIG](https://github.com/open-telemetry/community/blob/main/projects/go-compile-instrumentation.md), +with the intention of providing a unified, vendor-neutral approach that picks +the best aspects of each solution and benefits the community as a whole. + +This SIG will focus on: + +- Developing compiler plugins or enhancements that inject instrumentation code + automatically, ensuring minimal runtime performance overhead and compatibility + with existing Go projects. +- Providing standardized instrumentation patterns aligned with OpenTelemetry and + other monitoring frameworks. + +We look forward to seeing this new SIG in operation, and cannot wait for the +fruits of this awesome collaboration! From 1640c806037ab04c72e85315f86f16d6806f3789 Mon Sep 17 00:00:00 2001 From: Daniel Gomez Blanco Date: Thu, 16 Jan 2025 23:56:38 +0000 Subject: [PATCH 2/7] Fix link --- content/en/blog/2025/go-compile-time-instrumentation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2025/go-compile-time-instrumentation.md b/content/en/blog/2025/go-compile-time-instrumentation.md index e4f20a479f66..590552abd287 100644 --- a/content/en/blog/2025/go-compile-time-instrumentation.md +++ b/content/en/blog/2025/go-compile-time-instrumentation.md @@ -19,7 +19,7 @@ you want to "build your own scales", or the OTel equivalent): - Use a separate binary that analyzes your Go process and attaches eBPF programs to hooks in your application (see - https://github.com/open-telemetry/opentelemetry-go-instrumentation/). + [opentelemetry-go-instrumentation](https://github.com/open-telemetry/opentelemetry-go-instrumentation/)). - Manually configure instrumentation libraries in your code (see an [example](https://opentelemetry.io/docs/languages/go/getting-started/#instrument-the-http-server) to instrument `net/http`). From 1f542e94ccea0136e9c5275bee19141a7bece90f Mon Sep 17 00:00:00 2001 From: Daniel Gomez Blanco Date: Fri, 17 Jan 2025 13:50:41 +0000 Subject: [PATCH 3/7] Address pull request comments --- .../2025/go-compile-time-instrumentation.md | 22 +++++++++- static/refcache.json | 40 +++++++++++++++++++ 2 files changed, 61 insertions(+), 1 deletion(-) diff --git a/content/en/blog/2025/go-compile-time-instrumentation.md b/content/en/blog/2025/go-compile-time-instrumentation.md index 590552abd287..dff626297cff 100644 --- a/content/en/blog/2025/go-compile-time-instrumentation.md +++ b/content/en/blog/2025/go-compile-time-instrumentation.md @@ -45,6 +45,11 @@ We are really grateful to Alibaba and Datadog for these donation proposals. This continues to demonstrate the convergence of the wider industry towards the standards defined by OpenTelemetry. +Compile-time instrumentation leverages the standard Go toolchain’s `-toolexec` +mechanism to re-write Go source code before it is passed to the Go compiler, +adding instrumentation in all relevant places (including dependencies as well as +the Go standard library). + The most exciting part of this announcement is that it won't be Alibaba's or Datadog's solution that "wins". In the true spirit of open source collaboration, these two organisations have decided to join forces and commit the necessary @@ -53,7 +58,10 @@ resources to bootstrap a new with the intention of providing a unified, vendor-neutral approach that picks the best aspects of each solution and benefits the community as a whole. -This SIG will focus on: +The initial output from this SIG will soon replace +[instrgen](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/dafdad14b7858c7f491c8cb72e4bc7deaf9378e3/instrgen/README.md), +Go SIG's initial experimental approach to provide compile-time instrumentation. +In the longer term, this SIG will focus on: - Developing compiler plugins or enhancements that inject instrumentation code automatically, ensuring minimal runtime performance overhead and compatibility @@ -61,5 +69,17 @@ This SIG will focus on: - Providing standardized instrumentation patterns aligned with OpenTelemetry and other monitoring frameworks. +If you are interested in contributing, or you simply want to find out more, +here's some useful information about the SIG: + +- GitHub repository: + [opentelemetry-go-compile-instrumentation](https://github.com/open-telemetry/opentelemetry-go-compile-instrumentation) +- CNCF Slack: + [#otel-go-compt-instr-sig](https://cloud-native.slack.com/archives/C088D8GSSSF) +- Meetings: Every other Thursday UTC: 08:00 – 09:00 (subscribe to this + [Google Group](https://groups.google.com/a/opentelemetry.io/g/calendar-go) for + calendar invites, or visit the + [OpenTelemetry Calendar](https://calendar.google.com/calendar/embed?src=c_2bf73e3b6b530da4babd444e72b76a6ad893a5c3f43cf40467abc7a9a897f977%40group.calendar.google.com)) + We look forward to seeing this new SIG in operation, and cannot wait for the fruits of this awesome collaboration! diff --git a/static/refcache.json b/static/refcache.json index fc84032837fe..38ef8b326e02 100644 --- a/static/refcache.json +++ b/static/refcache.json @@ -1631,6 +1631,10 @@ "StatusCode": 206, "LastSeen": "2025-01-15T13:17:28.410343-05:00" }, + "https://calendar.google.com/calendar/embed": { + "StatusCode": 401, + "LastSeen": "2025-01-17T13:21:32.505559Z" + }, "https://calendly.com/euwg-user-feedback-session/end-user-feedback-session": { "StatusCode": 200, "LastSeen": "2024-03-12T07:55:41.183146657Z" @@ -1823,6 +1827,10 @@ "StatusCode": 200, "LastSeen": "2024-08-20T08:40:41.563366481Z" }, + "https://cloud-native.slack.com/archives/C088D8GSSSF": { + "StatusCode": 200, + "LastSeen": "2025-01-17T12:58:36.720595Z" + }, "https://cloud-native.slack.com/archives/CJFCJHG4Q": { "StatusCode": 200, "LastSeen": "2024-08-09T10:45:09.309237-04:00" @@ -3339,6 +3347,10 @@ "StatusCode": 200, "LastSeen": "2025-01-06T11:23:54.720969-05:00" }, + "https://en.wikipedia.org/wiki/International_System_of_Units": { + "StatusCode": 200, + "LastSeen": "2025-01-17T12:58:25.584113Z" + }, "https://en.wikipedia.org/wiki/JSON": { "StatusCode": 200, "LastSeen": "2025-01-06T11:23:32.692041-05:00" @@ -4235,6 +4247,10 @@ "StatusCode": 200, "LastSeen": "2024-08-06T15:18:42.027775+02:00" }, + "https://github.com/alibaba/opentelemetry-go-auto-instrumentation": { + "StatusCode": 206, + "LastSeen": "2025-01-17T12:58:30.442533Z" + }, "https://github.com/alolita": { "StatusCode": 206, "LastSeen": "2025-01-13T12:11:00.000033-05:00" @@ -4787,6 +4803,10 @@ "StatusCode": 206, "LastSeen": "2025-01-13T12:41:52.17647-05:00" }, + "https://github.com/datadog/orchestrion": { + "StatusCode": 206, + "LastSeen": "2025-01-17T12:58:33.133351Z" + }, "https://github.com/dattto": { "StatusCode": 200, "LastSeen": "2024-12-17T15:37:33.267299-05:00" @@ -7115,10 +7135,18 @@ "StatusCode": 206, "LastSeen": "2025-01-07T10:33:12.688711-05:00" }, + "https://github.com/open-telemetry/opentelemetry-go-compile-instrumentation": { + "StatusCode": 206, + "LastSeen": "2025-01-17T12:58:34.746413Z" + }, "https://github.com/open-telemetry/opentelemetry-go-contrib": { "StatusCode": 206, "LastSeen": "2025-01-07T10:31:43.730419-05:00" }, + "https://github.com/open-telemetry/opentelemetry-go-contrib/blob/dafdad14b7858c7f491c8cb72e4bc7deaf9378e3/instrgen/README.md": { + "StatusCode": 206, + "LastSeen": "2025-01-17T13:21:30.685258Z" + }, "https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CONTRIBUTING.md#code-owners": { "StatusCode": 206, "LastSeen": "2025-01-16T11:38:11.841222-05:00" @@ -9839,6 +9867,10 @@ "StatusCode": 200, "LastSeen": "2024-06-18T10:36:39.590318+02:00" }, + "https://groups.google.com/a/opentelemetry.io/g/calendar-go": { + "StatusCode": 200, + "LastSeen": "2025-01-17T12:58:38.699931Z" + }, "https://groups.google.com/a/opentelemetry.io/g/calendar-maintainer-meeting": { "StatusCode": 200, "LastSeen": "2024-06-18T10:36:41.220362+02:00" @@ -11235,6 +11267,14 @@ "StatusCode": 200, "LastSeen": "2025-01-06T11:32:22.440332-05:00" }, + "https://opentelemetry.io/docs/languages/go/": { + "StatusCode": 206, + "LastSeen": "2025-01-17T12:58:27.368618Z" + }, + "https://opentelemetry.io/docs/languages/go/getting-started/#instrument-the-http-server": { + "StatusCode": 206, + "LastSeen": "2025-01-17T12:58:28.687325Z" + }, "https://opentracing.io": { "StatusCode": 206, "LastSeen": "2024-12-18T06:36:29.862015-05:00" From 4c0f0192ff780ac69368f86d83d49b483b22b337 Mon Sep 17 00:00:00 2001 From: Daniel Gomez Blanco Date: Fri, 17 Jan 2025 13:54:42 +0000 Subject: [PATCH 4/7] Add cSpell ignore words --- content/en/blog/2025/go-compile-time-instrumentation.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/blog/2025/go-compile-time-instrumentation.md b/content/en/blog/2025/go-compile-time-instrumentation.md index dff626297cff..4133a443bc52 100644 --- a/content/en/blog/2025/go-compile-time-instrumentation.md +++ b/content/en/blog/2025/go-compile-time-instrumentation.md @@ -3,6 +3,7 @@ title: Alibaba and Datadog Join Forces on Go Compile-Time Instrumentation linkTitle: Go Compile-Time Instrumentation date: 2025-01-17 author: OpenTelemetry Governance Committee +cSpell:ignore: instrgen toolexec --- Standards are only useful if they're widely adopted, and adoption is only From 1198b6e7564cf4d387275f6a4908063840966861 Mon Sep 17 00:00:00 2001 From: Daniel Gomez Blanco Date: Fri, 17 Jan 2025 14:29:56 +0000 Subject: [PATCH 5/7] Update calendar link --- .../en/blog/2025/go-compile-time-instrumentation.md | 10 +++++----- static/refcache.json | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/content/en/blog/2025/go-compile-time-instrumentation.md b/content/en/blog/2025/go-compile-time-instrumentation.md index 4133a443bc52..17882bfed6bf 100644 --- a/content/en/blog/2025/go-compile-time-instrumentation.md +++ b/content/en/blog/2025/go-compile-time-instrumentation.md @@ -3,7 +3,7 @@ title: Alibaba and Datadog Join Forces on Go Compile-Time Instrumentation linkTitle: Go Compile-Time Instrumentation date: 2025-01-17 author: OpenTelemetry Governance Committee -cSpell:ignore: instrgen toolexec +cSpell:ignore: instrgen toolexec --- Standards are only useful if they're widely adopted, and adoption is only @@ -77,10 +77,10 @@ here's some useful information about the SIG: [opentelemetry-go-compile-instrumentation](https://github.com/open-telemetry/opentelemetry-go-compile-instrumentation) - CNCF Slack: [#otel-go-compt-instr-sig](https://cloud-native.slack.com/archives/C088D8GSSSF) -- Meetings: Every other Thursday UTC: 08:00 – 09:00 (subscribe to this - [Google Group](https://groups.google.com/a/opentelemetry.io/g/calendar-go) for - calendar invites, or visit the - [OpenTelemetry Calendar](https://calendar.google.com/calendar/embed?src=c_2bf73e3b6b530da4babd444e72b76a6ad893a5c3f43cf40467abc7a9a897f977%40group.calendar.google.com)) +- Meetings: Every other Thursday UTC: 08:00 – 09:00 (subscribe to + [this Google Group](https://groups.google.com/a/opentelemetry.io/g/calendar-go) + for calendar invites, or read more about + [our community calendar](https://github.com/open-telemetry/community/?tab=readme-ov-file#calendar)) We look forward to seeing this new SIG in operation, and cannot wait for the fruits of this awesome collaboration! diff --git a/static/refcache.json b/static/refcache.json index 38ef8b326e02..3511ddbee9f4 100644 --- a/static/refcache.json +++ b/static/refcache.json @@ -1633,7 +1633,7 @@ }, "https://calendar.google.com/calendar/embed": { "StatusCode": 401, - "LastSeen": "2025-01-17T13:21:32.505559Z" + "LastSeen": "2025-01-17T14:06:45.293165Z" }, "https://calendly.com/euwg-user-feedback-session/end-user-feedback-session": { "StatusCode": 200, From d81a62fe9b0accf5f88f105286888e101655ebcc Mon Sep 17 00:00:00 2001 From: Daniel Gomez Blanco Date: Fri, 17 Jan 2025 20:28:32 +0000 Subject: [PATCH 6/7] Update content/en/blog/2025/go-compile-time-instrumentation.md Co-authored-by: Przemyslaw Delewski <102958445+pdelewski@users.noreply.github.com> --- content/en/blog/2025/go-compile-time-instrumentation.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/blog/2025/go-compile-time-instrumentation.md b/content/en/blog/2025/go-compile-time-instrumentation.md index 17882bfed6bf..87d1e72364ea 100644 --- a/content/en/blog/2025/go-compile-time-instrumentation.md +++ b/content/en/blog/2025/go-compile-time-instrumentation.md @@ -62,6 +62,7 @@ the best aspects of each solution and benefits the community as a whole. The initial output from this SIG will soon replace [instrgen](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/dafdad14b7858c7f491c8cb72e4bc7deaf9378e3/instrgen/README.md), Go SIG's initial experimental approach to provide compile-time instrumentation. + In the longer term, this SIG will focus on: - Developing compiler plugins or enhancements that inject instrumentation code From 76305264a99fc69e40ea16482282eb00518dffb0 Mon Sep 17 00:00:00 2001 From: Daniel Gomez Blanco Date: Sat, 18 Jan 2025 21:15:04 +0000 Subject: [PATCH 7/7] Code review comments --- .../en/blog/2025/go-compile-time-instrumentation.md | 3 ++- static/refcache.json | 12 ++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/content/en/blog/2025/go-compile-time-instrumentation.md b/content/en/blog/2025/go-compile-time-instrumentation.md index 87d1e72364ea..0e1f7b879b22 100644 --- a/content/en/blog/2025/go-compile-time-instrumentation.md +++ b/content/en/blog/2025/go-compile-time-instrumentation.md @@ -61,7 +61,8 @@ the best aspects of each solution and benefits the community as a whole. The initial output from this SIG will soon replace [instrgen](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/dafdad14b7858c7f491c8cb72e4bc7deaf9378e3/instrgen/README.md), -Go SIG's initial experimental approach to provide compile-time instrumentation. +OpenTelemetry's initial experimental approach to provide Go compile-time +instrumentation based on `-toolexec`. In the longer term, this SIG will focus on: diff --git a/static/refcache.json b/static/refcache.json index e88f7e06f3a4..07622a52c384 100644 --- a/static/refcache.json +++ b/static/refcache.json @@ -6411,6 +6411,10 @@ "StatusCode": 206, "LastSeen": "2025-01-16T14:23:17.554218-05:00" }, + "https://github.com/open-telemetry/community/blob/main/projects/go-compile-instrumentation.md": { + "StatusCode": 206, + "LastSeen": "2025-01-18T20:45:35.792292Z" + }, "https://github.com/open-telemetry/community/blob/main/reports/ADA_Logics-collector-fuzzing-audit-2024.pdf": { "StatusCode": 206, "LastSeen": "2025-01-16T14:23:17.415384-05:00" @@ -6459,6 +6463,14 @@ "StatusCode": 200, "LastSeen": "2024-10-15T15:52:36.027937+01:00" }, + "https://github.com/open-telemetry/community/issues/2344": { + "StatusCode": 206, + "LastSeen": "2025-01-18T20:45:29.87596Z" + }, + "https://github.com/open-telemetry/community/issues/2497": { + "StatusCode": 206, + "LastSeen": "2025-01-18T20:45:32.577654Z" + }, "https://github.com/open-telemetry/community/issues/828": { "StatusCode": 206, "LastSeen": "2025-01-13T11:43:27.58776-05:00"