upgrade opentelemetry-exporter-prometheus-remote-write to use protobuf 5.26

[michaelschuett-tomtom]

Description

This change for my case it to allow us to use this in a project that needs to use protobuf 5.29. I have tested in that project that everything still works as expected so the version limit is only based on the restriction in the produced pypi package not on any code limitation.

I originally was looking at saying the package uses =~5.29 however this conflicts with another random package I had installed streamlit 1.30.0 requires protobuf<5,>=3.20, but you have protobuf 5.29.3 which is incompatible. Which made me think it might be nice to let people use all known working versions of protobuf with this package instead of arbitrarily deciding what one is correct.

Fixes #3179

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

I tested locally by running

$ PACKAGE_NAME=opentelemetry-exporter-prometheus-remote-write VERSION=1.0 ./scripts/build_a_package.sh
$ pip install --force-reinstall ./dist/opentelemetry_exporter_prometheus_remote_write-0.51b0.dev0-py3-none-any.whl

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

I don't think so?

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: michaelschuett-tomtom / name: Michael Schuett (df710aa, ab31103, e5e2826, b6acca1, 7f8b83c, c2fd3a3, 720e018, 2c53a1b, 414a68f)
• ✅ login: emdneto / name: Emídio Neto (7daebc7)

[emdneto]

Hey: thanks for the PR. I think we should at least add a test using protobuf 5, although tests pass and there is no guarantee for cross-version runtime. What we did in opentelemetry-proto was to support only one major version, protobuf 5.

@aabmass do you think we should wait for the protobuf Q12025 release for this kind of issue?

[aabmass]

Which made me think it might be nice to let people use all known working versions of protobuf with this package instead of arbitrarily deciding what one is correct.

It's not arbitrary like @emdneto mentioned, protobuf makes it very clear that this is not officially supported. This PR lets the user decide but I fear people will come here when things break and expect us to fix it. I would prefer to do what @emdneto mentioned and just support protobuf 5. It's been out for almost a year now.

Is there a not a newer streamlit version that works with protobuf 5?

[michaelschuett-tomtom]

Which made me think it might be nice to let people use all known working versions of protobuf with this package instead of arbitrarily deciding what one is correct.

It's not arbitrary like @emdneto mentioned, protobuf makes it very clear that this is not officially supported. This PR lets the user decide but I fear people will come here when things break and expect us to fix it. I would prefer to do what @emdneto mentioned and just support protobuf 5. It's been out for almost a year now.

Is there a not a newer streamlit version that works with protobuf 5?

I am completely fine with just having 5.x supported. steamlit isn't a specific issue for me but just something I ran into when testing in a few different environments. I'll update this to be only protobuf 5.x.

[michaelschuett-tomtom]

https://protobuf.dev/support/version-support/#python picked 5.26 based on this page.

[michaelschuett-tomtom]

@emdneto more than happy to write some tests although I am not sure exactly what kind of test to write here to improve coverage. I ran tox -e test-exporter-prometheus-remote-write and it passed outside of one test that looks like it needs a local server spun up HTTPConnectionPool(host='victoria', port=8428).

I do see that we are loading the protobufs in the tests opentelemetry.exporter.prometheus_remote_write.gen.types_pb2 so any breakage at least around imports changing should be caught. However this obviously isn't great coverage but I am not sure if we just want to load in all the protobufs or are looking for more of an integration test to be added where we are actually sending them off to the server.

[aabmass]

If we're just doing protobuf 5 I think his comment is resolved. Current tests should be ok

[aabmass]

Actually I think we need to regenerate the protobuf code too?

[michaelschuett-tomtom]

Actually I think we need to regenerate the protobuf code too?

Just regenerated with protoc 29.3

[emdneto]

Actually I think we need to regenerate the protobuf code too?

Right. There's a script in the opentelemetry-exporter-prometheus-remote-write/proto directory called generate-proto-py.sh @michaelschuett-tomtom did you use that script? If so, could you please create a requirement.txt or a note in the README file with the pinned version of grpcio-tools you've installed to run the script? Noticed you already did while I was writing the comment. Thanks

[emdneto]

LGTM with Changelog entry

[michaelschuett-tomtom]

LGTM with Changelog entry

Was thinking to add this under Breaking changes in the main CHANGELOG.md since upon update in the case someone has pinned protobuf to a specific version it will cause issues... But this is also true for any dependency update so I'm not 100% sure if this should go under another area.

[michaelschuett-tomtom]

These small PRs always turn out to have the most foot guns 😄 I was just testing some more locally to make sure I don't break anything for people and was running the tests with different versions of presumably supported protobuf versions as defined in the requirements (~=5.26). Because I generated with 29.3 protobuf will blow up with anything that isn't python protobuf version 5.29.3 or later.

Although we could just say this is the min version we should likely be nice to people since they are all still supported. I'll regenerate using v26 of protoc so that we actually support ~=5.26.

Learning on the go here although i have used protobuf before I haven't had to care much about versions since I controled client and server and generate them at the same time.

With the PR as it currently is installing 5.26 python protobuf throws the following.

google.protobuf.runtime_version.VersionError: Detected incompatible Protobuf Gencode/Runtime versions when loading opentelemetry/exporter/prometheus_remote_write/gen/remote.proto: gencode 5.29.3 runtime 5.29.2. Runtime version cannot be older than the linked gencode version. See Protobuf version guarantees at https://protobuf.dev/support/cross-version-runtime-guarantee.

[emdneto]

https://protobuf.dev/support/cross-version-runtime-guarantee.

Yeah. In that case, if we want to support starting from 5.26 we must re-generate it with protoc 26. You can also add a second test-requirements.txt file with protobuf 5.29, which should also work. With that we are sure 5.26 and newer runtime versions are covered (which is guaranteed according to this)

[michaelschuett-tomtom]

Done, updated docs and change log and regenerated protos with v26.