Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Trim lakeFS / lakeFSFS compatbility tests bivector #8299

Open
arielshaqed opened this issue Oct 23, 2024 · 0 comments
Open

Trim lakeFS / lakeFSFS compatbility tests bivector #8299

arielshaqed opened this issue Oct 23, 2024 · 0 comments
Labels
tech-debt

Comments

@arielshaqed
Copy link
Contributor

arielshaqed commented Oct 23, 2024 •
edited
Loading

As of #8928 we test lakeFSFS with 50 versions of lakeFS. That's a bit much:

  • Not all lakeFS versions are there.
  • This is an integration test, so necessarily flaky. Indeed we already have occasional flakes. Every version we add increases the probability of flaking out - and this is superlinear, because adding load of any kind makes flakes more likely.
  • It adds little value. For a patch release in particular there should be no difference.

I would like to codify a policy for these tests:

  • Test nothing before 1.x (so remove 0.113.0, which AFAIR is 1.0.0 by its pre-release name).
  • Test only the latest patch version of each minor release (lakeFSFS compatbility test: Test only the latest patch version of each minor release #8308).
  • @treeverse/product : declare a support window for lakeFS vs. lakeFSFS. During that window we might add patch releases, but lakeFSFS would continue to work.
    • Version pairs outside of the window would be supported just like any other software using the lakeFS API: if a lakeFS release breaks a version, we remain committed to fixing it.
    • I would be happy to declare some minor versions of lakeFS as "LTS" and commit to an X-month window of support. I suggest X=12.
    • Commit to a Y-month window of support for non-LTS minor versions, Y << X. I suggest Y=3.
@arielshaqed arielshaqed mentioned this issue Oct 23, 2024
Merged
arielshaqed added a commit that referenced this issue Oct 23, 2024
@arielshaqed
List lakeFS versions on separate lines of lakeFSFS compatiblity test
2e3be9b 
It's getting _really_ long.  Also correct the comment about what being on
the list actually means.

Relevant to #8299.  Resolves one portion of it.
arielshaqed added a commit that referenced this issue Oct 23, 2024
@arielshaqed
Test only the latest patch version of every minor 1.x release
6dff91d 
For any minor version, only the latest patch version needs to keep running
with with lakeFS.  After all: suppose a new lakeFSFS version does not work
with a lakeFS minor version.  If this were an issue, then we would release
some patch release for that lakeFS version.

This brings lakeFSFS support closer to that of any other lakeFS client: we
keep compatibility through the API and to fix any server-side breakages, a
new patch release.

Part of #8299, but not all of it.
@arielshaqed arielshaqed mentioned this issue Oct 23, 2024
Merged
arielshaqed added a commit that referenced this issue Oct 27, 2024
@arielshaqed
List lakeFS versions on separate lines of lakeFSFS compatiblity test
0ad88f4 
It's getting _really_ long.  Also correct the comment about what being on
the list actually means.

Relevant to #8299.  Resolves one portion of it.
arielshaqed added a commit that referenced this issue Oct 27, 2024
@arielshaqed
Test only the latest patch version of every minor 1.x release
682e382 
For any minor version, only the latest patch version needs to keep running
with with lakeFS.  After all: suppose a new lakeFSFS version does not work
with a lakeFS minor version.  If this were an issue, then we would release
some patch release for that lakeFS version.

This brings lakeFSFS support closer to that of any other lakeFS client: we
keep compatibility through the API and to fix any server-side breakages, a
new patch release.

Part of #8299, but not all of it.
@arielshaqed arielshaqed mentioned this issue Oct 27, 2024
Closed
arielshaqed added a commit that referenced this issue Oct 31, 2024
@arielshaqed
List lakeFS versions on separate lines of lakeFSFS compatiblity test
8fdc196 
It's getting _really_ long.  Also correct the comment about what being on
the list actually means.

Relevant to #8299.  Resolves one portion of it.

Rebased to include 1.40.0.
arielshaqed added a commit that referenced this issue Oct 31, 2024
@arielshaqed
Test only the latest patch version of every minor 1.x release
08609ad 
For any minor version, only the latest patch version needs to keep running
with with lakeFS.  After all: suppose a new lakeFSFS version does not work
with a lakeFS minor version.  If this were an issue, then we would release
some patch release for that lakeFS version.

This brings lakeFSFS support closer to that of any other lakeFS client: we
keep compatibility through the API and to fix any server-side breakages, a
new patch release.

Part of #8299, but not all of it.
arielshaqed added a commit that referenced this issue Oct 31, 2024
@arielshaqed
Trim patch versions from lakeFSFS / lakeFS compatibility tests vector (…
e50348e 
…#8300)

* List lakeFS versions on separate lines of lakeFSFS compatiblity test

It's getting _really_ long.  Also correct the comment about what being on
the list actually means.

Relevant to #8299.  Resolves one portion of it.

Rebased to include 1.40.0.

* Test only the latest patch version of every minor 1.x release

For any minor version, only the latest patch version needs to keep running
with with lakeFS.  After all: suppose a new lakeFSFS version does not work
with a lakeFS minor version.  If this were an issue, then we would release
some patch release for that lakeFS version.

This brings lakeFSFS support closer to that of any other lakeFS client: we
keep compatibility through the API and to fix any server-side breakages, a
new patch release.

Part of #8299, but not all of it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tech-debt
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@arielshaqed @itaiad200