Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a read-write store #1577

Open
chrisstaite opened this issue Feb 6, 2025 · 1 comment · May be fixed by #1581
Open

Create a read-write store #1577

chrisstaite opened this issue Feb 6, 2025 · 1 comment · May be fixed by #1581

Comments

@chrisstaite
Copy link
Contributor

Similarly to the size partitioning store, there's cases where you'd like to change the backend store dependant on the operation direction. For example, a dev AC and prod AC, you can flow prod to Dev but not vice versa. Another example is worker nodes, currently they store their results in the local CAS for no reason, the local filesystem store should only be populated on read operations, not write.

This would be a trivial store which would act like a diode when paired with a noop store.
@chrisstaite
Copy link
Contributor Author

Tricky implementation detail: when paired with a fast-slow store this would need to know if the original request is a read or write operation. Perhaps this might be better served as an option on the fast-slow store. Like cache_on: {read,write,both}.

chrisstaite added a commit to chrisstaite/nativelink that referenced this issue Feb 8, 2025
@chrisstaite
Fast slow store directions
f014616 
There are multiple use cases where we don't want a fast-slow store to
persist to one of the stores in some direction.  For example, worker
nodes do not want to store build results on the local filesystem, just
with the upstream CAS.  Another case would be the re-use of prod action
cache in a dev environment, but not vice-versa.

This PR introduces options to the fast-slow store which default to the
existing behaviour, but allows customisation of each side of the fast
slow store to either persist in the case or get operations, put
operations or to make them read only.

Fixes TraceMachina#1577
@chrisstaite chrisstaite linked a pull request Feb 8, 2025 that will close this issue
Open
6 tasks
chrisstaite added a commit to chrisstaite/nativelink that referenced this issue Feb 8, 2025
@chrisstaite
Fast slow store directions
69e7103 
There are multiple use cases where we don't want a fast-slow store to
persist to one of the stores in some direction.  For example, worker
nodes do not want to store build results on the local filesystem, just
with the upstream CAS.  Another case would be the re-use of prod action
cache in a dev environment, but not vice-versa.

This PR introduces options to the fast-slow store which default to the
existing behaviour, but allows customisation of each side of the fast
slow store to either persist in the case or get operations, put
operations or to make them read only.

Fixes TraceMachina#1577
chrisstaite added a commit to chrisstaite/nativelink that referenced this issue Feb 8, 2025
@chrisstaite
Fast slow store directions
194e7e4 
There are multiple use cases where we don't want a fast-slow store to
persist to one of the stores in some direction.  For example, worker
nodes do not want to store build results on the local filesystem, just
with the upstream CAS.  Another case would be the re-use of prod action
cache in a dev environment, but not vice-versa.

This PR introduces options to the fast-slow store which default to the
existing behaviour, but allows customisation of each side of the fast
slow store to either persist in the case or get operations, put
operations or to make them read only.

Fixes TraceMachina#1577
chrisstaite added a commit to chrisstaite/nativelink that referenced this issue Feb 8, 2025
@chrisstaite
Fast slow store directions
251dda7 
There are multiple use cases where we don't want a fast-slow store to
persist to one of the stores in some direction.  For example, worker
nodes do not want to store build results on the local filesystem, just
with the upstream CAS.  Another case would be the re-use of prod action
cache in a dev environment, but not vice-versa.

This PR introduces options to the fast-slow store which default to the
existing behaviour, but allows customisation of each side of the fast
slow store to either persist in the case or get operations, put
operations or to make them read only.

Fixes TraceMachina#1577
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fast slow store directions chrisstaite/nativelink
1 participant
@chrisstaite