Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send specified cache for remote inference: EDSL changes #1517

Open
johnjosephhorton opened this issue Jan 23, 2025 · 0 comments
Open

Send specified cache for remote inference: EDSL changes #1517

johnjosephhorton opened this issue Jan 23, 2025 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@johnjosephhorton
Copy link
Contributor

When a user runs a job, if they have explicitly included a cache e.g.,

jobs.run(cache = my_cool_cache, ...) 

and the job is run remotely, that cache should be sent to coop and used as the cache for running the job.
Similarly, if the user explicitly says cache = False we should not use a cache remotely.
If the user leaves the cache blank, we should use the coop universal cache (TBD).

This will require changes to:

  • Jobs
  • JobsRunnerAsyncI
  • Coop
  • JobsRemoteInferenceHandler

It will also require a modification to the coopr endpoint. Probably first step is to modify the coopr endpoint so it does not throw an exception when that cache is part of the payload.

Optional

In the job running log interface, we should probably also indicate the that job is being run with a cache in the logger / display.

@johnjosephhorton johnjosephhorton added the enhancement New feature or request label Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants