Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disk cache backend #403

Open
argaen opened this issue May 13, 2018 · 3 comments
Open

Disk cache backend #403

argaen opened this issue May 13, 2018 · 3 comments
Labels

Comments

@argaen
Copy link
Member

argaen commented May 13, 2018

Vote here with 👍 if you are interested on having this

Implementation would be done with https://github.com/Tinche/aiofiles

@argaen argaen added the feature label May 13, 2018
@N-Coder
Copy link

N-Coder commented May 27, 2018

dogpile.cache provides similar (but blocking) caching functionality with a focus on what is called cached_stampede in this library. They also have interchangeable backends and already implemented file-system backed storage and locking in addition to redis and memcached. As aiofiles just wraps the blocking file-system IO calls in run_in_executor, we could simply skip aiofiles, reuse the dogpile.cache DBM-based file backend and wrap it's methods in run_in_executor. Their approach to fs-based locking (in addition to the user customizable mutex creation) also looks promising.

@argaen
Copy link
Member Author

argaen commented May 27, 2018

with a focus on what is called cached_stampede in this library

You can protect against cache_stampede with locking provided in aiocache too http://aiocache.readthedocs.io/en/latest/locking.html.

we could simply skip aiofiles, reuse the dogpile.cache DBM-based file backend and wrap it's methods in run_in_executor

If there are enough votes for this feature I'll dig on how to implement it however, without knowing how dogpile is doing it, I don't want to add it as a dependency of this project. If your point was on just reusing the code and wrapping it, yeah could have a look on how to do it.

PS: Obviously contributions are welcome if you want to :)

@paulo-raca
Copy link

I've implemented an sqlite backend that should cover the disk-backed storage: #486

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants