Request for Guidance on Calculating On-Disk Usage per Node in a Cluster

[jkfran]

Describe the issue

I'm currently working with a Qdrant cluster deployed on Kubernetes, and I have a need to calculate the on-disk usage of the cluster, specifically how much space each node is using. This information is crucial for planning and managing backups, monitoring, and overall resource management.

Details

While the official documentation provides useful information on in-memory size calculation, I couldn't find specific guidance on how to measure or estimate the on-disk usage. I would like to know how much space each node is using, as this would enable me to plan my storage requirements more effectively, especially when utilizing Persistent Volume Claims (PVCs) in a Kubernetes environment.

Questions

1. Is there a recommended way or formula to calculate the on-disk usage per node within a Qdrant cluster?
2. Are there any built-in tools, metrics, or API endpoints that I can utilize to obtain this information?
3. If not, could you please consider this as a feature request? Having this functionality would greatly aid in managing and scaling Qdrant deployments.

Thank you for your assistance and for considering this request. Any guidance or recommendations would be greatly appreciated.

[generall]

Hi @jkfran, the exact disk usage depends on many parameters and there are no exact formula to do that.
There is only a very rough estimation: size of your raw data (vectors + payload) + 50%.

You can estimate the usage better by uploading some fraction of your data to Qdrant and measuring ./storage size. Usually, the disk usage grows linearly with inserting more data