Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upsize_limit for cache does not take into account the size of directories #140
Comments
|
You didn’t miss anything in the docs. I simply haven’t accounted the directory sizes because I don’t know good and easy way to do so. The directory structure could change with a different Disk so maybe it’s best to exclude them. I’d lean toward a simple docs update saying it excludes directory sizes. I do think it’s worth calling out. Others have been concerned by the large number of directories. When I created diskcache, I had an abundance of disk space so I don’t worry about it much. |
|
Blurb added to Caveats in a8a014c to be released in v5 |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have noticed that the
volumemethod does not return the exact size of the cache, but in some scenarios there could be a discrepancy of more than 100 MB.From the code, I gather that
volume = page_size * page_count + total_file_size, wherepage_size * page_countis an estimate of the database size andtotal_file_sizeis the size of all file-backed entries.The problem is that this does not take into account the space occupied by directories. There are two levels of directories, whose names correspond to one byte in the hash, so there can be at most 65536 directories. Each directory being 4KB in my case, this gives at most 256MB of space which is unaccounted for.
This is not a problem in my scenario but the behavior was confusing, so I was wondering if I missed the explanation somewhere or if it is implied by the current documentation of
volume.Version: 3.1.1
OS: Ubuntu