I’m using baderdb to store a lot of hierarchical data using a key indexing scheme like “key1:key2:key3” etc. If I’ve got a lot of data under something like “users:10204:items”, is that prefix going to take up storage space multiplied by the number of items I’m going to have (tens of thousands)? In my head I thought that because the keys are lexicographically sorted there would be some structural sharing going on but I realized that isn’t necessarily the case.
@krstoff Badger uses prefix difference encoding for storage optimisation. It divides each SST into multiple blocks. In each block first key is called as base key. Base key is stored fully. For next keys only difference of keys with base key is stored.
3 Likes