Length of Keys - Performance Concerns?

Users Badger
1

I’m implementing Badger in my app to be used as a global cache store. It fits the purpose but because I have a quite a number of jobs running at the same time, I use prefixes to partition the database for specific jobs.

The concern I have is that each job is identified by a UUID (16 bytes) then an additional, say, 8 bytes to indicate what the key refers to (ie. maybe a timestamp or such). Would this be too long to be used as a key? Is there any performance concerns I should be worried about? I’m most worried that if the key is too long, it may cause it to be escaped to the heap.

2

Hi @b_t,

I think Badger should be able to support the key size you are referring. @ibrahim would be able to provide a more comprehensive answer.

1 Like
3

Internally, the keys get prefix diffed (in the SSTables). So, there’s quite some savings there on key sizes.

4

Thanks @mrjn

Just for the sake of discussion, what if the keys are all completely random? Would there be a chance of leaking to heap?

5

The keys are kept sorted in SSTables. So, doesn’t matter how random they are.

Not sure what you mean by “leaking to heap.”

1 Like
6

Ah okay.

I was referring to the fact that if a key is too large to store in the stack, I’d assume it would be escaped to the heap and the pointer to the key would be stored instead. As I understand, this all happens through the Go runtime but I’m trying to reduce the possibility of it happening.

7

Your key (16 bytes UID + 8 byte + more) will escape to the heap since it will be passed to badger. Since byte slice is just a header to an underlying array, there will be only one copy of your actual key in heap but multiple copies of the slice header (3 word size).

Badger will store the actual key to the SST and value log. For larger values (more than options.ValueThreshold), the SST will store the key and pointer to the value in the vlog file.

2 Likes
8

Thanks for clarifying!

1 Like