gonzojive commented :
The backup file is 100GB (99,969,891,936 bytes)
In this case, backup.go’s Load function. seems to be a major offender. It does not account for the size of the values at all. Added logging shows huge key/value accumulation and no flushing:
I0326 09:16:43.884962 5195 taskstorage.go:147] not flushing with 1269 entries, 73.2K key size, 3.2G combined size, 9.6M limit
I’m guessing there are many places where value size is not accounted for when making memory management decisions.