Provide simple option for limiting total memory usage

Issues
, , ,
15

jarifibrahim commented :

@gonzojive How big are your values? The memory profile you shared shows that y.Slice was holding 15 GB of data. That’s unusual unless you have a big value.

I can’t find a way to force the OS to reclaim the memory freed by Go, which seems to use MADV_FREE on recent linux version (- The Go Programming Language). It would be helpful to force the OS to reclaim such memory get a more accurate picture of what’s going on.

debug.FreeOSMemory() debug package - runtime/debug - Go Packages is what you’re looking for

From runtime package - runtime - Go Packages,

    // HeapIdle minus HeapReleased estimates the amount of memory
    // that could be returned to the OS, but is being retained by
    // the runtime so it can grow the heap without requesting more
    // memory from the OS. If this difference is significantly
    // larger than the heap size, it indicates there was a recent
    // transient spike in live heap size.
    HeapIdle uint64

So heapIdle - heapreleased in your case is

>>> (9454788608-3498221568) >> 20
5680

which is 5.6 GB. That’s the amount of memory golang runtime is holding.