@Phill240 @Lash dgraph data directory (p or w) contains sst and vlog files. These are generated by Badger. SST file stores the (key,value) or (key, value pointer) where value pointer is (file, offset, len) in a vlog file. Your value will be stored in the LSM tree (the sst file) if it is less than a threshold (default is 1 KB). The vlog file is the Write-Ahead-Log. All operations are logged to the vlog file and then they are cleaned up later.
The compression option in dgraph affects the SST files only and since the majority of the disk space is occupied by the vlog files you’re not seeing any significant size difference. You should look at the total size of SST files with different compression settings, you should see a difference. @Phill240 if you have the data directories, can you share the total size of SST files with different compression levels?
Compressing vlog files isn’t easy and badger doesn’t support it yet. This is something we might support in the future.
We’re also working on making vlog files pure WAL so that we can reclaim disk space faster.
https://github.com/dgraph-io/badger/pull/1445
You should not delete the vlog files. Your values could be stored in the vlog files and if you delete it, you’ll have data loss. The DB would start, but you would see erroneous results. Dgraph runs ValueLogGC which is supposed to free up the disk space for you.