Bulk loader becomes slow when memory gets full

Problem solved, I guess (just guess, I have no idea how to verify) the problem is mainly related to the strategy of OS memory management, then the second factor is number of go routines.

My personal server(64G, 32 core, 480G+1T SSD) previous installed with centOS 7, and I found once the memory required larger than physical memory than it started consuming swap, when swap used up, bulk crashed with OOM.

However, when my laptop (Ubuntu 17.10) with lower memory (16G, 8 core CPU, 250G SSD) didn’t OOM and worked much faster with the same data. I suspect it may be the OS issue, so I reinstall my server with Ubuntu 17.10, then I found though it still crashed with OOM, it didn’t consume swap when the memory required become larger than 64G.

Then I followed Manish’s advice to reduce go routine number from 32(default) to 16, and it worked pretty well and stable.

MAP 01h16m06s rdf_count:963.1M rdf_speed:210.9k/sec edge_count:2.786G edge_speed:610.1k/sec

As you see, only around 1 hour, it finished near 1 billion RDF and the speed is around 600k, compared my last test in centOS, it cost more than 20 hours to 1 billion and the speed declined to 30k.

P.S.:
Anyway, it is just a guess, I am not sure it is the centOS problem, I just record what I observed, anyone with similar could have a try