Hi,
I reran the reduce phase with the command:
dgraph bulk --v 3 --skip_map_phase --replace_out --mapoutput_mb=2048 --num_go_routines=12 --ignore_errors --out $DGRAPHOUT --tmp $DGRAPHTMP --map_shards=1 --reduce_shards=1 -z $ZEROURL -f $DATADIR.rdf.gz --format rdf -s $SCHEMA --http 0.0.0.0:8080
Attached is the goroutine stack dump after hang on REDUCE for 1 hour. Before it hanged for around 40 hours, and then I canceled the task.
From the dump I don’t see any errors, but a lot of handleBlock. My computer almost don’t have any CPU occupation for dgraph bulk load process, and the directories of out and tmp didn’t change since it stucked at 100%, which means no io write event. And my computer still have a lot of available memory.
By the way, I can successfully run the bulk loader if I use a much smaller dataset but same schema. And I removed all of the characters other than [a-zA-Z0-9.- ] when I preprocess the data.
Please let me know if you have any suggestions. Thanks!
goroutine?debug=2.gz (134.7 KB)