so you mean, if an alpha is in a faulty state I need to remove it from the cluster and create a new one?
Yes. If you have a bad alpha and want to fix it, don’t delete the p directory. Simple restarts, restarts with config changes are good. But don’t touch the p and w directory.
Because dgraph did not automatically sync its nodes when one alpha is empty?
As I said before, dgraph has no way of knowing that all the data is gone by error. (For all it can just be a new db or someone dropped or data). So you should remove the alpha from the raft alpha gorup, and then add a new one. In that case we would automatically stream a new snapshot.
Seems to be like this, as I removed now all volumes beside one, and the result was, that all alphas start now, but nearly all data is gone.
You need to manually copy the good data from good alpha to bad alpha if you are rebuilding the dgraph cluster youself.
this does not make sense to me.
If one node is left with correct data it should synchronize to the others.
Now it seems, some data is synced, and most data is lost. but this is not how I understand HA.
I mean it means, I do not need HA in this scenario, I can just run a single instance, as this would be the same effect.
Dgraph will do provide you HA, if you remove the node from the alpha group, and then add a new node. Basically when you just delete the p directory, we currently don’t have any way to detect it. If we can detect it, we can get a snapshot from the leader. This is kind of what removing and adding a new node would do. It would trigger a new snapshot.
Also, what I notice on my local machine is, when a node has issues to start, dgraph connection is very slow.
What issues?