Data losing

[diggy (Dgraph Bot)]

Moved from GitHub dgraph/5240

Posted by H4midR:

What version of Dgraph are you using?

v1.2.2

Have you tried reproducing the issue with the latest release?

No

What is the hardware spec (RAM, OS)?

ubuntu 18.04 - CPU 4Core - 8GB Ram

Dgraph configurations

they registered as service in /etc/systemd/system/dg0.service and …
for example

[Unit]
Description=my main dgraph zero server
After=network.target
StartLimitBurst=5
StartLimitIntervalSec=10

[Service]
Type=simple
Restart=always
RestartSec=1
User=root
ExecStart=/usr/local/bin/dgraph zero --config /db/z0/config.json

[Install]
WantedBy=multi-user.target

1 Zero [
{
“my”: “localhost:5080”,
“replicas”: 3,
“wal”: “/db/z0/zw”
}
]
3 Alphas [
{
“my”: “localhost:7080”,
“zero”: “localhost:5080”,
“lru_mb”: 2048,
“postings”: “/db/a0/p”,
“wal”: “/db/a0/w”
},{
“my”: “localhost:7079”,
“zero”: “localhost:5080”,
“lru_mb”: 2048,
“postings”: “/db/a1/p”,
“wal”: “/db/a1/w”,
“port_offset”: -1
},{
“my”: “localhost:7078”,
“zero”: “localhost:5080”,
“lru_mb”: 2048,
“postings”: “/db/a2/p”,
“wal”: “/db/a2/w”,
“port_offset”: -2
}
]

Expected behaviour and actual result.

current Dgraph index 0x16993f
I have a really edgy and complicated data including 2000 exams and ~50000 Questions and 200000 choices. each node has a ~owner . in other word there is a person who owned all those nodes.
there are some other things he owed too.
i’ve got nothing when runing:

{
   data(func:uid(UserUid)) {
         owner @filter(type(Exam)) {
            uid
            name
         }
   }
}

bot i have this one

{
   data(func:uid(UserUid)) {
         owner @filter(uid(ExamUid)) {
            uid
            name
         }
   }
}

and this one

{
   data(func:type(Exam)) @filter(uid(ExamUid)) {
         uid
   }
}

It happened to me twice and i restore from backup each time.
i zipped the p&w and also export the database after crashing and zipped it to and here they are
the exported data is not complete. i mean don’t contain the whole data. so i couldn’t restore it.
export.zip
w.1.zip
the p.1.zip

Main Questions

fixing data is not important . the most important thing is that what should i do to prevent this happen again?

can Simultaneous requests cause this?

[diggy (Dgraph Bot)]

MichelDiz commented :

I think the problem is in the way that you are creating the data. I also encountered some problems that confused me.

For example, in your query, you use the “owner” predicate with the first letter in lower case. However, in your dataset, it has no lower case predicate. And Dgraph is case sensitive. Also, there is no “tuid”, maybe it was a typo.

When switching from “owner” to “Owner” I still didn’t get data. I decided to appeal to expand using “expand (_all _) {expand (_all _) {expand (_all_)}}” to check the data structure.

What I found is that all data is there, but its type schema does not match what is in the dataset. Or the Type Schema doesn’t reflects the structure of your dataset.

Also, you have repeated predicates with different cases. It is considered good practice to adopt a standard. e.g. camelCase or Pascal Case. So you avoid confusion.

Another thing, avoid zipping Dgraph folders. Always use Export for this. If you rely on the files, there may be compatibility issues in some cases unintentionally. It’s good to avoid. But if you know what you’re doing, okay tho.

Anyway, the problem is in the way you insert the data and how you are designing the structure. If you share these details I may be able to help.

[diggy (Dgraph Bot)]

H4midR commented :

@MichelDiz
thanks brother for notations.
“tuid” was typing mistake.i updated it.
the types defined PascalCase and pridicates are cameCase
the structure was fine and working perfect before.

that export file is not correct due to exported after crash.
here is one old correct one.
but due to avoiding high load to database by programming mistake the types didn’t defined completely, so the expand(all) wont work at all.
here is one good export.
the data are OK. and schema is correct.
export.OK.zip

i will prepare the full types definition for you and add it here as soon as possible.

i’m ready to get access to server to you and will be Grateful for your helping and appreciate for that.

[diggy (Dgraph Bot)]

H4midR commented :

some of types are

type Person {
    xid
    name
    fallows
    owner
    buy
    used
    shared
}
type Exam {
    name
    component
}
type Question{
    xid
    name
    choice
}
type ExamSessionGroup {
    xid
    endDate
    startDate
    exam
    examSession
}
type ExamSession {
    xid
    corrected
    result
    answer
    question
    scoring
}

[diggy (Dgraph Bot)]

H4midR commented :

2 Notes : that may cues this.
Max Open File was 1024 ;
*referring to official documentation it’s better to be 65535 .
the server configuration is not enogph:

low Memory which cues some daily Out Of Memory Crash
*the recommended minimum is 8GB.

is it just it or other thing may inflect it too?
there are some bigger projects who are working fine with Dgraph which can be count this as one.

[diggy (Dgraph Bot)]

MichelDiz commented :

About “Max Open File was 1024” this is standard on the OS. In the documentation, it is recommended that the user increase the limit https://dgraph.io/docs/deploy/#too-many-open-files

Regarding memory, it is good that you do load tests. Imagine scenarios and create a “load test” for these scenarios and see how your instances react. The recommendation in the minimum documentation is for minimum loads. Intense use and giant loads require more resources, so you have to adapt to the demand you need.

The Type seems better now.

Now, after the new dataset. I’m not sure what the issue is.
Are you saying that after a crash Dgraph has modified the dataset?
In your previous dataset you had an “Owner” predicate and in this new one is “owner”. Did the crash change the predicate case? that’s weird. In the previous dataset the data was there, but with different predicates.

With the new dataset provided, I got:

{
   data(func: uid(0x5c8e2)) {
    uid
    owner(first:1) @filter(type(Exam)) {
    uid
    name
  }
   }
}

{
  "data": {
    "data": [
      {
        "uid": "0x5c8e2",
        "owner": [
          {
            "uid": "0x509a7",
            "name": "آزمون آنلاین تستی عربی دهم رشته انسانی | درس 8: يا مَنْ فِی الْبِحارِ عَجائِبُهُ"
          }
        ]
      }
    ]
  }
}

So, there’s no issue now right?

[diggy (Dgraph Bot)]

MichelDiz commented :

I have a really edgy and complicated data including 2000 exams and ~50000 Questions and 200000 choices. each node has a ~owner . in other word there is a person who owned all those nodes.

A question. I didn’t quite understand that part. Are you saying that a single user holds 252,000 objects? Sometimes the browser may crash with a certain amount of information on the screen. That was what happened?

[diggy (Dgraph Bot)]

H4midR commented :

About “Max Open File was 1024” this is standard on the OS. In the documentation, it is recommended that the user increase the limit Self-managed cluster - Deploy

Regarding memory, it is good that you do load tests. Imagine scenarios and create a “load test” for these scenarios and see how your instances react. The recommendation in the minimum documentation is for minimum loads. Intense use and giant loads require more resources, so you have to adapt to the demand you need.

The Type seems better now.

Now, after the new dataset. I’m not sure what the issue is.
Are you saying that after a crash Dgraph has modified the dataset?
In your previous dataset you had an “Owner” predicate and in this new one is “owner”. Did the crash change the predicate case? that’s weird. In the previous dataset the data was there, but with different predicates.

With the new dataset provided, I got:

{
   data(func: uid(0x5c8e2)) {
    uid
    owner(first:1) @filter(type(Exam)) {
    uid
    name
  }
   }
}

{
  "data": {
    "data": [
      {
        "uid": "0x5c8e2",
        "owner": [
          {
            "uid": "0x509a7",
            "name": "آزمون آنلاین تستی عربی دهم رشته انسانی | درس 8: يا مَنْ فِی الْبِحارِ عَجائِبُهُ"
          }
        ]
      }
    ]
  }
}

So, there’s no issue now right?

yes , the export.OK.zip was exported a month ago it’s ok and data are fine.
the Owner pridicate was a mistake from old preview versions. you can simply drop it. forget about the Owner .
the “owner” is correct and your result is fine .
my main question is what was the main reason of this happened?
the file attached to the first comment was exported after the crash and it wasn’t complete, some predicates disappeared.

[diggy (Dgraph Bot)]

H4midR commented :

I have a really edgy and complicated data including 2000 exams and ~50000 Questions and 200000 choices. each node has a ~owner . in other word there is a person who owned all those nodes.

A question. I didn’t quite understand that part. Are you saying that a single user holds 252,000 objects? Sometimes the browser may crash with a certain amount of information on the screen. That was what happened?

the browser is not important , it’s something usual that browser can’t handle a canvas with too many nodes.
the go-client ( dgo ) couldn’t get data from this query :

{
   data(func: uid(0x5c8e2)) {
    uid
    owner(first:1) @filter(type(Exam)) {
    uid
    name
  }
   }
}

the point is that this person who owned this 250000 nodes is the admin of website, other users owned things too. other users was fine while the admin lost his owned data.some of this users buied some objects from admin. so they could see that nodes and other part of website working well , but the admin who owned that 250000 nods couldn’t see them.