Partial edge removal with 20.03.4-rc2 and 20.07.0-rc1

Issues Dgraph
, ,
1

What version of Dgraph are you using?

Doesn’t work: 20.03.4-rc2 and 20.07.0-rc1
Works: 20.03.3

What is the hardware spec (RAM, OS)?

Linux Debian Stable 16Go RAM

Steps to reproduce the issue (command/config used to run Dgraph).

Edit: The bug is different than described here, details in this post.

Remove an edge having @reverse and @count indexes (not sure @count is involved with the issue).

Expected behaviour and actual result.

Expected: Edge is removed both ways.
Actual: Normal edge is still present, reverse edge has been removed.

Schema extract :

type Game {
  [some fields...]
}
type Answer {
  game_answer
}
game_answer:  uid  @reverse @count  .

Json delete:

[{"uid":"0x8d569","game_answer":{"uid":"0x8d35f"}}]

A query proves the incoherent state :

{
  Answers(func: type(Answer)) @filter(gt(count(game_answer), 1)) {
    uid
    games as game_answer {
      uid
    }
  }
  Games(func: uid(games)) {
    uid
    ~game_answer { 
      uid
      game_answer { uid }
    }
  }
}

Response :

{
  "data": {
    "Answers": [
      {
        "uid": "0x8d569",
        "game_answer": {
          "uid": [
            "0x8d35f",
            "0x8d6e0"
          ]
        }
      }
    ],
    "Games": [
      {
        "uid": "0x8d35f",
        "~game_answer": [
          {
            "uid": "0x8d360",
            "game_answer": {
              "uid": "0x8d35f"
            }
          },
          {
            "uid": "0x8d9f4",
            "game_answer": {
              "uid": "0x8d35f"
            }
          }
        ]
      },
      {
        "uid": "0x8d6e0",
        "~game_answer": [
          {
            "uid": "0x8d569",
            "game_answer": {
              "uid": [
                "0x8d35f",
                "0x8d6e0"
              ]
            }
          },
          {
            "uid": "0x8d6e1",
            "game_answer": {
              "uid": "0x8d6e0"
            }
          }
        ]
      }
    ]
  }
}

As you can see, answer 0x8d569 points to 0x8d35f but the reverse edge is not present.
Only the reverse edge has been removed.
More over, game_answer is of type uid and thus should not contain a list.

2

That’s not an issue. It is a known behavior. You should delete in both directions. It would be very expensive for Dgraph to do this. You can use Upsert Block for this. See this comment
https://github.com/dgraph-io/dgraph/issues/3447#issuecomment-562793852

PS. Also take a read on Pawan’s comment, it is the last comment.

UPDATE (Using v20.07.0-rc1):

I noticed that you are running the mutation from the parent to child. I have tried to reproduce it, but I can’t. It is all working as it should.

type Game {
  name
  <~game_answer>
}
type Answer {
  name
  game_answer
}

name: string .
game_answer:  uid  @reverse @count  .

{
   "set": [
      {
         "uid": "_:Game_1",
         "dgraph.type": "Game",
         "name": "Game 1"
      },
      {
         "name": "Answer 1",
         "dgraph.type": "Answer",
         "game_answer": {
            "uid": "_:Game_1"
         }
      }
   ]
}

{
   "delete": [
      {
         "uid": "0x4",
         "game_answer": {
            "uid": "0x3"
         }
      }
   ]
}

{
  Answers(func: type(Answer)) @filter(eq(count(game_answer), 1)) {
    uid
    name
    games as game_answer {
      uid
    }
  }
    A2(func: type(Answer)) {
    uid
    name
    game_answer {
      uid
    }
  }
  Games(func: uid(games)) {
    uid
    name
    ~game_answer { 
      uid
      game_answer { uid }
    }
  }
  G2(func: type(Game))  {
    uid
    name
    ~game_answer { 
      uid
      game_answer { uid }
    }
  }
}

Result

{
  "data": {
    "Answers": [],
    "A2": [
      {
        "uid": "0x4",
        "name": "Answer 1"
      }
    ],
    "Games": [],
    "G2": [
      {
        "uid": "0x3",
        "name": "Game 1"
      }
    ]
  }
}
3

@MichelDiz so why did it work in the previous version? I also tried to reproduce it by adding extra assertions to one of our tests but the tests deletes in both directions.

@myo Is there a minimal dataset you could provide that consistently reproduces the issue in your setup? thanks.

In the meantime, as Michel suggested, you can try manually removing the edges in both directions.

4

I didn’t tested on previous. It is working on v20.07.0-rc1. At first, I thought that the case was about from child to parent delation. But it is the other way around.

If you delete the reverse edge from the child. It wouldn’t delete in the parent. In that case you have to delete from the origin and them delete the target node itself.

5

Ok. Yeah, I think deleting the main edge should remove the reverse one. It does in my test which is why I am confused.

6

Thx @MichelDiz & @martinmr for your answers.

@MichelDiz, please note that game_answer in my data is from an answer to a game, not the opposite. Then when I delete the normal edge, dGraph only removes the reverse edge.

I’m well aware of the delete behaviour you mention first, my code is doing well since a few years on that.

@martinmr I’ll try to do a minimal example, but I think it won’t be easy.

7

Yeah, I have noticed. In my sample example you can see that I take it into account. Maybe it is something else you’re doing.

8

Wait a minute. Ahh, I see.

@martinmr he is using 1:1 relation. But for some reason, it creates a list of UIDs. In my end, I had converted to 1:Many relation ( game_answer: [uid] @reverse @count .). For sure the case is there. So probably it is a bug in the 1:1 relation behavior. Maybe this happened during his upgrade(Maybe he didn’t export and reimported RDF. Or maybe he did, but Dgraph ignored the new 1: 1 behavior for some reason). Cuz as it is 1:1 relation, it shouldn’t be a list of UIDs. The predicate “uid” should never be a list. (That looks like the @normalize was used in the query).

It should look like this

"game_answer": {
          "uid": "0x7"
        }

@myo do you have any idea why your edge has 2 UIDs? as a single edge list?

In some time ago I have seen this kind of bug. But it just never happened again.

I did a second test, without converting game_answer to 1:Many. But I couldn’t reproduce. For this fact, I think it is related to the upgrade process.

9

The bug happens during my tests, which always clear dgraph by stopping it, removing its folders, starting it, and bulk importing my test data set (RDF format).
That game_answer index has always been a 1:1 so I don’t know how I could get a one to many relationship.

10

Could you share just that part of your RDF? (just that slice) So I do a bulkload and other tests to make sure where the bug is.

11

Yes, I tested it with a 1:1 relationship. There was a recent change to uid predicates but nominally it only affects how reverse edges are added, not how they are deleted.

1 Like
12

Actually, the change I made is not what is causing this. I cherry-picked that change into the 20.03 branch but it’s not been merged. So it cannot be the cause if the issue is also in 20.03.4-rc2.

13

I wrote a minimal go program that reproduces the issue.
The problem is slightly different than I thought.

It happens when I remove an edge and add another one from the same node in a single transaction.
The result is the former one is not removed, thus leading to 2 outgoing edges, even if it is defined as 1:1 in the schema.

If I use 2 separate transactions, it works fine.

dgraph_edge_not_removed.go (3.3 KB)

14

@martinmr, @MichelDiz, have you reproduced the bug with my code?
It seems to me it is a quite serious data consistency issue.

15

Sorry, I missed your first message. We’ll take a look at it but for now use two separate transactions.

16

Ok Martin. I’ll stay with 20.03.3 for now because I need to do these mutations in a single transaction.

17

I confirm the issue is still here with stable 20.07.0 and 20.03.4.

18

Hello the team,
Any update on this?

19

I no longer have the bug during my tests with 20.07.1-rc1 but oddly my minimal example still reproduces the issue.

21

Hey @myo,

It seems the issue is not specific to the dgo client, I’ve found that the issue occurs with raw HTTP too. I’ve created a shell script for this: test_with_txn.sh (1.7 KB)

Interestingly, the issue does not occur if I perform both the delete and set operations in one mutation (i.e. without a transaction): test_without_txn.sh (1.3 KB)

Our team will investigate further and get back to you.

1 Like