Type filtering on reverse [uid] predicates with expand(_all_)

Issues Dgraph
1

Version: 20.11.2
Client: Go

This may be a bug, but I’ll start this out as a question. We have a very particular set of requirements on the graph structure we store in dgraph. Nearly all the [uid] predicates we have in our Types are @reverse indexed and what we want to do is select a node and query for everything connected to it by either a forward or reverse [uid] predicate. We have a custom versioning scheme on top of this that sandwhiches an extra node in between two uids and we assign a least one Type we call “ConnVersion” and if that version is the latest, we also add a Type “Latest”. The reason we add the latest type is we don’t know a priori which predicates we want to find the connection for so we expand(all) to get outgoing predicates from that node. This means we can’t filter based on a boolean prediate such as “latest” and instead filter on a Type “Latest” that is set on the latest ConnVersion node. In the end it looks something like this. For a connection between node A and B:

schema:

<predicate_name>: [uid] @reverse .

type PredicateType {
    <predicate_name>
    <~predicate_name>
}

type ConnVersion {
}

type Latest {
}

A —predicate_name—> C —predicate_name—>B
A —predicate_name—> C’ —predicate_name—>B

A, B, and C, and C’ has some type “PredicateType” that includes both <predicate_name> and <~predicate_name>.
C also has a Type ConnVersion and Latest.
C’ has dgraph.type ConnVersion, but DOES NOT have draph.type Latest

Let’s assume this is the entire graph.

If I query for connections on A, I do query like

{
  connections(func:uid(0xa)) {
    uid 
    expand(_all_) @filter(type(ConnVersion) AND type(Latest)) {
      uid
      expand(_all_) {
        uid
        expand(_all_) {
	  uid
        }
      }
    }
  }
}

Which gets me all forward and reverse connections that connect A and B, but removes the path that includes C’. This is perfect.

The problem comes when I do the same exact query, but starting with node B:

{
  connections(func:uid(0xb)) {
    uid 
    expand(_all_) @filter(type(ConnVersion) AND type(Latest)) {
      uid
      expand(_all_) {
        uid
        expand(_all_) {
	  uid
        }
      }
    }
  }
}

This ONLY returns B and nothing else. If I remove the @filter:

{
  connections(func:uid(0xb)) {
    uid 
    expand(_all_) {
      uid
      expand(_all_) {
        uid
        expand(_all_) {
	  uid
        }
      }
    }
  }
}

I get everything, but I also get C’. The point of those filters is so that I only get C, not C’. This feels like a bug to me, but we thought maybe this was by design? If so, why? Are there any workarounds for our use case that you can think of?

2

Hi Daniel,
Could you share the schema as you retrieve it from your Dgraph instance. ( response to DQL query schema {} to check what as been saved). I’m surprised that you can declare the existence of the reverse edge in the type definition. That’s something I need to check with our engineers because I didn’t think expand would work on reverse relationships.

That’s were I have doubts about what we actually support.
We need to clarify this part first.

About your use case, I think using facets on the relationship with a date or latest info would be more aligned with Dgraph design. Have you tried this approach? That said there are usually several good solutions to the same problem with a graph:)
Note : this is not related to the @reverse and expand(all) with filter problem that you have. We’ll have a look…

3

Ok. I did some test. It seems that we have a defect to solve here.
Your use case can be even simplified (I just created a type “B” with a predicate “in” with @reverse.
I declared B with in and ~in predicate. and 2 nodes <0x1> in <0x2>
Expand all works on 0x2 and follows the reverse. expand with filter does not.

The defect is reported as an issue in our git repo. Add other filters to expand() · Issue #5803 · dgraph-io/dgraph · GitHub
Could you have a look and add a comment with your case. We will set a proper priority to this old issue.

In the meantime is it an option for you to explicitly write your predicates and reverse predicates (instead of expand(all) in your queries ? or write the behavior of expand in your go client at least for the part using filters on a reverse edge : building your DQL query string based on the schema and repeat the block for all predicates and reverse instead of expand(all) ?

4

Raphael, I think that issue may be reporting on something slightly different. Filtering on Type should work for forward connections, but not reverse. So in your above example:

{
  q(func: uid(0x1)) @filter(type(B)) {
    uid
    expand(_all_) {
       uid
    }    
  }
}

would work and return 0x1 and 0x2 in the resulting JSON response, but

{
  q(func: uid(0x2)) @filter(type(B)) {
    uid
    expand(_all_) {
       uid
    }    
  }
}

would only return 0x2 Same query aside from the UID, type filter is just fine in the forward direction, but doesn’t work in the reverse direciton. I can add this to that issue, but I think it’s fundamentally a different problem.

As for why we aren’t using facets for this, that’s actually where we originally started, but since facets are not indexed, it presented a very possible bottleneck for our use case. I haven’t outline all the details of our particular strategy for versioning, but shoving in the extra node in the middle gets us around the indexing issue.

We can explicitly build a query that has all the predicates for the types on a node, but we were trying to avoid the added latency of an extra query to determine what predicates to build. That’s the only other solution we came up with, but the type filtering would result in simpler queries to dgraph, which we feel makes things a bit easier to follow, debug, and maintain.

5

Although on that issue if we could filter on more than type, we could instead filter on the datetime/latest fields we have on the connection version node, which would be nice.

6

It looks very connected to me @Daniel_Wilson. I think your query

is not returning the expected value because expand(_all_) @filter(...) does not follow the reverse edge from B to C. And it should.
I have created a repro case in postman and added it to the issue. I also set the kind/bug as we should either have expand(_all_) @filter(...) working or at least we should prevent user from declaring the reverse relationship in the Type definition. (As you know, expand(all) is using the Type definition).
Let’s see what our engineering can do. Thanks for the use case details.

7

ok. I got your point #5803 is an enhancement request (more filters) and your’s is a bug.
I’ll open it…[BUG]: expand(_all_) @filter(type(X)) does not work on reverse relationships · Issue #8558 · dgraph-io/dgraph · GitHub