Not able to query across tree structure with millions of links

Users Dgraph
,
7

@LCL I have no definite answer here. You could explore the following options:

Query optimization:

1- you don’t need to ‘enter’ inBuckets and create a map (variable) of uids.
2- you can count the inBuckets in the first var
So you can use

   var(func: eq(Parameter.key, "someKey")) {
        buckets as count(Parameter.inBuckets)
    }

Now buckets is a map of uid(key) → count of buckets. You can sum the values

query {
    var(func: eq(Parameter.key, "someKey")) {
        buckets as count(Parameter.inBuckets)
    }

    result() {
      count: sum(val(buckets))
     }
}

It should be faster, but I did not test on realistic data set, I just recreated you schema and played with few parameters to test the queries. If you have an extract of your dataset to share in RDF we can do some testing.

Architecture

The other option to explore is the architecture to leverage predicate sharding with multiple Alphas so that the relationships inBuckets, inFIles, and parameters will get a chance to be served by different Alpha nodes. I’ll ask engineers about their thoughts on this use case.
Let me know if the updated query runs faster.