Streaming response

Users Dgraph
1

Hi!

I failed to find an answer to my question, so I am here.

I wonder if dgraph can stream response. The need is we are planning to use dgraph for catalog service (users, groups, etc) and some responses can be a little too bulky. Something like

get all users who are members of group ALL or one of its child group (child of child, child of child of child, etc)

A catalog may have millions of users and returning them in one big group is not the right thing to do. Does dgraph have this kind of functionality?

2

No, there’s no such functionality.
You can do instead, is pagination.

1 Like
3

Well, better than nothing yet proper streaming seems to be a better match for these kind of tasks. Hope this will work fast enough, although I am highly suspicious about it: sorting means prior fetching, i.e. the database get the full response first, then sort it, than return Nth, (N+1)th, …, Mth elements. Each time for each page and the number of collected elements can be quite large at that.

4

you can wrapper a “stream” with pagination.
every time you pull your stream, you get one page as your stream element.

5

Pagination requires ordering, ordering requires all data to be collected first. This means costly traversal procedure on each request.