429 throttle limit - cloud dedicated instance

[dmai (Daniel Mai)]

Dgraph Cloud backends have max pending query limits set to prevent the backend from getting overloaded. That said, new dedicated backends these days are now automatically configured with higher limits. @RickSalmon your dedicated backend has now been updated and you shouldn’t see the too many requests message as often.

@RickSalmon

1. does the limit apply to a single client or all the clients together? (for context, I have multiple server apps using dql to talk to dgraph)

The limit applies for all clients. The limit is specifically per Alpha in a backend.

@azafoo

• what is the recommended way to diagnose which queries are triggering these 429s?
• If dgraph is struggling in a mixed read/write scenario, what are best practices we can put in place to compensate?
• is there a reference implementation (in any language, but we use typescript), that shows how we should handle these 429s which are not based on any fixed rate limits that I’m aware of?

@azafoo @jj0b This 429 Too Many Requests error happens when the current pending queries count reaches the configured limit. The metrics for dgraph_pending_queries_total shows the current pending queries for any particular Dgraph Alpha. Also, you can inspect the traces to see the current list of running or high-latency queries.