Badger with a sufficient block cache would help a lot. But, probably not get you all the way there — for that, Badger would need to serve a key directly from Ristretto, instead of from the block cache.
So, what you could do is, store data in Badger, and use Ristretto on top of it for caching — do also set decent cache within Badger.