Swig on RocksDB

Users
1

Currently, we use GoRocksDB which uses C API from Facebook’s RocksDB.

The problem is that when we want some new functionality, e.g., blockcache getusage, the function might not be in RocksDB C API or GoRocksDB. To add it to both can be time-consuming.

Another problem is that GoRocksDB is currently not compatible with the latest version of RocksDB. We have to help maintain GoRocksDB.

Here are some alternatives:

  1. Fork someone’s C API. Fork someone’s Go wrapper. Maintain both ourselves.
  2. Fork someone’s Go wrapper but continue to use someone’s C API.
  3. Do away with C API. Try to swig the C++ directly. This seems tricky and may require some extra C++ code around a small subset of functions to make it more swig-friendly. We have done a bit of exploration here.

I personally prefer either (1) or (3), and maintain everything ourselves end-to-end.

Firstly, there are really not that many RocksDB functions that we need I believe, and we can grow these wrapper code over time.

Secondly, RocksDB core functions seem quite stable and I doubt we need to do much maintenance. The more functions you try to use, the more maintenance you need, but we can reduce our usage.

Thirdly, these wrapper code are not always the fastest. C API does some copying and some Go wrappers (not GosRocksDB) will do an additional copy probably with cgo.GoString(C string). If we maintain our wrappers, we can explore further optimization and potentially gain an edge here.

What do you all think?

3 Likes
2

I’m in favor of option 1. Ensuring that we’re compatible with the latest version of rocksdb, and whatever functions we do use, they’re optimized is important to us. So, it’s best if we build a very small subset of RocksDB functionality that we need, and maintain the entire Go <--> C <--> C++ for RocksDB.

3

Thanks Manish. I am in favor of option 1 too. We can work on that.

1 Like
4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.