Rust-rocksdb: Make `cf_handle` lock-free

Created on 13 Mar 2019  ·  12Comments  ·  Source: rust-rocksdb/rust-rocksdb

The cf_handle function is often called very frequently and it's implementation requires acquiring a read-lock to an underlying map. If this is called in a hot-path, it can have performance implications. We should investigate switching this to an approach that doesn't require a lock. For example, we could make create_cf and drop_cf and any other function that mutates the internal map require a mutable reference to DB and make the caller responsible for acquiring any locks in order to mutate the DB.

Most helpful comment

Yes, it is. We plan to release it in near future.

All 12 comments

like this?

use std::collections::HashMap;

#[derive(Debug)]
struct ColumnFamily(i32);

struct DB {
    cfs:HashMap<i32,ColumnFamily>,
}

impl DB {
    fn new()->DB{
        let mut cfs = HashMap::new();
        cfs.insert(1,ColumnFamily(1));
        cfs.insert(2,ColumnFamily(2));
        DB{
            cfs
        }
    }
    fn get(&self,key:i32)->&ColumnFamily{
        self.cfs.get(&key).unwrap()
    }
    fn drop_key(&mut self,key:i32){
        self.cfs.remove(&key);
    }
}

fn main() {
    let mut f = DB::new();
    let a = f.get(1);
    println!("{:?}",a);
    f.drop_key(1);
    let c = f.get(2);
    println!("{:?}",c);
}

FWIW, in paritech's fork cf_handle is Copy, which is not technically not safe (OTOH one would argue it's hard to misuse), but avoids using locks.

@hjiayz Yes, very similar to that.

When I tried making this change (i.e. take &mut self when creating/dropping column families) I ran into problems with the way ColumnFamily is defined. It will store a mutable reference in the PhantomData which makes code like this fail:

#[test]
fn property_cf_test() {
    let n = DBPath::new("_rust_rocksdb_property_cf_test");
    {
        let opts = Options::default();
        let mut db = DB::open_default(&n).unwrap(); // Notice the `mut`
        let cf = db.create_cf("cf1", &opts).unwrap();
        let value = db.property_value_cf(cf, "rocksdb.stats").unwrap().unwrap();

        assert!(value.contains("Stats"));
    }
}

The error happen when making a method call on db that takes &self, such as db.property_value_cf() and passing in a ColumnFamily (which stores a &mut in the db field):

error[E0502]: cannot borrow `db` as immutable because it is also borrowed as mutable
  --> tests/test_property.rs:39:21
   |
38 |         let cf = db.create_cf("cf1", &opts).unwrap();
   |                  -- mutable borrow occurs here
39 |         let value = db.property_value_cf(cf, "rocksdb.stats").unwrap().unwrap();
   |                     ^^                   -- mutable borrow later used here
   |                     |
   |                     immutable borrow occurs here

I can't think of a very elegant way around this, but one option is to not return a ColumnFamily from create_cf and instead let consuming code fetch it with a call tocf_handle():

#[test]
fn property_cf_test() {
    let n = DBPath::new("_rust_rocksdb_property_cf_test");
    {
        let opts = Options::default();
        let mut db = DB::open_default(&n).unwrap();
        db.create_cf("cf1", &opts).unwrap();
        let cf = db.cf_handle("cf1").unwrap(); // Get the CF
        let value = db.property_value_cf(cf, "rocksdb.stats").unwrap().unwrap();

        assert!(value.contains("Stats"));
    }
}

Would changing create_cf() to return Result<(), Error> be an acceptable change?

@dvdplm Adding lifetimes to ColumnFamily has ended up being a bit of a nightmare from an ergonomic perspective. I'd like to pursue an approach that doesn't need a lifetime but also doesn't run the risk of using a column family pointer after it's been deleted. That change will likely impact how we go about removing locks from cf_handle - you probably won't see this issue.

Agree with @iSynaptic. Obviously returning a reference to ColumnFamily is a safe way only in this case.

Maybe #298 can be merged then?

314

Is it resolved by #314? Any plans to release if so?

Yes, it is. We plan to release it in near future.

@aleksuss what is the status of releasing it? :)

@bkchr I think we will do it next week.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

rohitjoshi picture rohitjoshi  ·  10Comments

jonhoo picture jonhoo  ·  22Comments

zach-schoenberger picture zach-schoenberger  ·  7Comments

freehere107 picture freehere107  ·  12Comments

iSynaptic picture iSynaptic  ·  31Comments