Rust-rocksdb: Support WriteBatch serialization and deserialization.

Created on 7 Jan 2021  ·  4Comments  ·  Source: rust-rocksdb/rust-rocksdb

Hello, i want to implement replication between rocksdb instances. As far as i understand, now rust-rocksdb supports all needed apis, except one last feature - writebatch serialization. I want to create WriteBatch from bytes and serialize it to bytes, to be able to send it over network for replication.

Is any way to achieve it? Or i can make PR with my solution.

Most helpful comment

I need something like:

impl WriteBatch {
    /// Tries to serialize writebatch as raw bytes vector with null values
    /// Its useful for sending write batches over network.
    pub fn try_into_raw(&self) -> Result<Vec<u8>, Error> {
        let data;
        unsafe {
            let mut repsize1: size_t = 0;
            let rep: *const i8 = ffi::rocksdb_writebatch_data(self.inner, &mut repsize1);
            let cs = slice_from_raw_parts(rep as *const u8, repsize1);
            data = cs.as_ref().ok_or_else(|| {
                Error::new("writeBatch reference is Null".to_string())
            })?.to_vec();
        }
        Ok(data)
    }

    /// creates writeBatch from raw input
    pub fn from_raw(data: &[u8]) -> Self {
        let u8slice = unsafe {
            &*(data as *const _ as *const [i8])
        };
        WriteBatch {
            inner: unsafe { ffi::rocksdb_writebatch_create_from(u8slice.as_ptr(), data.len() as size_t) }
        }
    }
}

With raw value of writeBatch, i would be able to send it over network to the secondary database replica.

All 4 comments

Sorry for the late reply. Honestly, I'm less involved with anything related to RocksDB now, so I don't fully understand your situation. Could you please elaborate on what api needs to be added?

I need something like:

impl WriteBatch {
    /// Tries to serialize writebatch as raw bytes vector with null values
    /// Its useful for sending write batches over network.
    pub fn try_into_raw(&self) -> Result<Vec<u8>, Error> {
        let data;
        unsafe {
            let mut repsize1: size_t = 0;
            let rep: *const i8 = ffi::rocksdb_writebatch_data(self.inner, &mut repsize1);
            let cs = slice_from_raw_parts(rep as *const u8, repsize1);
            data = cs.as_ref().ok_or_else(|| {
                Error::new("writeBatch reference is Null".to_string())
            })?.to_vec();
        }
        Ok(data)
    }

    /// creates writeBatch from raw input
    pub fn from_raw(data: &[u8]) -> Self {
        let u8slice = unsafe {
            &*(data as *const _ as *const [i8])
        };
        WriteBatch {
            inner: unsafe { ffi::rocksdb_writebatch_create_from(u8slice.as_ptr(), data.len() as size_t) }
        }
    }
}

With raw value of writeBatch, i would be able to send it over network to the secondary database replica.

I guess it makes sense to add such api. Would you like to submit a pull request?

Sure.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

iSynaptic picture iSynaptic  ·  12Comments

valeriansaliou picture valeriansaliou  ·  4Comments

iSynaptic picture iSynaptic  ·  5Comments

zach-schoenberger picture zach-schoenberger  ·  5Comments

spacejam picture spacejam  ·  3Comments