Rust-rocksdb: The prefix iterator behaves weirdly

Created on 8 Nov 2018 · 6Comments · Source: rust-rocksdb/rust-rocksdb

Hi,
If you give the prefix_iterator a longer prefix than what you declared in Options (set_prefix_extractor).
You don't get an error and it doesn't ignore the rest of the prefix, it behaves like a prefix together with a From.
like this:

set_prefix_extractor(len)
myp = &[1u8; len+1];
prefix(myp[..len]) + From(myp)

And an actual working example:

let prefix_extractor = rocksdb::SliceTransform::create_fixed_prefix(1);
let mut opts = Options::default();
opts.create_if_missing(true);
opts.set_prefix_extractor(prefix_extractor);
let db = DB::open(&opts, &tempdir).unwrap();


db.put(&[0,0,0,0], &[0,0,0,0]).unwrap();
db.put(&[0,0,0,1], &[0,0,0,1]).unwrap();
db.put(&[0,1,0,1], &[0,1,0,1]).unwrap();
db.put(&[0,1,1,1], &[0,1,1,1]).unwrap();
db.put(&[0,1,2,1], &[0,1,2,1]).unwrap();
db.put(&[2,0,0,0], &[2,0,0,0]).unwrap();
db.put(&[2,2,2,2], &[2,2,2,2]).unwrap();

let me = db.prefix_iterator(&[0,1,0]);
for (key, value) in me {
    println!("Saw {:?} {:?}", key, value);
}

This will print these results:

Saw [0, 1, 0, 1] [0, 1, 0, 1]
Saw [0, 1, 1, 1] [0, 1, 1, 1]
Saw [0, 1, 2, 1] [0, 1, 2, 1]

So the first byte behaves like a real prefix(giving an exact match), but the rest are neither a prefix nor ignored, they're being used as a "From" (you can see this ignores everything that starts with [0,0] but includes bigger bytes like [0,2])

Is this the wanted behavior or is this a bug?

bug

Source

elichai

All 6 comments

I suspect that the behavior you are seeing is because of how the core library is implemented. I'm not sure if this is by design or "undefined behavior".

iSynaptic on 19 Nov 2018

👍1

@elichai Okay to close or do you think this is a issue with this language binding?

iSynaptic on 29 Nov 2018

hmm I actually didn't check if it exists in the C++ code, So idk.
Maybe i'll check soon when I have the time (not a C++ dev)

elichai on 9 Dec 2018

@iSynaptic Not sure if this deserve a separate issue,
The put functions of write_batch doesn't contain any try! and yet they return a result.
Is this intended? why?
https://github.com/rust-rocksdb/rust-rocksdb/blob/master/src/db.rs#L1147

elichai on 9 Dec 2018

@elichai I think it's intended behavior, as no chance of errors is implementation detail. In the future, we'll be able to change this internal implementation without changing user-facing API

vitvakatu on 10 Dec 2018

👍1

@elichai I now agree that prefix_iterator behaves weirdly, but as designed. In #254, I added a test that validates the behavior and includes a comment explaining the behavior (quoted below). Based on the behavior I _think_ you want, you should probably use a regular iterator and provide your own ReadOptions. This is enabled in #253.

Explanation: db.prefix_iterator sets the underlying options to seek to the first key that matches the entire prefix. From there, the iterator will continue to read pairs as long as the prefix extracted from key matches the prefix extracted from prefix.

@vitvakatu @aleksuss I think we need to take a look at all the functions that construct iterators and simplify them as much as possible - they are a bit confusing. I have some ideas and I can put a PR together.

iSynaptic on 3 Feb 2019

Was this page helpful?

0 / 5 - 0 ratings