Jdbi: Support R2DBC

Hi Vaughn, thanks for the pointer. Reactive is very new and exciting but my understanding is that the current implementation of JDBC, and in fact the database server software itself, makes it a poor candidate for reactive concepts at the lower layers. The database server has worker processes that take queries and produce results, and each process will work on its query until completion. There's no ability to asynchronously submit 10 queries and await their results.

This means that as you implement JDBC reactive, you eventually find that you must implement an old style thread pool that holds your connections / handles, services queries, and then emits results into the reactive framework. This means that you can have some of the benefits of reactive, in particular you can write reactive code, but it seriously limits the scalability benefits that reactive attempts to achieve.

So we'd definitely welcome API integrations, but unless the state of the JDBC world has moved on since I last investigated, it'll be a somewhat limited wrapper that just hides the fact that the code underneath is still old school threaded code.

I appreciate the offer to get up to speed on your platform, but our library prides itself on being almost entirely standalone and not part of any platform or framework. So while we might look for inspiration and are always willing to collaborate towards shared goals, we're a fiercely independent project :)

Hey, Steven! Thanks for your comments. Currently we solve the reactive-over-JDBC by serializing requests within actors in our vlingo-symbio implementations. You can create as many JDBCObjectStoreActor as makes sense for your connection limitations, but the actors themselves access JDBC synchronously. It's more that the client of the ObjectStore implementations running on actors don't block while queries are being fulfilled.

I don't have a thorough understanding of R2DBC, but it is touted as being a full JDBC replacement and works completely asynchronously as a database driver. There are currently implementations for Postgres and MS SQL Server only, but probably will grow in time. I was wondering what it would be like for Jdbi to implement over R2DBC. I understand if this doesn't make sense for you now, but the reference is here in case it does at some future time. No stress :)

Interesting, they actually do provide their own drivers. So maybe there is some benefit to be had here!

I'm curious to explore this further: benchmarks, proof of concept code, etc. There seem to be some limitations: their r2dbc driver for postgres advertises that e.g. BLOB and CLOB types don't work yet, and there's no mention at all of stored procedures or CALL.

But at least my focus personally right now is elsewhere so this may need some community input and attention to move forward, as I don't have time right now to open an exploration here, and I don't imagine other core members are looking to jump on it either.

Looking forward to hearing more, even if just votes from other members of the community that this is a needed feature.

Great to see this ticket.

I wanted to quickly drop two things to clarify the state of R2DBC:

• It's still a young initiative and things like BLOB/CLOB and Stored Procedure support are to be implemented before we go GA.
• Drivers are fully reactive and non-blocking meaning the whole I/O part is non-blocking. There's no offloading of blocking work into thread pools.

In any case, R2DBC aims for an SPI to be picked up by client implementations such as Spring Data R2DBC, R2DBC Client (a tiny wrapper on top of R2DBC SPI that lend some ideas from jdbi) and more to come.

Just wanted to chime in and mention that my original prototype/demonstration client layer, r2dbc-client was designed based on Jdbi's APIs and ethos. If you do choose to investigate further, I think you'll find it familiar 😄.

I have seen folks expose Rx stuff out of JDBI ( @hgschmie for example ). I am not sure what "support" for R2DBC would mean.

Are you talking about implementing R2DBC on top of JDBI, having JDBi consume R2DBC Connections, swapping JDBI to focus on Rx interfaces, or... something else?

I think it would be "support R2DBC drivers to be used by JDBI, exposing the JDBI interfaces" ?

R2DBC is an async replacement for JDBC.

@brianm R2DBC isn't JDBC on ThreadPools. R2DBC means two things:

1. R2DBC drivers are implementations using a non-blocking transport layer returning org.reactivestreams.Publisher types for each I/O-bound operation. They do not use JDBC drivers underneath but implement wire protocols from scratch.
2. R2DBC is a standardized (vendor-independent) SPI allowing to build client libraries on top. Driver implementations can be interchanged like it is done today for JDBC.

Are you talking about implementing R2DBC on top of JDBI, having JDBi consume R2DBC Connections

Yes, this is what it basically means. Having an R2DBC-specific JDBI module returning Project Reactor/RxJava2/Zerodep types.

I'm pretty sure I get what R2DBC is, and I think it is a Very Good Thing.

I am trying to figure out what "JDBI support" for it is!

Maybe write a code sample of what you expect to do via JDBI's API?

How about supporting the following for starters:

Jdbi jdbi = Jdbi.create("r2dbc:h2:mem:///test"); // (H2 in-memory database), obtain ConnectionFactory via ConnectionFactories.get(String url)

// Reactor style:
Flux<User> users = jdbi.withHandle(handle -> {

    return handle.execute("CREATE TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")

        .then(handle.execute("INSERT INTO user(id, name) VALUES (?0, ?1)", 0, "Alice"))

        .then(handle.createUpdate("INSERT INTO user(id, name) VALUES (?0, ?1)")
            .bind(0, 1)
            .bind(1, "Bob")
            .execute())
         .then(handle.createQuery("SELECT * FROM user ORDER BY name")
              .mapToBean(User.class).many());
});

// RxJava style
Flowable<User> users = jdbi.withHandle(handle -> { … });

The interface-based support could look like:

public interface UserDao {
    @SqlUpdate("CREATE TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")
    Completable createTableRxJava();

    @SqlUpdate("CREATE TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")
    Mono<Void> createTableProjectReactor();

    @SqlQuery("SELECT * FROM user ORDER BY name")
    @RegisterBeanMapper(User.class)
    Flowable<User> listUsersRxJava();

    @SqlQuery("SELECT * FROM user ORDER BY name")
    @RegisterBeanMapper(User.class)
    Flux<User> listUsersProjectReactor();
}

I'm not opinionated about RxJava vs. Project Reactor, therefore I tried to list variants using both APIs.

Could somebody clarify what exactly is being proposed here?

Is R2DBC a layer on top of JDBC, or its own thing? Because Jdbi is coupled to JDBC in the extreme.

R2DBC is not a layer on top of JDBC. R2DBC is a non-blocking API to access SQL databases and it is its own thing (R2DBC drivers are typically written from scratch implementing vendor-specific wire-protocols using a non-blocking I/O layer). If you will, R2DBC is the reactive specification of how to integrate with SQL databases.

The proposal here would be having an API that looks and works like Jdbi but uses underneath R2DBC drivers. Instead of returning scalar objects, the API would return Reactive Streams types.

Thanks for the clarification. I expect the first step here would be to prototype a Jdbi fork or branch that serves minimally as a PoC showing how to implement the above examples on top of R2DBC. From there we would have to determine a path towards integration in parallel to existing JDBC support, or establish that it would be better as a hard fork. As Matthew points out above we are tightly coupled to JDBC currently and changing that is likely to both be a lot of hard work as well as potentially require breaking changes.

I am excited to have this on the project's long-term radar but I don't expect it to move quickly without significant community involvement.

@stevenschlansker We've created initially r2dbc-client that was inspired by JDBI. What do you think about a pull request/PoC that introduces a r2dbc module (jdbi-r2dbc) as a contribution of R2DBC Client back into JDBI along the lines of:

ConnectionFactory cf = …;

Rdbi rdbi = new Rdbi(cf);

rdbi.inTransaction(handle ->
    handle.execute("INSERT INTO test VALUES ($1)", 100))

    .thenMany(rdbi.inTransaction(handle ->
        handle.select("SELECT value FROM test")
            .mapResult(result -> result.map((row, rowMetadata) -> row.get("value", Integer.class)))))

    .subscribe(System.out::println);

Very cool. Yes, if the work can be integrated in a way that is reasonably seamless, we'd happily incubate it in a module!

It would be nice if somehow the Rdbi api "came from" Jdbi -- in that when you do a jdbi.installPlugin(...).registerRowMapper(...) then any Rdbi instances that "come from" that Jdbi also get their configuration. It'd also be nice to be able to do things like @SqlUpdate CompletableFuture<ResultType> doSomeWork(...); or whatever the appropriate reactive type is.