Knex: how to configure to use the standard ES6 Promise?

Curious: you want to replace a faster library with a slower built-in? On the client it wouldn't matter but in node speed is a factor. What's your reasoning for this?

@johanneslumpe It is factor in some applications, with knex I highly doubt that used Promise library has any significant effect to performance. It has been discussed that we should write all Promise code to use only A+ APIs so that bluebird wouldn't be needed.

After that it should be easy to override, which Promise library is used.

I agree it feels pointless in a library such as knex. Given that knex's codebase already imports its own promise.js file internally, it would technically still be very easy to implement, provided the API is the same _(which I'm not sure it is?)_. In this file one could default to global promises, else require configured library, or something like that.

it's all about giving users a choice; for Node V6+ users, give a choice of managing 3rd party dependencies as minimum as possible.

Curious: you want to replace a faster library with a slower built-in?

what you're saying might be true in the past, but how about now or 6 months later? (I did some search and can only find out some 2015 benchmarks, if you have more recent comparison, please post some links)
I believe the whole point of making Promise the ES6 standard is for people to easily use Promise, not having to rely on a 3rd party library, and Node or V8's core team can't be blind of the performance difference for ever, as long as the two projects' open source licenses are compatible, they can even borrow some code; or just give them some time I believe the builtin Promise can be faster and better.

see Amazon's AWS-SDK: it also default to use whatever Promise is globally available; while give user a choice to configure the favorite Promise library as well

http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-making-requests.html#Support_for_Promises

it's all about choices

On second thought, it won't be as easy as changing a single file. Knex currently relies a lot on Bluebird utility functions such as .timeout, .tap, .reduce, .map and so on, which im assuming and expecting not to exist in ES6 Promises.

I am interested in supporting ES6 promises. Ideally we'd require a second argument to the Knex constructor that takes a Promise constructor. Back compatibility would be achieved like so:

const Promise = require('bluebird');
const knex = Knex(myKnexConfig, Promise);

Perhaps we could conditionally alias map, filter etc. based on their presence on supplied Promise.prototype?

I think this should be pretty low on the priority list, and it'd require quite a bit of internal changes, not to mention the decrease in performance (though admittedly haven't seen benchmarks in awhile) and the fact that folks might be relying on the fact that the returned promises is a bluebird (for conditional catch, etc).

I'd be more inclined to wait until async/await lands in a stable node to look at addressing this.

for people want the minimum 3rd party dependencies, the native Promise can be better,

like @google-cloud and many other libraries, you can make it default to use native Promise and accept a promise parameter for whom want to use 3rd party libraries

https://googlecloudplatform.github.io/google-cloud-node/#/docs/google-cloud/

var gcloud = require('google-cloud')({
  promise: require('bluebird')
});

@tgriesser I know this is an old issue, but we've now got async/await in both stable and Carbon.

Given that it is preferred for async/await to work with native promises (per the spec, async functions must return a native promise) is this any higher priority?

If native promise support is something that Knex would like to have happen, but it's not currently on the radar, would a PR be welcome?

Thanks for your time.

@malexdev async functions do return native promises it doesn't have much to do with knex currently. It doesn't mean that await needs native promises to work properly. Could you clarify in which case this is a problem / benefit (except of dropping one dependency)?

That being said I'm not against of dropping bluebird, but it really does need quite a bit internal changes. We might need to implement some methods that are currently exposed from bluebird to knex APIs, unless old functionality is hidden under some configuration switch and by default native promises would be used. This way migrating wouldn't be impossible to people who has been relaying on fact that knex returns bluebird promises.

@elhigu The main benefit for me personally is that I'm using TypeScript with a TSLint rule enforcing native promises being used for await, so I have to wrap all Knex calls inside a Promise.resolve(). I realize this isn't anything to do with Knex specifically though, and is likely an issue unique to me.

Other than that in my opinion having less 3rd party dependencies is better, and as @c0b mentioned more options is never a bad thing.

I realize that it would be a lot of work, which is one reason I'm more than happy to spend time on this, if it's something that Knex is interested in moving towards.

Yeah, I landed here coming from a typescript issue - I'm using Knex as the SQL engine for my multi-storage datastore library, and while i'm ambivalent vis-a-vis native vs bluebird promises, I can't easily use typescript on knex for this reason. I treat knex thenables as following the native spec (I don't use any of the bluebird extensions), but typescript bugs me about returning a Bluebird when the method declaration is a Promise.

This is kind of two levels deep here, as we're dealing with both the promise implementation and the typings for knex (which are handled by different devs), but I'm basically stuck here - I'm technically breaking the type contract by returning a Bluebird when I've declared a Promise (I guess there's stuff in the Promise api that Bluebird doesn't support?) but I really don't feel like putting a bunch of return Promise.resolve(bluebirdthing) everywhere either.

I've spent enough time digging around in knex guts for the past year and working with promises in general that I'd be willing to take something up here and work on a PR to modularize out the Promise implementation if people want - would you be open to a PR? It'd end up being something like what @elhigu mentioned - re-implementing some of the utility functions to use whatever Promise constructor was passed in at instantiation so to avoid code-rewriting needs. Not sure about performance, of course, but that's something that can be benchmarked.

Having it all done up fancy with async / await would be cool too, and I'm not in a hurry to have this fixed (ultimately for my use case I just end up flagging things as any and dealing with those codebranches as if they were javascript instead of typescript).

@ericeslinger I don't see why real Promise implementation that is being used would cause problems with TypeScript, I've been mixing bluebird and native promises a year and half without any problems...

I just haven't been using any typings that would introduce types for bluebird, I just tell typescript typings that they are normal Promises and it doesn't see any difference (ofc. it will complain if I try to use bluebird's special methods).

Which typescript version and is there any example project to reproduce... e.g. github repo with npm start script.

This is kind of two levels deep here, as we're dealing with both the promise implementation and the typings for knex (which are handled by different devs), but I'm basically stuck here - I'm technically breaking the type contract by returning a Bluebird when I've declared a Promise (I guess there's stuff in the Promise api that Bluebird doesn't support?) but I really don't feel like putting a bunch of return Promise.resolve(bluebirdthing) everywhere either.

Are those knex typings from npm? Are bluebird typings from npm? Which packages? I'm having hard time to understand why that would happen. If there is separate Bluebird type it should be inherited from native promise and be ok to return from APIs that tells that they will return Promise. From the description it sounds that the problem is very broken typing implementations. And that is not relevant to this issue (typescript doesn't mind about real types, so it won't know what knex returns).

I'm getting my typings from the DefinitelyTyped repo via installing @types/knex. That definition pulls with it @types/bluebird, and all the knex methods are typed as returning Bluebird objects.

As a specific thing, I cannot do this:

function mungeData(v: any): DataItem {} 
function foo(): Promise<DataItem[]> {
  return knex('data').select()
  .then((rows) => rows.map(row => mungeData(row)))
} // error, cannot return Bluebird<DataItem[]> as Promise<DataItem[]>

using these typings. This is because knex.select().then() is typed to return a Bluebird in the DefinitelyTyped repo, and those chain together to make more Bluebirds, and saying something like return foo as Promise<any>() when foo is a Bluebird will fail (at least it fails in typescript 2.4), because Bluebirds aren't assignable to Promises (they lack [Symbol.toStringTag] according to this, so coercing one to the other would be in error, albeit a small error).

Instead I can change to

function bar(): Promise<DataItem[]> {
  return Promise.resolve<DataItem[]>(foo())
}

or do other tricks to wrap all calls to knex inside a native Promise.resolve() call. This will cause typescript to stop complaining downstream of my library functions, while still allowing me to use knex typings inside my library functions.

Prior to yesterday, I hadn't used @types/knex at all - I was just typing knex as an any. The code works fine either way at runtime (for my use case at least), it's just

@elhigu: The issue is not broken typing implementations.
TypeScript sets the type for async functions as Promise<[type]>, which is correct per JS spec.
Knex returns Bluebird<[type]>, which the typings accurately reflect.

I just haven't been using any typings that would introduce types for bluebird, I just tell typescript typings that they are normal Promises and it doesn't see any difference

This is lying to the compiler, as Knex functions do actually return Bluebirds. Not interested.
You're correct that Bluebirds are compatible with Promises, but part of the deal with TypeScript is that you actually return what you say you're returning.

When returning a Bluebird from a function that has been typed to return Promise, TypeScript complains because type Bluebird is not the same as type Promise.
There are various tricks we can do (such as what @ericeslinger mentioned about using any, or wrapping in Promise.resolve()) but at the end of the day tricks like that make us lose out on much of what TypeScript provides.

At the end of the day, the reality is there are at least two users now who are saying "Using native promises is important to us, and we're willing to put in the work to make the promise functionality more generic".

I realize you're just trying to help, but frankly instead of hearing "you could do it this way" I'd like to hear whether the promise changes proposed by myself / @ericeslinger / @c0b are acceptable so that I can either start on a PR or what.

@malexdev @ericeslinger Thanks for the more info! Looks like it actually is not possible inherit your own class from Promise so that might be the reason why returning Bluebirds from function that is typed to be Promise<> fails :(

@ericeslinger Anyways this is not a problem when you create asyncfunctions, since they automatically wraps results to native promises internally. Following complies without a problem, with typings from @types/bluebird and compiled to either ES2015 or ESNEXT.

import * as Bluebird from 'bluebird';

// declaring function async converts bluebird implicitly to native Promise
async function asyncReturningPromise(): Promise<string> {
    const blueBirdPromise = new Bluebird<string>((resolve, reject) => { 
        resolve('yay asyncReturningPromise');    
    });
    return blueBirdPromise;
}

// main func to run the code using async / await
Bluebird.resolve().then(async () => {
    console.log("await function returning promise (bluebird)", await asyncReturningPromise());

    const blueBird = new Bluebird((resolve, reject) => { resolve(); });
    const returnedFromAsync = asyncReturningPromise();

    console.log("Bluebird instanceof Promise:", blueBird instanceof Promise);
    console.log("async retval instanceof Promise:", returnedFromAsync instanceof Promise);
});

output:

await function returning promise (bluebird) yay asyncReturningPromise
Bluebird instanceof Promise: false
async retval instanceof Promise: true

So for now when you are using knex APIs for now you actually need to tell that you are returning Bluebird unless you are using async functions / methods, which wraps bluebird automatically to native Promises.

@malexdev

This is lying to the compiler, as Knex functions do actually return Bluebirds. Not interested.
You're correct that Bluebirds are compatible with Promises, but part of the deal with TypeScript is that you actually return what you say you're returning.

Actually deal with typescript is that it is enough that returned object implements the interface correctly, for example this is perfectly fine:

class FakePromise<T> implements Promise<T>  {
    [Symbol.toStringTag]: "Promise";
    then<TResult1, TResult2>(onfulfilled?: (value: T) => TResult1 | PromiseLike<TResult1> | null | undefined, onrejected?: (reason: any) => TResult2 | PromiseLike<TResult2> | null | undefined): Promise<TResult1 | TResult2> {
        return new Promise((resolve, reject) => { resolve('Im totally broken and fake!); });
    }
    catch<TResult>(onrejected?: (reason: any) => TResult | PromiseLike<TResult> | null | undefined): Promise<T | TResult> {
        throw new Error("Method not implemented.");
    }
}

// this works and  fake promise instance has nothing to do with native promise
function returningPromiseInterface(): Promise<string> {
    const fakePromise = new FakePromise<string>();
    return fakePromise;
}

// compiling this fails, because looks like Bluebird actually doesn't implement Promise interface correctly
function asyncReturningPromise(): Promise<string> {
    const blueBirdPromise = new Bluebird<string>((resolve, reject) => { 
        resolve('yay asyncReturningPromise');    
    });
    return blueBirdPromise;
}

It messes up instanceof, but typescript actually hasn't even promised that you returned native Promise instance, only interface.

When returning a Bluebird from a function that has been typed to return Promise, TypeScript complains because type Bluebird is not the same as type Promise.
There are various tricks we can do (such as what @ericeslinger mentioned about using any, or wrapping in Promise.resolve()) but at the end of the day tricks like that make us lose out on much of what TypeScript provides.

I would hate to see people having to do that kind of tricks / changing JS implementations just to satisfy bad typings.

At the end of the day, the reality is there are at least two users now who are saying "Using native promises is important to us, and we're willing to put in the work to make the promise functionality more generic".

I realize you're just trying to help, but frankly instead of hearing "you could do it this way" I'd like to hear whether the promise changes proposed by myself / @ericeslinger / @c0b are acceptable so that I can either start on a PR or what.

Thanks for understanding :) Changing knex to use native promises was already started and implemented to some point last year then it was changed back by @tgriesser so I would say that for now it is better not to start this change.

Also I still consider these typescript problems mentioned in this thread as a problems in typings declaration (why bluebird doesn't implement Promise correctly... I need to dig in more deep on that?), than problems in knex implementation.

That being said, I'm not opposed of getting rid of bluebird in some timespan just seeing two separate issues here.

:+1: for native Promises.

@elhigu:

Actually deal with typescript is that it is enough that returned object implements the interface correctly

Fair enough. I still stand by my opinion that less dependencies and more choice is better, but I now see what you meant about broken typings.

So still 👍 for native Promises (which I'm still willing to help with), but I see now my immediate issue can be resolved by fixing Bluebird typings. Thanks for the info.

So I've started to use TypeScript quite a bit and I both love it and now realize the issues here are a real pain point. As async/await gains more of a foothold in recent Node land the Bluebird utility fns (map, reduce, tap, bind, return) become less useful. I'd be cool with continuing to use Bluebird internally but officially "deprecating" the public api of knex query builder returning all of the additional utility methods.

With that merged, we could then upgrade the TypeScript definitions to drop the Bluebird typings (except for toBluebird), and change the Bluebird typings to Promise typings.

If anyone has bandwidth wants to tackle this, here's what I'm thinking for a plan of action:

• [ ] Add a deprecation warning for all of the Bluebird proxied methods (tap, map, reduce, bind, return).
• [ ] Add a .toBluebird() method which will be a migration path for those who want to continue using Bluebird (they can do so with a simple find/replace of all of the calls of the above methods and just add it before those are called)
• [ ] Once this is merged/new release cut (0.15) we can update the Typescript definitions
• [ ] Eventually we can drop these methods entirely. This simpler API paves the way for eventually using native Promises & async/await when it makes sense.

Let me know if this makes sense and if anyone is interested in taking a shot at this.

Would definitely be interested to help with this.

Does this mean Knex would start maintaining its own TypeScript definitions? Would allow us to do some cool stuff with generics that autogenerated typings will never support.

I started this fork as a first attempt to add support for native Promises:
https://github.com/tgriesser/knex/pull/2523/files

Like this comment from 2016:

Curious: you want to replace a faster library with a slower built-in

Amazing how much can change in 2 years.

When returning a Bluebird from a function that has been typed to return Promise, TypeScript complains because type Bluebird is not the same as type Promise.

@malexdev Actually typescript uses structural typing (flow uses nominal typing and would work the way you describe) so as long as your type fulfills the Promise interface it is compatible with Promise whether it explicitly extends/implements it or not.

How's this progressing? I reckon a good first step would be to factor out Bluebird specific method calls within knex (i.e. don't actually remove it yet). Removing bluebird and providing an option for a custom Promise constructor would follow (and give folks using Bluebird methods an upgrade path).

I don't starting work on the first step if there are no objections. Existing work appears to have died down.

@qubyte I don't think there is an active effort to do the change, incremental changes were made here and there, but that's about it.

Ok. In my next chunk of free time I'll make some small-as-possible changes to factor out each method.

@tgriesser Any opinions when we should go forward with this one (if ever)? To me next April would sound like a reasonable time for it when Node 6 LTS reaches end of line.

Interesting information in 2018:

promises-native-async-await has better performance than promises-bluebird in node 10.
Reference: https://github.com/petkaantonov/bluebird/tree/master/benchmark

So performance is not a reason to keep bluebird anymore. We should go for async/await.

promises-native-async-await has better performance

that's also what I strongly believed in 2016 that the native way would improve much faster, just because it's the core of Nodejs community, has more number of people caring about it, more than any 3rd party libraries

While, the ticket was filed asking for choices, there are so many competing Promise implementations, it's just not good to bet on bluebird forever

Is there any update on this?

@cfanoulis Same still stands. When April comes, we can drop support for Node 6 and start removing bluebird.

any 2019 update? /cc to some core contributors or maintainers or anyone @here who cares about it : @johanneslumpe @tgriesser @wubzz @elhigu from https://github.com/tgriesser/knex/graphs/contributors?type=c&from=2018-01-01&to=2019-12-31

On the other hand, JavaScript community is such a dynamic, vibrant and sometimes cruel world, every 3 or 2 years (or even faster) there are replacements for something we were familiar before, think about Grunt, Gulp => Webpack, the tools, libraries, frameworks are fully competing at every level, So, to older libraries if you're stopping bringing in innovations, or slowing down supporting new standards (think about ES2019 async/await iterators...) you will eventually be replaced

I just did some simple research, it looks like at DB ORM level there are also many alternatives, TypeORM might be a good one ... (I stop here not to say more...)
https://bestofjs.org/tags/db
https://bestofjs.org/projects/typeorm

@c0b no need to cc. I get mails from all the comments anyways. @kibertoad just said all that had to be said in his last comment... This issue also has nothing to do with knex supporting async/await ES2019 features better and knex is not an ORM so I'm not sure what that comment was really about.

If you need a good ORM, I can recommend objection.js. It is implemented on top of knex too, here is pretty nice thread about it https://github.com/Vincit/objection.js/issues/1069

At some point this knex will be replaced, but not with any ORM. It could be replaced by some other query builder, which has cleaner code base and more consistent API. Like for example knex 1.0 maybe ;)

Also if knex is replaced I would be totally fine with that, less work for me :D

There's WIP on that I believe: https://github.com/tgriesser/knex-next

Just wanted to also mention that not using native promises results in https://github.com/nodejs/node/issues/22360 when using async_hooks which causes the current context to be lost.

Trust me, we don't need additional reasons to move, we want to do it as bad as you all do :). However, we still need to release a couple more fixes to Node 6 branch, and then (finally), we'll drop it and start gradually phasing bluebird out.

After #3227 is merged, we can finally get started!

I know you mentioned before a you could use help in this migration, if thats still the direction you want to go, could we help in any way?

I'm thinking: make a Project, add a couple tasks, and see if anyone (maybe I have time) could get assigned and set some dates?

@chaffeqa Will create some more granular tasks soon, have #3250 up for a first round of easy changes. Mainly we need to replace bluebird.tap, bluebird.method and bluebird.try usages with something native-based. If you have some time already, you could try branching off #3250 and taking a look at any remaining 'bluebird' requires (I would recommend starting with non-dialect-specific ones so that you could quickly validate functionality still working by running test:sqlite without any Docker setup).

@qubyte If you would like to contribute, now is the time!

@kibertoad am I safe to use async/await now?

You mean in knex codebase? Sure. In your own one you always were :-D

https://github.com/tgriesser/knex/issues/3253
https://github.com/tgriesser/knex/issues/3254
https://github.com/tgriesser/knex/issues/3255
https://github.com/tgriesser/knex/issues/3256
https://github.com/tgriesser/knex/issues/3257
https://github.com/tgriesser/knex/issues/3268
https://github.com/tgriesser/knex/issues/3286
are up for grabs.

Sorry for being MIA this past week, things are ramping up for our company so I'm having to focus there.

Wanted to close the loop on some of the discussion we had on one of the bigger blocks to the upgrade: replacing the Disposer usage.

Its a pretty deep wole when you start going down it, so will take some good engineering to provide a good copy / abstraction. I worry the perf overhead of something may be pretty large (lots more objects created as the promise chain grows).

I actually started on a few POC's, and I think this is the most straightforward of them:

class DisposablePromise extends Promise {

  disposerFunc = null;
  originalResource = null;

  then(onFulfilled, onRejected) {
    const $onFulfilled = this.wrap(onFulfilled);
    return super.then($onFulfilled, onRejected).copyContext(this);
  }

  copyContext(promise) {
    this.disposerFunc = promise.disposerFunc;
    this.originalResource = promise.originalResource;
    return this;
  }

  disposer(disposerFunc) {
    this.disposerFunc = disposerFunc
  }

  isDisposable() {
    return !!this.disposerFunc
  }

  wrap(onFulfilled: any) {
    const $onFulfilled = (result: any) => {
      if (this.disposerFunc && !this.originalResource) {
        this.originalResource = result
      }
      if (result instanceof Promise) {
        return onFulfilled(result);
      } else {
        const res = onFulfilled(result)
        if (this.disposerFunc) {
          this.disposerFunc(this.originalResource)
        }
        return res
      }
    };

    return $onFulfilled;
  }
}

And another:

      var DisposablePromise = function DisposablePromise() {
          var self = DisposablePromise.convert(Promise.resolve());
          return self;
      };
      DisposablePromise.convert = function convert(promise, props) {
          promise.__proto__ = DisposablePromise.prototype;
          return props ? Object.assign(promise, props) : promise;
      };
      DisposablePromise.prototype = Object.create(Promise.prototype);
      DisposablePromise.prototype.constructor = DisposablePromise;
      DisposablePromise.prototype.then = function then(resolve, reject) {
          var returnVal = Promise.prototype.then.call(this, resolve, reject);
          return DisposablePromise.convert(returnVal);
      };
      DisposablePromise.prototype.catch = function _catch(err) {
          var returnVal = Promise.prototype.catch.call(this, err);
          return DisposablePromise.convert(returnVal);
      };
      DisposablePromise.prototype.finally = function finall(obj) {
          var returnVal = Promise.prototype.finally.call(this, obj);
          return DisposablePromise.convert(returnVal);
      };
      DisposablePromise.prototype.disposer = function disposer(disposerFunc) {
        var returnVal = Promise.prototype.finally.call(this, obj);
        return DisposablePromise.convert(returnVal);
      };

But haven't had time to prove them out.

I think it may be worthwhile to actually explore other options (keep bluebird but convert it to use native promises internally?) due to the fact that this feature must be in the repository (unless you can think of better approaches... async iterators? Would love to hear any thoughts by the bluebird team even on abstracting that functionality, though my gut says its pretty tied to the bluebird implementation hooks.

I'd say if we can figure this part out, the rest of those tasks are pretty straight forward.

@chaffeqa Np, appreciate you still finding the time to get back to this!
I highly doubt bluebird people will be open for suggestions to seriously reengineer their implementation, they have repeatedly reiterated the point that at this point they are interested in stability above everything, and they do recommend people to actually use native promises unless one really needs advanced features provided by Bluebird.
Considering that Node 8 seems to be the most popular version of Node.js right now (based on official Node.js download statistics), I'm afraid we can't move to async iterator-based approach just yet.
What downsides do you see to Knex implementing DisposablePromise internally? Since it extends native Promise, I assume it brings none of the Bluebird drawbacks with it, and nothing in userspace needs to know about it?

@ericeslinger FWIW, TS typings shouldn't be a problem in master anymore, we are typing our promises as the native ones now to discourage users from relying on Bluebird features. This may cause issues down the road when native Promises implement something that Bluebird promises don't, so we still want to replace promises used as much as possible. Any contributions along these lines would be greatly appreciated :)

Nuts I figured as much 😞
I agree doing something like the DisposablePromise is probably the way to go in this case, especially since the item that is really needed is still in proposal.

The downside is that it's going to be very important to engineer something like DisposablePromise in a judicious way... and frankly I dont even know if my implementation works 😆 (I have so much trouble with thinking async for some reason ha).

If there is anyone else on this thread that would like to take a stab at this issue <3 u longtimes!

@chaffeqa How complicated is the Bluebird implementation? Maybe we can simply extract it and add on top of native promise?

@chaffeqa Worst case scenario - we can remove all other Bluebird usages and keep this one due to its complexity if we deem it too risky to touch. Not ideal, but eventually using is going to happen.

unfortunately pretty complicated... the implementation piggybacks on the fact that bluebird controls the promise lifecycle. I think the best approach is to see what it's trying to do (which is pretty close to the link on using above) and create as simple and performant a shim as possible for it.

The problem is that the pipeline must be a Bluebird style promise, which if I understand correctly, does not adhere to the performance of the native promise (and you therefore lose all the tracing + native async functionality).

I'd much rather do something that under the hood uses native promises for the async parts, but provides the ability to bind context and implement needed usage like a disposer.

FYI another thing in my mind is: there is actually minimal use of usage and .disposer in knex, so maybe the approach works better to move that to a higher level?

Worth an experiment :)

oooo also an option I found based on https://github.com/petkaantonov/bluebird/issues/1593

Either way, I think a good step forward was what you started on a previous branch, where we isolate all the Promise usage that actually is a BluebirdPromise, that way we can start playing around with dropping in replacements like DisposablePromise or BluebirdNativePromise.

@chaffeqa You mean the Bluebird.setScheduler(fn => Promise.resolve().then(fn)) part?
Overall conversion is proceeding really smoothly! If we could keep Disposers in Bluebird while making them use native promises under the hood, that might actually be a good solution.

OK, so these ones still need attention:
https://github.com/tgriesser/knex/issues/3257
https://github.com/tgriesser/knex/issues/3286
https://github.com/tgriesser/knex/issues/3256

Help would be most appreciated!

Just wanted to also mention that not using native promises results in nodejs/node#22360 when using async_hooks which causes the current context to be lost.

Workaround is to use https://github.com/TimBeyer/cls-bluebird patch.

Just for information, LTS for Node v8 ends this year.

@Bessonov Context? How bumping min node to 10 affects this issue? Note that we dropped node 6 support already.

I'm not familliar with knex code base, but maybe there are some features, which can help you to get bluebird away. For example, node 10 has a support for Promise.finally.

But anywa, I'm happy to see the progress on this topic :+1:

About disposer pattern - could we just add optional callback for things, which returning disposable promise?
(Just as with transactions)

getDisposableConnection(config, cb) {
    const connection = await getConnection(config)

   // user want autodisposable connection
    if (cb) 
      Promise.resolve(cb(connection)).then(() => connection.dispose())
   // user will dispose by himself
   return connection
}

Which level of promise library independence we need?
1) all use native promises
2) internals native promises, user can set own promise lib for interface
3) user can set promise lib for internals and interface

What is the current state of this issue. Generally knex work now with async await but typescript will report warning that we awaiting method that is not native promise.

So to answer the original issue question. Current workaround is to simply await and add something like // tslint:disable-next-line: await-promise

@maximelkin I vote for the option 1. In long term I hope every promise library would be obsolete.

id second that, at this point we are beyond promise polyfills even for the majority of browsers

@Bessonov currently on knex depends libraries (and maybe projects), which requires exactly bluebird

we should give some fallback solution for them

@Bessonov currently on knex depends libraries (and maybe projects), which requires exactly bluebird we should give some fallback solution for them

It doesn't matter if knex's users are dependent on bluebird. Knex can still use native promises and they will interoperate just fine with bluebird promises. We should absolutely not give any fallbacks.

So this issue started with request for feature with choosing promise implementation.
Out of nowhere, it mutated in to removing bluebird for no reason, and breaking all dependents. Without any warning, changelog, option for fallback and major release.

But I suppose that all 1.5 typescript users are happy now.

So this issue started with request for feature with choosing promise implementation.
Out of nowhere, it mutated in to removing bluebird for no reason, and breaking all dependents. Without any warning, changelog, option for fallback and major release.

At least earlier with knex 0.x versions has been considered as major releases with potentially breaking changes, so only updating to 0.20.x should have been considered safe upgrade (semver is really loose when version number < 1).

Removing bluebird has been on the table for a long time, it is not only about this issue.

removing bluebird for no reason

Removing bluebird has not been for no reason. You can still externally use bluebird with knex promises. One big reason to drop bluebird has been that async functions implicitly creates native promises, so in future keeping on using Bluebird would have required extra bluebird wrapping code to be added in knex API for no reason what so ever.

Without any warning, changelog,

Agreed. I hauled through latest changelogs... Sadly it really looks like we have failed to list breaking changes between versions. We need to be more careful when writing changelogs to really point out the changes, which breaks the old APIs. For example many of the typings changes actually will break old TS code.

There was the same issue on ioredis project https://github.com/luin/ioredis/commit/da60b8b. They wanted to support native promises - and guys made a really good solution - they added an option to support any custom promise library and they use native promise by default. Why not? Setting a custom promise library is fast and doesn't require patching all application code.

keeping on using Bluebird would have required extra bluebird wrapping code to be added in knex API for no reason what so ever.

Yup. But why not wrap module calls in bluebird (or any other promise library) if it was explicitly specified? That's one simple wrapper, zero overhead, and it would allow users to use whatever promise library they want. If no one needs bluebird, no one would use this options, and you can safely deprecate it in time.

Also, I saw an opinion that

In long term I hope every promise library would be obsolete.

But IMHO there are two wrong assumptions:

• Bluebird is used because it's faster.
• Bluebird is used as a pollyfill.

I think that's not the case for really complex applications, which go beyond async-await one liners.
Bluebird has many features that are absolutely necessary for complex async flow - like timeouts, custom error handling, mapping with concurrency, cancellation, reducing and so on. All those features can be implemented in native promises, but that's a lot of useless boilerplate. In 2020, we are still using bluebird in Node 12 because we don't want all this boilerplate.

Why not? Setting a custom promise library is fast and doesn't require patching all application code.

Anything which uses async-await internally is going to coerce promises to native promises, so your options become either wrapping the output of each method in the custom promise or banning async-await in internal code. It's not as small an undertaking as it might seem on first inspection.

@qubyte

It's not as small an undertaking as it might seem on first inspection.

No, that's just as simple as I already told. You make a wrapper for exported external functions and that's all. About 10 lines of code. And write all internal code in any way you want.

@jehy : Feel free to submit a PR for those 10 lines of code if you see a straightforward way to implement them.

I'll also spend some time today trying to come up w/ a work-around.

For what it's worth, much of the API of bluebird is duplicated with the same or close API using native promises by these packages: https://github.com/sindresorhus/promise-fun

For what it's worth, much of the API of bluebird is duplicated with the same or close API using native promises by these packages

~50 packages instead of 1? Seriously?

Yes, though most of the time only a few are required (p-map for example). Your mileage may vary of course. It's offered only as one potential route to what you want.

@jehy : Here is something you can try as a temporary work-around within your application code:

const Bluebird = require('bluebird');


const prototypesNeedingDecoration = [
  require('knex/lib/query/builder').prototype,
  require('knex/lib/schema/builder').prototype,
  require('knex/lib/transaction').prototype,
  require('knex/lib/raw').prototype,
];

const corePromiseMethods = ["then", "catch", "finally"];


function decoratePromiseMethods(target) {
  for(const m of corePromiseMethods) {
    const original = target[m];

    target[m] = function(...args) {
      return Bluebird.resolve(original.apply(this, args))
    }
  }  
}

function hackBluebird() {
  for(const target of prototypesNeedingDecoration) {
    decoratePromiseMethods(target);
  }
}


hackBluebird();

This is not really an adequate solution to the overall problem. There are other temporary objects created within knex that would need to be decorated in a similar manner.

Also, disclaimer: the work-around ☝️ has had very little testing. So, you should re-run your application's tests to ensure that nothing has broken.

Just wanted to add my 2 cents here: i really appreciate all the work put into this migration, regardless of the negative feedback.

From our app's perspective, knex was the last library forcing us to require Bluebird, and conforming to full native promise support means that:

1. we no longer have sullied stack traces
2. we reduced our SSR weight by a decent amount
3. we improved perf since native async await is now more performant than bluebird (and growing more and more!)

its such a huge win to continue to run towards the es standard... and I know that isnt easy for library maintainers so i wanted to shout out to you guys and thank you for taking on such a burden!

for those suffering from the change: I'd love to help since we have benefitted, so please reach out if you need help debugging or migrating!

@chaffeqa Thank you for this feedback, it means a lot!

@jehy : Have you had a chance to try to work-around that was proposed? If so, did it resolve your immediate issues?