knex 0.16 doesn't support knexfiles written in TypeScript

Created on 14 Jan 2019  ·  85Comments  ·  Source: knex/knex

Environment

Knex version: 0.16.3
Database + version: docker postgres:10.4
OS: Windows 10 Pro & Docker node:8.14.0

Bug

  1. knex migrate:make --knexfile knexfile.ts
  2. Error message: Unexpected token import

Works normally on 0.15.x

Most helpful comment

I was facing the same issue with Knex version 0.16.3.

import * as Knex from 'knex';
^^^^^^

SyntaxError: Unexpected token import
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:617:28)
    at Object.Module._extensions..js (module.js:664:10)

This solution is not encouraged but I simply fixed it by simply adding ts-node/register at the beginning of knexfile.ts file.

require('ts-node/register');
//

All 85 comments

0.16 now has TypeScript types included in the repo (see #2845 ). You probably need to remove the @types/knex from your project for this to work correctly.

I uninstalled @types/knex but I still got the error.

/usr/src/app/src/db/knexfile.ts:1
(function (exports, require, module, __filename, __dirname) { import { database } from '../config'
                                                              ^^^^^^

SyntaxError: Unexpected token import
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:617:28)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at initKnex (/usr/src/app/node_modules/knex/bin/cli.js:62:25)
    at Command.commander.command.description.option.action (/usr/src/app/node_modules/knex/bin/cli.js:172:24)
    at Command.listener (/usr/src/app/node_modules/commander/index.js:315:8)
    at emitTwo (events.js:126:13)
    at Command.emit (events.js:214:7)
    at Command.parseArgs (/usr/src/app/node_modules/commander/index.js:654:12)
    at Command.parse (/usr/src/app/node_modules/commander/index.js:474:21)

knex migrate:make --knexfile knexfile.ts isn't that trying to run typescript code with plain node? I have no idea how that could have worked with 0.15 either.

How does your knexfile look like? is there any import statements?

import * as knex from 'knex'
import * as path from 'path'
import { env } from './env'

const database = {
  client: 'postgresql',
  connection: env.databaseUrl,
  migrations: {
    directory: path.resolve('../db/migrations'),
    tableName: 'knex_migrations',
  },
  seeds: {
    directory:  path.resolve('../db/seeds'),
  },
} as knex.Config

export = database

works 100% on 0.15

I have no idea why that would work with 0.15. As the error Unexpected token import tells you, the import * as knex from 'knex' is not understood by Node.js (not in the configuration you're using, at least).

You'll either need to compile your knexfile to JavaScript, use a Node.js version that understands the import syntax (see https://nodejs.org/api/esm.html ), or revert the file to use the old require() syntax.

I suppose there was a bug in knex 0.15 so that it never read that knexfile. Doesnt look like bug in 0.16 that shouldnt work.

so that it never read that knexfile

It does a 100% read the knexfile, I use it in more than a single project (from the top of my head I can think of 4, in production) with different migration folders and connection data.

On version 0.15 it was compiling or maybe using ts-node automatically to run the file.

I just tested it... you are right, both versions are trying to register ts-node stuff. Sounds like that feature is not tested. Not sure if even documented.

nope... now really tested it with necessary stuff installed on node 8

Mikaels-MacBook-Pro-2:test mikaelle$ npm install [email protected] ts-node-register
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ [email protected]
+ [email protected]
updated 2 packages and audited 1037 packages in 2.135s
found 0 vulnerabilities

Mikaels-MacBook-Pro-2:test mikaelle$ echo "import * as knex from 'knex'; export = {client: 'pg'}" > knexfile.ts
Mikaels-MacBook-Pro-2:test mikaelle$ node_modules/.bin/knex migrate:make --knexfile knexfile.ts -x ts foo
Failed to load external module ts-node/register
Failed to load external module typescript-node/register
Failed to load external module typescript-register
Failed to load external module typescript-require
Failed to load external module @babel/register
/Users/mikaelle/Projects/Vincit/yessql/dodo-objection/test/knexfile.ts:1
(function (exports, require, module, __filename, __dirname) { import * as knex from 'knex'; export = {client: 'pg'}
                                                              ^^^^^^

SyntaxError: Unexpected token import

cannot reproduce, but it is totally possible feature to be implemented, since knex already uses ts-node to compile actual typescript migration files and seeds

I've setup a full example in case you want to investigate why it works on 0.15 and not on 0.16

Branch: knex16 has 0.16.3 installed
https://github.com/brunolm/knex16bug/tree/knex16

Branch: knex15 has 0.15.2 installed
https://github.com/brunolm/knex16bug/tree/knex15

Switch to the branch you want to check and run on the root folder:

docker-compose up

If it works you should see a migration called test in /api/src/db/migrations

after installing also some more packages (ts-node ts-node-dev typescript) it seems that both, 0.15 and 0.16 works just fine here... but indeed your docker setup with node 0.16 is failing and I have no idea why

Mikaels-MacBook-Pro-2:test mikaelle$ rm -fr node_modules/*
Mikaels-MacBook-Pro-2:test mikaelle$ npm install knex typescript ts-node
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ [email protected]
+ [email protected]
+ [email protected]
added 295 packages from 184 contributors and audited 1213 packages in 6.59s
found 0 vulnerabilities

Mikaels-MacBook-Pro-2:test mikaelle$ node_modules/.bin/knex migrate:make --knexfile knexfile.ts test
Requiring external module ts-node/register
Created Migration: /xxx/migrations/20190119003358_test.js
Mikaels-MacBook-Pro-2:test mikaelle$ cat knexfile.ts 
import * as knex from 'knex'; export = {client: 'pg'}
Mikaels-MacBook-Pro-2:test mikaelle$ 

I modified your docker-compose.yml to make pretty much the same and it fails like this:

version: '3'

services:
  api:
    image: node:8.14.0
    command: bash -c 'rm -fr api/node_modules; npm i knex ts-node typescript; node_modules/.bin/knex migrate:make --knexfile ./src/db/knexfile.ts test'
    working_dir: /usr/src/app
    volumes:
      - ./api:/usr/src/app
api_1  | npm WARN [email protected] No repository field.
api_1  | 
api_1  | + [email protected]
api_1  | + [email protected]
api_1  | + [email protected]
api_1  | updated 3 packages and audited 1176 packages in 9.804s
api_1  | found 0 vulnerabilities
api_1  | 
api_1  | /usr/src/app/src/db/knexfile.ts:1
api_1  | (function (exports, require, module, __filename, __dirname) { import { database } from '../config'
api_1  |                                                               ^^^^^^
api_1  | 

If I moved knexfile in your container to /usr/src/app it did try to compile correctly... I cannot reproduce that behavior locally though. Here everything works even if knexfile.ts is inside subdirectories...

I think I found maybe a bug on 0.16, or just something weird.

On 0.15 launch is called with configPath
https://github.com/tgriesser/knex/blob/0.15.2/bin/cli.js#L260

But on 0.16 it's called with

    knexfile: argv.knexfile,
    knexpath: argv.knexpath,

https://github.com/tgriesser/knex/blob/0.16.3/bin/cli.js#L303-L304

LiftOff calls var env = this.buildEnvironment(opts); which calls findConfig({ passing configPath (which is no longer provided). Internally it uses configPath and makes no reference to knexfile or knexpath.


I have typescript and ts-node installed in the project and it works when I have v0.15 installed (like the example I provided on the Git repository.

I did some debugging and found this, but I still didn't figure out why it works on 0.15

[email protected]

Logging the result of this line

var config = require(env.configPath);

https://github.com/tgriesser/knex/blob/0.15.2/bin/cli.js#L55

I get this

api_1    |   require(/usr/src/app/src/db/knexfile.ts)------------ { client: 'postgresql',
api_1    |   connection: 'postgres://user:password@db/api-db',
api_1    |   migrations:
api_1    |    { directory: '/usr/src/app/src/db/migrations1337',
api_1    |      tableName: 'knex_migrations' },
api_1    |   seeds: { directory: '/usr/src/app/src/db/seeds' } }
api_1    | this.config { extension: 'js',
api_1    |   loadExtensions:
api_1    |    [ '.co',
api_1    |      '.coffee',
api_1    |      '.eg',
api_1    |      '.iced',
api_1    |      '.js',
api_1    |      '.litcoffee',
api_1    |      '.ls',
api_1    |      '.ts' ],
api_1    |   tableName: 'knex_migrations',
api_1    |   schemaName: null,
api_1    |   directory: '/usr/src/app/src/db/migrations1337',
api_1    |   disableTransactions: false }

The migration directory is saying directory: '/usr/src/app/src/db/migrations1337',, which is exactly what I have on knexfile.ts

And it's directly requiring the file .ts

require('/usr/src/app/src/db/knexfile.ts')

[email protected]

The require is to the exact same file, but it fails

// /usr/src/app/src/db/knexfile.ts
env.configuration = require(resolvedKnexfilePath);

[email protected]

I uninstalled typescript and ts-node. Now I get this:

api_1    | Failed to load external module ts-node/register
api_1    | Failed to load external module typescript-node/register
api_1    | Failed to load external module typescript-register
api_1    | Failed to load external module typescript-require
api_1    | Failed to load external module @babel/register
cli.on('requireFail', function(name) {
  console.log(chalk.red('Failed to load external module'), chalk.magenta(name));
});

https://github.com/tgriesser/knex/blob/0.15.2/bin/cli.js#L254

I don't see this message at all on 16, it just breaks.

@elhigu I figured it out, I made a PR, can you check it?
https://github.com/tgriesser/knex/pull/3005

Reopening since this probably is a bug somewhere... even that I don't know how to reproduce it on osx.

FYI, I ran brunolm's example and knex 0.15.2 works while knex 0.16.3 does not work

@scorbin yes, I was able to reproduce that too with docker, but not locally on osx. That is why I reopened this.

Same problem here (TypeScript migrations + 0.16.3).
When Migrator#latest is called, its starts with migrationListResolver#listAllAndCompleted which in turn calls listAll(config.migrationSource) where migrationSource having loadExtensions = [".co", ".coffee", ".eg", ".iced", ".js", ".litcoffee", ".ls", ".ts"], even if I { …, loadExtensions: ['.js'], … } in the migrator configuration.
As a consequence, it picks both the .ts files and .js and tells the migration to go on with the .ts files (since they're told to be not applied yet, obviously), which... breaks when #getMigration is finally called to evaluate the script (since it calls #require with the migration script which is TypeScript).

Hope this helps...

With the following little change, things seem back to normal:

function listAllAndCompleted(config, trxOrKnex) {
  return _bluebird.default.all([listAll(config.migrationSource, config.loadExtensions), listCompleted(config.tableName, config.schemaName, trxOrKnex)]);
}

I didn't validate it is 100% compliant with the migrator's design though - one more knowledgeable than me would have to validate! :)

I was facing the same issue with Knex version 0.16.3.

import * as Knex from 'knex';
^^^^^^

SyntaxError: Unexpected token import
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:617:28)
    at Object.Module._extensions..js (module.js:664:10)

This solution is not encouraged but I simply fixed it by simply adding ts-node/register at the beginning of knexfile.ts file.

require('ts-node/register');
//

Should be fixed in 0.16.4-next2

Just tried 0.16.4-next2. Sad to see the same error

@brunolm can you confirm that 0.16.4-next2 still fails this?

@pvoisin Does it work for you with 0.16.4-next2?

@kibertoad I'm getting the exact same error on 0.16.4-next2.

OK, so the proper fix for this would be to fully remove dependency on knexfile from migration:make (since it not really needs it).

I don't want to beat the dead horse, but @kibertoad said it _should_ be fixed. Where was the evidence of this fix? Have regression tests been written? Saying it should be fixed, closing the issue, and having the same issue present is not the way releases should be pushed.

@juliancoleman this has been difficult to reproduce. But indeed there should have been regression test added in the fix (didnt check if there actually was and test just didnt fail on ci env).

Difficult to reproduce is actually fair as I've looked back and have seen that no one, including myself, has added steps to reproduce.

  1. Add [email protected] to dependencies
  2. Create a knexfile.ts
  3. (optionally using ObjectionJS) instantiate knex config on Model.knex
  4. Add the following script to package.json

    • "knex": "knex --knexfile ./path/to/Knexfile.ts"

  5. Make migrations

    • yarn knex migrate:latest

  6. Run into SyntaxError

I've verified these steps on a project I just started the other day, which brought me to this issue

    "knex": "^0.16.4-next2",
git clone [email protected]:brunolm/knex16bug.git
cd knex16bug
git checkout knex16
npm run api-i
docker-compose up
api_1    | > knex migrate:make --knexfile ./src/db/knexfile.ts "test"
api_1    |
api_1    | /usr/src/app/src/db/knexfile.ts:1
api_1    | (function (exports, require, module, __filename, __dirname) { import { database } from '../config'
api_1    |                                                               ^^^^^^
api_1    |
api_1    | SyntaxError: Unexpected token import
api_1    |     at createScript (vm.js:80:10)
api_1    |     at Object.runInThisContext (vm.js:139:10)
api_1    |     at Module._compile (module.js:617:28)
api_1    |     at Object.Module._extensions..js (module.js:664:10)
api_1    |     at Module.load (module.js:566:32)
api_1    |     at tryModuleLoad (module.js:506:12)
api_1    |     at Function.Module._load (module.js:498:3)
api_1    |     at Module.require (module.js:597:17)
api_1    |     at require (internal/module.js:11:18)
api_1    |     at initKnex (/usr/src/app/node_modules/knex/bin/cli.js:62:25)
api_1    |     at Command.commander.command.description.option.action (/usr/src/app/node_modules/knex/bin/cli.js:172:24)
api_1    |     at Command.listener (/usr/src/app/node_modules/commander/index.js:315:8)
api_1    |     at emitTwo (events.js:126:13)
api_1    |     at Command.emit (events.js:214:7)
api_1    |     at Command.parseArgs (/usr/src/app/node_modules/commander/index.js:654:12)
api_1    |     at Command.parse (/usr/src/app/node_modules/commander/index.js:474:21)
api_1    | npm ERR! code ELIFECYCLE
api_1    | npm ERR! errno 1
api_1    | npm ERR! [email protected] migrate-make: `knex migrate:make --knexfile ./src/db/knexfile.ts "test"`
api_1    | npm ERR! Exit status 1
api_1    | npm ERR!
api_1    | npm ERR! Failed at the [email protected] migrate-make script.
api_1    | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
api_1    |
api_1    | npm ERR! A complete log of this run can be found in:
api_1    | npm ERR!     /root/.npm/_logs/2019-03-18T21_39_10_408Z-debug.log

My PR fixes it, I just don't have the knowledge to fully test it etc, if someone could look into it...

https://github.com/tgriesser/knex/pull/3005

I don't want to beat the dead horse, but @kibertoad said it should be fixed. Where was the evidence of this fix? Have regression tests been written? Saying it should be fixed, closing the issue, and having the same issue present is not the way releases should be pushed.

I apologize, it was a late hour here in Amsterdam and my choice of words was really not the best. It should have been "Please let me know if 0.16.4-next2 fixes the problem", as, TBH, I would have been very surprised if it did. There was a unit test written that verifies exactly what this fix actually fixes, and now I'm fairly sure that I'm not misunderstanding anything and that problem is exactly what I thought it is, and the only way how it could have worked previously is through bypassing knexfile entirely. This is a system behaviour that I intend to improve in the nearby future, but it requires a bit of prerequisites to be delivered before. I apologize for the inconvenience. Just give me some time. In the meantime, using ts-node looks to be a proper solution - one might argue that it's actually less hacky than not depending on knexfile, since it allows you to use knexfile as a single source of truth regarding the location of migration files.

@kibertoad

and the only way how it could have worked previously is through bypassing knexfile entirely.

This is not true, see my previous comments. Everything works on 0.15.

@brunolm But can you explain on a technical level how Node.js may not choke while parsing Typescript file even hypothetically? I'll check the PR out, but I suspect that it achieves same thing ultimately (although by different means) - bypasses the knexfile.

@kibertoad I am presently using ts-node. I'm not sure why this happens on a technical level

If I had the time to check the source, I would, but I've been using Knex for years and I just trust it works at this point. I reverted back to 0.15 and everything works... Except this, of course

@brunolm But can you explain on a technical level how Node.js may not choke while parsing Typescript file even hypothetically? I'll check the PR out, but I suspect that it achieves same thing ultimately (although by different means) - bypasses the knexfile.

if you do this you can see it fully working on knex 0.15, not ignoring the knexfile.ts file

git clone [email protected]:brunolm/knex16bug.git
cd knex16bug
git checkout knex15
npm run api-i
docker-compose up

however if you run the exact same thing on knex 0.16 you will see the error. If you clone this I suggest cloning in a different folder so you can make sure you're on the right version

git clone [email protected]:brunolm/knex16bug.git
cd knex16bug
git checkout knex16
npm run api-i
docker-compose up

I verified those examples of @brunolm earlier I was never able to create reproduction case for this that would fail on macos, but I was able to reproduce docker setup.

So the problem currently would be to be able to create test case for knex CI, which actually also fails on 0.16 and works with 0.15. Since CI is running some old ubuntu, maybe that test case works (meaning fails) there just fine.

Quick update on my end. Not sure if this is useful or not, but my issue was that I was using multiple tsconfig.json files in my project, and knex was using the default. When I overrode this functionality by directly launching knex using ts-node, I was able to get around this error.

My updated package.json:

// ...
"knex": "ts-node --project tsconfig.server.json $(npm bin)/knex --knexfile ./src/server/knexfile.ts",
// ...

How I run migrations now:

npm run knex -- migrate:latest

Hopefully this helps someone.

@FindAPattern would you mind posting your tsconfig? Here's mine. I only use one, since I have no need to build as I serve with ts-node

{
  "compilerOptions": {
    "noEmit": true,
    "rootDir": "src",
    "target": "es6",
    "module": "commonjs",
    "moduleResolution": "node",
    "declaration": true,
    "inlineSourceMap": true,
    "resolveJsonModule": true,
  },
  "include": [
    "src/**/*.ts",
  ],
  "exclude": [
    "src/**/*.spec.ts"
  ]
}

Hi! Is there any progress on this?

@brunolm How are you solve it? Any workaround?

@algil I didn't need to update or start a new project yet, so I'm still using 0.15 for the most part.

But I did make a PR that solves this issue, You can fork knex and apply these changes to it: https://github.com/tgriesser/knex/pull/3005

It's not so simple, unfortunately. I'll try to come up with a fix that doesn't break any use-cases this week.

Maybe it will help someone. Fixed when change in tsconfig.json "module": "es2015" to "module": "commonjs"
Command in package.json => "migrate": "ts-node -P ./tsconfig.json node_modules/.bin/knex migrate:latest --knexfile knexfile.ts".

Maybe it will help someone. Fixed when change in tsconfig.json "module": "es2015" to "module": "commonjs"
Command in package.json => "migrate": "ts-node -P ./tsconfig.json node_modules/.bin/knex migrate:latest --knexfile knexfile.ts".

Not optimal solution but works fine.

@Areshetcov @Meldiron I've been using commonjs module as you can see above. I think what really needs to happen is a deeper support, not necessarily causing a ton of user configuration. Unless that truly is not an option

Considering how this issue is related to a release that's now behind, is this currently an issue with 0.17.6?

@juliancoleman Using ts-node is still the best solution, as knex still tries to open knexfile.js and explodes if it is not in Javascript format. There were improvement to handle TS extensions more gracefully in more cases (e. g. migration generation or resolving knexfile in default location), however. Which particular issue are you having?

In any case, thank you for reminding me about this issue. I'll try to take another swing at it on the weekend, hopefully I can at least understand why it worked in the past :D

When I made that PR I noticed that it is supposed to work not only with TypeScript, but also with babel and some other stuff, there's a package that handles all that - if I remember correctly.

My PR might have some useful hints https://github.com/tgriesser/knex/pull/3005/files

The issue I have is when I run npm run knex migrate:latest, I get an unexpected token '{'. I use ts-node only for this API. I don't run it through any bundler and I don't compile the TS. It's probably not a good thing to _do_ that, but that's what I'm doing for now. I am, however, aware that Typescript cannot import third-party typescript files. They must be built first. I've experienced this elsewhere, not just with Knex

I am, however, aware that Typescript cannot import third-party typescript files.

Do you mean that ts-node cannot load typescript files with require()? Migrator uses require() to load those migration files dynamically...

Just chiming in here if somebody else has the same issue I had: I have [email protected] and was trying to run essentially* knex --knexfile src/knexfile.ts which failed as it apparently tries to read the knexfile as JS. Using knex --cwd src works as intended.

*) actual commandline was node -r dotenv/config node_modules/knex/bin/cli [...] but that probably doesn't matter.

@ilkka Could you clarify what "works as intended" in this context mean? Considering that you are running node and not ts-node, it's not Knex failing to parse knexfile.ts, it's Node. Unless you mean that filename resolution is working wrong.
Does knex --cwd src actually manage to open the knexfile in question and does correctly react to configuration changes in it?

Ah, I'm sorry for being unclear. When I use the --cwd parameter when running knex from my npm scripts, it reports "Requiring external module ts-node/register", reads my knexfile, and works. The behavior is the same if I execute it as npx knex, outside an npm script.

@ilkka Interesting. I don't think knex is ever attempting to load ts-node on its own, so it must be something else from the stack that you are using. If you could identify what it exactly is, would be possible to figure out what we did to break compatibility with that.

I believe the message about requiring the module is printed by cli.js here, caused by a Liftoff event (from before 'require' was renamed to something else in Liftoff). Liftoff in turn is using a package called "rechoir" to register loaders or whatever, and rechoir's README has this note about how it will "automatically load and register transpilers like coffee-script". So... maybe just having ts-node in my path in this specific instance is enough to make this work? It's quite a convoluted system.

Anyway, now I realize that we could have just changed node to ts-node in our npm script and it would also have worked.

Thanks for looking into it. I'll see if I can make liftoff behave in these cases :D
but yeah, using ts-node directly should work in all cases.

Had same error. Turned out I have copied wrong tsconfig.json. Now it's working. Relevant configs:

package.json (workspace)

  "scripts": {
    "migrate:make": "knex --cwd src migrate:make -x ts"
  },
  "dependencies": {
    "knex": "0.19.0",
    "pg": "7.11.0"
  }

package.json (root):

        "ts-node-dev": "1.0.0-pre.40",

(ts-node version is: 8.3.0)

tsconfig.json (workspace):

{
    "extends": "../../tsconfig.node.json",
    "compilerOptions": {
        "rootDir": "./src",
        "outDir": "./build",
    }
}

tsconfig.node.json (root):

{
    "compilerOptions": {
        "target": "es2015",
        "moduleResolution": "node",
        "esModuleInterop": true,
        "strict": true,
        "alwaysStrict": true,
        "declaration": true,
    }
}

src/knexfile.ts:

import { Config } from 'knex'

export = {
    client: 'pg',
    connection: {
      database: 'db',
      user: 'user',
    },
} as Config

Command to run:

yarn migrate:make my_migration_name

Problem still persists on [email protected]

git clone [email protected]:brunolm/knex16bug.git
cd knex16bug
git checkout knex19
npm run api-i
docker-compose up
api_1    | /usr/src/app/src/db/knexfile.ts:1
api_1    | (function (exports, require, module, __filename, __dirname) { import { database } from '../config'
api_1    |                                                               ^^^^^^ 

@brunolm why are you so ignorant?

diff --git a/api/package.json b/api/package.json
index c0f8bff..0906f51 100644
--- a/api/package.json
+++ b/api/package.json
@@ -8,7 +8,7 @@
     "dev": "ts-node-dev --respawn --poll --no-notify src/index.ts",
     "\n# Database": "",
     "migrate": "knex migrate:latest --knexfile ./src/db/knexfile.ts",
-    "migrate-make": "knex migrate:make --knexfile ./src/db/knexfile.ts",
+    "migrate-make": "knex migrate:make --cwd src/db",
     "seed": "knex seed:run --knexfile ./src/db/knexfile.ts",
     "seed-make": "knex seed:make --knexfile ./src/db/knexfile.ts",
     "\n# Testing": "",
api_1    | > [email protected] migrate-make /usr/src/app
api_1    | > knex migrate:make --cwd src/db "test"
api_1    | 
api_1    | Requiring external module ts-node/register
api_1    | Working directory changed to /usr/src/app/src/db
api_1    | Created Migration: /usr/src/app/src/db/migrations/20190723173751_test.ts

Specifying --knexfile still doesn't work.

Using --cwd src/db instead works.

Documentation is not clear that you have to use cwd over knexfile.

I didn't check the function signature, when I made that PR I changed to pass the correct parameters, it is possible that it is still passing the wrong parameters.

Hi guys

I also had this error
I think, the problem is that option --knexfile is wrongly set a directory for knexfile.ts
So I set explicit directions with--cwd for the directory with knexfile.ts

It works for me: "knex migrate:make --cwd src"
(It is equal this: "cd src knex migrate:make")

I tried knex migrate:make --knexfile knexfile.ts -x ts new_script
I get following error

import knex from 'knex';
^^^^
SyntaxError: Unexpected identifier

when adding cwd

internal/process/main_thread_only.js:29
binding.chdir(directory);

my knexfile.ts looks like below

import knex from 'knex';
export const database: knex.Config = {
  client: 'postgresql',
  connection: process.env.databaseURL,
  migrations: {
    extension: 'ts',
    directory: './ds/migrations',
  },
  seeds: {
    directory: './ds/seed',
  },
};

any clue?

For fellow googlers, I had the "unexpected token export" error when trying to migrate up/down with the latest knex 0.19 in full typescript mode.

I turned out I had both a tsconfig.json and a .babelrc in the working directory, I suspect one of those interfered with the transpilation.

When I moved the migration folder and the knexfile.ts in a clean directory, it worked again ✅ .

Hi guys. So as @mikl said, the problem is, that you are trying to run typescript code on node interpreter. I ran into this problem today when I was trying to create a new migration.

I've solved this issue by running knex through ts-node (https://npmjs.com/package/ts-node).

To make this work just add this script inside your package.json file :)

"migrate:make": "ts-node ./node_modules/.bin/knex migrate:make --knexfile <PATH_TO_YOUR_KNEXFILE>"

Replicate this to migrate:latest, seed:run etc... :)
Then just run your new script!

Solution

Instead of --knexfile use --cwd

-    "migrate-make": "knex migrate:make --knexfile ./src/db/knexfile.ts",
+    "migrate-make": "knex migrate:make --cwd src/db",

Solution

Instead of --knexfile use --cwd

-    "migrate-make": "knex migrate:make --knexfile ./src/db/knexfile.ts",
+    "migrate-make": "knex migrate:make --cwd src/db",

Thanks !! It helps me a lot.

Why is this closed ? It's totally broke when working with knexfile.ts, tried all I could from this thread ... (latest version + typescript 3.6.4)

Why is this closed ? It's totally broke when working with knexfile.ts, tried all I could from this thread ... (latest version + typescript 3.6.4)

Open new issue and provide reproduction code (for example a link to example project where your problem occur). This was closed because @brunolm found his solution to his problem.

It's still broken with --cwd:

knexfile.ts:6
export default {
^^^^^^

SyntaxError: Unexpected token export

So, I won't' raise a new issue as I don't know how it's even supposed to be working. A minimal full typescript example in the docs would be ... godsend
Objectin.js typescript example didn't bother to use knex in typescript, so I think they had the same problem ...

Well ...What I wanted to have :

  • all my source code in ts
  • knexfile.ts in typescript
  • migrations in typescript (to have createTable autocomplete api while coding)
  • the hability to make all this work in CLI (knex migrate) AND using api inside my app.ts (auto migrate at server startup)

I think this last point was the more a pain ...

What I had to do to make it all work :

  • in tsconfig set allowJs=true + declaration=false (without this knex would try to execute .d.ts files ...)
  • knexfile.ts : it seems mandatory to use export = instead of the typescripts exports ... So yes I have a ts file but in my app.ts I can't import it and had to require it ..
  • app.ts : loading only .js compiled files worked : knex.migrate.latest({ loadExtensions: ['.js'], });
  • package.json : as well, only js compiled files worked (inside my /dist) : "db:migrate": "knex migrate:latest --cwd ./dist/config --env development --knexfile knexfile.js"

And I forgot some, I'm sure of it ..

Now I'm not quite satisfied as it feels totall hacky

@ctiaffay-conserto you could try this example (replace knexfile with cwd)
https://github.com/brunolm/knex16bug/tree/knex16

Solution

Instead of --knexfile use --cwd

-    "migrate-make": "knex migrate:make --knexfile ./src/db/knexfile.ts",
+    "migrate-make": "knex migrate:make --cwd src/db",

It works but... why? It doesn't look right solution as --knexfile supposed to work.

@ShGKme these are the changes needed to make it work: https://github.com/knex/knex/pull/3005

But no one wants to address it, since cwd works I'm happy enough to just accept that.

I spent the afternoon looking into this because I was encountering a closely-related initialization problem. I believe that @brunolm 's assessment is correct: the Liftoff#launch(..) method is being invoked w/ the wrong parameters. You can see the details in #3005

Roughly speaking, Liftoff appears to behave as follows:

// If the configPath was specified, then use it.  Otherwise, try to infer it.
const configPath = opts.configPath || inferConfigPath(opts);

function inferConfigPath(opts) {
  // configName represents the expected name of the config file, minus its extension.
  // For example:  "knexfile"
  // If no configName was specified, then attempt to infer it from the name instead.
  // In our case, since `name === "knex"`, this will result in `configName = "knexfile"`
  const configName = opts.configName || (opts.name + "file");

  return findPathFor(configName, {
    withPossibleExtensions: [".js", ".ts"],
    inDirectory: opts.cwd,
  });
}

Since there is currently a bug in the way that Liftoff#launch(..) is being invoked, the value of configPath is _always_ being inferred. Consequently, Liftoff will not kick off the appropriate preload scripts. (Specifically: ts-node/register will fail to load)

@briandamaged Could you provide a fix pr that wouldn't break existing tests?

@kibertoad + @brunolm : I'll try to put something together either later today or tomorrow. I'm still making sure I understand the big picture first. From what I can tell, it looks like the Knex CLI might be duplicating some of the functionality that is already provided by the Liftoff library.

@briandamaged Could be. As long as cli tests pass so that we know we are not introducing breaking changes for users, feel free to refactor as deeply as necessary.

@brunolm @ShGKme @mmiszy @ilkka This should work better in 0.20.9 thanks to amazing work of @briandamaged. Please give it a try and let me know if the changes were helpful for you.

@kibertoad All works, thanks a lot 👍

@kibertoad thank you so much! I can confirm it's working 100% now!

api_1    | > knex migrate:make --knexfile ./src/db/knexfile.ts "test"
api_1    |
api_1    | Requiring external module ts-node/register
api_1    | Working directory changed to /usr/src/app/src/db
api_1    | Created Migration: /usr/src/db/migrations/20200210194631_test.ts

@brunolm Haha, @briandamaged is the real hero. Glad it works fine now!

I still get this error with NodeJS 14.0.0, the command knex migrate:make test, and the following file:

// knexfile.ts

export const config = {

  development: {
    client: "postgres",
    connection: {
      filename: "./dev.sqlite3"
    }
  },

  staging: {
    client: "postgresql",
    connection: {
      database: "my_db",
      user: "username",
      password: "password"
    },
    pool: {
      min: 2,
      max: 10
    },
    migrations: {
      tableName: "knex_migrations"
    }
  },

  production: {
    client: "postgresql",
    connection: {
      database: "my_db",
      user: "username",
      password: "password"
    },
    pool: {
      min: 2,
      max: 10
    },
    migrations: {
      tableName: "knex_migrations"
    }
  }
};

I get this error:

Failed to load external module ts-node/register
Failed to load external module typescript-node/register
Failed to load external module typescript-register
Failed to load external module typescript-require
Failed to load external module @babel/register
(node:6468) UnhandledPromiseRejectionWarning: C:\Users\Flori\WebstormProjects\OragesAuthentication-Backend\knexfile.ts:3
export const config = {
^^^^^^

SyntaxError: Unexpected token 'export'
    at wrapSafe (internal/modules/cjs/loader.js:1101:16)
    at Module._compile (internal/modules/cjs/loader.js:1149:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1205:10)
    at Module.load (internal/modules/cjs/loader.js:1034:32)
    at Function.Module._load (internal/modules/cjs/loader.js:923:14)
    at Module.require (internal/modules/cjs/loader.js:1074:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at openKnexfile (C:\Users\Flori\WebstormProjects\OragesAuthentication-Backend\node_modules\knex\bin\cli.js:26:16)

EDIT: This was fixed by adding ts-node as a dependency. Sorry for the inconvenience.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

saurabhghewari picture saurabhghewari  ·  3Comments

mishitpatel picture mishitpatel  ·  3Comments

nklhrstv picture nklhrstv  ·  3Comments

ghost picture ghost  ·  3Comments

lanceschi picture lanceschi  ·  3Comments