Jest: Please consider adding native support for ES modules

Jest has its own require implementation (https://github.com/facebook/jest/blob/master/packages/jest-runtime/src/index.js), so it would be way more involved than just supporting the syntax and default to looking at .mjs. I'm also against activating experimental flags.

It might be possible to automatically transpile import/export, but implementing the semantics is a huge undertaking, and probably blocked for a year because of support for node 6.

I agree with SimenB. We need a number of hooks from the node team to be able to make this work together with the vm module. However, I think we should support this in the meantime, but not by mirroring the full native implementation but rather by using babel and compiling it to require inside of babel-jest. I think for testing purposes this will work fine and we do not have to provide the same guarantees the node runtime needs to provide anyway.

So just adding babel-plugin-transform-es2015-modules-commonjs & babel-plugin-dynamic-import-node to babel-jest?

Yeah, that's what I was thinking.

Guys, what about integrating https://github.com/standard-things/esm ? It's fast and maintain a lot of edge cases.

@TrySound what would that look like concretely? Can you make a prototype?

We still have our own require-implementation (needed for mocks), so I don't think that'd help much.

And we need to work both with Node's rules and the browser's rules.

I'd be very happy to be corrected and have it work perfectly for us :D

@std/esm apparently should just work with jest: https://twitter.com/jdalton/status/930257653292400640

Could anyone give it a whirl and come back with a PR for the docs? 🙂

I guess users would like to have support everywhere, but I found it works only for dependencies of test files.

// test.js
require = require('@std/esm')(module, { esm: 'js', cjs: true });
const utils = require('./utils');

// utils.js
export { default as update } from './update';

It's better, but not ideal.

https://github.com/tarunlalwani/jest-overriding-require-function-stackoverflow

So just adding babel-plugin-transform-es2015-modules-commonjs & babel-plugin-dynamic-import-node to babel-jest?

I don't think this is a great solution, as it doesn't perform any "missing export" checks that are so valuable with ES modules. For example, in React repo I started running build more often just because Rollup finds these mistakes but Jest with es2015-modules-commonjs does not.

@std/esm apparently should just work with jest

It would be pretty great to invest some time into their interop. Pretty sure this hacky solution will break eventually: https://stackoverflow.com/questions/46433678/specify-code-to-run-before-any-jest-setup-happens. But if it's just a matter of exposing something on Jest side, would be cool to see it supported.

@SimenB, in my mind the immediate step would not be too complex. What is urgent is to allow people to work with .mjs module, even if babel is helping behind the test scene. Otherwise people might have to find different testing solution if they want to use .mjs.

1. Fix the testMatch property to allow .mjs file to be found
   (currently it doesn't work even the regex seems correct already, maybe there's a hard code somewhere rejecting .mjs extension)
2. Pass --experimental-modules flag when running node.js so that it will run natively
3. Jest should treat .mjs the exact same way as .js today (e.g. in jest require) -- the import statements are already allowed inside .js today, so only need to allow .mjs extension.

The final solution might be complex and takes time, but it is inevitable isn't it?

Hello, has someone been able to fix this error?

We are using .mjs with "node --experimental-modules" option. Any workaround?

We are using .mjs with "node --experimental-modules" option. Any workaround?

That's experimental and not fully fleshed out. There is lots of churn still with basic things, like how to import a builtin module, still up in the air. Projects like AVA have started allowing the use of @std/esm as their loader pipeline if used (bypassing Babel). Maybe jest could take a similar approach.

Supporting @std/esm is something we want to do, help to implement it is more than welcome!

@SimenB Can you chat sometime on hangouts?

Hi @SimenB 👋

An esm user has contributed esm + Jest demo and I figured it might be a good starting point for creating a more official cow path.

Update:

The esm + Jest demo has been updated with basic module name mapping support.

That's pretty cool! Thanks for sharing.

We'll have to figure out where we want the integration to be. How would it handle CSS (or other non-js assets) files? Should it just be a transform? What about the built-in babel transform? How should Jest behave when it comes to the incoming loaders, if it does affect anything?

It seems like there may be a benefit for a community contrib esm-enabled alternate jest runner (or an unofficial / experimental flag) so we can make progress on something like that. Would there be interest in that from jest team?

require is not implemented in a runner, it's in the runtime itself. Any contributions towards making it pluggable is very much welcome (ref #848).

I'm sure if you can get the example code @jdalton linked to to work without issues (or close to it), it should be simple enough to load in the esm loader behind a flag in jest itself. One thing I see as an issue is that it wants the real module global, not the fake one we create. Not sure if it means modules can leak between tests? I don't know what esm does with it under the hood. It also doesn't handle mocks, so mocks with import would still break

May just need some tweaking of how Jest taps into CJS. For example when mocking the module object, instead of using a plain object, it could use require("module") and then wrap/overwrite module.require to intercept requests and juggle as needed.

Update:

I'm now working on enabling Jest out of the box with esm (with hopefully nothing Jest has to do differently). I'll continue to experiment over the weekend but early signs look good.

@jdalton Any update with the esm compatibility?

Hi @JasonCust, wow my comment has gotten some attention!

I have made progress in identifying the work needed to enable Jest support in esm. In my experiments I got Jest loading and evaluating tests written in ESM. The work required on the esm loader side is to make the way we handle vm.Script use more generic. At the moment we hook into that primarily for REPL use which assumes a single module. We have to make that a bit more generic for Jest support to shake out. It doesn't look like Jest will have to change anything. The refactor of our vm.Script support is still on my TODO and will still be tackled before I release things like experimental WASM support. At the moment I've been squashing bugs that have popped up around improved APM and mocking support.

Thanks for the update @jdalton as I am excited to be able to use esm with Jest. So as to not bother you while you are working on those things, is there a task for esm that we can follow along with the progress?

You can keep subscribed to this thread of follow the repo. Jest support will be in the the v3.1.0 release so you could keep a look out for that version.

@jdalton Any news about supporting Jest in esm?

Hi @deepj!

It's still on my list and the items I can tackle before getting this done are shrinking. I've been improving supplemental Jest support of testing esm modules within Jest _(though CJS within Jest tests)_. There's still nothing critically blocking implementation on Jest's side _(so no work for them to do)_. Microsoft, my employer, is a heavy Jest user too so it's one of my day-job goals to improve this as well. I'm hoping to tackle it soon.

@jdalton Good to know. And thanks for your work. I appreciate it!

Really looking forward to this feature :speak_no_evil:

Been trying to make this feature work for the past two hours. There are so many partial solutions, half-documented answers, all different from each other... It used to be easy to test node.js packages.

So just adding babel-plugin-transform-es2015-modules-commonjs & babel-plugin-dynamic-import-node to babel-jest?

What happened to that idea? Any issue in implementing it?

Hi @jdalton. I was wondering if you could keep us up to date about status of this issue. Sorry if I bother you, but I am waiting for this for a while, and it would be better if we'll receive an update at least for last/next six months. Thank you :)

Hi @SurenAt93!

Thank you for your patience. I'm hoping to have a release out by the end of the month with Jest support. With that you'll specify esm as a Jest transform.

Cool. How is that transform different than Babel?

@kenotron

Using Jest's transform option is a way for me to side-load the esm loader. So while it is technically transforming source too, it's also wiring up the esm loader.

If the question is more what's the difference between Babel and esm. Babel is a collection of packages that transforms source, one of the targets could be ESM syntax. The esm loader is a zero dependency loader that simulates the runtime environment. So esm supports things like dynamic import(), passes relevant test262 specs (temporal dead zones, live bindings, upfront-errors, etc.), and supports loading a mix of CJS/ESM/WASM to taste by configuration.

@jdalton Thank you for you work and support!

@tomheller ;)

Been trying to make this feature work for the past two hours. There are so many partial solutions, half-documented answers, all different from each other... It used to be easy to test node.js packages.

I agree.

I created a simple Vue project, which also manifests the problem.
https://github.com/igasparetto/vue-jest-test
I wasn't able to get it to work.

I followed the instructions from the following pages:

• https://vuejs.org/v2/guide/unit-testing.html
• https://medium.com/@kevin_peters/unit-testing-vue-js-components-with-jest-86e14ef499da
• Please see this thread: https://github.com/vuejs/vue-cli/issues/1584

My machine (not sure it should matter):

• Windows 10 Pro 64-bit
• Node v8.9.4
• Vue: 3.2.3
• npm: 6.5.0

@kenotron

Using Jest's transform option is a way for me to side-load the esm loader. So while it is technically transforming source too, it's also wiring up the esm loader.

If the question is more what's the difference between Babel and esm. Babel is a collection of packages that transforms source, one of the targets could be ESM syntax. The esm loader is a zero dependency loader that simulates the runtime environment. So esm supports things like dynamic import(), passes relevant test262 specs (temporal dead zones, live bindings, upfront-errors, etc.), and supports loading a mix of CJS/ESM/WASM to taste by configuration.

@kenotron could you provide us an update please?

@igasparetto, it's @jdalton's work that should enable this. I hadn't tried that solution just yet.

Latest status there I think is: https://twitter.com/jdalton/status/1080627279124934661

Yep! Sorry it's taking longer than I wanted. Locally I now have all relevant test262 tests passing. In the course of getting those up the mocking related scenario tests slipped so I have to pick them back up before I can release. It isn't insurmountable, it just takes a bit of time. I'm presenting at Covalence Conf on Jan 16th and hope to have a release by then. In other news, npm tink early adopted esm for its ESM syntax support 🤝.

Jan 16th and hope to have a release by then

@jdalton I hope the presentation went well.

Have you got an update on this release please?

@igasparetto It's in https://github.com/standard-things/esm/issues/97#issuecomment-454985896 and in https://github.com/standard-things/esm/issues/706 ☺️

I'm leaving a small "data point" here to help you in your planning. I realize you are all donating your time and skills to address this issue. As a dev myself, I appreciate your contribution.

I just spent most of yesterday, a Saturday no less, to come up to speed on javascript modules on both the client and server side. ESM, CommonJS, AMD, what a confusing mess. I was never able to get a jest test to work with ESM loading a (node) module using a .mjs extension. I could load the same module successfully client side. I could create a node "client" that used the module with an import statement. I couldn't configure jest correctly to consume the same import statement, with or without babel, with or without esm. I eventually switched to ava, and just got it to work following a recipe on their website. Yes, I followed a recipe, I don't fully understand the machinery that made all the parts work. But at least I can now write ESM loading javascript modules and associated unit tests. I think. I'm extrapolating on the basis on one single success. They also have a recipe to connect ava to webstorm. But at least they are presenting recipes to mere mortals like me.

I also realize this message will read like I'm whining (which in part I am). I see all the work that's gone into jest. But this ESM support thing is a killer and a deal breaker for me. Thought you'd appreciate some feedback, but if not, ignore this or ask me to delete it and I will.

Any news in the light of the new modules support in the v12 release?

@dandv I've been doing some investigation into if Jest can support native ES modules in node 12. It will involve Jest using the vm.SourceTextModule API, which requires a few CLI flags to be passed to node:

--experimental-modules --es-module-specifier-resolution=node --experimental-vm-modules

The API is super low-level too.

TBD.

Discussion in https://github.com/nodejs/node/issues/27387

Update: been told to wait until the design of Node's module loader is locked down. In the mean time, standard-things/esm#706 might be the best bet.

jest is a very nice testing library. support for esm really all it needs to be complete!

Update: been told to wait until the design of Node's module loader is locked down. In the mean time, standard-things/esm#706 might be the best bet.

this does not work with jest, sadly.

@jdalton We are using lodash-es, which is the ES Module Exports build of lodash. Our project is an Angular v7 project, which uses Webpack v4 under the hood as its bundler. For those that don't know, lodash-es is tree shakeable with Webpack v4! ( ◔ ౪◔)⊃━☆ﾟ.*・.

Unfortunately this is causing us problems given that Jest has no ES Module support yet. We are hoping this feature will be part of Jest soon. Would anyone happen to know how to get lodash-es working with Jest?

Jest fails with the error message:

node_modules\lodash-es\lodash.js:10
    export { default as add } from './add.js';
    ^^^^^^

    SyntaxError: Unexpected token export

Our jest.config.js

We are also using jest-preset-angular npm package.

module.exports = {
  testMatch: ['**/+(*.)+(spec|test).+(ts|js)?(x)'],
  transform: {
    '^.+\\.(ts|js|html)$': 'ts-jest'
  },
  resolver: '@nrwl/builders/plugins/jest/resolver',
  moduleFileExtensions: ['ts', 'js', 'html'],
  collectCoverage: true,
  coverageReporters: ['html']
};

Unfortunately this is causing us problems given that Jest has no ES Module support yet. We are hoping this feature will be part of Jest soon. Would anyone happen to know how to get lodash-es working with Jest?

Tell Jest not to ignore lodash-es when transforming:

  "transformIgnorePatterns": [
    "[/\\\\]node_modules[/\\\\](?!lodash-es/).+\\.js$"
  ],

@azz do you have any idea when the design of Node's module loader will be locked down?
Because:

npx -n '--experimental-modules' jest func.spec.js

would be so darn cool and easy-life.

Since Node.js v12.0.0 was launched on 2019-04-23 everything changed again

I wrote the npm package jest-esnext back when things were broken with babel
It seems to work for Node.js 12+ unchanged, ie. "type: module" in package.json and extension .js
The strategy in jest-esnext is to do all transpiling in a dependency so that a stable babel environment is achieved. The consuming project does not need babel

https://www.npmjs.com/package/jest-esnext

https://github.com/haraldrudell/ECMAScript2049/tree/master/packages/jest-esnext

@haraldrudell per the instruction it is not clear how to use the package, it says: create file next to package.json named jest.config.js content module.exports = require('jest-esnext'), but what if I already have a config? How to integrate?

this is the file used

https://github.com/haraldrudell/ECMAScript2049/blob/master/packages/jest-esnext/config/jest.config.js

you might be able to replace the contents of _default with the one from your jest.config.js

Hi guys,
node 12.13.0 LTS's finally released... any news about that matter?

@mtsmachado8 I'm afraid some time is still needed by v8 team on the matter because I see ESM modules still being flagged as experimental... they failed on the roadmap for it maybe.

https://nodejs.org/api/esm.html

For unflagged ESM this PR is in place
https://github.com/nodejs/node/pull/29866

@azder @haraldrudell so basically in your solution you do Babel transformation for all JS files including ones in node_modules?

In my case I had to use your preset directly because I haven't found how to configure transformer like so:

const babelPreset7Esnext = require('babel-preset-7-esnext');
const babelJest = require('babel-jest');

module.exports = babelJest.createTransformer(
    babelPreset7Esnext(undefined, {decorators: {legacy: true}})
);

Node now has module support toggled by default

https://github.com/nodejs/node/pull/29866

ECMAScript Modules default support landed in 13.2.0

https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V13.md#13.2.0

We won't be working on this until the loaders are available. Without them, Node doesn't have the hooks Jest need to provide proper support. See https://medium.com/@nodejs/announcing-core-node-js-support-for-ecmascript-modules-c5d6dc29b663 (I'm unable to link directly to the section, but it's the "Work in Progress" one at the bottom)

For those who use native modules and want use jest.
While you working on this i suggest a quick fix for node v. 13.2.0:
babel-plugin-transform-default-import
Usage (in _package.json_):

{
  "babel": {
    "presets": [
      [
        "@babel/preset-env",
        {
          "targets": {
            "node": "current"
          }
        }
      ]
    ],
    "plugins": ["transform-default-import"]
  },
}

Libs need to install:
npm i --save-dev @babel/core @babel/preset-env babel-plugin-transform-default-import

Note: you maby not need to use babel-plugin-transform-default-import if you have no libs that have babel named export(or you don't use it)

@infodusha Awesome :) . Thank you for this.

In my project it works without the plugins:

npm i --save-dev @babel/preset-env

babel.config.js (this one had to be named so, not .babelrc):

module.exports = {
    presets: [
        [
            '@babel/preset-env',
            {
                targets: {
                    node: '13.2',
                },
                modules: 'commonjs',
            },
        ],
    ],

    plugins: [],
};

the trick is to separate files into directories and use appropriate package.json in them:

in test dir, I used

{
  "type": "commonjs"
}

while in src dir:

{
  "type": "module"
}

@azder But if you import package that provides commonjs named import in native modules you need to import it with default import and in babel you need to use named import. Likely, you have no packages like that, or packages that provide es6 export, but not all packages are ready to do it tomorrow

@infodusha Likely I have no what now? Did you try what I wrote and find issue with it or do you just assume I have problems using it?

Do provide an actual example since I'm not in the clear what you mean by "import package that provides commonjs named import in native modules".

So far I've had no problems writing all files with .js file extension where in the ./src directory are "type":"module" and the ./test directory "type":"commonjs" with this:

const imported = require('../src/module.js').default;
const {anotherOne} = require('../src/module.js');

That’s because Jest silently transpiles ES Modules to CommonJS code.

This is what I think should test ES Modules natively:

(async () => {
    const [
        { default: imported },
        { anotherOne },
    ] = await Promise.all([
        import('../src/some-module.js'),
        import('../src/another-module.js'),
    ]);

    // Rest of your test goes here.
})();

@azder those are workarounds.

Make a directory mymodule with package.son type=module and two files in it: first.js and second.js.
Then try to import { first } from "mymodule";

You need the exports field in place in the json to use Node ESM, and no package nowadays has it (i.e: lodash).

Your example may seem to work but it breaks as soon as either one of some-module.js or another-module.js try to import a named module: they're going to break on cascade.

@damianobarbati You _NEED_ "type": "module" in package.json, without it, all .js files in your module will be loaded as CommonJS.

exports is only used to limit what is exposed to external consumers and for conditional exports, it has absolutely no effect on whether .js files will be parsed as CommonJS or ESM.

@damianobarbati you're wrong, as per what @Exe-Boss said, btw,

That’s because Jest silently transpiles ES Modules to CommonJS code.

Yes, I rely on Jest quirks, that's the reason I was using babel.config.js without even adding babel to the devDependencies

@damianobarbati

Please do note, I have a working project, in it I'm using Jest transpiled, while the src directory has the module type, note, ./src not the root where the babel config file is (that one is CJS due to quirks).

Also, you are right as I do not import anything from NPM that is CJS as I don't think NPM (or the package authors) are quite ready for mix and match.

The goal in my project is to have only ESM (to cut down the tooling fat), so Jest is the only exception that gets transpiled on its own.

@azder something like this? https://github.com/damianobarbati/node-jest/tree/feature/azder-test

@damianobarbati

Something like that, here's the project https://github.com/azder/clip . Do note there isn't "dependencies" in my package.json as per the last sentence of the blog post excerpt, I decided not to mix ESM and CJS modules from NPM.

This way for the needs of Jest, it does transpile what requires from my ESM modules, but maybe more babel config will be required to deal with the node_modules directory.

https://medium.com/@nodejs/announcing-a-new-experimental-modules-1be8d2d6c2ff

Currently, it is not possible to create a package that can be used via both require(‘pkg’) and import ‘pkg’. There are efforts underway to address this, and may involve changes to the above. In particular, Node.js might choose a field other than “main” to define a package’s ES module entry point. While we are aware that the community has embraced the “module” field, it is unlikely that Node.js will adopt that field as many of the packages published using “module” include ES module JavaScript that may not evaluate in Node.js (because extensions are left off of filenames, or the code includes require statements, etc.). Please do not publish any ES module packages intended for use by Node.js until this is resolved.

Please see #9430 which I've just opened to track support in Jest. I'll keep this issue open for discussion.

@SimenB this is a good sign! This and mention of modules in Jest 25 release notes gives a hope to see the support sooner.

@SimenB if I remember correctly Jest was unable to use NPM packages that are published as ESM only, which forced to enable Babel for some node_modules packages too. Has this changed?

You'll need to transpile import/export until support lands, both user and library code

@SimenB:

You'll need to transpile import/export until support lands, both user and library code

For our case the best way so far is this, since all our packages have /es/ dir, but that's fragile and may not fit for other projects:

transformIgnorePatterns: ['node_modules/(?!.*?/es/.*\\.js)'],

Jest does not pick up those packages w/o transformation despite type: module, just like you said.

You'll need to transpile import/export until support lands, both user and library code

Any rough timetable on this?
from node 14 release:

It is our belief that the current implementation offers a future proof model to authoring ESM modules that paves the path to Universal JavaScript. Please read more in our documentation.

The ESM implementation in Node.js is still experimental but we do believe that we are getting very close to being able to call ESM in Node.js “stable”. Removing the warning is a huge step in that direction.

as such I am reluctant to export NPM packages (which may well be consumed by tests implementing the JEST testing framework) using commonJS for much longer.

We're shipping an experimental version from Jest 25.4. Quite a few bug fixes in 25.5, but it's still not where it should be. You can follow the progress in #9430