Jest: [bug] duplicate manual mock found in separate directories

+1

In my case, after clear the cacheDirectory/var/folders/dm/vt920lmd6tzdq_709zkykwx40000gn/T/jest and re-install npm dependencies, these messages have been disappeared.

Here's the offending code:

https://github.com/facebook/jest/blob/cd8976ec50dbed79cfe07f275052cdd80d466e73/packages/jest-haste-map/src/index.js#L98

But it looks like the behavior might be explicitly wanted as there is a test confirming it:

https://github.com/facebook/jest/blob/8de90b320c87a0a36d68f6bd8177620a985df269/packages/jest-haste-map/src/__tests__/__snapshots__/index-test.js.snap#L15

Which was added in:

https://github.com/facebook/jest/commit/cfade282fbbe2737b6dd2cee1cf3da3ee8624512

I wonder why we are using basename rather than the whole path as the key?

/cc @flarnie

This means that basenames for modules need to be globally unique in a project when using manual mocks. For my use case, it means I can't do something like:

import { MyWhatever } from 'models/MyWhatever/schema';
import { MyOtherWhatever } from 'models/MyOtherWhatever/schema';

and use manual mocks at the same time. Jest will currently see them both as mocking schema and complain.

Though the workaround is trivial (s/schema/MyWhateverSchema/), it feels like a bug to rename and restructure non-test code to make jest happy 🐞 .

Yes this does suck indeed. The manual mocking system is really not good and I'm happy to accept PRs that will improve the situation, assuming we can make sure we don't break all of FB (but I can take care of that :) )

Cool. I might find some time tomorrow to cook up a patch, but no promises though 😅

@cpojer is there a particular reason for this behaviour?

Could it have anything to do with the fact that fb uses unique filenames for modules? I don't see otherwise how not allowing two mocks with the same name makes any sense...

Yes, mocks are "global" as well. This is a terrible design that we have to live with, unfortunately. At FB we have 4000+ mock files in the wrong location (and often there isn't even a proper location). It is likely we will fix this early next half, so this should improve in Jest. I'm happy to support PRs that improve the behavior in Jest for open source – if we can retain the old behavior for Jest at FB for now.

@cpojer how about a flag? Would you accept a pr that enables/disables this with a flag?

Yeah, it should be a config option. But I'm not just talking about the warning, I'm also talking about the feature in general.

@cpojer right - which parts of jest does this touch?

The resolution code is called from jest-runtime: https://github.com/facebook/jest/blob/master/packages/jest-runtime/src/index.js and is somewhere in jest-resolve and jest-resolve-dependencies.

@cpojer thanks for the pointers :+1:

@cpojer what about some global override, like JEST_USE_BASENAME_FOR_CACHING, to switch this behaviour?

At least, we can enjoy non-unique filenames, and it will not break anything in FB.

Of course, it's a temporary solution.

I mean, this is in some /etc/profile or ~/.bashrc

export JEST_USE_BASENAME_FOR_CACHING="true"

(or some file with env)
and then

$ jest

or this one, without env file modifications:

$ JEST_USE_BASENAME_FOR_CACHING="true" jest

What you think? It's a sort of hack or it's ok? :wink:

I just tried with a new repo, using two versions of jest (^15.0.0 and ^17.0.0) and, although the latter gives the warning, the test behaves as expected.

@ColCh I don't think the issue here is with the cache, probably a more suitable name could be JEST_USE_BASENAME_FOR_MOCKING.

@cpojer if FB code has the restriction on the uniqueness of the names, using the full path as a key for the map of mocks shouldn't bring problems.

Am I right or there is something I'm not seeing?

The two solutions I see are:

• modify getMockName to accomodate the option to use the basename or the full path
• remove that function altogether

it would be nice to see @cpojer answer on it

Hey everyone, sorry for the delay, I'm pretty backed up with a ton of stuff right now.

I think I'm fine if you guys decide to do whatever breaking change in Jest that is necessary to improve this system. Manual mocking is really messed up and doesn't work well. One thing we kind of want to do is limit the scope of "haste modules" (internal FB module system) using a config option, like "haste_modules": ['path/a', 'path/b']" and then only look at haste modules in those folders, including this weird mocking behavior. If anybody wants to make this change, that would be amazing.

One thing to be figured out then, is this: If all manual mocks are local, like __mocks__/a.js maps to a.js, what do we do with node_module mocks? For this there are a couple of ways:

• Introduce a new __node_modules_mocks__ folder but that is ugly.
• The top level __mocks__ folder as seen from rootDir (project root) could act as a global folder.

So to summarize:

• Let's fix manual mocks in Jest!
• Let's namespace haste modules and limit them to certain folders/regex whatever.
• Make sure that the current mocking behavior still works for FB, even if that means we have to whitelist certain folders to make haste work (it would just look like ["<rootDir>"] for us for now, I guess)
• Figure out how to still be able to mock node modules.

What do you think?

In order:

• Let's fix manual mocks in Jest!

HURRAY :smile: :tada:

• Let's namespace haste modules and limit them to certain folders/regex whatever.
• Make sure that the current mocking behavior still works for FB, even if that means we have to whitelist certain folders to make haste work (it would just look like [""] for us for now, I guess)

I'm not sure I understand the needs with haste.
What you mean is to give the possibility to say "those modules are haste modules"?
If we have four modules: /path_1/a, /path_1/b, /path_2/a, /path_2/c, and the setting is

"haste_modules:" ["/path_1/a", "/path_2/c"]

only /path_1/a and /path_1/b are restricted to exist only in /path_1, so /path_2/c is valid, and /path_2/a raises an error/warning.

I'd say, targets could easily be specific files and entire directories, even with single/double *.

• Figure out how to still be able to mock node modules.

I'd maintain the current behavior:

If the module you are mocking is a node module (eg: fs), the mock should be placed in the same parent directory as the node_modules folder.

My thoughts:

haste modules:

I think , haste_modules is nice, it fits just like collectCoverageFrom and other options: array of globs
In case, if you have all src as _haste_ modules, and just one directory is non-haste:

haste_modules: [
  "src",
  "!src/foo"
]

node_modules

@EnoahNetzach what if someone have same names for app module and module from node_modules?

To make it work without haste... hm, I think it can be described like:

given node module project/node_modules/react, mock will be inside of project/__node_modules_mocks__/react.js
if you have a file project/react.js, then use project/__mocks__/react.js

(of course, react.js is an example. here can be any filename among of all modules, which can be installed from npm)

Really, mocking node_modules module is a rare case by my experience, so... may be _rareness_ may compensate _ugliness_ in particular case of mocking node_modules ?

Anyone mocking modules inside of node_modules often?

what about to think further

as I noticed, for react-native projects, we often have to mock application modules and leave modules from node_modules unmocked (e.g. lodash)

this means, we have:

• manual created mock component for each dumb component (dumb component are layout components)
• a long list of jest.mock call in every test file

__what I want to say__: it would be very nice to have ability to _auto mock_ modules on some paths.

It can be implemented around well-known data structure array-of-jest-globs in config, and filtering modules upon it.

I will describe that step by step

Given this config entry

"autoMockingPaths": [
  "src/components/dumb/**/*.js",
]

and this code at src/screens/app.js:

import _ from 'lodash';
import Button from '../../components/dumb/button.js';

// blah blah AppScreen implementation skipped

and this test code for screen at src/screens/__tests__/app-test.js:

import AppScreen from '../app.js';

describe('AppScreen', () => /* testing app screen */);

We come to this situation, in context of app-test.js:

• AppScreen is not mocked
• lodash is not mocked
• Button, which is required by AppScreen, is mocked

... You can answer, how it will play with automock config entry?

simply saying, automock: true is equivalent to:

"autoMockingPaths": [
  "<rootDir>"
]

auto mocking... wait!

may be just introduce special value for automock? at least it will not break people's configs

for example, with this config entry:

automock: "app"

jest will auto mock all application modules, and leave actual versions for modules from node_modules

what do you think about app level modules automocking, @cpojer ? I find it very efficient for my particular case.

I fully agree with "haste_modules".

We personally don't use automocking that much, so I can't say what's better, my wild guess is that the "autoMockingPaths" var could be useful and elastic enough.
On the contrary I find "automock": "app" too stiff (jest already disabled automocking by default).

The __node_modules_mocks__ could be an option, I agree that rareness compensate for ugliness (in my particular case, we rarely mock node_modules, and when we have to do it, we use jest.mock(...)).
The only caveat is: what happens when you have a nested node_modules folder (e.g. src/node_modules), do you have to mock its modules from the global __node_modules_mocks__, a nested version of it, or normally with __mocks__ co-located?

may be just throw, if someone has same module name in node_modules and app modules

e.g.

app/express.js as app module (may be I'm doing train game)
and app/node_modules/express as web server from npm
throw new Error("can't mock express.js file - it duplicates one from node_modules")

in this case, __mocks__ can be used for node_modules, but dev have to rename own modules on such collisions

nah... this is more ugly than __node_modules_mocks__, isn't it?

What I meant is: what if you have an npm installed module x and later, deeper in your codebase define a module x in a nested node_modules folder?

The naming collision is usually handled in node by preferring the nearest, but I don't know how this works in haste.

I'm bringing this up because projects like Create React App are using it, or will do in the near future.

On a side note, @cpojer is this issue somehow related?

Let's keep this focused on changing how haste works (whitelist/blacklist rather than on by default). I do think I'd prefer to maintain that <rootDir>/__mocks__ should be the default for node module mocks. We could make this a configuration option as well: "globalMocks" that defaults to <rootDir>/__mocks__. Is anybody willing to work on this?

I should be able to work on this no sooner than the next weekend.

I can work on PR this sunday, I'm kinda free

@cpojer just to recap - create globalMocks config entry with default value of <rootDir>/__mocks__. This option regulates use of node-haste within jest by specifying path? Or it will be array of paths?

I think these are some larger changes on the way to get this resolved but I think we need both the singular globalMocks option (could be a string or array of strings) and the hasteModules option which would be an array of paths of haste modules. Most of this code lives in jest-haste-map and jest-resolve. I'm not 100% sure what the right solution will look like yet.

From: Max Sysoev notifications@github.com
Sent: Friday, December 9, 2016 8:18:44 AM
To: facebook/jest
Cc: Christoph Pojer; Mention
Subject: Re: [facebook/jest] [bug] duplicate manual mock found in separate directories (#2070)

I can work on PR this sunday, I'm kinda free

@cpojerhttps://github.com/cpojer just to recap - create globalMocks config entry with default value of /__mocks__. This option regulates use of node-haste within jest by specifying path? Or it will be array of paths?

-
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/facebook/jest/issues/2070#issuecomment-265958606, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAA0KAMFc34iKqBDLHZzgaGHqyc3WkAzks5rGQ7kgaJpZM4Kt2DW.

Sorry, my workstation became broken, and apparently no way to get it working in couple of months (1-2 months). I even lost my project on this PR :( So, sorry for taking responsibilities.

What about just a config option to change the behavior of getMockName?

Not too familiar with the internals of jest, but it looks like that is the simplest solution to fix the issue without breaking jest for FB.

This is going to be more complicated than I originally thought. To me, manual mocks should replace the file they're closest too, ie. something like this:

{ 'aws-sdk': '/Users/project/__mocks__/aws-sdk.js',
  'slack': '/Users/project/__mocks__/slack.js',
  '/Users/project/db/index': '/Users/project/db/__mocks__/index.js',
  '/Users/project/slack/index': '/Users/projects/slack/__mocks__/index.js' }

require('aws-sdk') should resolve to the /Users/project/__mocks__/aws-sdk.js this is a mock for a node_module.

require('./db') (or any path to db) should resolve to: /Users/project/db/__mocks__/index.js.

My understanding of the way to setup jest manual mocks (and there should probably be more documentation if something like the above is used) was that they should be as close to the file being mocked as possible within a __mocks__ directory.

Manual mocks are defined by writing a module in a __mocks__/ subdirectory immediately adjacent to the module. (https://facebook.github.io/jest/docs/manual-mocks.html)

Given that, the above behavior makes the most sense to me.

Thoughts?

This is an initial pass at implementing something like the above: https://github.com/facebook/jest/compare/master...mhahn:issue-2070-new-mock-resolution

It breaks the majority of the jest unit tests because it no longer allows for pure "named" mocks like the following: https://github.com/facebook/jest/blob/master/packages/jest-runtime/src/__mocks__/createRuntime.js

IMO, something like createRuntime is a test utility that should be in a test utils folder and imported in the tests that need it. The way it is implemented as a manual mock doesn't really make sense to me as something that should be preserved.

One option is to add a config variable that will switch between the existing behavior and cleaned up version of the changes above. I'm not really sure what that should be called though.

We cannot break the current behavior as we rely on it extremely heavily at Facebook.

@cpojer i agree. i didn't really understand this comment after reading through the code: https://github.com/facebook/jest/issues/2070#issuecomment-265027510. maybe we could support whitelisting a list of directories that will resolve to the current behavior?

what about two new config options:

• fullPathMockResolution (defaulted to false to keep existing behavior)
  coupled with:
• namedMockDirectories: if fullPathMockResolution is enabled, any directories within that array will be resolved with the existing behavior. For the FB use case, this would just be [<rootDir>] like you mention above.

This lets devs opt into the full path resolution so it doesn't require changes by existing jest installs and also enables the existing behavior for specific directories if they want that.

cc @voideanvalue this may be something you'll have to think about (namespace manual mocks as part of the new haste implementation).

+1, is there a way to fix this?

+1

Any updates on how to fix this problem? This turns out to be really painful if you follow a structure like this:

project/
├── models
│   ├── index.js
│   ├── __mocks__/
│   │   ├── index.js/
├── challenges
│   ├── index.js
│   ├── __mocks__/
│   │   ├── index.js/

Right now I have to manually mock a module in the test if I'm using two with a 'conflicting' name even though they actually don't have conflicting names since the path should be considered.

Cheers,
Dominik

@cpojer Any updates here guys? Is there a workaround?

A little workaround for me is to mock the modules with require

jest.mock('models/index', () => require('models/index/_mocks_/index'));

I renamed the __mocks__ folder name to _mocks_ so that jest does not catch the files.

One day when this will work I will rename _mocks_ to __mocks__ and remove the require part from jest.mock

Firstly: Thanks for all the awesome work.
Secondly: This is really, quite frustrating. I find myself forced to use jest.mock in the test file when the mocked file shares a name with any other file in the entire codebase that also gets mocked. The hoisting disallows the actual mock to be imported too, so it forces you to duplicate the mock in any two tests that require that file to be mocked, adding to the fragility of the test suite. Are we looking at addressing this? Does FB use this stuff? If yes, then how? 😞

+1 very frustrating to see these warnings. Can't wait for a fix.

I think the issue I'm having is related:

if I have a jest.mock('src/utils/history'), it also incorrectly mocks the 'history' node_module.

https://github.com/cwmoo740/jest-manual-mocks-node-modules

Guys any update?

@masoudcs I think this got rid of the warning:

package.json

"jest": {
  /* other settings ... */
  "modulePathIgnorePatterns": ["<rootDir>/node_modules/react-native/Libraries/Core/__mocks__"]
}

Add the "duplicate" folders in the modulePathIgnorePatterns array and the warning will be gone

Thanks @brunolm

So it is just a warning, correct?!
I just want to make sure this would not cause something bad happen, i.e. mocking index.js in multiple directories:
src/app/modules/module1/__mocks__/index.js and src/app/modules/module2/__mocks__/index.js

I don't know if it matters, but when I was running it was saying it would pick only the other one and this that I listed would be ignored either way.

So I think it is fine to tell it to ignore, it wasn't using it anyway.

Yes just like you said, and we did not face any issues till now.
Hopefully, it is doing what it has to do! :)
Thanks

I tried what this last two commits suggested but didn't remove the warning.

jest.mock('dir/index', () => require('dir/__mocks_/index'));

I'm seeing this warning because I use GitBook (./_book directory), even though I have testPathIgnorePatterns: ['/_book/', ...otherStuff] in my configuration. Skimming through this issue, I don't see a clear cut workaround. I just want to stop seeing the warnings - any help would be appreciated.

+1

Any update on this?

Also still looking for a fix so that my test files can be immported in a regular fashion, not by using the more dirty way that was suggested above.

jest.mock('models/index', () => require('models/index/_mocks_/index'));

Our directory structure looks the same as @dkundel 's referenced here https://github.com/facebook/jest/issues/2070#issuecomment-301332202 with models and components in their own namespaces with index.js being the default export.

Would prefer to not silence warnings or throw all mocks into a flat directory. Some of our mocks are deeply nested, and the workaround suggested would likely look like
jest.mock('pages/index/components/Component', () => require('pages/index/components/Component/_mocks_/index'));
in our structure.

Any word?

@karomancer I've submitted PR #6037 which will allow you to use a configuration to remove the warnings. As of yet, it hasn't been merged; I'm waiting for a response from the contributors.

This issue is super frustrating, but I think I've found a nice workaround that results in a cleaner naming convention.

In package.json

{
  "jest": {
    "setupFiles": [
      "<rootDir>/test.mocks.ts"
    ]
  }
}

/* test.mocks.ts */

// modules mocked before every test
// use `jest.unmock(...)` to undo for any single test case
const mockedModules = [
    "./path/to/module1/index.ts",
    "./path/to/module2/index.ts",
];

mockedModules.forEach((path) => {
    const mockPath = path.replace(/\.ts$/g, ".mock.ts");
    jest.mock(path, () => require(mockPath));
});

This will allow you to mock anything.ts by creating a sibling anything.mock.ts and adding the path to the original in the top-level test.mocks's mockedModules array.

Why this issue tagged as enhancement?

This seems to work for me. in jest.config.js:

module.exports = {
  // ...
  modulePathIgnorePatterns: ["<rootDir>/.*/__mocks__"]
};

I'm not sure of the the scope or impact of this change, because I have a small project.

@amccloud thanks! This solved my problem! Details below.

I have a module helpers in my project root directory. The helper is exporting function parseNodeFromString.
I've created in some other module local file helpers. Then I mocked it for one of my tests. And all of the tests using the function parseNodeFromString started to fail with the following error:

FAIL  src/some_dir/bla/tests/SomeClass.test.js
  ● Test suite failed to run

    TypeError: (0 , _helpers.parseNodeFromString) is not a function

What about this issue? Seem like @amccloud solution is correct.

modulePathIgnorePatterns: ["<rootDir>/.*/__mocks__"]

This solution works although it made my top level mocks for node modules fail. So I changed it to not ignore my root __mock__ folder with: "modulePathIgnorePatterns": ["<rootDir>/src/react/.*/__mocks__"],. Still it's quite weird that mocks are not just unique based on the full path from the root. It's pretty common to have: users/helper.js & posts/helper.js. The warning does take quite some space, and I don't want to completely hide actual warnings.

So what is the current status of PR? Is there any proper solution or just some hacks?

In my case, the mock module was copied to Dist dir with every build.

Looks like this is a problem with typescript being unable to exclude paths/patterns that belong to deeper dir structure. Its still a problem with "typescript@^3.4.5".

To fix this I started cleaning my Dist dir with "rimraf dist" before every test run.

"test:unit": "npm run clean && stencil test --spec --snapshot",

I know its a hack, but works.

Hey i solved this here is what happened, and maybe it can help you duplicate this.

three solutions or scenarios:

1, I was editing my app on my text editor twice meaning, I was running pod install / update and react-native run-ios from two different windows. and I got this error, I tried searching for the duplicate files in Xcode and on my app but i couldn't find any. so I simply deleted the app from the simulator and re-ran react-native run-ios and it worked, It turned out that two node_modules had been duplicated as such: node_modules and node_modules0 in my scr file.

2, sometimes when I press random keys on my mac the node_modules are duplicated in for example the SRC folder so this also be the case, so it makes sense to have a look at your folders for any node_modules duplications.

3, I could'nt get my app to start until I launched and terminated a different app on the same simulator and then I rebooted the app with the duplication which then fired up without any errors.

Still nothing? Cant use modulePathIgnorePatterns in CRA

3 years and 10 months

Alternatively, it would be nice if we could force Jest to throw an error when this issue comes up, rather than just warning. Warnings are easily ignored; on my project, I'd prefer that an error be thrown, so the dev who introduced it has to deal with it.