Yarn: Competing lockfiles create poor UX

Add config to package.json to indicate

It could be a good use case for the engine field 🤔

Round tripping package-lock.json → yarn.lock → package-lock.json is lossy, but that's likely unimportant. yarn.lock → package-lock.json → yarn.lock should not be lossy, AFAIK.

From the npm point of view, I favor the option where if yarn sees a package-lock.json and not a yarn.lock that it imports it and removes the package-lock.json. And similarly, if npm sees a yarn.lock and no package-lock.json it should do the same, importing and removing the `yarn.lock.

This would allow users of both tools to seamlessly switch back and forth without leaving the user's project in a confusing state (where files appear to be from both).

I'm a bit concerned with this, because it means that neither package-lock.json nor yarn.lock will be able to add metadata to the files (Yarn currently doesn't, but why not), removing some freedom to our formats in the process.

This issue also raises the question of interoperability between Yarn and npm in general, originally discussed in another thread - I feel like trying to be too interoperable might deserve both of us, since users will then have the expectation that everything will work exactly the same way on both project, which I feel is a dangerous assumption (one obvious example is that workspaces will silently break if someone run npm on a workspace-powered project).

@yarnpkg/core, thoughts?

I like @iarna’s idea to make both tools migrate from one lock file to
another on install automatically.
Makes sense to delete the imported lockfile after the migration to avoid
both package managers competition in one project.

Both tools could print warnings and maybe reqire user prompt to proceed.

I also agree with Mael’s point that achieving 100% compatibility will lock
us from exploring new features, I think we should treat it as a one-off
migration path, that could be a rather small feature in install.js on our
side.

On Wed, Apr 11, 2018 at 9:49 PM Maël Nison notifications@github.com wrote:

I'm a bit concerned with this, because it means that neither
package-lock.json nor yarn.lock will be able to add metadata to the files
(Yarn currently doesn't, but why not), removing some freedom to our formats
in the process.

This issue also raises the question of interoperability between Yarn and
npm in general, originally discussed in another thread - I feel like trying
to be too interoperable might deserve both of us, since users will then
have the expectation that everything will work exactly the same way on both
project, which I feel is a dangerous assumption (one obvious example is
that workspaces will silently break if someone run npm on a
workspace-powered project).

@yarnpkg/core https://github.com/orgs/yarnpkg/teams/core, thoughts?

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/5654#issuecomment-380677110, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACBdWI9jnLJeFqH8v2T-AB74sQO1PMIjks5tntzrgaJpZM4TQ5-B
.

Pinging @imsnif since he has expressed interest in implementing the "Convert the other lockfile" solution a few weeks ago. He may even have some working code by now.

Thanks for the ping!

So yes, I'm actually in the midst of working on exactly this. I was planning on writing up a yarn RFC and pinging all stake holders next week.

So briefly: I've done some work on converting yarn.lock <==> package-lock.json. There are some losses in the process, but very little of them are logical. In my eyes, this is totally acceptable if we're talking about the 'one time conversion' scenario mentioned above. If we'd like to discuss this further, I can go into more details.

I currently have some rudimentary code that does this: https://github.com/imsnif/synp
It's not 100% and relies on an existing and current node_modules folder for the project to be converted. I'm in the finishing stages of a re-write (on branch 2.0.0) that will get package state from the registry (using the awesome pacote lib).

What I want to do once that is done is start talking about how (and if?) we'd like to implement this exactly in both yarn and npm.

Would love to hear everyone's thoughts on this.

In my eyes, this is totally acceptable if we're talking about the 'one time conversion' scenario mentioned above

I don't think it will be that common. I might be wrong, but I feel like the more common use case will be projects using Yarn, and one of the developer copies/pastes the command from a README to add a dependency, using npm instead of yarn in the process.

In this context, I'm still not convinced it's a good thing to lose data and change the package layout silently by trying to do the right thing - they will probably not even notice it until things break (we could have a confirmation prompt, but I'm expecting people that copy/paste those commands to also blindly confirm install prompts). Worse, it could work on their own machine, but break on their collegues' ones once they switch back to Yarn.

@arcanis - I think all the scenarios and edge-cases are very much worth discussing. In the scenario you mentioned I very much agree, but I think there are other scenarios where there should be a clear bias toward one lockfile.

I see this feature as mainly being used in CI environments, where I think it can shine (as OP mentioned). In the very least though, I think both tools should be aware of the other tool's lockfile? Leaving the specifics of when they decide to make the conversion up to them. What do you think?

I see this feature as mainly being used in CI environments, where I think it can shine (as OP mentioned).

Do you have an example? CI systems are supposed to act in a very deterministic way, accidentally commiting a converted lockfile is something that should be caught at worse at PR-time imo, CI is too late for this.

In the very least though, I think both tools should be aware of the other tool's lockfile?

Maybe. Using the engine version to force a specific package manager looks cleaner to me, both projects would just have to keep a list of exclusive engines (like "yarn" cannot be satisfied on "npm", and inversely).

In my mind, converting the lockfiles on the fly also isn't very scalable. For example, we've been only talking about the package-lock.json file until now, but I think pnpm has its own lockfile called shrinkwrap.yaml. Would be nice of we could find a way to deal with both of them and any other format that might come up with one stone.

I see this feature as mainly being used in CI environments, where I think it can shine (as OP mentioned).

Also note that if I understood correctly, npm is currently working on a CI-specific very light package manager, which might not work very well with things requiring heavy business logic such as a lockfile converter.

Regarding CI:
My initial thought was: warn when adding a new package with the opposite tool, convert when doing a fresh install (or more conservatively: fail by default on a fresh install and verbosely allow conversion through an import command or config flag for install).

When I previously worked in a mixed npm/yarn environment, we would essentially do this manually with git histories. It was a painstaking process and I'm sure we were/are not alone in this.

As for scalability:
I'd like to account for this as well.
When the conversion is performed, an intermediary logical and physical package tree (if the lockfile has one) are created. When converting to the desired lockfile, those trees are used (or created with sane defaults in case we're converting from yarn and do not have a physical tree).
This way, we can easily convert between other lockfile types as (all we need to do is provide conversions to/from a logical/physical tree for each type).

This is obviously a little ahead of the tool's capabilities right now, but I do not see implementing this as a big issue.

(as for npm's new ci tool - I see what you mean, but I think this is a little ahead of the current discussion?)

or more conservatively: fail by default on a fresh install and verbosely allow conversion through an import command

Yep, I'd be totally for adding this conversion logic into the yarn import command, it seems like a useful feature that can be extracted into a separate package if/when we get to implement Yarn plugins :) My comments are focusing on the default behavior we should adopt.

As of npm@6, you should be able to convert package-lock.json -> yarn.lock losslessly, without having to run an install first. It should contain all the data you need, identifiers, etc (npm@6's pkglock has version ranges in the requires field, which matches what yarn.lock uses?)

Oh, and you also need https://github.com/yarnpkg/yarn/pull/5042 to be shipped before this is true, as well, since npm doesn't have the sha1 in the pkglock in many cases.

@arcanis - fantastic! So would a PR with:

1. A warning when installing and finding a package-lock.json file
2. A warning when adding and finding a package-lock.json file
3. Adding the ability to convert from package-lock.json through the import command (and deleting the file thereafter)
   Be acceptable? (just so we have something we can use to start the conversation on specifics)

@zkat - that's great about npm@6! If it is up to me, I'd opt for using that and falling back to manifest files in order to support all lockfile versions. Could I ask if you'd be open to implementing similar (or maybe somewhat different) behaviour for npm?

Also note that if I understood correctly, npm is currently working on a CI-specific very light package manager, which might not work very well with things requiring heavy business logic such as a lockfile converter.

If by "currently working on" you mean "have shipped" then yes. =)

As you suggest, currently loading yarn.locks is beyond its scope (as it doesn't know how to layout package trees) but if we all end up with a library that can do that layout I'm not opposed to having it support loading yarn.locks. (Ideally this would be a library shared between it and Yarn.) Obviously for its case it shouldn't remove the existing lock file. It's purely read-only as far as package metadata goes.

A warning when installing and finding a package-lock.json file

I'm concerned that just having warnings when the an unsupported kind of lock file exists won't help with the problems raised by @jmorrell that Heroku is having?

I'm concerned that just having warnings when the an unsupported kind of lock file exists won't help with the problems raised by @jmorrell that Heroku is having?

A warning would be an improvement, but an error with good messaging would be ideal for users imo. I can only speak for myself, but I have several projects, some that use yarn and some that use npm, and I often find myself typing the wrong one for that project. A quick error means that I can immediately use the one I should have from the beginning.

If it's a warning, it should be big and obvious. IME users are pretty trained to expect to ignore the output of their package manager as long as it worked.

I've reached out to some people who have more experience with beginning developers who will hopefully chime in on what that experience looks like to someone who's just getting started.

CI systems are supposed to act in a very deterministic way, accidentally commiting a converted lockfile is something that should be caught at worse at PR-time imo, CI is too late for this.

I agree. Ideally by the time it arrives at the build / CI service the application is not in an indeterminate state. The only way I would feel comfortable not failing the build in this case was if there was an accepted way for the user to indicate their preference in package.json, but that solution also puts more burden on the user.

I'm concerned that just having warnings when the an unsupported kind of lock file exists won't help with the problems raised by @jmorrell that Heroku is having?

It would help them by warning the users that they're doing something they probably don't want to before they actually push on Heroku. I'm not sure how much it would actually change the numbers, but that's something we can just check in a few weeks and see if we need to go further. And since the required changes are relatively small, that might be a good first iteration.

Honest question for those proposing a warning: Is there a situation where running yarn install or yarn add while a package-lock.json is present would not be a mistake? Ditto for npm install and yarn.lock

Migrating from npm to Yarn, or from Yarn to npm, comes to mind. I'd prefer to avoid adding friction when one wants to try out other package managers.

One thing to also keep in mind is that warnings and errors are not mutually exclusive, or at least not with the engine field. Projects without pinned package managers could print a warning, and projects with a pinned package manager could error. This way users could freely switch from one package manager to the other, until they make their choice and configure their package.json accordingly.

Another advantage of the engine field is that it would allow you to know for sure which version of the package manager has to be used. I guess it's configurable somewhere, but with this system it would be part of the regular workflow.

Another advantage of the engine field is that it would allow you to know for sure which version of the package manager has to be used. I guess it's configurable somewhere, but with this system it would be part of the regular workflow.

Is there an engine field or are you referring to engines? I can't find any docs referring to engine. Apologies if I'm missing something.

We support specifying versions in the engines field: https://devcenter.heroku.com/articles/nodejs-support#specifying-an-npm-version

Ex:

  "engines": {
    "npm": "5.6.x"
  }

There are a couple concerns that I have with using this to determine which package manager to use:

• The way things look today: a non-trivial number of users have both npm and yarn defined, or an npm version, but then they have a yarn.lock. Future versions of npm and yarn could enforce this, but users who don't upgrade frequently will continue to use current versions for a long time.

• Most users never touch this config. They forked an old boilerplate that specified Node 0.10 even though they use whatever they downloaded off of nodejs.org on their machine. This causes a non-trivial number of issues on Heroku.

• Teams often do not coordinate which versions they have installed. The version of Node or npm or yarn they have installed is often whatever was current when they started on a project or the last time one of them broke and forced them to reinstall. Not that this is ideal, but my understanding is that the majority of npm / yarn users are beginners, and this adds a new hurdle to getting started.

Ex: "I cloned the example hello-world project, downloaded Node from https://nodejs.org, and it didn't work"

All that said, I would love, love, love if the tools enforced strict versions and recorded it in engines. It would prevent many of the errors that I see every day, but that feels like a much, much larger change.

All that said, I would love, love, love if the tools enforced strict versions and recorded it in engines. It would prevent many of the errors that I see every day, but that feels like a much, much larger change.

AFAIK Yarn enforces this field strictly unless you pass a flag to disable that.

Teams often do not coordinate which versions they have installed. The version of Node or npm or yarn they have installed is often whatever was current when they started on a project or the last time one of them broke and forced them to reinstall.

This is quite dangerous, especially when using Yarn since it only guarantees consistency across the same major versions of Yarn.

Not that this is ideal, but my understanding is that the majority of npm / yarn users are beginners, and this adds a new hurdle to getting started.

How about Yarn (and also npm, if they like the idea) adding a "yarn": "^<current_version>" entry into the engines field to help introduce some safety without too much friction? We can automatically add this when running yarn init, yarn import or when we create a lockfile from scratch.

I don't know much about Yarn, but I think the best solution would be to add a package-lock.json parser into Yarn and don't create a yarn.lock if it exists.

I think that the end goal should be to have one lockfile format shared among Yarn and npm.

@thatlittlegit You'll be pleased to know about https://github.com/yarnpkg/yarn/pull/5745 then. (npm will be doing the same in reverse, so we'll end up in a place where it doesn't matter which kind of lock file you're working from or which tool you pick.)

Edited to add: A single lockfile format is not, I think, a likely outcome as there are different tradeoffs between the lockfile formats and lack of consensus as to which one's are best. I think having both tools being able to mutually understand one another's lockfiles is sufficient.

Beyond that, there are a ton of advantages of having multiple actively used tools that make different tradeoffs because it means user use cases can be better met than can be with a single general solution.

@BYK Some history on the behavior of the engine field would probably be helpful here:

In npm@1 and npm@2, we had a package.json called engineStrict in addition to the engine fields. If engineStrict were true then the engine field was used as _resolution constraint_ and before we applied the semver range to the list of versions, versions with incompatible engines would be filtered out. This would let you do things like npm install foo and get [email protected] even if [email protected] was published if [email protected] wasn't supported on your platform. This _seems_ desirable, but in practice was super confusing, particularly because docs on the website were only for the most recent version.

If engineStrict were false then resolution was done without regard to the engine fields and warnings were issued if the result wasn't compatible.

There was ALSO an engine-strict config option, which did the same thing as the package.json property, but applied it to ALL packages.

Starting in npm@3, we dropped support for the engineStrict package.json property and changed the behavior of the engine-strict config option. The config option now turns the warnings that you get with it off into errors, but does not impact resolution.

The engine fields are enforced not just for the top level project but transitively. Introducing a yarn engine flag would imply that dependencies can only be installed with yarn and I don't think that's the case here.

Since in the next yarn release we should have the ability to import package-lock.json to yarn.lock - I'd like to go back to discussing the warning/error issue.

The way I see it: I don't think there's a good reason for a single package to have both a package-lock.json and a yarn.lock file. Most projects I've seen who have this knowingly view it as an issue that needs to be fixed rather than a desired situation (but I'm of course open to be corrected).
As I understand from @iarna's explanation above, the engine field won't be a good solution to explicitly specify which package manager a package is using.
So IMO, this is a situation that should produce a verbose error that would invite the user to either delete one file or convert it.

However, I think producing such an error would be a clear breaking change. On the yarn side, I'd propose starting out with a warning (as mentioned above) and slowly deprecating this behaviour until the next major version in which it would be changed to an error (maybe adding a config flag that would allow both files to exist simultaneously).

What does everyone think? @jmorrell, @BYK, @arcanis, @iarna?

It seems odd to me that any kind of project should specify what package management tool should be used. Correct me if I'm wrong, but I see Yarn as a drop-in replacement that hinges mostly on end user preference. Is Yarn trying to replace NPM entirely as the de facto standard, or is it attempting to coexist?

@BrainBacon You only get the benefits of locked dependencies if everyone uses the same package manager that respects the same lockfile and uses the same semantics. If a project has a yarn.lock and someone then runs an npm install, that person could still end up with a different set of dependencies. So no, it's not dependent on end-user preference, unfortunately - preferably, everyone on a project uses the same package manager.

This also means that it does not make sense to have two lockfiles in a project. They are unlikely to resolve to the same dependency tree, and make it unclear to contributors which package manager to use.

(The drop-in replacement part was especially true when npm did not produce lockfiles, as there was no information to get lost there. It's also still true if #5745 really means that Yarn can now produce a lockfile based on package-lock.json, but that only means that a _project_ can then easily replace npm by Yarn. However, since you still need to check in the new lockfile, all contributors will need to switch.)

Correct me if I'm wrong, but I see Yarn as a drop-in replacement that hinges mostly on end user preference.

Yarn isn't a straight up drop-in replacement in term of exposed API. The yarn add <pkg> vs npm install <pkg> is an obvious example of something where we do things differently. We do tend to have the same feature set, but in the end it's two different tools and sometimes we have different solutions to a same problem.

The yarn.lock vs package-lock.json is one of those differences and I believe neither Yarn nor npm are interested into using a single one, as they contain different information.

On the yarn side, I'd propose starting out with a warning (as mentioned above)

I'd be fine with this. Something like:

WARN Your project seem to contain lock files (package-lock.json, shrinkwrap.json) generated
WARN by other tools than Yarn. It is advised not to mix package managers, in order to avoid
WARN resolution inconsistencies caused by desynchronized lock files.

Does someone want to open a PR?

I'd be happy to open a PR that adds a warning :)

Is the desired end-solution here a helpful error when package-lock.json exists + yarn import? My understanding was that @iarna was advocating that each tool should remove / convert the opposite lockfile if it exists automatically.

Either solution would be a significant improvement for users, but I think it would be best if both yarn and npm chose the same strategy (though not strictly necessary if there isn't agreement and people feel strongly about it).

@jmorrell - my understanding was that we agreed automatic conversion is a bad idea here.

Mostly because it's hard to know what the delta between the two lockfiles should be.
Eg. when I see the above warning as a developer, what I'd probably want to do is figure out when the other lockfile was created, what packages were added to it that were not added to my main lockfile and then add them. Maybe using the import option as a reference, but not necessarily.

I hope the npm folks agree on this one?

Ideally, I'd like the warning to include something about this being a deprecated situation (not sure about the wording) and that future yarn versions will produce an error. Maybe linking to some detailed documentation about this and how to address it. @arcanis - do you think it's feasible, or would you rather stay with the warning behaviour?

If both tools issue an error when there are multiple lockfiles, I feel developers will catch on to this issue while it's still "small" and be able to address it quickly (with the help of importing as reference or without).

@arcanis npm add <pkg> is perfectly valid and does what Yarn does 👼

As far as pkglock vs yarnlock, the original intention of package-lock.json was to create a format that included all the data either package manager needed. In fact, pkglock as of npm@6 can be transparently converted into a yarn.lock without executing an installer, since it describes a full tree with all the relevant metadata. We did that _intentionally_ (such as the requires field), and we asked for feedback from both the Yarn and pnpm teams to make sure that whatever we did was sufficient for both of them to use either as an importable lockfile or as an alternative option. yarn.lock is, unfortunately, a lossy subset so the inverse is not true, but we'll be adding stuff to import it anyway. npm would even obey Yarn-calculated tree shapes (unlike Yarn, which would not).

In fact, pkglock as of npm@6 can be transparently converted into a yarn.lock without executing an installer, since it describes a full tree with all the relevant metadata.

I don't think it can (but maybe I'm mistaken, feel free to point any misconception). The Yarn lockfile doesn't support by design multiple ranges resolving to a single version. If you have multiple packages each depending on lodash@^1.0.0, they will all use the exact same version. It's not only an optimization, but also how we encode things in our lockfile.

The npm lockfile being a tree, this property isn't guaranteed, and multiple identical ranges might end up using different versions (which is fine, since it might allow slightly better optimizations by exploiting the Node resolution rules). In such circumstances, the package-lock.json -> yarn.lock conversion would be lossy, since we wouldn't have any other choice than to merge them into a single one.

On the other hand, a Yarn lockfile can be transformed into a package-lock.json relatively without trouble, because our rules are a stricter subset of yours (our merged ranges can be transformed into a tree without loss, a tree cannot be transformed into a merged range without loss). Note that I'm only talking about the dependency tree itself, I'm not familiar with your metadata fields.

To be honest I'm not entirely sure whether we say the same thing or something entirely different, I just wanted to clarify a bit the situation 🙂

Ideally, I'd like the warning to include something about this being a deprecated situation (not sure about the wording) and that future yarn versions will produce an error. Maybe linking to some detailed documentation about this and how to address it.

@imsnif I'm not convinced about this being an error in non-CI environments (no question about CI - this is something that should definitely trigger an error). I'd prefer to stay on a warning that simply describe the current state, and explain why it's not ideal.

A good thing is that @jmorrell will be able to provide us precise metrics to see whether the warning has been enough, and based on that we'll choose our next move.

Is the desired end-solution here a helpful error when package-lock.json exists + yarn import?

Yep, coupled with the "upgrade-to-error-on-CI" mode* I think it's a reasonable first step!

(* This doesn't exist yet, I'll mention it in the #5773 Yarn 2.0 WG)

@arcanis - About lockfiles: I think what @zkat is referring to (but please @zkat correct me if I'm wrong or misunderstanding) is that package-lock.json saves the physical as well as logical tree, while yarn.lock only saves the logical tree. So as mentioned earlier, converting package-lock.json => yarn.lock => package-lock.json will lose the physical tree data. IMO in most cases this is fine, but there are exceptions: eg. bundled dependencies.

My personal opinion is that both package managers can benefit from syncing their construction of a physical tree. From what I've seen, we can probably extract some well defined rules about how a physical tree is created and both follow them (ideally with a module maintained by all stakeholders). I think this will both solve the compatibility issue, some issues yarn has with bundled dependencies, while still allowing each package manager to keep its unique quirks. (Though admittedly, I'm quite new to this discussion, so there might be things I'm unaware of - my apologies if I'm waking some sleeping dogs).

To the issue at hand though: @arcanis - I'll try to offer just one more argument here in favour of blanket errors in this situation, and if you disagree we can stay with the warnings as far as I'm concerned:
While it is indeed quite important to have this caught on the CI level, I think an error there would be significantly harder to debug (on average) than an error in the developer's local environment.
If a developer gets an error on their local environment for using the wrong tool, they (likely) won't go even one step further. They'd just say "oops", and use the other tool. It will save a tremendous amount of time, in place of catching it on the CI after developing a feature and then backtracking.
A warning (as was mentioned previously here) might get swallowed up in quite a few others that developers tend to ignore if the exit status is 0.
What do you think?

Also - in the situation I described above the "foreign" lockfile might be understandably placed in .gitignore and so the CI won't even get a chance to know about it. As an absent-minded developer, I might think "Well, it gives me some obscure warning on my local environment, but it passes CI - so it's probably just a local issue on my end - ah well"

A warning (as was mentioned previously here) might get swallowed up in quite a few others that developers tend to ignore if the exit status is 0.

My point is that this affirmation isn't backed by any data, even though we do have a way to collect some to then make an informed decision. In this context, choosing the more radical option that will break peoples' workflow seems to me a bit futile and potentially harmful 😕

Additionally, I think you're kinda overlooking the user story of "I'm using a package manager and want to try another one", which I think is quite important for everyone involved, both package manager maintainers and users alike. It wouldn't be great if people weren't able to easily try out npm 6 from their Yarn project (or conversely, to try Yarn from their npm project).

My personal opinion is that both package managers can benefit from syncing their construction of a physical tree.

This is a different topic, but I disagree. I think there's a misconception that Yarn is npm, which is incorrect. If you want to import a project into another through a dedicated command I'm all for it, but silently converting (or reading) lockfiles from a third-party format is a recipe for disaster:

• it will ignore configuration files
• it will break the physical tree the next time a package is added/removed
• any feature unique to a package manager won't work (workspaces, resolution override, link:, ...)

My point is that this affirmation isn't backed by any data, even though we do have a way to collect some to then make an informed decision. In this context, choosing the more radical option that will break peoples' workflow seems to me a bit futile and potentially harmful confused

True - it's not backed by any data. I'm definitely speculating here (sorry if I did not emphasize this). While we might be able to get a sense for the trend of the warning fix, we won't be able to get a sense of how convenient this is to users and whether there is a more convenient solution. If we rely on data, I don't think there is any result other than "there was absolutely no change in the Heroku deploy error rate" that will steer us toward the blanket error solution.

I agree it's quite important not to break people's workflows. Which is why I proposed the error as a breaking change in 2.0.0, with wording in the warning that would alert the user that this situation is deprecated. I also think we can permit the use of a --force or config parameter to override this behaviour. (I believe this also addresses your trying out the other package manager concern?)

This is a different topic, but I disagree. I think there's a misconception that Yarn is npm, which is incorrect. If you want to import a project into another through a dedicated command I'm all for it, but silently converting (or reading) lockfiles from a third-party format is a recipe for disaster

I think you raise important concerns here which I'd love to address. But as you say, this is a different topic and I don't want to derail the conversation. Maybe we can discuss it in a separate issue?

I think the conversation here got derailed a bit so I want to tidy it up with some clarifications and a summary:

1. Yarn and npm have different resolution and physical tree creation strategies and this is a key factor for both projects to exist.
2. Both yarn.lock and package-lock.json files were created with specific goals in mind and as far as I can see, these goals don't fully align but have some overlaps.
3. The difference in these goals don't make a format superior to other in general, they just trade off different things.

Based on these, I think keeping both formats is actually critical for innovation and serving different needs. What is needed is ensuring project portability with minimal data loss between package managers without imposing a particular package managers strategy onto the other to allow free experimentation.

I see the acknowledgment and warning against other lockfiles along with proper conversion with a clear messaging about what is conserved and what is lost as a great value to users on each side. So now the question here in this issue is what is the best flow for users that have a package-lock file in their repo when they run yarn.

Looks like auto conversion is potentially dangerous and failing installation without a way out may hurt certain people. How about requiring a config to be set for yarn to know that having a package-lock file is intended in CI mode? This is a signal from the developers to yarn, saying "I know what I'm doing so please do not try to protect me against any discrapencies".

Thoughts?

How about requiring a config to be set for yarn to know that having a package-lock file is intended in CI mode? This is a signal from the developers to yarn, saying "I know what I'm doing so please do not try to protect me against any discrapencies".

@BYK - sounds great. With one addition: as @arcanis convinced me here and on discord, it might be good to also add a warning in non-ci mode when the config flag is not set. So that developers have a chance to know their CI build might fail before they push (and also to protect against package-lock.json being in .gitignore).

@arcanis as @imsnif mentioned: pkglock includes _both_ the logical and physical layout of the tree. By following the logical version of the tree (which includes the ranges that we deduped from), you can construct a yarn.lock in-memory with no installs or running installer logic at all. Just tree lookups :)

From NPM 5 release notes:

A new, standardised lockfile feature meant for cross-package-manager compatibility (package-lock.json)

It seems that the NPM team advertises package-lock.json as the universal solution; is it (technically) possible to join that force?

One solution I'd like to propose would be "use only package-lock.json when it already exists". That's basically how NPM "migrated" from shrinkwrap to a lock json file.

It seems that the NPM team advertises package-lock.json as the universal solution; is it (technically) possible to join that force?

We already discussed it, please check the rest of the thread if you haven't. This isn't going to happen in the short term future.

I think you're kinda overlooking the user story of "I'm using a package manager and want to try another one", which I think is quite important for everyone involved

This probably isn't a frequent thing, context switching. I picked Yarn a while ago and haven't looked back. I think it's better to throw some weight behind the seriousness of why the lock files are there. Make it an error that can be opted out of, rather than a warning that will be ignored. When someone commits a lock file, it's for a reason.

Our stats show that the majority of Yarn users also use npm. Mixed _projects_ are uncommon, but a single person working on multiple projects, some of which are npm and some of which are Yarn, is quite common. Further complicating the issue, install scripts that run npm directly are a thing.

I want to use yarn, but most of the projects I work on only have package-lock.json files. I cannot add yarn.lock files. yarn import is slow and/or broken. Currently my
_only_ option is to ditch yarn and switch to npm.

I understand that yarn is built by yarn-only developers for yarn-only projects. But what about the rest of us?

Hey @Spongman - yarn import should not be broken. It should now be able to import package-lock.json for you. If you're having issues please let us know!

I did #6103

Hey @Spongman - I commented in the issue, this specific case was due to a corrupted package-lock.json. I'd be happy to know about any other problems.

npm can use that package-lock.json file just fine. yarn needs to be more resilient.

Answered this in the other issue - I ask that the discussion move there so as to keep this issue on topic. Thanks!

I was carefully watching and participating in https://github.com/yarnpkg/yarn/issues/3614 (with currently 254 :+1:s). That issue has now been closed and the discussion moved here.

Ignoring practical challenges, in my opinion by far the best UX would be provided by an option not mentioned in the summary at the top, but mentioned by @thatlittlegit:

I think that the end goal should be to have one lockfile format shared among Yarn and npm.

Which is also backed up by @BrainBacon's point:

It seems odd to me that any kind of project should specify what package management tool should be used.

Again, ignoring practical challenges for a minute, the theoretical counter-argument to this was made by @iarna in this thread (and @jhabidas in the other thread):

There are a ton of advantages of having multiple actively used tools that make different tradeoffs because it means user use cases can be better met than can be with a single general solution.

Personally, I don't think this argument should apply in this case, as I made in a comment on the other thread (with 95 :+1:s, 2 :-1:s):

Yes I agree, Yarn wants to maintain its independent space as far as possible, so it has the flexibility to provide a better experience. Which is why, for example, Yarn's CLI is deliberately not compatible with NPM's.

However, I think lock files are outside of the space where Yarn can reasonably maintain independence. package-lock.json and composer.lock are committed to the repository, alongside package.json and composer.json. These are not tool-specific files, they are instead project-specific files specifying the exact dependency versions that the project is guaranteed to work with.

...

For Yarn to be a useful tool, it has to allow developers to follow standard models in their projects, so the project can remain tool-agnostic. Which, after all, is why Yarn was built on top of package.json and not its own separate dependency file.

Having read most of this thread, I still agree with my initial assessment - if Yarn continues to consume the same package.json dependency file from the repository, ideally it should emit the same package-lock.json file into the repository, so the repository can remain tool-agnostic. If Yarn read its dependencies from yarn.json instead of package.json I wouldn't keep making this point.

I believe that we should admit that this is the ideal situation, the seamless user experience. However, I do understand @iarna's other point:

A single lockfile format is not, I think, a likely outcome as there are different tradeoffs between the lockfile formats and lack of consensus as to which one's are best

(although I disagree that "having both tools being able to mutually understand one another's lockfiles is sufficient").

If the ideal scenario of allowing repositories to be tool-agnostic is definitely not achieveable (and I see the many comments about how the different lockfiles are built in fundamentally different ways and contain different information), then where we appear to be going seems just about palatable, which is:

• NPM/Yarn will emit warnings if they spot the other's lockfile (and maybe a config option to make this into an error)
• Both tools provide mechanisms for converting one lockfile into another

I don't think either should, under any circumstances, convert and delete the other's lockfile. This would mean that new developers would be forever converting from one to the other and back, depending on what tool that developer happens to use (given that too many people commit with git commit -a).

I also don't agree with @imsnif's point:

I don't think there's a good reason for a single package to have both a package-lock.json and a yarn.lock file.

I think it's presumptuous to dictate what a developer runs on their computer, or in their production environment. It's at least courteous not to be more restrictive than you need to be. If your project is only ever run by your team of developers, and they all have exactly the same software running, great. But you may have a dev who finds it much easier to use NPM than Yarn for some reason. And especially if you make open source software, you want to make it as easy for community members to get up and running as possible. Given that NPM and Yarn are simply tools for installing the same dependencies from package.json, they should just work. A dev should be able to see package.json, have a tool to interpret it, and not need to worry further. Which used to be the case before this lockfile conflict came alone.

Hey @nottrobin - I think I understand where you're coming from. If we ignore practical challenges and philosophical differences, it would definitely be better for the various tools' users if they all consumed the same lockfile (this is my personal opinion, of course).

But since we can't ignore said practical challenges and mostly seemed to have agreed the philosophical differences have their place, I think the compromise we came to (the one you outlined above) is definitely "good enough".

Do note that a hidden assumption in said compromise is that the tool choice is one made per project rather than per developer's machine or production environment.

Do note that a hidden assumption in said compromise is that the tool choice is one made per project rather than per developer's machine or production environment.

Yes I think this is what most concerns me - although I think in the scenario I outlined above it is still possible to have a project support both Yarn and NPM.

I do think that ultimately that if the two projects abandon the mission of trying to allow a project to be tool-agnostic then really they should stop sharing a dependency file. Yarn should switch to reading yarn.json rather than package.json. Anything else is just messy and confusing.

If npm lockfiles already have a superset of the dependency relationships that Yarn lockfiles support, why not support both in Yarn? Yarn could switch to package.json and any additional fields that may be desired by Yarn could be added in the future (similar to how some SVG editors like Inkscape manage editor metadata). Then Yarn could have the same dependency format as npm and be back compatible with yarn.lock, converting npm lockfiles to whatever structure Yarn wants in memory lossily.

I do think that ultimately that if the two projects abandon the mission of trying to allow a project to be tool-agnostic then really they should stop sharing a dependency file. Yarn should switch to reading yarn.json rather than package.json. Anything else is just messy and confusing.

Perhaps, perhaps not. Certainly, this would require a massive breaking change of the tool (yarn). Less radical changes with more direct and measurable benefits have been passed over.

I'm not saying it's a bad idea, I'm just saying I don't feel it's a practical one.

I'm not saying it's a bad idea, I'm just saying I don't feel it's a practical one.

I agree that it seems like a big ask. But what I'm saying is that this seems like a pivotal issue for the project.

Up until now, Yarn has been a tool that developers could choose to use instead of NPM to install Node dependencies for any project with a package.json. Now, as you point out, it's requiring projects to explicitly choose between Yarn and NPM. That's a huge change, and shouldn't be made accidentally. This is the crunch point to make that call.

To my mind, there are 3 ways forward:

1. Continue to be a drop-in replacement for NPM by finding a way to align Yarn with NPM on all project-level interfaces (standardise the lockfile)
2. Deliberately diverge from NPM by using explicitly different project-level interfaces (e.g. yarn.json and yarn.lock)
3. Double down on providing half of NPMs interface, and a half different interface. This is actually the same as point 2., but while looking to most people like point 1.

I believe the third option here confuses the whole Node space, and weakens both tools considerably.

It can still be back compatible. Yarn could just have a built in npm lockfile convertor so when it sees package-lock.json, it will keep it there and convert it to yarn.lock's format in memory. As far as I'm aware, npm can't do this because Yarn's lockfile has less information than npm's, so in my opinion it would be best for Yarn to standardize with npm.

@nottrobin I think you're mostly correct in your analysis, however:

Now, as you point out, it's requiring projects to explicitly choose between Yarn and NPM. That's a huge change, and shouldn't be made accidentally.

I think this has always been the case, or at least w.r.t. the primary benefit that Yarn was initially touted to bring: reproducibility of dependency trees. You can check in your yarn.lock, but that is practically useless if other developers add/remove dependencies without respecting that lockfile.

@nottrobin - I agree with @Vinnl. As I mentioned above, while I don't want to be in a position of telling anyone how they should work, I think using both yarn and npm for installing dependencies in the same project is an antipattern.

Even though both tools could technically put in a lot of work to make this work, I don't feel this is something we as maintainers should encourage. There are countless other benefits to having interchangeable lockfiles (as the various discussions in the different threads show) but I don't think this is one of them, personally.

but that is practically useless if other developers add/remove dependencies without respecting that lockfile.

Yes, I suppose to some extent Yarn was in that muddy water from the start - the existence of yarn.lock did already mean it partially had its own interface. I think that was always a bit of a messy thing to do, which served Yarn's purpose of wanting people to go from NPM to Yarn, but not back.

But it was a decision I felt more comfortable making for my project, because at least I knew that NPM was still entirely supported - since it didn't provide locking in the first place, it would continue to work as well as it always did.

Now that's changed, because NPM does lock dependencies. If I choose to bind a project to Yarn, I will now be keeping yarn.lock up-to-date, and not package-lock.json, so it is no longer true that someone can use NPM effectively on my project.

It sounds like you're saying that yarn is no longer intended to be a drop-in replacement for npm? That it's only supposed to be used by yarn-only projects?

I think if that's the case, then you owe it to everyone to make this fact clear on the yarn homepage - use this or npm, not both.

It sounds like you're saying that yarn is no longer intended to be a drop-in replacement for npm?

It never was. The interface somewhat mimicked npm to make it easier for users to understand how to use it, but from the very beginning some things worked differently. The main reason some people thought it was a drop-in replacement was that npm was just lacking the features to compare.

Now that npm is catching up on some aspect and decides to implement things differently, it simply starts to show that we have different approaches and make different decisions (for example the recent offline mirror feature npm implemented isn't compatible with our existing one). So in short: it never was "safe", it just happened to accidentally work.

I think if that's the case, then you owe it to everyone to make this fact clear on the yarn homepage - use this or npm, not both.

As a matter of fact, we do have migration instructions. Your remark made me notice that it unfortunately contains a wrong paragraph that might give people the wrong impression 🙁 We would gladly accept a PR to change this paragraph to something more in line with our recommendations (ie. to use Yarn consistently across teams, in order to make sure that everyone can benefit from the various features that might be used by the project).

It never was

err... you need to talk to your marketing people, then. https://yarnpkg.com/lang/en/docs/

Yarn interops directly with many features of npm, including its package metadata format, allowing for a painless migration.

We don't have marketing people, but we do accept nice PRs 🙃

In this particular case it doesn't sound too wrong though. We do interop with many features, just not with all of them, and the migration is painless in most cases (at worse it's a yarn import away).

migration is painless

i'm not really interested in migration. i'm looking for what was promised here (these guys _definitely_ have marketing people): https://code.fb.com/web/yarn-a-new-package-manager-for-javascript/

It has the same feature set as existing workflows while operating faster, more securely, and more reliably.

yarn today is not that.

AFAICT there are 4 classes of users here:

• those that _only_ use yarn in their projects,
• those that bought the facbook sales-pitch (above) but haven't yet hit any of these lock issues,
• those that are painfully trying to work around the incompatibilities by manuall converting lock files when necessary, or using other hacks to get it to work,
• those that just gave up on yarn and switched back to npm.

i really don't want to be in the last category, but it seems that's where i'm being pushed.

@Spongman what's stopping you from that last one? We can probably fix it ;)

@Spongman I'm not affliated with Yarn, so I guess I can be a little more blunt: there's really no point in stating in this issue that you think the wording is wrong. If you think the wording is wrong, go to that page in GitHub, click the Edit button, and submit a pull request with better wording. arcanis made it clear above that they're open to that.

(I guess you probably can't edit the blog post, but the website is probably the most important here.)

I can see from @imsnif and @arcanis' responses that the official position here seems to be that Yarn was "never intended" to continue to work seamlessly alongside NPM.

But I totally agree with @Spongman that this is the impression Yarn gave, and I really don't think that was an accident at the time. That was its genius - you could have improved speed, security etc. while still completely supporting the official NPM standards.

In either case, this issue makes Yarn's position on this much more clear for me than it was previously, and of course, Yarn's maintainers can choose to take it whatever direction they choose.

But I think you're drastically under-estimating the number of people who used Yarn precisely because they believed it maintained compatibility with NPM (at the project level), and would never have made the switch otherwise. I believe that the 254 :+1:s and 10 :heart:s on https://github.com/yarnpkg/yarn/issues/3614 and the 57 upvotes on "Should I commit yarn.lock and package-lock.json files?" make this abundantly clear.

If Yarn abdicates any responsibility on that front I think it will lose not only @Spongman and my teams, but many others besides.

Frankly I really don't understand the problem you have with using only Yarn, or only npm. What you're saying is basically "hey, I can't force my team to use Yarn, so I'll force them to use npm". That doesn't make any sense to me. If you use Yarn's features then make everyone use Yarn, and if you want to use the npm features, then make everyone use npm. It's as simple as this.

Any inbetween means that at the very least your builds are not consistent accross your team, which goes against the premise of Yarn, as mentioned before. One of your engineers could start using workspaces, and it would work, but would break on npm. Same of the offline mirror. Same for the dependency resolutions. Even worse, since some fields are entirely unknown by npm, it would silently do the wrong thing.

As for the communication, the fb blog post doesn't mention a drop-in replacement. Let me quote the part where Yarn is introduced. It literally says that it replaces the workflow. I guess you got confused by the "remains compatible with the npm registry", which is a fair point you should bring to npm, not to us (there's the npm cli, the npm registry, and of course npm Inc itself).

Yarn is a new package manager that replaces the existing workflow for the npm client or other package managers while remaining compatible with the npm registry.

what's stopping you from that last one? We can probably fix it ;)

@zkat That's helpful, thank you.

@nottrobin - I can't speak for yarn's original intentions because I wasn't around at the time. I was however working in a mixed yarn/npm environment with several dozen repositories.

I can say that it was perfectly clear to all developers involved back then that the choice of yarn/npm was a per-repo choice, just like the choice of express/hapi, mobx/redux, etc. This became even more clear as npm@5 came out with its own lockfile format and some developers decided to start using it with new repositories.

When a developer would install a dependency with the wrong tool, it would create a mess even before npm@5, because that dependency would not be consistently locked. It caused problems in our various environments and it was very clear to all involved that this was a mistake*.

I realize this might be confusing - and I understand this may not be 100% clear to everyone through no fault of their own, but I don't feel it's right to drastically change the tool to fit that misunderstanding. The way I see it, yarn is a drop in replacement for npm on a per-repo rather than per-box basis.

*Granted, that particular situation of multiple non-workspaced repos with mixed yarn/npm tooling was not ideal on a Human level rather than a tech level for exactly those mistake potentials and it was eventually fixed. I use this example here to show the two tools can work side-by-side if approached correctly.

Frankly I really don't understand the problem you have with using only Yarn, or only npm.

Yes, you @arcanis and @imsnif have made clear that you don't understand. The only point I'm making is that lots of people (look at the :+1:s) made the same "mis-"interpretation and want Yarn to work alongside NPM, regardless of whether you understand that or not. If Yarn is not the tool for us, so be it.

(Just one final point - @imsnif it's entirely ridiculous to compare a tool for installing Node dependencies to a project choice like express vs hapi, mobx vs redux. Those are fundamental characteristics of your app. If you can't see the obvious difference, no wonder you don't understand my point.)

Anyway, I'm done now. I think I've made my point as well as I can.

For open source projects its never a question of forcing "the team" to use yarn or npm. It's a question of what is most convenient for everyone. In the real world npm is king so, unless yarn can play nice in an npm world, then it's dead.

^ This

I'm afraid that's a misconception as well. During the past year, Yarn went from 20% of the total amount of requests to the npm registry to 40% in April. Last time I heard the stats (straight from the npm folks, since our stats recently broke when the npm registry went to Cloudflare), it was about 48% of the requests. In the actual real world, Yarn and npm coexist, plain and simple.

I'd now appreciate if we could go back to the topic at hand, which is: who's willing to write a PR to write a warning when we detect a package-lock.json file at install time? Thanks.

Psst, @arcanis - https://github.com/yarnpkg/yarn/pull/5920

Awesome :heart_eyes:

Between this and your work on yarn import, can we close this issue then?

I thought we'd wait for @jmorrell to give us some stats of how this affects the problem on Heroku so that we can decide if this is enough or we'd like to change something (warning/error, etc.) wdyt?

EDIT: afaik the warning has not been released yet, so we still have some time to wait.

I'm afraid that's a misconception as well.

How so? Volume of traffic at npm tells you nothing about whether or not the project is open source.

a more accurate assessment would be to count which github projects have 0,1 or 2 of (yarn.lock, package-lock.json). personally i have _never_ seen an open-source project (of any appreciable size) on github that has a yarn.lock file and _no_ package-lock.json (except the obvious).

IMO if you're going to keep separate lock files, then BOTH package managers should detect the other's lockfile and ERROR (possibly with an override flag).

All - I'd kindly ask that we stay on topic and take any non-directly-related comments offline (eg. our Discord channel). There are a lot of people subscribed to this thread and in fairness, I feel the npm<>yarn discussion does not belong here. Thanks!

I havent read the whole tread, but IHMO user should be at least notified that there is an ambiguity:

if npm sees a yarn.lock file then npm should print something like:
"WARNING: this project seems to use yarn, maybe you should use 'yarn' instead of 'npm install'"

if yarn sees package-lock.json file then yarn should print something like:
"WARNING: this project seems to use npm, maybe you should use 'npm install' instead of 'yarn'"

And as suggested above, a way to "prefer" package manager (in package.json).

I can say that it was perfectly clear to all developers involved back then that the choice of yarn/npm was a per-repo choice, just like the choice of express/hapi, mobx/redux, etc.

This was not clear to me at all. Up until now I thought it was a developer-local choice, and multiple developers could coexist on a single repo using whichever one they liked, while it being _a little more convenient_ to use only one consistently. The express/hapi etc examples are good though and make it clear that that's not a choice. This should have more visibility.

Add a field in package.json to indicate which package manager a project should use

"package-manager": "yarn"

I think this is the best solution, iff it's implemented in all package managers.

The package managers then MUST 100% refuse to proceed if they're called on the wrong project, just as you would expect an error if you required redux but then tried calling mobx functions on it. They should issue an error, not a warning, and inform the user how to proceed with the correct package manager.

Add a field in package.json to indicate which package manager a project should use
"package-manager": "yarn"

I think this is the best solution, iff it's implemented in all package managers.

If you want npm to consider this I encourage you to open a discussion in the appropriate venue, but I'll say that I don't like this direction. I would like to see convergence between package managers not divergence.

I would recommend against some new "package-manager" field, instead recommending the existing engines stanza which already has prior art.

Both the yarn and npm docs mention (in passing) usage of engines.npm and engines.yarn to specify the versions of each that are expected to be used. Yarn will even error out if the yarn version doesn't satisfy engines.yarn.

https://yarnpkg.com/en/docs/package-json#engines-
https://docs.npmjs.com/files/package.json#engines

Even if neither npm or yarn check engines for the "competing" manager (which would be nice), using this field is still a great way to communicate to other developers which manager is expected to be used. (if the existance of package-lock.json or yarn.lock isn't enough of a clue)

I would like to see convergence between package managers not divergence.

Agreed. Either convergence, or refusal.

but the current ambiguous, erroneous middle-ground is harmful.

I still think that just throwing an error in the presence of the others' lockfile would require the least friction for existing projects.

If you want npm to consider this I encourage you to open a discussion in the appropriate venue

Thank you, I've just signed up. There's a recent thread in this direction: https://npm.community/t/npm-install-should-warn-about-the-presence-of-yarn-lock-files/1822

My vote is for erroring out full-stop instead of just warning, so not sure if I should append to that thread of start a new one.

I would like to see convergence between package managers not divergence.

Same here. Though they are incompatible right now, so the divergence has already happened and is already causing friction to many users and projects.

Whatever convergence comes out of further talks amongst the teams is going to take time. When it does arrive, the proposed errors can be lifted and users can start switching and mixing package managers without a cinch.

Adding the errors now avoids further confusion and doesn't stop efforts in the direction of convergence.

I would recommend against some new "package-manager" field, instead recommending the existing engines stanza which already has prior art.

That sounds reasonable. I'm not attached to any particular method of detecting the presence of an incompatible package manager.

To add illustration, my use case in particular is gatsbyjs. The situation is the exact same as reported by @gaearon here:

To Create React App users, this is extremely confusing because their project gets created with Yarn (if Yarn exists on the system) but then they follow the docs of some library that tells them to npm install, and that blows away the whole tree.

To avoid this problem, even though gatsby uses yarn, it also adds a package-lock.json to default starters. But then users end up with both lock files and start seeing warnings. It's easy to just delete one of them, but it muddles up the onboarding experience.

Thanks everyone for your input.

@jmorrell - https://github.com/yarnpkg/yarn/pull/5920 was released in 1.9.2 on Jul 25. Been a little over a month, I know it's not a lot of time (and surely not everyone has upgraded) - but do you maybe have some insights to share regarding the double-lockfile errors on Heroku since then?

@imsnif Thanks for the email reminder! Sorry for missing the update here.

Here is a graph of the number of apps that experienced this particular failure each week in 2018.

(I've redacted the y scale, but this represents 100's of developers and shows the trend)

1. The dip in July comes from an unrelated issue with S3

2. Looking at the trend after July 25, the warning seems to have had a substantial effect on the number of people experiencing this error. I am somewhat surprised by the magnitude of the shift.

@jmorrell - this is really great!
I'll admit I'm also quite surprised at this.

I think (on the yarn side) this issue can be considered resolved. Do you agree?

i still think the advice of removing package-lock.json to fix the error is misleading - this will in general result in a different set of package versions being used, and will (again, in general) lead to incompatibilities and breakages. also, this advice is only useful for those who make decisions about which package manager a project is using. general contributors following this advice are also going to run in to issues.

IMO if a package-lock.json exists and yarn can't guarantee that the versions it installs are the same as those that npm would install, then it should fail with an error.

Thanks again for your input @Spongman. I feel this has all been well discussed in the (very long) thread above us and I don't think it makes sense to restart it again. I think we all understand your position. Thanks for participating.

Unless @jmorrell tells me otherwise in the next week or so, I'm going to consider this done and close this issue.

@imsnif Taking a different track and looking at failures from September so far (Sep 1 - Sep 20), this is still the number 1 reason that Node builds on Heroku fail. It happens twice as much as the next most common known failure case: package.json being invalid JSON. Once again redacting exact numbers, here is how it stacks up to other issues.

I haven't seen any movement from npm on this, so I'll create a PR adding a warning there too and see it gets accepted.

I think (on the yarn side) this issue can be considered resolved. Do you agree?

I don't think this has been resolved for users, and the user experience still isn't great. But I can see how the numbers change once npm also gives feedback to a user hitting this case.

I'll leave it up to you whether you want to close this or make further changes. I'm happy to open a new issue in a few months if this is still a big problem for users.

@jmorrell - seeing as this issue is attracting a lot of attention and the conversation keeps restarting and even repeating itself, I think it would be better to close it now with the information we have.

If, with npm providing a warning as well, we still see an issue, I'd definitely be willing to start the error discussion again (or look into other solutions). Please, do ping me personally in that case.

Thanks for bringing this up, contributing and making all these changes happen! I personally feel yarn is better after this issue.