Yarn: Extracting the tar content of undefined failed

Same issue, it's blocking my CI too, we updated recently to yarn 1.9.2

@opiation The error is indeed random but I may have found the cause: do you have distant git URLs in your package.json without .git at the end? We had two of those and adding .git fixed the issue. Not sure why the error message doesn't directly state this is is the problem though.

Also, maybe related: https://github.com/yarnpkg/yarn/issues/5536

@adrienharnay, can you define what you mean by _distant_? For the record, here's the package.json I used. There's only one github dependency and I still get errors without it. I'm not sure how I would append .git to non-git dependencies, unless I've misunderstood your suggestion.

Distant wasn't the right word, I just meant packages installed from Git 🙂

Could you try this?

"storybook-addon-markdown": "https://github.com/mihalik/storybook-addon-markdown.git"

As per my previous comment, I still encounter the issue without storybook-addon-markdown dependency. Thus, I don't believe this issue stems from yarn improperly handling git URLs.

Indeed, I read too quickly. Well, that fixed our bug, but I have no idea about yours 😕 Sorry

@opiation Did you update you yarn.lock file too? Cause I had to do that

@Titozzz, I encounter this error with and without the yarn.lock file. I have deleted and recreated the lock file several times to no avail.

I get this too and I dont have any packages from git.

I wanted to get around this issue (https://github.com/yarnpkg/yarn/issues/6256) by using the tarball versions of the packages but indeed the error above is thrown for tarball urls on self-hosted Github enterprise.

github.com hosted tarballs somehow did work. e.g.
https://github.com/luwes/chameleon/archive/grasshopper-v0.0.1-beta.4.tar.gz

I'm seeing the same issue with a project we have. However when I remove the deps that run a prepare script as part of the install (due to being git urls) then it works. These happen to be pointing git urls, but I think it's actually the prepare that seem to kick off more yarn install processes that seem to subvert the mutex flag for some reason. I wonder if that's because the other processes are kicked off by the root process rather than by different root processes. I don't know if this information helps or if its actually way off base. But I figured I would share what I've found regardless.

@khendry I got the issue again, and you are right, it comes from git dependencies which have a prepare script in their package.json! :+1:

I've been tracking this down with a project we have and so far narrowed it down to the concurrent install the git-fetcher starts here. If the package that is being installed by the git-fetcher has any of the same dependencies of the currently installing package a race condition is created where the duplicated packages will be untarred to offline cache at the same time.

I haven't seen enough of the code base to understand where/what the correct fix is, but that's the start of the issue.

Any news on this? We're facing this issue too.

Same problem. It's impossible to use yarn with CI. Every 2nd build failed with this error 😞

try delete node_modules,

yarn cache clean
yarn install --network-concurrency 1

Thanks for sharing this. It's at least a workaround 🤗 , but no real solution if you want your build time reasonable fast 😅

We did try using the --network-concurrency flag with no success. So that does not actually resolve this particular issue. The flag addresses concurrency at a higher level than where the issue occurs.

For me --network-concurrency 1 solves the problem. I know it temporary fix, but it works. But value must be exactly 1.

I spoke too soon. I had asked my team mate if we'd tried this, rather than actually trying myself and he was _very_ confident that we had...he was mistaken and miss-read the previous post thinking it was related to the mutex flag, not network concurrency. We've since re-tried and can confirm this does seem to also address the issue for us.

setting --network-concurrency 1 does not actually work for me.

right now, the only workaround I've come across involves completely regenning yarn.lock. The error I get is:

2.054 Performing "GET" request to "https://<private-artifactory-npm-registry>/@myorg/eslint-plugin-import/-/@myorg/eslint-plugin-import-3.0.0.tgz".
verbose 2.519 Error: https://<private-artifactory-npm-registry>/@myorg/eslint-plugin-import/-/@myorg/eslint-plugin-import-3.0.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
    at MessageError.ExtendableBuiltin (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:237:66)
    at new MessageError (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:266:123)
    at Extract.<anonymous> (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:59446:14)
    at emitOne (events.js:121:20)
    at Extract.emit (events.js:211:7)
    at Extract.module.exports.Extract.destroy (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:135306:17)
    at Extract.module.exports.Extract._final (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:135364:34)
    at callFinal (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:70270:10)
    at _combinedTickCallback (internal/process/next_tick.js:139:11)
    at process._tickCallback (internal/process/next_tick.js:181:9)

Update: I've just found that using --skip-integrity-check allows me to bypass this error. While obviously that's really a solution. This looks like kind of an important bug in the integrity check logic.

Im using [email protected], [email protected]

@arcanis @rally25rs more details on this error:

So, this seems pretty odd to me that it's failing integrity checksum, considering the sha1's are the same:

Error: sha1-Sl7Hxk364iw6FBJNus3uhG2Ay8Q= integrity checksum failed when using sha1: wanted sha1-Sl7Hxk364iw6FBJNus3uhG2Ay8Q= but got sha1-AHoWKXweP+Pg9aZkGBsAjFruGaM=. (77 bytes)
    at Transform.on (/Users/shargrove/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:32831:19)
    at emitNone (events.js:111:20)
    at Transform.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:139:11)
    at process._tickCallback (internal/process/next_tick.js:181:9)

Update: After seeing above, I confirmed that --skip-integrity-check bypasses this error. Looks like a more serious bug in integrity check logic.

@opiation out of curiosity, can you paste your package.json? Are you using the following "override" technique anywhere?

"dependencies": {
  "foo": "npm:@myorg/foo"
}

For ex, I'm using it:

"eslint-plugin-import": "npm:@myorg/eslint-plugin-import",

And, this is the package I see the error for.. So, I wonder if this is related?

@hulkish, as per my initial post, here's the gist I created with my package.json, yarn.lock and all the tests I ran that all produced the error described. To clarify, each line in failing_test.sh can encounter this error but not consistently. They may have to be tried more than once to encounter the error. Just to have it in this thread, I'll summarize each test below:

Failing tests

• yarn install
• yarn install --frozen-lockfile
• yarn install --pure-lockfile
• yarn install --mutex network
• yarn install --network-concurrency 1
• All of the above tests with rm yarn.lock beforehand
• All of the above tests in a node:alpine container with git installed (alpine at the time this thread was created)
• All of the above tests in a node:8-alpine container with git installed

As for the "override" technique, I am uncertain what you mean. If you are referring to the _protocol_-like prefix in the dependency value (like npm: in your example), then yes, one dev dependency uses a github package:

"storybook-addon-markdown": "github:mihalik/storybook-addon-markdown"

The errors are still encountered however even when I remove the dev dependency so this does not seem related.

Shout out to @holyxiaoxin - adding --network-concurrency 1 resolved this for my CI 👍

ping @imsnif ? It seems related to the integrity check, per @hulkish's comment

@khendry Not using prepare anymore on our git dependencies resolved this for our ci, while --network-concurrency 1, --child-concurrency 1 and --skip-integrity-check were not enough

We were able to fix this with npm config set always-auth true (as documented here). As best I can tell, npm by default will provide your credentials _only_ for publishing packages, not for fetching them. For some reason yarn previously didn’t respect that setting, but it now does.

I recently had this issue, using yarn 1.12.3 and node 10.13.0. After trying many of the solutions above, to no avail, deleting/regenerating the yarn.lock file worked.

I've been getting a similar issue. Removing the yarn.lock like @mvonballmo suggested was the only thing that got it running. It's still not fully working though.

yarn install v1.12.3
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/iconv-lite/-/iconv-lite-0.4.23.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOSPC: no space left on device, write"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

yarn install v1.12.3
info No lockfile found.
[1/4] Resolving packages...
warning celebrate > joi > [email protected]: This version is no longer maintained. Please upgrade to the latest version.
warning xo > eslint > file-entry-cache > flat-cache > [email protected]: CircularJSON is in maintenance only, flatted is its successor.
[2/4] Fetching packages...
error https://registry.yarnpkg.com/babel-runtime/-/babel-runtime-6.26.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOSPC: no space left on device, write"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Hi friends,

So, judging by the different errors reported here - this actually seems like it can potentially be several different issues:
ENOSPC: no space left on device, write,
wanted sha1-Sl7Hxk364iw6FBJNus3uhG2Ay8Q= but got sha1-AHoWKXweP+Pg9aZkGBsAjFruGaM= (btw, takes a hard look, but these are not the same),
the file appears to be corrupt: "Unexpected end of data", etc.

While I appreciate these can be happening in a similar place, they might be caused by entirely different issues and/or environments. The integrity check (specifically the untarStream on error callback - thanks for the detailed debug @hulkish!) is a funnel that can gather many errors, and it's a little hard to give feedback beyond the actual error to the user.

The above is especially true for the integrity migration (populating an old-style yarn.lock with the new integrity fields), since this one-time process (one time assuming it succeeds ofc) is more network intensive than a normal install (it loops through all packages without an integrity field and fetches their registry manifest).

The theories about a race condition are interesting and very definitely a possibility, I'd be happy to look into them further. I'm afraid though that @opiation 's reproduction did not work for me. I'm now running my 7th local installation of it and it still works without issue (I did not run the script, rather I just ran yarn to install it with that package.json and yarn.lock - I understand this still caused the issue for you?)

@opiation - can you still reproduce this issue? Under the same conditions? Maybe we can go down a level of resolution and you can tell me everything you do down to the commands to make this happen?

Anyone else on this thread has a setup they can share that reproduces this issue even partially consistently? I'd be very happy to get to the bottom of this.

I encountered the same error message in my CI system:

2018-11-12T04:32:13.0386630Z error https://pkgs.dev.azure.com/JeremyTCD/_packaging/Main/npm/registry/cheerio/-/cheerio-0.22.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
2018-11-12T04:32:20.4838361Z 
2018-11-12T04:32:20.4852626Z     yarn install v1.12.3                                                                    
2018-11-12T04:32:20.4853491Z     [1/4] Resolving packages...                                                             
2018-11-12T04:32:20.4855400Z     [2/4] Fetching packages...                                                              
2018-11-12T04:32:20.4856037Z     info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Managed to figure it out my particular issue though. Thought I'd leave a note here for anyone who encounters something similar:

Cause

I called yarn install on my local machine after adding a new dependency to my project ([email protected]). Due to a local .npmrc, yarn restored the dependency from a private registry of mine. The generated yarn.lock contained the following lines:

[email protected]:
  version "0.22.0"
  resolved "https://pkgs.dev.azure.com/JeremyTCD/_packaging/Main/npm/registry/cheerio/-/cheerio-0.22.0.tgz#a9baa860a3f9b595a6b81b1a86873121ed3a269e"
  dependencies:
  ...

Note how the package was resolved from a private repository. On my CI machine, I didn't have a .npmrc with the credentials for the private registry. This was the cause of the error message:

https://pkgs.dev.azure.com/JeremyTCD/_packaging/Main/npm/registry/cheerio/-/cheerio-0.22.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"

I fixed my local .npmrc and regenerated my yarn.lock:

[email protected]:
  version "0.22.0"
  resolved "http://registry.npmjs.org/cheerio/-/cheerio-0.22.0.tgz#a9baa860a3f9b595a6b81b1a86873121ed3a269e"
  integrity sha1-qbqoYKP5tZWmuBsahocxIe06Jp4=

Note how the package is now resolved from the default NPM registry. The error stopped ocurring once I did this.

Fix

If the cause of your issue is the same as mine, you can:

• Add required credentials to your CI machine or
• Tweak your local .npmrc (yarn config list will print the registry that yarn restores from), then regenerate yarn.lock.

Notes

Perhaps the error message could be more specific.

Edit: Initially thought rolling back Yarn would solve the issue - accidentally linked my faulty commit to this issue. Yarn wasn't the issue in the end.

TL;DR: try deleting your yarn.lock file and generating it again.

I got an error when trying to build on Netlify: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"

Deleting the node_modules folder and the yarn.lock file and then generating them again via yarn install got me a new yarn.lock file with different dependencies. With this new file Netlify succesfully built my project.

@imsnif agreed that there are seemingly multiple different issues being reported here. I believe I have a repro case from a project I'm working on that triggers the problem outlined by @khendry here

I'm seeing the same issue with a project we have. However when I remove the deps that run a prepare script as part of the install (due to being git urls) then it works. These happen to be pointing git urls, but I think it's actually the prepare that seem to kick off more yarn install processes that seem to subvert the mutex flag for some reason. I wonder if that's because the other processes are kicked off by the root process rather than by different root processes.

Sharing repro steps below in the hopes it allows you to reproduce the problem. Let me know if you need more information.

Repro steps

1. With node v10.3.0 and yarn v1.12.3, in a new test folder, download the package.json and yarn.lock from this gist
2. run rm -rf ~/.cache/yarn* node_modules/ && yarn install --frozen-lockfile --network-concurrency 16 (clear cache and previously install node modules for reliable environment. Set concurrency high to increase chances of hitting issue)
3. observe output like the following:

yarn install v1.12.3
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning Pattern ["object-assign@latest"] is trying to unpack in the same destination "/home/ocderby/.cache/yarn/v4/npm-object-assign-4.1.1-2109adc7965887cfc05cbbd442cac8bfbb360863/node_modules/object-assign" as pattern ["object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4","object-assign@^4.1.1","object-assign@^4.1.0","[email protected]","object-assign@^4.1.0","object-assign@^4.1.1","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.1","object-assign@^4.1.1","object-assign@^4.0.1","object-assign@^4.0.1","object-assign@^4.1.0","object-assign@^4.0.1","object-assign@^4.0.1","object-assign@^4.0.1","object-assign@^4.1.0","object-assign@^4.0.1"]. This could result in non-deterministic behavior, skipping.
info No lockfile found.
[1/4] Resolving packages...
warning eslint > file-entry-cache > flat-cache > [email protected]: CircularJSON is in maintenance only, flatted is its successor.
warning jest > jest-cli > prompts > [email protected]: Please upgrade to kleur@3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit <https://github.com/lukeed/kleur/releases/tag/v3.0.0\> for migration path(s).
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.4.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/home/ocderby/.cache/yarn/v4/npm-lodash-4.17.4-78203a4d1c328ae1d86dca6460e369b57f4055ae/node_modules/lodash/_shortOut.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Other notes

I've tried a variety of things, here are my notes:

1. The issue does not reproduce 100% of the time for me. As noted above, increasing the network concurrency used seems to make the issue more likely to occur.
2. Using a version of react-textarea-autosize published on the package registry makes the problem go away (seeming to confirm what @khendry reported above)
3. Setting --mutex file doesn't seem to help here at all
4. As reported above, if I limit network concurrency to 1 (via --network-concurrency 1 argument), everything installs properly, albeit slower.
5. I've reproduced this on node v8.12.0, with both yarn v1.9.4 and v1.12.3. This was running on the circleci/node:8-stretch docker image running on Circle CI 2.0.

I start seeing this error recently after updating yarn to 1.12.3.
See my travis-ci build failure https://travis-ci.org/ankurk91/vue-cleave-component

$ yarn install --non-interactive
yarn install v1.12.3
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
error https://registry.yarnpkg.com/har-validator/-/har-validator-5.1.2.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command "yarn install --non-interactive" failed and exited with 1 during .

It is happening with only with [email protected].
I will post back if i got success somehow.
PS.
It was specific to the har-validator package.

I also get
error https://registry.yarnpkg.com/har-validator/-/har-validator-5.1.2.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
with curl i got 404 for https://registry.yarnpkg.com/har-validator/-/har-validator-5.1.2.tgz
but in my browser i can download it.
One of my server if i downgrade yarn to 1.12.1 it start to work but on the other server even if i downgrade the error remains the same (I remove yarn cache dir in both case).
Is that possible that is some kind of cloudflare (config) issue?

No, this specific instance (yours and the one by @ankurk91) are caused by har-validator having been unpublished (cf #6694).

I get this error in my CI environment only, after having added another repo as a dependency ("@team/myproject": "git+ssh://[email protected]/team/myproject.git#master",) . I can confirm that

• adding --network-concurrency 1 to my CI script solves the issue, but of course makes the build very slow
• running yarn install --network-concurrency 16 provokes the error locally as well

Neither cleaning cache nor resetting yarn.lock made a difference for me

EDIT: It seems the --network-concurrency 1 fix is not consistent, unfortunately 😢

same error here,
easy to reproduce:
yarn upgrade typescript@^2.8

then:
yarn upgrade [email protected]

I did ctrl+c while installing this last package.. and when I try 'yarn upgrade' again I get:

yarn upgrade v1.12.3
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error https://registry.yarnpkg.com/typescript/-/typescript-2.8.4.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, stat '/Users/u/Library/Caches/Yarn/v4/npm-typescript-2.8.4-0b1db68e6bdfb0b767fa2ab642136a35b059b199/node_modules/typescript/lib/lib.d.ts'"
info Visit https://yarnpkg.com/en/docs/cli/upgrade for documentation about this command.

UPDATE: The below was due to corrupt metadata in our Sonatype Nexus install, and is hence not a Yarn issue. Leaving for context.

Seeing this for multiple packages in our CI environment. Yarn 1.12.3 and Node 11.1:

responsive-props-1.2.2.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?"

styled-components-breakpoint-2.1.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?"

I had a similar problem but I get ....the file appears to be corrupt: "EBUSY: ...".
I cleared my whole yarn cache an re-ran it and still got the same error so it seems like yarn is creating files and locking them for itself.

This is on Windows 10.

yarn install v1.10.1 [1/4] Resolving packages... [2/4] Fetching packages... error https://registry.yarnpkg.com/fbjs/-/fbjs-0.8.17.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "EBUSY: resource busy or locked, open 'c:\\src\\yarn\\cache\\v2\\npm-fbjs-0.8.17-c4d598ead6949112653d6588b01a5cdcd9f90fdd\\lib\\UserAgent.js'" info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

I did a workaround by running "yarn --pnp" which worked. Strange since that should be newer and probably more unstable code.

Removing yarn.lock was what got it working for me.

Hey everyone, just had the same issue. Resolved by removing .npmrc from home dir.

rm ~/.npmrc

@binchik - this is the only thing that worked for me.

Thank you @binchik, that did the trick for me. 👍

So after rewinding back to the series of events leading up to yarn failing, I believe I executed an npm script in a package.json that was something like this:

"audit": "npm audit"

Which is totally silly, because I never use npm in that project. After this command everything (including npm) would just start to have random faults and never complete, in line with the experience of others in this thread.

If someone reproducing the error could investigate and figure out what exactly is causing the problem it would be super helpful! I've tried but I can't reproduce it 🙁

Some pointers:

• We need to figure what goes into untarStream when it fails - my hypothesis is that maybe we're trying to process a json response as a tarball (https://github.com/yarnpkg/yarn/blob/master/src/fetchers/tarball-fetcher.js#L146-L150)

• the only thing I'm think that could matter in the .npmrc is the auth token. I would appreciate if someone could confirm that the problem disappear simply by removing the auth token line from the .npmrc (rather than the whole file)

FWIW, I encountered this problem today. A few things:

• Removing .npmrc fixed it. The only thing in the file had to do with the auth token.
• npm install failed too, and it logged a 401 unauthorized error.
• After removing the .npmrc file, npm install worked again.

@deleteme per my findings, this sounds more like a byproduct of the bug, rather than the cause.

I've encountered with and without a .npmrc or .yarnrc

Given that this problem suddenly appears much more than usual and that it's during the npm registry being especially flaky, it's quite likely my hypothesis isn't far off

@arcanis just started having this problem today. I can confirm that by removing that npmrc auth token line the issue was resolved.

There's no ~/.npmrc created in my case. But regenerating yarn.lock worked for me.

Simply,

$ rm yarn.lock && yarn

EDIT: Faced this issue twice only to end up landing here. :smile:

In my case, I use CircleCI, circleci/node:10.11.0 docker image, and [email protected], and there is no ~/.npmrc. Thank you @achillesrasquinha. It works for me.

I have been facing this issue over a week. yarn install --network-concurrency 1 solved issue but it is very very slow.

In btw, this info could be helpful to anyone.
I was using a custom npm package (in house) in my project. Always I am getting same issue like .cache/v4 but showing different package names every fail. After spending much time, I find one random observation.
My project and custom npm package is using same yarn build for build the bundle. I have updated my custom package build script name to another name as yarn build:p. Then it is start working. I ran build many times. It was not failed. Not sure How these 2 are dependent but worked for me.

Removing .npmrc only didn't do it for me. I also had to remove my yarn.lock file like @davidalee mentioned. I don't know why he's getting thumbs down for it 🤷‍♂️

Not sure if removing .npmrc had any effect for me.

I'm not really fan of deleting the yarn.lock file so what I did is just remove the har-validator package from the yarn.lock and then re-run yarn which fixed the issue for me.

Yes rm yarn.lock work for me. Facing issue with package har-validator-5.1.2.

error https://registry.yarnpkg.com/har-validator/-/har-validator-5.1.2.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"

Hi, har-validator-5.1.2 was unpublished from npm as stated here https://github.com/ahmadnassri/node-har-validator/issues/112#issuecomment-437378269 , thus you need to upgrade your dependencies via yarn upgrade(this has probably the same effect as removing yarn.lock that was recommended by the others).

I suppose this issue can be closed.

Removing yarn.lock did not work for me, as mentioned in my initial issue report. Nor did removing .npmrc. Besides, as far as I am aware, the node:10-alpine docker image does not have or create a .npmrc file.

Lastly, the error is not limited to the har-validator package. In fact, I've never encountered it with that package. I did encounter it with packages lodash, fbjs, react, and a host of others.

I summarized my attempts that still reliably reproduce this issue in a previous comment. For the record, when testing with docker, I can reproduce the problem including only the package.json thus, no yarn.lock, no .npmrc, no node_modules. I can still reproduce this issue on my local machine, on a GCE instance and with Gitlab.com's CI. Neither --network-concurrency=1 nor --skip-integrity-check seem to resolve the issue for me. Thus, I would hesitate to recommend closing this issue, especially since all the above-mentioned tests work using npm install, assuming that yarn install should be a drop-in replacement for npm install given the same package.json.

The problem is that the npm registry is generally unstable and returns errors (at a higher rate when multiple requests are firing apparently - maybe some kind of per-ip throttling?). For some reason they are not properly caught by Yarn, which blindly tries to hash them and compare them to the expected hash - which fails.

So there is a bug in Yarn (we should print a more helpful error), but given that the real problem is how flaky the npm registry is, it's not my priority at the moment (I'd definitely review a PR, though!).

As for why it doesn't happen with npm: they retry their requests until they work. Yarn has a mechanism to do that, but not on the part that specifically compute the hash.

I'd suggest to use an offline mirror to stop relying on the npm registry for your installs.

https://github.com/yarnpkg/yarn/pull/6817 will "fix" that by showing the actual status code returned by the registry. I'd prefer it to be stable rather than blindly retry until it works so I haven't added the retry code, but if there's no improvements to the horizon we might have to do that.

In the meantime I'll close this discussion, as the error messages will change and this thread grown quite large (we can open new ones to discuss each status code individually).

There's no ~/.npmrc created in my case. But regenerating yarn.lock worked for me.

Simply,

$ rm yarn.lock && yarn

thank you,
rm -rf ./yarn.lock && yarn
it's work for me!

in case it helps anyone:

• this same error occurred for me, when I had forgotten to log in to npm (doh!)

For me the issue was resolved with service docker restart (Ubuntu 18.04).

I've been experiencing intermittent and non-deterministic errors like this one. I restart my build, nothing else having changed and it works. Does anyone have any alternatives to yarn?

I starting getting this same error on every build (errors on a different npm module each time) after making a PR to update our base docker image from node:8.12.0 to node:8.13.0. I inspected these node docker images and discovered that the pre-installed yarn version was changed from v1.9.4 to v1.12.3. See: associated git commit. I tried some of the suggested fixes in this thread with no luck in resolving the error. I was able to fix the issue by simply downgrading the yarn version in my Dockerfile to v1.9.4. I know this version of yarn has been problematic for others, but for me the more recent yarn version is triggering the issue. I'll note that I am using an .npmrc file that provides credentials to access private modules via jfrog artifactory and we have artifactory set up to mirror/proxy all npm modules.

Why is this closed? Still breaking CI

In the meantime I'll close this discussion, as the error messages will change and this thread grown quite large (we can open new ones to discuss each status code individually).

I'll go ahead and lock this thread since it seems to me it reached past its usefulness. As a reminder:

• If you have this error message, you are very likely using an old release. Upgrade to 1.13+ to get the true error message. Chances are the registry is returning an HTTP 500 for some reason.

• If you still get errors that seem to come from Yarn itself open a new thread and detail how to reproduce the problem. If you don't provide a reproduction we won't be able to provide a fix, and will likely have to ask you to investigate yourself.