I'm having linking dependancies take 200+ seconds with this https://github.com/macdja38/pvpsite/blob/master/package.json on Windows 10, off an SSD with a decent i7.

It seems the performance issue may be caused by windows Defender, disabling it while possibly inadvisable reduced 200s to somewhere closer to 50.

I think that there should be a better solution than reducing security.

Some other users have confirmed that disabling it just in the root directory of your project works, but I can't confirm as Windows Defender broke itself a bit when I tried to do that.

I've got the same with issue with git repo with about 30 dependencies.
Windows 10
node v5.5.0
yarn 0.16.1

Disabling Windows Defender reduced linking time significantly

Probably should be "solved" by this PR?

Yeah there's not a lot we can do here unfortunately :( Virus scanners scan all files, and the npm ecosystem has a lot of small files. Small files generally have a bit more overhead on NTFS compared to other file systems such as EXT4 or ZFS, but it's exacerbated by virus scanners.

Having said that, Yarn should _still_ be faster than npm, it just won't be as fast as on Linux or Mac.

I am on a Mac without having any antivirus scanners installed. But I still see the same problem, linking taking considerable amount of time even with a simple angular-js app.

I have this issue too. It took 174s on Ubuntu.

I started having this problem only after I upgraded from 0.17.8 to 0.17.19. Mac without antivirus.

Strange thing as well is that I need to go throw linking process each time when I delete a package as well. Npm does that faster. And linking really takes long time.

This can be reproduced with this package.json (on Heroku):

{
  "name": "yarn-link-slowness",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "axios": "^0.15.3",
    "lodash": "^4.17.2",
    "react": "^15.4.1",
    "react-dom": "^15.4.1",
    "react-player": "^0.12.1",
    "react-redux": "^4.4.6",
    "react-router": "^3.0.0",
    "react-router-redux": "^4.0.7",
    "react-scripts": "^0.8.4",
    "redux": "^3.6.0",
    "redux-auth-wrapper": "^0.9.0",
    "redux-logger": "^2.7.4",
    "redux-promise-middleware": "^4.2.0",
    "redux-thunk": "^2.1.0"
  },
  "engines": {
    "node": "7.2.1",
    "yarn": "0.17.8"
  }
}

With yarn 0.17.8, the install takes 37s. With yarn 0.17.10, the install takes 97 seconds. No other changes (new Heroku apps each time).

✨ Done in 45.10s.

    "autoprefixer": "6.3.6",
    "babel-core": "6.7.6",
    "babel-jest": "13.0.0",
    "babel-loader": "6.2.4",
    "babel-plugin-transform-class-properties": "6.9.1",
    "babel-plugin-transform-object-rest-spread": "6.8.0",
    "babel-preset-es2015": "6.6.0",
    "babel-preset-react": "6.5.0",
    "bluebird": "3.3.5",
    "cardmask": "github:aj0strow/cardmask#v1.0.0",
    "chai": "3.5.0",
    "classnames": "2.2.5",
    "copy-webpack-plugin": "2.1.3",
    "core-js": "2.4.1",
    "css-loader": "0.23.1",
    "enzyme": "2.3.0",
    "file-loader": "0.8.5",
    "force-case-sensitivity-webpack-plugin": "0.1.1",
    "jest": "13.0.0",
    "jest-cli": "13.0.0",
    "json-loader": "0.5.4",
    "lodash": "4.11.1",
    "moment": "2.13.0",
    "ms": "0.7.1",
    "node-sass": "3.4.2",
    "postcss-loader": "0.9.1",
    "raw-loader": "0.5.1",
    "react": "15.2.0",
    "react-addons-css-transition-group": "15.2.0",
    "react-addons-test-utils": "15.2.0",
    "react-css-transition-replace": "2.0.1",
    "react-dom": "15.0.1",
    "react-redux": "4.4.5",
    "react-router": "2.3.0",
    "react-textarea-autosize": "4.0.3",
    "recompose": "0.20.2",
    "redux": "3.5.1",
    "redux-actions": "0.10.0",
    "redux-thunk": "2.0.1",
    "reselect": "2.5.3",
    "sass-loader": "3.2.0",
    "sinon": "1.17.4",
    "style-loader": "0.13.1",
    "webpack": "1.13.0",
    "webpack-dev-server": "1.14.1",
    "whatwg-fetch": "1.0.0",
    "zxcvbn": "4.3.0"

Please, can anyone explain what yarn does in "Linking dependencies" step exactly?
Because max number on this step vary from ~1000 to ~65000 for same project on different machines. What does this number mean?

Having this issue as well. Adding a dependency with yarn add seems to trigger "Linking dependencies" and it takes forever. Had to switch back to npm for now.

node: 6.9.2
OS: Windows 10

node: 7.3.0
OS:Windows 10 64
Same for me

Same here. Links 23420...something, and takes about one and a half minutes on a good day.

Yarn 0.19.1
NodeJS 7.3.0
Windows 10

yarn add moment takes 105 seconds. It has no dependencies. :/

EDIT: Turning off Windows Defender does reduce time to ~30 to ~50 seconds. I tried excluding the directory I'm working in, but that didn't seem to help.

Just ran a fresh copy of create-react-app and it took 876.37s. I understand you don't have much/any control over how anti-virus works but my experience with NPM and CRA was much faster on Windows.

On windows 10 just use Ubuntu bash shell as a general advice.

On windows 10 just use Ubuntu bash shell as a general advice.

Disk I/O is extremely slow in Windows Subsystem for Linux, it's a known limitation at the moment

but my experience with NPM and CRA was much faster on Windows.

@JeffBaumgardt - Interesting... Yarn is slower on Windows, but it should still be faster than npm!

@Daniel15 it probably should be, but it's not. Node installs and de-installs were smaller for me. So I would do npm add <packages> --save-dev, delete yarn.lock and run yarn and that be fast than running yarn add <packages> -D a single time. Now hat everyone is on yarn, of course I don't want to delete the lock and force everyone to upgrade their bundle. Instead the following has been great:

cc @echobnet

For anyone on windows 10 + windows defender

1. First click settings

   

2. Scroll down to exclusions

   

3. Run yarn cache dir to get the location of your cache folder

   • Add this cache folder to the exclusion list
   • Add your project node_modules folder to the exclusion list

4. Speed-up for a react project x 3-10

@SleeplessByte or you can simply add yarn and node to excluded processes.

Not just a problem on windows. I've been seeing horrendous link times on my Mac Pro.

OS: OS X 10.11.6 (El Capitan)
Node: 7.6.0
Yarn: 0.20.3

I can confirm that it is _very_ slow on Mac 10.12.3, too. Not related to windows.

And it seems my setup is _way_ slower than others in this thread. yarn sometimes tries to link around 600.000 files in small projects. This can take more than 30 minutes. I currently try it with a clean cache and nightly (v0.22.0-20170226.1626). I use the official registry as well as a custom private registry for certain scoped packages. However my colleagues don't suffer from this problem, so I don't think the custom private registry is the problem (and fetching packages is already finished anyway). We also have some relative files with path: protocol in our package.json.

I have a lot of problems installing https://github.com/google/material-design-icons which has _a lot of small files_ which seem to be troublesome for other people as well (https://github.com/google/material-design-icons/issues/518). I don't know if my hardware is broken handling a lot of small files or something like that or if this is related at all. Again my colleagues don't have as much problems installing https://github.com/google/material-design-icons as I do.

Update

I'm not sure... it _looks_ like installing a package with file: puts a module in the cache containing node_modules/ and other stuff. This is _really_ a problem if you have multiple examples all containing their own node_modules/. It seems .npmignore, etc. is ignored for file: installs. This probably boils down to https://github.com/yarnpkg/yarn/issues/2165, if the solution is to _not_ cache locally resolved files at all. If I open my cache ($ yarn cache dir) and look for modules why where install with file: and they contain a node_modules directory or other big directories, I can speed up the linking phase by removing these directories manually. Now everything seems to install with good speed.

[3/4] Linking dependencies...
Done in 947.71s. 

Had to wait this amount of time adding any new package with yarn add ...
Win7 /w Yarn v0.21.3
Got material-design-icons package in my app.
I think this one is related https://github.com/yarnpkg/yarn/issues/990

@kuncevich everything works fine on my end, during runtime of you yarn add start task manager ctrl + alt + esc check which program uses the highest cpu, in my case it was antivirus so I had to exclude not only the %appdata% directories but also the the project directory and things became fast again

@kuncevic you might be affected by the bug I found on windows : https://github.com/yarnpkg/yarn/pull/2958

Essentially yarn can always copy every file in node_modules for every operation.

@asolopovas in my case it is node.exe like 10-26 %:

My AV is not an issue even if I just turned it off completely I cant see any yarn speed improvements.

node -v 6.9.2

@kuncevic update your node to 7 and check if that makes things faster, otherwise @vbfox is pointing in right direction.

Essentially yarn can always copy every file in node_modules for every operation.

@vbfox could you elaborate as to why? I'm looking at the verbose output of yarn and finding that almost all of the time is spent creating directories and copying files, as it seems to do each one individually, rather than, say, creating a directory for each _package_ and then copying the whole package, which should be a fair bit faster, or even just symlinking all the packages which might be faster again. Are these not safe to do?

@danpalmer the linking phase works in essentially 3 big steps:

1. Find every file that need to be in node_modules
2. Check this list versus what is already there and find what need to be copied around from cache to node_modules
3. Do the copy

Due to a libuv/nodejs bug (utime is used by yarn and the bug is that it set the milliseconds to zero) files copied by yarn in a previous run are always found to be different (Version in cache have a normal modified time but all files in node_modules have a zero-for-milliseconds version) so phase 2 always report that everything changed.

As the bug is fixed in node 7.1 it's pretty easy to fix if you aren't limited to LTS (First yarn operation on a repo will be slow as the files were created with the buggy utime but all following will be a lot faster). My PR essentially fix this by ignoring the milliseconds part of file times on windows when comparing them.

Regarding copying whole package it's not an operation that exists on current filesystems AFAIK they all work at the file level.
The best windows provide is a FileCopy API (I have a PR to use it in yarn: https://github.com/yarnpkg/yarn/pull/2960) but it's just a little bit faster than using the native nodejs stream API.

As for symlinking I don't know why it isn't done, i'm not knowledgeable enough on javascript package managers (A few modifications are done to the package files like removing test folders but symlinking individual files wouldn't make that different) but as it isn't the case on linux/macos either (where it's a lot more common than on windows) there might be a good reason.

My experiment with upgrading to Node 7.8.0: https://github.com/yarnpkg/yarn/issues/990#issuecomment-290288375

1. Find every file that need to be in node_modules
2. Check this list versus what is already there and find what need to be copied around from cache to node_modules
3. Do the copy

Considering that most of the time people are relinking swaths of libraries when switching between branches - maybe there's a better way to go about this?

Could you create a unique ID for each build of node_modules and then sym link the entire directory from a cache? This way, switching branches back and forth is really just symlinking different node_modules

Granted, you'll be writing a lot to disk since now you're caching every version of node_modules you run up against, but could there be optimizations there to sym-link to directories so that you're really only storing a tree of sym-links?

Forgive me for being naive, I'm not the most educated when it comes to the unix and even-less-so Windows file systems, but I'd be happy to dig into this, as an educational exercise, and try and provide a proof of concept of this idea if it's not obviously flawed in any way.

pnpm and ied employ some of the techniques you mention I think, not entirely sure, tried them some time ago but they either caused issues on Windows or were not fast as yarn.

Me too just had very long time to create-react-app with yarn
windows server 2012
node 7.9.0
yarn 0.22
Done in 554.08s.

But in other cases it is much faster if no React installations included

I don't observe long linking time recently. Running
Yarn - v0.23.2
node - 6.10.2 or 7.9.10 (using nvm to switch)

Tried this on a mac and archlinux (Manjaro)

I can confirm that adding node and yarn to the Windows Defender exclusions reduced the Linking time in around 60% on my windows machine.

+1

[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 180.22s.

Switching to node 7.9.0 didn't speed it up for me. Adding 'yarn', 'node' and 'npm' to windows defender (with and without .exe extensions, not sure what is required) did speed it up 3x for me, but still 50% longer than npm install.

Also removing all protection from anything running in node or any packages being installed doesn't seem like a good idea to me...

Adding my experience - adding node.exe/yarn.exe to windows defender's exception list cut our yarn install time in half (from 60s to 30s).

I'm seeing this as well, makes it frustrating to iterate quickly while developing a package because it takes so long to update a single package.

yarn install v0.24.5
[1/4] Resolving packages...
[2/4] Fetching packages...
warning [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 338.20s.

Distributor ID: Ubuntu
Description:    Ubuntu 14.04.5 LTS
Release:    14.04
Codename:   trusty

  "dependencies": {
    "autoprefixer": "^6.7.7",
    "axios": "^0.16.1",
    "babel-core": "^6.24.1",
    "babel-loader": "7.x",
    "babel-preset-env": "^1.4.0",
    "coffee-loader": "^0.7.3",
    "coffee-script": "^1.12.5",
    "compression-webpack-plugin": "^0.4.0",
    "css-loader": "^0.28.0",
    "element-ui": "^1.3.3",
    "extract-text-webpack-plugin": "^2.1.0",
    "file-loader": "^0.11.1",
    "glob": "^7.1.1",
    "js-yaml": "^3.8.3",
    "node-sass": "^4.5.2",
    "path-complete-extname": "^0.1.0",
    "postcss-loader": "^1.3.3",
    "postcss-smart-import": "^0.6.12",
    "precss": "^1.4.0",
    "rails-erb-loader": "^5.0.0",
    "rails-ujs": "^5.1.0",
    "sass-loader": "^6.0.3",
    "style-loader": "^0.16.1",
    "turbolinks": "^5.0.3",
    "vue": "^2.3.0",
    "vue-loader": "^12.0.2",
    "vue-router": "^2.5.3",
    "vue-template-compiler": "^2.3.0",
    "webpack": "^2.4.1",
    "webpack-manifest-plugin": "^1.1.0",
    "webpack-merge": "^4.1.0"
  },
  "devDependencies": {
    "element-theme": "*",
    "element-theme-default": "^1.3.3",
    "eslint": "^3.19.0",
    "eslint-config-airbnb": "^14.1.0",
    "eslint-plugin-import": "^2.2.0",
    "eslint-plugin-jsx-a11y": "^4.0.0",
    "eslint-plugin-react": "^6.9.0",
    "nodemon": "^1.11.0",
    "webpack-dev-server": "^2.4.5"
  }

:(

Adding in my +1 on Mid-Summer 2010 MacBook Pro (Sierra 10.12.4) using yarn 0.24.5, node 7.10.0, and npm 4.2.0:

λ yarn add bootstrap-sass
yarn add v0.24.5
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
success Saved lockfile.
success Saved 1 new dependency.
└─ [email protected]
✨ Done in 123.52s.

"dependencies": {
    "@angular/animations": "^4.1.3",
    "@angular/common": "^4.0.0",
    "@angular/compiler": "^4.0.0",
    "@angular/core": "^4.0.0",
    "@angular/forms": "^4.0.0",
    "@angular/http": "^4.0.0",
    "@angular/material": "^2.0.0-beta.5",
    "@angular/platform-browser": "^4.0.0",
    "@angular/platform-browser-dynamic": "^4.0.0",
    "@angular/router": "^4.0.0",
    "bootstrap-sass": "^3.3.7",
    "core-js": "^2.4.1",
    "font-awesome": "^4.7.0",
    "material-design-icons": "^3.0.1",
    "materialize-css": "^0.98.2",
    "rxjs": "^5.1.0",
    "zone.js": "^0.8.4"
},
"devDependencies": {
    "@angular/cli": "1.0.1",
    "@angular/compiler-cli": "^4.0.0",
    "@types/jasmine": "2.5.38",
    "@types/node": "~6.0.60",
    "codelyzer": "~2.0.0",
    "jasmine-core": "~2.5.2",
    "jasmine-spec-reporter": "~3.2.0",
    "karma": "~1.4.1",
    "karma-chrome-launcher": "~2.0.0",
    "karma-cli": "~1.0.1",
    "karma-coverage-istanbul-reporter": "^0.2.0",
    "karma-jasmine": "~1.1.0",
    "karma-jasmine-html-reporter": "^0.2.2",
    "protractor": "~5.1.0",
    "ts-node": "~2.0.0",
    "tslint": "~4.5.0",
    "typescript": "~2.2.0"
}

Switching back to npm install fixed it for me.. I was getting - u'stream': u'[3/4] Linking dependencies in yarn and no error in NPM..

Something has gone wrong with the latest build perhaps

Running this through Docker.

@iwarner npm 5.0 is a good choice.

I run yarn in Vagrant (Ubuntu Xenial) and by Jenkins. There are two sub-projects with package.json.
npm -v 3.10.10
node -v 6.10.1
yarn install v0.21.3

We switched from npm to yarn some time ago, because we had a timeout problem (4hours was not enough) for npm install.

Now yarn works about 30% of the time, but for 70% of the time we get 4h timeout at some point. It might the first yarn install, or the second, or we might timeout while running unit tests (jest).

This is a duplicate of https://github.com/yarnpkg/yarn/issues/990, there is a comparison chart and seems like latest versions of Yarn made some good progress there.
If it is still a problem please file a new issue with repro steps and a comparison with latest npm

success Saved lockfile.
Done in 1737.79s.

Ubuntu 16.04
i5, 8 GB RAM

:(

Windows 10 v 1709 + SSD + PowerShell + Node 6.12.2:
Yarn install was fast until the last package, seemed to get stuck on a preinstall command.
Followed the instructions here to add exclusions for windows defender, but also I had my source checked out to %USERPROFILE%source which slowed it down drastically. Checked out in c: it was heaps faster.

Any solution for Ubuntu platform? I literally have to think twice before adding a package.

Ubuntu is super fast for me, no slowness at all.

On Fri, 23 Feb 2018, 11:13 Basant Besra, notifications@github.com wrote:

Any solution for Ubuntu platform? I literally have to think twice before
adding a package.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/1496#issuecomment-367897260, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAcMheTtAYOsXcrnej_f2F8bY5D3nDT2ks5tXizngaJpZM4Kh3OZ
.

This is super annoying. I literally changed one line in a module, republished it under a new version and yarn add module took over 5 minutes.

It'd be quicker just to manually update my packages using a text editor

I also experience this issue:

success Saved lockfile.
success Saved 1 new dependency.
info Direct dependencies
└─ @material-ui/[email protected]
info All dependencies
└─ @material-ui/[email protected]
Done in 93.43s.

My system is Linux manjaro 4.14.31-1-MANJARO #1 SMP PREEMPT Wed Mar 28 21:42:49 UTC 2018 x86_64 GNU/Linux

NodeJS: v9.9.0
Yarn: v1.5.1

Also super slow for me Done in 254.32s.

node v8.10.0
npm 5.6.0

OSX 10.11.6 (15G19009)

I have switched back to [email protected] are things are working pretty great.

We are using the offline cache feature to avoid this problem most of the time, but as soon as the package.json or yarn-lockfile change, then we are back to this problem. Linking takes 10 minutes on our Linux machine. I do not think this is windows specific.

This is definitely not a Windows only issue (which should be evident from all the posts from people on non-Windows machines)!

I'm on macOS High Sierra 10.13.4, using node 10.1.0 (npm 5.6.0) and yarn 1.6.0. Using yarn, installing a dependency took ~40s. I switched to npm and it took about ~10 seconds. I'll stick to npm for the time being.

Same for our centos 7 box. Any updates on this?
yarn: v1.7.0
npm: v5.7.1

it is happening to me on 1.9.2 on mac on node 10

For me on macOS HighSierra, Avast FileShield was causing the issue. I've added the yarn executable as excluded path, using which yarn. It seems ok now, I'll give an update if it returns.

it is happening to me on 1.9.2 on mac on node 10

Same here. Yarn 1.9.2, node 10.6.0 on High Sierra.

@bestander this is not a Windows issue. I can repro on my Mac with Yarn 1.9.4. This issue should be reopened.

@davidgoli , better open a new issue, this is a new one and should be triaged separately

Yarn is quite slow for any environment I ran it. Debian, Mac, Windows. The new issue was open? or the RFC that get rid of node_modules will solve this?

i have same issue, already switch to npm but still got bug

I have also same issue with Yarn. Any solution found?

This is a duplicate of #990, there is a comparison chart and seems like latest versions of Yarn made some good progress there.
If it is still a problem please file a new issue with repro steps and a comparison with latest npm

This not a duplicate, this issue is not only about Windows. Opening a new issue would cause a loss of context.

I have same issue!

yarn install
yarn install v1.16.0
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
[###############################################---------------------------------------------------------------------------------------------] 22778/67399

Done in 179.59s.

MacOS / Docker

Vagrant 2.2.4
Guest: Ubuntu 18.10 (GNU/Linux 4.18.0-25-generic x86_64)
Host: MacOS 10.14.5 Mojave

yarn 1.16.0
npm 6.9.0

MacBook Pro (Retina, 13-inch, Early 2015)
Processor 2.7 GHz Intel Core i5
Memory 16 GB 1867 MHz DDR3

yarn install v1.16.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 1552.45s.

I actually can't run yarn install without losing 25+ minutes of productivity. It's absurd. I don't buy that this is a Windows issue. It seems very likely to me that there is some problem when running in a virtualised environment. Perhaps to do with synced folders / checking file state on the guest OS?

yarn v1.17.3
node v10.16.3
npm 6.9.0

Windows

I have tried to put my App project folder location at the same disk section as yarn cache dir.
yarn cache dir -> C:UsersAppDataLocalYarnCachev4

The result:
old location -> D:myApp Done in 747.17s.
new location -> C:myApp Done in 488.97s.

C and D are the same physical disk.

Mac

However Mac is faster than Windows Done in 121.37s

I think the bottleneck is the disk read/write speed?

OS X 10.15
yarn v1.22.4
node v12.13.0
npm v6.12.0

I am still experiencing this. Project is located in a mounted encrypted disk image. Installing a single package takes several minutes with a relatively small package.json. Haven't benchmarked it but npm feels much faster.

Edit: Turns out that changing yarn's default cache folder to be on the same encrypted volume fixed this for me. This was really the root of the issue.

Just got hit by this too and I'm running:

OS: Ubuntu 18.04.2
Yarn: 1.22.4
Node: 14.7.0
NPM: 6.14.7