Gatsby: [fsevents bug] Stuck at "source and transform nodes" / "createPagesStatefully" on MacOS

Updated - I tested commenting out the gatsby-node.js file and the build progressed on one occasion, but then I tried again and doing this and it got stuck in the same spot again.

Is there any chance this is due to my ancient computer, 2010 Macbook Pro (upgraded to 8GB Ram and an SSD), hasn't stopped me yet but I know its days are limited. This is a hobby for me and I have young kids so the dollars haven't been there to spend on a new computer since this one has worked fine with the upgraded Ram and SSD.

I have tried to revert back to gatsby 2.13.52 which was the last version I was stable on, I have also tried reverting to react 16.8.6.

Interestingly I had it build successfully one time when I reverted react to 16.8.6 but after that first time it hung up at the same spot which is really odd behaviour to me.

I can also, rarely, for no reason I can discern get it to build fine. There does not seem to be a rhyme or reason for when this happens. I spent a couple of hours looking for patterns or errors that might be causing this but didn't find anything. I reviewed https://github.com/gatsbyjs/gatsby/issues/6654 and tried some of the items in there to no avail.

This is got to me one of the most odd bugs I have ever encountered because the behaviour seems to change without there being discernible change in the code. In some cases the build hangs at source and transform nodes(60% of time), in some cases it hangs at Create Pages Statefully(20%), in some cases it builds successfully (20%). I have had it display all three of these behaviours without changing a line of code.

I have replicated this behaviour in gatsby-starter-blog as well, which is really odd. Again it makes me think this is an issue on my end locally. Interestingly it only hangs at createPagesStatefully.

I am now starting to think that it has something to do with a library that got updated by homebrew automatically - which one I have no idea but it is the only thing I can think of which I have not tested.

Here is a list of the things I have tried so far:

• Changing node versions, tried 12, 10 and 8

• Reverting to a previous point in my commit history that was working - it still fails now.

• Commenting out plugins and areas of the gatsby-config

• Commenting out the contents of my gatsby-node.js

• Test it again on Netlify, still built successfully, which is what makes me think it must have something to do with my computer.

• Deleting the 4 pages in my src/pages directory and putting in a very basic index.mdx file

• Deleting all the node_module and lock files, reinstalling

• Restarting computer

• Creating a fresh yarn workspace with new clones of the theme/starter from Github.

• Testing gatsby-starter-blog, similar behaviour but it hangs on createPagesStatefully

Hello,

I can do little but confirm that I'm seeing this issue on a number of gatsby starters. And, as you point out, it sometimes just decides to build or to stop building, hanging at either "source and transform nodes" or createPagesStatefully.

Quite frustrating and leading to trying the most outrageous attempts to fix it.

I did not witness this problem but this sounds awkward and I would really like to know the reason for this behaviour on your side.

Preparation
You should try to use the node debugger to get to the root of the problem. A task in your package.json would look like this. If you are using VSCode you can enable "Auto Attach" to use the internal debugger together with this task (make sure you use the internal terminal for this purpose)

"start:debug": "node --inspect=127.0.0.1:9232 node_modules/.bin/gatsby develop",

Debugging will of course work with any IDE just make sure you attach your debugger correctly.

1. Variant: Minimal Reproductiion
You have mentioned createPagesStatefully as the source of the problem. If you are sure that's the source of the problem maybe you could create a very tiny project to reproduce it. Ditch all the starters, just use gatsby directly and use createPagesStatefully through gatsby-node.js with some examples that mimic the things your starters are doing. Then start gatsby and debug it through node inspect.

That way you might find where it hangs.

2. Alternative: Inside your project/starter
You could of course try to debug the problem inside your project with the same technique. But maybe you have multiple problems that are stacking and blur your view on the cause of problems. So I would always try to get a minimal reproduction before starting to debug.

Good luck.

So...I ran into some weird behaviour with the lock files. Maybe someone who knows more could point me in the right direction. In the process of trying to get to a minimal working build I basically stripped down to a "hello world" Gatsby install and it still wasn't working which was really odd.

Where it got even more strange was that the actual gatsby-starter-hello-world does work and build fine on my computer. BUT...if I delete the lock file and get yarn to rebuild a new lock file then the build starts failing, hanging at source and transform nodes. I could get my stripped down and minimal implementation to build if I copied the lock file from "hello-world" and used this. So my current theory is that there is some kind of version problem in my lock files that is causing this issue but I am stuck here.

I also deleted all of my homebrew installs and reinstalled node, yarn, git, etc. Just to make sure it wasn't somekind of funny business there.

Thanks @ehowey for raising this.... I thought it was just me because it is quite intermittent (but happens more than 50% of the time). Did the same as you to try and debug the situation. When I hang on ⠙ source and transform nodes it usually means having to kill my terminal.

If I find something I'll let you know. And I'll watch this thread too.

@georgiee - thank you for the --inspect info. I'm going to see if I can get node debugging working with WebStorm.

I also like your idea of a minimal reproduction. But that will take time while I understand Gatsby a little deeper.

Currently it's hanging on "source and transform nodes". Rarely, it makes it to createPagesStatefully and hangs there. Otherwise it builds normally.

I am currently facing the same problem after doing "yarn upgrade" to fix a vulnerable dependency. Here's my set up:

System:
OS: macOS 10.15
CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 12.8.1 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.10.3 - /usr/local/bin/npm
Languages:
Python: 2.7.16 - /usr/local/bin/python
Browsers:
Chrome: 76.0.3809.132
Firefox: 68.0.2
Safari: 13.0
npmPackages:
gatsby: ^2.13.42 => 2.14.7
gatsby-background-image: ^0.8.3 => 0.8.6
gatsby-image: ^2.2.7 => 2.2.15
gatsby-plugin-manifest: ^2.2.4 => 2.2.10
gatsby-plugin-netlify: ^2.1.4 => 2.1.10
gatsby-plugin-netlify-cms: ^4.1.6 => 4.1.12
gatsby-plugin-offline: ^2.2.5 => 2.2.10
gatsby-plugin-react-helmet: ^3.1.2 => 3.1.5
gatsby-plugin-react-svg: ^2.1.1 => 2.1.2
gatsby-plugin-sass: ^2.1.3 => 2.1.12
gatsby-plugin-sharp: ^2.2.9 => 2.2.18
gatsby-plugin-typography: ^2.3.2 => 2.3.5
gatsby-plugin-webfonts: ^1.1.0 => 1.1.0
gatsby-remark-images: ^3.1.10 => 3.1.19
gatsby-remark-relative-images-v2: ^0.1.5 => 0.1.5
gatsby-remark-responsive-iframe: ^2.2.5 => 2.2.10
gatsby-source-filesystem: ^2.1.6 => 2.1.18
gatsby-transformer-remark: ^2.6.12 => 2.6.19
gatsby-transformer-sharp: ^2.2.5 => 2.2.12

I am currently facing the same problem after doing "yarn upgrade" to fix a vulnerable dependency. Here's my set up:

Update: I manage to fix things by restoring my old "yarn.lock" file.

@hbthen3rd

Update: I manage to fix things by restoring my old "yarn.lock" file.

My experience is that the problem can go away for no clear reason only to return later. You may find the problem returning despite restoring yarn.lock. Keep us posted.

Here is a minimal reproduction using gatsby-starter-hello-world:

https://github.com/ehowey/gatsby-test-lockfiles

The current lock file included in the repo is the one failing for me. I have also included copies in the repo of yarn-working.lock and yarn-notworking.lock. Hopefully that makes it easier for someone to troubleshoot.

My current environment:

System:
OS: macOS High Sierra 10.13.6
CPU: (2) x64 Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 10.16.3 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.10.3 - /usr/local/bin/npm
Languages:
Python: 2.7.10 - /usr/bin/python
Browsers:
Chrome: 76.0.3809.132
Firefox: 67.0.2
Safari: 12.1.2
npmPackages:
gatsby: ^2.13.73 => 2.15.0
npmGlobalPackages:
gatsby-cli: 2.7.40

I am experiencing the same issue.

Some direction I have found:

1. Downgrading gatsby-plugin-sitemap from 2.2.9 to 2.2.6 solved the issue with yarn develop.

   • It is odd because gatsby-plugin-sitemap should not affect development build.

2. yarn build still doesn't work but when I remove gatsby-plugin-sitemap form my config, yarn build works again.

@sharvit I can report that this did not work in my case. Glad it fixed it for you, I do think it has something ultimately to do with the lock files and some weird version issue deep in the bowels of the lock file.

I have managed to get back to a working build, most of the time, maybe 8/10 times by reverting to an old lock file and doing some copy and paste. I can now also CTRL+C to force quit the build if it is hanging which I couldn't do at one point in this process. I don't know what fixes it in the lock file but I think the clues are in the repository I posted above with two copies of a lock file, one that works and one that doesn't. This is an odd beast of a bug. Usually in computer land things reassuringly work or don't work.

@steinitz Any luck on your end? I had what you mention happen. It seemed to get better, pretty good but not perfect and now is failing almost every time again. Pretty frustrating.

@ehowey

As you might imagine, due to the on-again off-again nature of this problem, I'm hesitant to report. Here's a case in point:

After deleting the node_modules directory, deleting yarn.lock then running
npx yarn-upgrade-all
and
yarn install
all was well.

Then, just now, in response to your message, I tried
yarn start
Result: it hangs again at
source and transform nodes

I suppose there are two sensible things to do:

1. take @georgiee's advice to get Webstorm's debugging working with
   node --inspect and see if I can spot any obvious problems
2. set Gatsby aside for a while to see if some smart person solves the problem

Update:

ctl-C'd the above stuck process (that didn't used to stop the stuck process which was doubly annoying).
Then yarn start got stuck on createPagesStatefully.
ctl-C'd again, another yarn start - stuck on source and transform nodes
ctl-C'd one more time (for the hell of it) - this time yarn start worked and it's running

Don't know what to make of that, but there it is.

This is similar to what I am seeing, tonight it seems worse maybe building successfully 1/10 times or less. From a programming/coding perspective this is super odd behaviour. Anecdotally it seemed to be working better a few days ago at 2.15.1 then today at 2.15.6. I also wonder what commonalities we all share that is causing this bug to happen. Can you run the gatsby info --clipboard command and post it?

It obviously isn't super widespread otherwise there would be a flood of reports but it also isn't just me like I originally thought. We all seem to be using yarn as well, which is a requirement for me as I am working on themes in a yarn workspace. I am still able to reproduce this on the gatsby-starter-hello-world so I believe it is some dependency issue or conflict in the core gatsby files.

@ehowey here's the gatsby info you requested

System:
OS: macOS 10.14.6
CPU: (4) x64 Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 12.9.1 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.11.2 - /usr/local/bin/npm
Languages:
Python: 2.7.16 - /usr/local/bin/python
Browsers:
Chrome: 76.0.3809.132
Firefox: 68.0.1
Safari: 12.1.2
npmPackages:
gatsby: ^2.14.3 => 2.14.7
gatsby-image: ^2.2.14 => 2.2.15
gatsby-plugin-feed: ^2.3.9 => 2.3.9
gatsby-plugin-i18n: ^1.0.1 => 1.0.1
gatsby-plugin-less: ^3.0.2 => 3.0.4
gatsby-plugin-manifest: ^2.2.9 => 2.2.10
gatsby-plugin-offline: ^2.2.10 => 2.2.10
gatsby-plugin-react-helmet: ^3.1.5 => 3.1.5
gatsby-plugin-robots-txt: ^1.5.0 => 1.5.0
gatsby-plugin-sharp: ^2.2.18 => 2.2.18
gatsby-plugin-sitemap: ^2.2.9 => 2.2.9
gatsby-remark-images: ^3.1.19 => 3.1.19
gatsby-remark-prismjs: ^3.3.9 => 3.3.9
gatsby-source-filesystem: ^2.1.18 => 2.1.18
gatsby-transformer-remark: ^2.6.19 => 2.6.19
gatsby-transformer-sharp: ^2.2.12 => 2.2.12
npmGlobalPackages:
gatsby-cli: 2.7.40

I was having the same issue: site built flawlessly on Netlify, but hung on my development machine with both gatsby build and gatsby develop.

After playing around with package versions, I noticed that reverting gatsby-plugin-sitemap from version 2.2.10 to 2.2.9 fixed the problem for me. Oddly enough, some people here seem to have issues with 2.2.9, which could mean the problem resides somewhere else.

Edit: Spoke too soon, Gatsby still hangs at the "source and transform nodes" and "createPagesStatefully" steps, though much less frequently.

@goblindegook this seems a common pattern with this particular issue. Because the issue comes and goes, seemingly with the weather, the time of day, time since boot up... you can believe something you've done has fixed it - only to have it recur.

Experiencing this issue too, downgraded gatsby-plugin-sitemap to 2.2.6 and that seems to have fixed it

FWIW, I'm also experiencing this but use neither yarn nor gatsby-plugin-sitemap.

My gatsby info in case it helps:

  System:
    OS: macOS 10.14.6
    CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 12.9.1 - /usr/local/bin/node
    npm: 6.11.2 - /usr/local/bin/npm
  Languages:
    Python: 3.7.4 - /usr/local/opt/python/libexec/bin/python
  Browsers:
    Chrome: 76.0.3809.132
    Firefox: 68.0.1
    Safari: 12.1.2
  npmPackages:
    gatsby: ^2.15.1 => 2.15.1 
    gatsby-cli: ^2.7.41 => 2.7.41 
    gatsby-image: ^2.2.15 => 2.2.15 
    gatsby-plugin-catch-links: ^2.1.6 => 2.1.6 
    gatsby-plugin-google-tagmanager: ^2.1.7 => 2.1.7 
    gatsby-plugin-manifest: ^2.2.12 => 2.2.12 
    gatsby-plugin-offline: ^3.0.0 => 3.0.0 
    gatsby-plugin-react-helmet: ^3.1.5 => 3.1.5 
    gatsby-plugin-sass: ^2.1.12 => 2.1.12 
    gatsby-plugin-sharp: ^2.2.18 => 2.2.18 
    gatsby-remark-images: ^3.1.19 => 3.1.19 
    gatsby-source-filesystem: ^2.1.18 => 2.1.18 
    gatsby-transformer-remark: ^2.6.19 => 2.6.19 
    gatsby-transformer-sharp: ^2.2.12 => 2.2.12 
    gatsby-transformer-yaml: ^2.2.7 => 2.2.7 
  npmGlobalPackages:
    gatsby-cli: 2.7.41

For me, cleaning the cache with gatsby clean solved the issue.

I am still doing some hunting, trying to figure this out. Still a problem for me. Does anyone know if this might relate to the switch from babel 7.0.0 -> 7.5.5. This switch happened around the time I started experiencing this bug and coincides with me beginning to see problems...I have been trying to get yarn resolutions to work in order to peg the babel versions at 7.0.0 but haven't had success with this yet.

I think I can offer some insight--I started having this problem when, halfway through adding plugins to a project, I updated gatsby-cli in another terminal window. Running gatsby clean in my project worked.

from https://github.com/gatsbyjs/gatsby/issues/6385#issuecomment-531949380 - I am also seeing this issue but gatsby clean did not solve it. It seems like it might just be the CLI printout is hanging, which is why resizing the terminal fixes it?? 🤷‍♂ 😕 ❓❗️

Resizing the terminal window does seem to fix it for me! I have not tested it super extensively, can some other people having this problem try this as well? What an odd bug and potentially odd solution.

I am facing the exact same issue—gatsby gets stuck on "source and transform nodes" or "createPagesStatefully" often, and I'm _not_ using any source plugins. I only recently encountered the "resize terminal window" fix and am 140% baffled as to how in the heck this fixes it, but it does. This seems like a pretty serious issue!

@JaKXz - Thank-you! This was driving me nuts. The fix does seem to be resizing the terminal window in VS Code. Just drag it up or down a bit and the develop task chugs along happily. I tested in a couple of different cases both yarn and npm, workspaces and not. Seemed to be working in all those cases for me. It also seems to freeze or hang at createPagesStatefully more often than source and transform nodes now for me. I am going to leave the issue open for now, this may need to be fixed in a less "hacky" way by someone with more know how than me.

@ehowey I have the same issue and moving the terminal window in vscode works (couldn't believe it when reading this issue until I tried myself).
Do you know if it only happens on VS Code?

I have this issue on iTerm 2, so it's not VS Code specific.

My issue has also been in iTerm 2

Webstorm terminal, Mac Terminal, iTerm

Did the resizing terminal fix work for all of you in different dev environments?

On my end, resizing the terminal worked on iTerm and VSCode (took a few tries to reproduce the issue on iTerm)

The resizing terminal trick works reliably for me in iTerm 2.

Yep, resizing iTerm 2 works perfectly

If resizing works, I suspect this bug is related buffer, somewhere needs stdout flush.

This kind of seems like a kernel+shell related issue. Probably some way node interacts with your kernel and/or your shell. I only say this because I can't seem to replicate the issues using Linux or Windows. I can't seem to find any known issues, so either a) it's some combination of code patterns unique to Gatsby + the interaction with the system, or b) I just don't know the right question to ask yet.

If resizing the terminal fixes the issue, then it might be some glitch with between node and kqueue causing a block in the event loop. Resizing the terminal sends the process a SIGWINCH signal, which interrupts the event loop, which frees it up and allows it to continue.

Can you try running kill -SIGWINCH ${pid} on the running process when it freezes? Should act the same as resizing the terminal.

I'm also interested in whether or not this happens in all shells. Based on the information so far, this has been failing in bash and zsh, and this is probably one of the common factors between all of the terminal emulators. Can you guys try one or more of sh, csh, ksh, tcsh, etc... ?

If the problem occurs in all of the shells, then I would guess that it's the way the kernel is handling node's event loop. This could also be why it's an intermittent problem. If some function gets less processor time (maybe because of variable system load), it might block for too long, and maybe the kernel tries to reuse an event node somewhere, resulting in a deadlock? I'm not super familiar with the internals of the api, but I bet that source and transform nodes is fairly filesystem IO intensive. That means there's probably a lot of offloading happening, so who knows what's really happening on lower levels.

I think it would be a good idea to try to narrow down the surface area of this bug. It occurs most frequently in source and transform nodes, it looks like, so maybe try narrowing it down to what plugin is blocking. Try adding the following lines to node_modules/gatsby/dist/utils/api-runner-node.js:

+ 347       if (api === `sourceNodes`) {
+ 348         debugger;
+ 349       }
  350       resolve(runAPI(plugin, api, Object.assign({}, args, {
  351         parentSpan: apiSpan
  352       })));

Then run node inspect node_modules/.bin/gatsby develop. It will break as soon as it starts, so you have to hit c when you get to the debug> prompt, and wait for it to continue. When it breaks on that debugger line, write exec console.log(plugin), and note what it says in resolve. Then hit c to continue. Just do this until it hangs... if it hangs.

Because of the nature of the event loop, I bet that it won't even hang while debugging. Those interrupts are probably enough to keep it on a track. Async bugs can be a real pain to track down. If it doesn't hang while using the debugger, replace that debugger line with:

reporter.log(plugin.resolve);

Hopefully that will turn something up. It would be nice to see what plugin is causing the block. If we can figure that out, we might be able to figure out what needs to be optimized/refactored in order to sort this out.

Resizing works for me too, I use zsh as my VSCode shell.

@Js-Brecht - Thank-you for taking the time for such a detailed response. I was not sure exactly where I was supposed to input kill -SIGWINCH ${pid}. I couldn't do it during the build process.

Withe the debugger the following code came out...beyond me to make much sense of it. I actually got stuck in the debugger but .exit still worked.

⠋ source and transform nodes
break in /Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/utils/api-runner-node.js:365
 363       return new Promise(resolve => {
 364         if (api === `sourceNodes`) {
>365           debugger;
 366         }
 367         resolve(

< {
<   resolve: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/internal-plugins/internal-data-bridge',
<   id: 'a5079d69-ba80-53dc-82f9-0f440bd5448c',
<   name: 'internal-data-bridge',
<   version: '1.0.0',
<   pluginOptions: { plugins: [] },
<   nodeAPIs: [ 'sourceNodes', 'onCreatePage' ],
<   browserAPIs: [],
<   ssrAPIs: []
< }
< ⠋ source and transform nodes
debug> undefined
debug> c
break in /Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/utils/api-runner-node.js:365
 363       return new Promise(resolve => {
 364         if (api === `sourceNodes`) {
>365           debugger;
 366         }
 367         resolve(
⠦ source and transform nodes
break in /Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/utils/api-runner-node.js:365
 363       return new Promise(resolve => {
 364         if (api === `sourceNodes`) {
>365           debugger;
 366         }
 367         resolve(
{
<   resolve: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-plugin-mdx',
<   id: '479d2e25-5a33-5a59-a290-cf1877d39ee5',
<   name: 'gatsby-plugin-mdx',
<   version: '1.0.43',
<   pluginOptions: {
<     plugins: [ [Object] ],
<     extensions: [ '.md', '.mdx' ],
<     defaultLayouts: {
<       default: '/Users/erichowey/Coding/gatsby-catalyst/themes/gatsby-theme-catalyst-core/src/components/layout.js'
<     },
<     gatsbyRemarkPlugins: [ [Object], [Object], [Object] ],
<     remarkPlugins: [ [Function: slug] ]
<   },
<   nodeAPIs: [
<     'sourceNodes',
<     'onCreateNode',
<     'onCreatePage',
<     'onCreateWebpackConfig',
<     'resolvableExtensions',
<     'preprocessSource',
<     'onCreateBabelConfig',
<     'onPreBootstrap',
<     'onPostBootstrap'
<   ],
<   browserAPIs: [ 'wrapRootElement' ],
<   ssrAPIs: [ 'wrapRootElement' ],
<   pluginFilepath: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-plugin-mdx'
< }
< ⠦ source and transform nodes
debug> undefined
{
<   resolve: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-plugin-mdx',
<   id: '479d2e25-5a33-5a59-a290-cf1877d39ee5',
<   name: 'gatsby-plugin-mdx',
<   version: '1.0.43',
<   pluginOptions: {
<     plugins: [ [Object] ],
<     extensions: [ '.md', '.mdx' ],
<     defaultLayouts: {
<       default: '/Users/erichowey/Coding/gatsby-catalyst/themes/gatsby-theme-catalyst-core/src/components/layout.js'
<     },
<     gatsbyRemarkPlugins: [ [Object], [Object], [Object] ],
<     remarkPlugins: [ [Function: slug] ]
<   },
<   nodeAPIs: [
<     'sourceNodes',
<     'onCreateNode',
<     'onCreatePage',
<     'onCreateWebpackConfig',
<     'resolvableExtensions',
<     'preprocessSource',
<     'onCreateBabelConfig',
<     'onPreBootstrap',
<     'onPostBootstrap'
<   ],
<   browserAPIs: [ 'wrapRootElement' ],
<   ssrAPIs: [ 'wrapRootElement' ],
<   pluginFilepath: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-plugin-mdx'
< }
< ⠦ source and transform nodes
debug> undefined
debug> c
break in /Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/utils/api-runner-node.js:365
 363       return new Promise(resolve => {
 364         if (api === `sourceNodes`) {
>365           debugger;
 366         }
 367         resolve(
{
<   resolve: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-transformer-sharp',
<   id: '822bdf7b-a95a-5885-9351-158705910ac3',
<   name: 'gatsby-transformer-sharp',
<   version: '2.2.16',
<   pluginOptions: { plugins: [] },
<   nodeAPIs: [
<     'onCreateNode',
<     'setFieldsOnGraphQLNodeType',
<     'onPreExtractQueries',
<     'sourceNodes'
<   ],
<   browserAPIs: [],
<   ssrAPIs: [],
<   pluginFilepath: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-transformer-sharp'
< }

⠋ source and transform nodes
break in /Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/utils/api-runner-node.js:365
 363       return new Promise(resolve => {
 364         if (api === `sourceNodes`) {
>365           debugger;
 366         }
 367         resolve(

< {
<   resolve: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/internal-plugins/internal-data-bridge',
<   id: 'a5079d69-ba80-53dc-82f9-0f440bd5448c',
<   name: 'internal-data-bridge',
<   version: '1.0.0',
<   pluginOptions: { plugins: [] },
<   nodeAPIs: [ 'sourceNodes', 'onCreatePage' ],
<   browserAPIs: [],
<   ssrAPIs: []
< }
< ⠋ source and transform nodes
debug> undefined
debug> c
break in /Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby/dist/utils/api-runner-node.js:365
 363       return new Promise(resolve => {
 364         if (api === `sourceNodes`) {
>365           debugger;
 366         }
 367         resolve(
{
<   resolve: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-source-filesystem',
<   id: '339022db-842c-55bd-8e87-d84bd74a175a',
<   name: 'gatsby-source-filesystem',
<   version: '2.1.26',
<   pluginOptions: { plugins: [], name: 'src', path: 'src/' },
<   nodeAPIs: [ 'sourceNodes', 'setFieldsOnGraphQLNodeType' ],
<   browserAPIs: [],
<   ssrAPIs: [],
<   pluginFilepath: '/Users/erichowey/Coding/gatsby-catalyst/node_modules/gatsby-source-filesystem'
< }
< ⠋ source and transform nodes
debug> undefined
⠹ source and transform nodes
debug> exec console.log(plugin)
c
c

macOS Sierra, using Terminal, resizing worked for me.

@ehowey Just so I know I'm understanding you correctly, it did hang while you were in the debugger? In that case, it looks to me like gatsby-source-filesystem is the culprit, which makes sense to me... especially because of existing, related, gatsby issues.

I'd like to see if the plugin can handle a test run. If it hangs while running tests, then I think that will be an easy way to see where it is failing. If not, well... will just have to do some dissecting/debugging, which will be difficult for me, since I don't have MacOS.

Can you download the main gatsby repo, and run the gatsby-source-filesystem tests?

> git clone [email protected]:gatsbyjs/gatsby.git gatsby-js
> cd gatsby-js
> yarn bootstrap
> yarn jest -- -i './packages/gatsby-source-filesystem/src'

Also, are you doing all of this with your minimal reproduction repository? I see that gatsby-plugin-mdx was run twice, so that tells me that it is not a barebones repository. In a perfect world, it shouldn't matter, but I think it would be better to run as simple a setup as possible. If you can't get it to fail reliably with the minimal repo, then use whichever one fails most consistently (in the same place, every time (or close to))

😉

kill -SIGWINCH ${pid} must be run from another shell instance. When the build hangs, open another terminal window/tab, and run the command from there. This little snippet should work:

pid=$(ps -ef |grep -E 'node.*index\.js develop' |awk '{ print $2 }').
kill -SIGWINCH $pid

# Can also be run like this
kill -SIGWINCH $(ps -ef |grep -E 'node.*index\.js develop' |awk '{ print $2 }')

If there are errors, try running the commands seperately:

# Run first
ps -ef grep -E 'node.*index\.js develop'
# Example output
     UID     PID    PPID  TTY        STIME COMMAND
********   39928       1 cons0    06:47:38 /path/to/node /path/to/gatsby-cli/lib/index.js develop
# Second column is the pid you want
kill -SIGWINCH 39928

If you come up empty-handed after running the tests, I think it will be useful to have a core dump, so we can get a stack trace. But, one step at a time.

@Js-Brecht Sorry for the slower responses...this is really just a side hobby I do in the spare time in the evenings.

So I ran the same thing inside of the the gatsby hello world starter - as bare bones as you can get. Sorry I ran it before in my main workspace on a project I had been having problems with. I have previously replicated it on starters though so I dont think it relates to a plugin but rather to something in the core of gatsby.

It hangs on source and transform nodes giving me the following output:

< success onPreBootstrap - 0.102 s
⠋ source and transform nodes
break in node_modules/gatsby/dist/utils/api-runner-node.js:362
 360       return new Promise(resolve => {
 361         if (api === `sourceNodes`) {
>362           debugger
 363         }
 364         resolve(

< {
<   resolve: '/Users/erichowey/Coding/my-hello-world-starter/node_modules/gatsby/dist/internal-plugins/internal-data-bridge',
<   id: 'a5079d69-ba80-53dc-82f9-0f440bd5448c',
<   name: 'internal-data-bridge',
<   version: '1.0.0',
<   pluginOptions: { plugins: [] },
<   nodeAPIs: [ 'sourceNodes', 'onCreatePage' ],
<   browserAPIs: [],
<   ssrAPIs: []
< }
< ⠋ source and transform nodes
debug> undefined

< success source and transform nodes - 25.137 s

Here is a dump from the gatsby info command in case that is helpful:

  System:
    OS: macOS High Sierra 10.13.6
    CPU: (2) x64 Intel(R) Core(TM)2 Duo CPU     P8600  @ 2.40GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 12.9.1 - /usr/local/bin/node
    Yarn: 1.17.3 - /usr/local/bin/yarn
    npm: 6.10.3 - /usr/local/bin/npm
  Languages:
    Python: 2.7.10 - /usr/bin/python
  Browsers:
    Chrome: 76.0.3809.132
    Firefox: 67.0.2
    Safari: 13.0.1
  npmPackages:
    gatsby: ^2.15.22 => 2.15.22 
  npmGlobalPackages:
    gatsby-cli: 2.7.47

As a side note - when I use your debug command it hangs on source and transform nodes. When I use gatsby develop it is mostly hanging at createPagesStatefully now for me. Sorry I know this is a really odd one. To be honest I am not sure how much work it is worth from your end of things. The resizing the terminal window trick works every time for me and others. It is a hacky fix but it must not be affecting that many users otherwise the issues would be flooded.

This has started happening over here as well. The _resize the terminal_ trick resolves it; very odd!

+1 Doesn't work in iTerm2, but work in Terminal 🤷‍♂️

@ehowey The majority of what is happening during the build is defined by one plugin or another. Gatsby ships with them as dependencies; I suppose they could be considered core plugins. The Gatsby core exposes an API, which looks for endpoints defined in the plugins, and passes them a series of arguments which include actions/objects that are defined in the core. So yes, what is happening could be happening in the core, but it first has to call out to a plugin, and those plugins define a specific context for the API. I'm trying to determine the context that is resulting in the build hanging, and also need to verify that the issue is not occurring in the plugin itself.

Can I get you to change these lines?

 360       return new Promise(resolve => {
-361         if (api === `sourceNodes`) {
-362           debugger
-363         }
+364         reporter.log(`${api}\t${plugin.name}`)
 365         resolve(

Please run this, and copy/paste the entire output (might even just attach it to your response as a .txt file). It will be considerably more verbose, and a lot of the information will probably be unnecessary, but 🤷‍♂.

After you've done that, I'd like to see if increasing node's thread pool makes a difference. I've noticed other issues that mention the same, or similar issues. Mostly they occur in source and transform, and some talk about that stage taking forever, or completely locking up. So, I want to see if it happens to be some kind of deadlock in filesystem io... asynchronous file system access is offloaded by node to different threads, and, by default, node only has a pool of 4 threads for userspace. If those fill up, and it needs to do any more offloading, the tasks will queue up in the main event loop, waiting for a thread. This could, potentially, bring the entire program to a standstill... until a thread becomes available. Gatsby maintains a cache on the filesystem, so perhaps there's a collision occurring somewhere, and if there's some kind of deadlock happening, then perhaps the threads are waiting for a timeout/interrupt before moving on, and if there's dozens (or hundreds, even) of filesystem access events, they could all be waiting for the same timeout, and causing everything to pile up. You can see that this could cause wait times to grow rather quickly.

Increasing the pool size might help alleviate some of the traffic... or, it could just happen the same way, just with more threads 😆.
Try the following command:

UV_THREADPOOL_SIZE=10 gatsby develop

@Js-Brecht Changing the thread pool size didn't seem to make much difference.
I get the following output from the standard gatsby develop command with the changes you mentioned above. Still on the basic hello-world gatsby starter.

Erics-MBP:my-hello-world-starter erichowey$ gatsby develop
success open and validate gatsby-configs - 0.050 s
success load plugins - 0.119 s
success onPreInit - 0.036 s
success initialize cache - 0.050 s
success copy gatsby files - 0.281 s
onPreBootstrap  load-babel-config
success onPreBootstrap - 0.131 s
sourceNodes     internal-data-bridge
success source and transform nodes - 0.105 s
success Add explicit types - 0.030 s
success Add inferred types - 0.186 s
success Processing types - 0.145 s
success building schema - 0.535 s
success createPages - 0.032 s
createPagesStatefully   dev-404-page
onCreatePage    internal-data-bridge
onCreatePage    prod-404
createPagesStatefully   gatsby-plugin-page-creator
createPagesStatefully   gatsby-plugin-page-creator
createPagesStatefully   gatsby-plugin-page-creator
createPagesStatefully   gatsby-plugin-page-creator
createPagesStatefully   gatsby-plugin-page-creator
createPagesStatefully   gatsby-plugin-page-creator
onCreatePage    internal-data-bridge
onCreatePage    prod-404
success createPagesStatefully - 9.098 s
success onPreExtractQueries - 0.030 s
success update schema - 0.129 s
success extract queries from components - 0.336 s
success write out requires - 0.057 s
success write out redirect data - 0.044 s
success onPostBootstrap - 0.045 s
⠀
info bootstrap finished - 16.437 s
⠀
success run static queries - 0.038 s
success run page queries - 0.109 s — 2/2 27.65 queries/second
onCreateWebpackConfig   webpack-theme-component-shadowing
onCreateWebpackConfig   webpack-theme-component-shadowing
success start webpack server - 1.506 s — 1/1 4.63 pages/second
 DONE  Compiled successfully in 4699ms

This is an example where it hangs on createPagesStatefully. I got it to continue the build by resizing the terminal window. What is internal-data-bridge any chance that could be the culprit in some way? That was featuring in both of the commands you have asked me to run so far.

Can you show what line it hung on?

createPagesStatefully dev-404-page

I am not 100%sure whether the dev-404-page part was there yet but it hung on the first instance of createPagesStatefully

Thanks. I'd like to try some more changes now, if you don't mind.

Please update the indicated lines in the following files:

node_modules/gatsby/dist/internal-plugins/internal-data-bridge/gatsby-node.js

- 114   chokidar.watch(pathToGatsbyConfig).on(`change`, () => {
+ 114   chokidar.watch(pathToGatsbyConfig, { useFsEvents: false }).on(`change`, () => {

node_modules/gatsby/dist/internal-plugins/dev-404-page/gatsby-node.js

- 30     chokidar.watch(source).on(`change`, () => copy()).on(`ready`, () => done());
+ 30     chokidar.watch(source, { useFsEvents: false }).on(`change`, () => copy()).on(`ready`, () => done());

node_modules/gatsby-page-utils/dist/watch-directory.js

  26               chokidar.watch(glob, {
  27                 cwd: path,
+ 28                 useFsEvents: false,
  29               }).on("add", function (path) {

node_modules/gatsby/dist/utils/get-static-dir.js

- 51   chokidar.watch(staticDir).on(`add`, path => {
+ 51   chokidar.watch(staticDir, { useFsEvents: false }).on(`add`, path => {

node_modules/gatsby/dist/query/query-watcher.js

- 237   watcher = chokidar.watch([slash(path.join(rootDir, `/src/**/*.{js,jsx,ts,tsx}`)), ...packagePaths]).on(`change`, path => {
+ 237   watcher = chokidar.watch([slash(path.join(rootDir, `/src/**/*.{js,jsx,ts,tsx}`)), ...packagePaths], { useFsEvents: false }).on(`change`, path => {

I have a suspicion that chokidar might be at fault here. It was upgraded to v3.x about a month ago, and it looks like they either started using fsevents, or they did something that caused some errant behavior with relation to fsevents. They have some open issues similar to the ones we are facing here, with node hanging when using chokidar.watch(). This seems to fit, since it is localized to MacOS due to fsevents being a Mac process, and the build seems to be failing in the modules where it is being used, or in modules that are writing out/processing files that are likely being watched.

chokidar.watch() exists in gatsby-source-filesystem, too, and that is where it was failing with your other repo, @ehowey; not to mention, gatsby-source-filesystem is not being called in your minimal repo, which kind of explains why it is making it past source and transform. There are instances of chokidar before internal-data-bridge, but I think on locations that are not affecting the build (e.g. query-watcher). However, I believe internal-data-bridge is the reason it was hanging while running the debugger; and in a straight-through run, it was likely affecting later stages of the build.

Let me know if this fixes the issues, or even if it makes it further than it has been. :crossed_fingers:

@Js-Brecht You are getting somewhere! I ran gatsby develop probably 15 times. It never hung on source and transform nodes or createPagesStatefully but it seemed to hang, maybe 2/10 times, on start webpack server. I could bump this along with the resizing terminal trick as well. Any chance you missed an instance of chokidar/fsevents that would relate to start webpack server?

As a side note it also seemed to move through the steps of the develop process much faster than before if that has anything to do with it?

That’s really good to hear. I did leave one instance of chokidar, on purpose, and it is right after it finishes the bootstrap and starts the server. It is in node_modules/gatsby/dist/commands/develop.js towards the end of the startServer() function. I’m not at a computer right now, or I would give you a diff.

You can find the exact line by doing:

cat node_modules/gatsby/dist/commands/develop.js |grep -n ‘chokidar’

I was pretty sure that if this fixed the bootstrap, then it would have to still hang at the server. Let me know if changing that makes it run reliably, and I’ll submit a PR and take care of letting people know.

I’m actually not surprised that it makes it run faster. The build on a barebones project usually takes me maybe a second, and I saw some crazy long runtimes for certain stages in your debug output. fsevents is supposed to speed things up, and take a lot of the load off the cpu, but there’s something funny going on that’s making it get stuck, instead.

@Js-Brecht My hat goes off to you. I have no idea how you pulled that rabbit out of a hat and bug fixed such a weird one from a distance but good work! I will wait for the diff to make changes to develop.js as I do not want to change the wrong thing, but I suspect that will fix it as it was hanging at that very last step when it starts the server a couple of times.

Here's the diff:

node_modules/gatsby/dist/commands/develop.js

- 337   chokidar.watch(watchGlobs).on(`change`, async () => {
+ 337   chokidar.watch(watchGlobs, { useFsEvents: false }).on(`change`, async () => {

I really appreciate your help, running all these tests. This has been a tricky one, but it has been an opportunity to dig into the internals of MacOS, and I always enjoy learning new things :slightly_smiling_face:

Please let me know how it goes; not completely out of the woods yet :sweat_smile:.

Works! I just ran gatsby develop 10+ times and it worked flawlessly. Thanks for your help in looking into this. Hopefully this makes for an improvement for the group of us experiencing this!

Great! Here in a little while, when I’ve got a couple minutes, I’ll put together a PR. Once it’s done, you’ll be able to use gatsby-dev-cli with my repo so that you have a working dev environment until a patch gets published (if you aren’t familiar with gatsby-dev-cli, I can help). I might try recruiting you to test the changes anyway, since I don’t have the right OS... same goes for everybody else on this thread experiencing this issue.

I’ll post back here when it’s done.

I've found another separate issue that also causes this symptom - https://github.com/gatsbyjs/gatsby/issues/17935

If anyone is using LokiJS and resizing the terminal doesn't fix it, it's quite likely the issue I've found.

Hey guys, @ehowey, please check out PR #17938. If anybody is willing to test, please do, and drop a line on the PR.

You can snag my repo, and use it as your gatsby source in your site by using gatsby-dev-cli. First, you need gatsby-dev-cli

# Use your package manager of choice, and install globally
npm install -g gatsby-dev-cli

Then, just clone the repo, and bootstrap it.

git clone --single-branch -b macos-fsevents-fix https://github.com/Js-Brecht/gatsby
cd gatsby
yarn bootstrap
# Set the target repo path for gatsby-dev
gatsby-dev -p "${PWD}"

Then go to the site you wish to use the repo in, and run gatsby-dev

# Disable file watching, unless you intend on making changes to the gatsby repo
gatsby-dev --scan-once

in my case, I'm using OSX and IntelliJ, I had to:

• Close the project in IntelliJ
• Try again (npm start)
• and reopen the project in Intellij
  I guess that the issue is related with IntelliJ indexing files

@steinitz, @rheinardkorf, @hbthen3rd, @sharvit, @JaKXz, @emilyaviva, @MaximeHeckel

Any of you guys willing to test #17938?

I should be able to take a look later tonight when I am home. Thanks for all your work on this!
On Oct 1, 2019, 10:23 AM -0600, Jeremy Albright notifications@github.com, wrote:

@steinitz, @rheinardkorf, @hbthen3rd, @sharvit, @JaKXz, @emilyaviva, @MaximeHeckel
Any of you guys willing to test #17938?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Any update on the fix for this? It's freezing at source and transform nodes for me and my friend as well after it tried [Firebase Source] Fetching data for ...

Resizing the terminal unfortunately doesn't seem to fix it for us

@rishabhaggarwal2 There's a similar issue that sounds like it might be what you're experiencing, where it will hang while retrieving data from an online source. Can you try using GATSBY_CONCURRENT_DOWNLOAD=10 gatsby develop?

Seeing this too. Can't run gatsby develop locally at all on the lumen startup. Keeps hanging on createPageStatefully or source and transform nodes

macOS Mojave 10.14.6
Gatsby CLI version 2.7.7
Gatsby version 2.14.1

For anybody else finding this issue, try

CHOKIDAR_USEPOLLING=1 gatsby develop

That should disable fsevents on MacOS, which seems to be a reliable workaround.

@Js-Brecht I confirm that it seems to fix it more permanently than the other workarounds mentioned here. Thanks for sharing.

@steinitz you say “more” permanently. Do you mean to say that it still happens when you use that switch?

@Js-Brecht No, sorry to be unclear.

I was alluding to the fact that other workarounds, including my own, seemed to work for a while but the problem returned. With your solution I provoked it by making changes and re-running gatsby develop (with your enhancement) but the build continues to work.

That said, this has been a rollercoaster ride so I remain emotionally prepared :smile: for the problem to return.

To be clear: so far, so good.

Oops, update: I relaunched WebStorm for a final test before posting this and now it's hung on source and transform nodes in the WebStorm terminal.

For anybody else finding this issue, try

CHOKIDAR_USEPOLLING=1 gatsby develop

That should disable fsevents on MacOS, which seems to be a reliable workaround.

@Js-Brecht I'm using ubutu 18.04 and I'm running into the same problem occasionally. Is there any suggestion for my OS? Would you provide some detail about the possible causes for this problem?

This has been solved thanks to the diligent work of @Js-Brecht and @RomanHotsiy. It was an upstream issue in fsevents, way beyond what I could have figured out on my own, and should be fixed in future updates once these changes are implemented and migrate down from fsevents into gatsby itself. For now a reliable workaround is to resize the terminal window, there is a way to fix this in your repo itself which is discussed here: https://github.com/gatsbyjs/gatsby/pull/17938 however you would have to redo this fix after any changes to your node_modules folder and might not be worth the work depending on how often you update your package versions.

I will leave the issue open until the fix migrates downstream into gatsby itself.

@Boty22 Ubuntu doesn't use fsevents, so it's probably something different. Some issues have been encountered when retrieving data from a remote location; check out #6654 and #17940 for some details on fixing concurrency issues.

Quick question: does resizing your terminal help? If so, then it could be _something_ similar to this issue.

@Js-Brecht resizing the terminal does not help for Ubuntu. I'm retrieving data from an AirTable table using the plugin gatsby-source-airtable BTW

That could be the problem. Check out this comment. If that doesn't work, might be better to open a new ticket

@steinitz sorry, I missed your comment. Can you try the fix described in #17938? More specifically, here and here. If it doesn't work, then there's probably more going on in your case.

Haven't had any issues with CHOKIDAR_USEPOLLING=1 gatsby develop since mentioned.

Thanks for the workaround @Js-Brecht

I'm seeing this too with 2.15.28 when I do gatsby build. Do i need to stop the gatsby develop in the other terminal? It's intermittently happening

Happened again without the dev server running. I've got a simple blog from following the blog tutorial though.

It seems to be almost every other run. I'm on a Mac btw

@canvaspixels does resizing the terminal window unstuck your build? If so, please try this and let us know if that helped https://github.com/gatsbyjs/gatsby/pull/17938#issuecomment-540661991

@RomanHotsiy that indeed sorts it! Thanks!

Hey everybody, the patched version of fsevents has been published. You should be able to simply delete your yarn.lock file, and run yarn again. Each of your dependencies should automatically pick up [email protected], which should resolve the issue.

Be careful - deleting your whole yarn.lock file can also cause other packages to be upgraded.

Another, more precise option if you're using yarn is to delete just the lines relating to fsevents in the yarn.lock and re-run yarn. This will cause only the affected package to be upgraded. So for example, I removed the following lines:

- 
- fsevents@^2.0.6:
-   version "2.0.7"
-   resolved "https://registry.yarnpkg.com/fsevents/-/fsevents-2.0.7.tgz#382c9b443c6cbac4c57187cdda23aa3bf1ccfc2a"
-   integrity sha512-a7YT0SV3RB+DjYcppwVDLtn13UQnmg0SWZS7ezZD0UjnLwXmy8Zm21GMVGLaFGimIqcvyMQaOJBrop8MyOp1kQ==

Another way of doing this is to use the resolutions feature of Yarn (https://yarnpkg.com/lang/en/docs/selective-version-resolutions/). By adding the following to your package.json and then running yarn, it will also upgrade transitive dependencies and update your yarn.lock file:

  "resolutions": {
    "fsevents": "^2.1.1"
  }

After you have done this and your lockfile is updated, you can remove the resolutions property from package.json again.

Edit: removed previous, incorrect version of instructions.

You can also run yarn upgrade chokidar@latest. That should rebuild the lock file with the correct version of fsevents, without touching anything else

Oh, wait. That's only if you have chokidar as a direct dependency 🙄 . Forgot. @karlhorky is right

I'm using npm. Deleting node_modules, running npm i --save [email protected] and then npm i did the trick for me. Took 19s to startup and I'm using the gatsby-lumen-starter as a base.

-- Edit:

I actually finished what I was working on, pushed to Netlify and it was completely unable to build because of fsevents (error [email protected]: The platform 'linux' is incompatible with this module).

I switched to yarn and had the same issue so I yarn removed fsevents entirely, and now local and netlify are both working...

Got the same issue and could not track the reason. This happen to me both on mac and pc. Could try to relate this to internet speed, however this happened to me also when I was connected to high speed network. What seems to be working for me right now is to add this in my env

GATSBY_CONCURRENT_DOWNLOAD=5

and running using --v8-pool-size=128

In my case it seems to be my firewall prompt on osx. If I'm fast enough to allow or deny connections coming from outside, it'll succeed flawlessly. Problem is, the prompt dialog pops up for a split second and disapears as Gatsby continues and then fails hanging on one of the many steps mentioned above.
Options:

• disable firewall (not having that)
• whitelist Gatsby (not consistent across projects and upgrades)
• find a way to halt on prompt until user selects option
• default bind address to localhost (should not require prompt to accept incoming connections).

@thomasthep The default bind address for the dev server is localhost already. It doesn't even listen for external connections unless you tell it to use your out-facing IP address (or 0.0.0.0). And even then, it doesn't start the dev server until after the bootstrap / build are done. If it's the bootstrap that is causing the firewall prompt, then it would have more to do with what plugins you are using, because Gatsby does not reach out to the internet in its default state.

There shouldn't even be any connections "coming from outside", generally speaking; even when a plugin is collecting information from an external source, the connection originates from your local host, not from the external source, and that is usually accepted as an "established connection" by most firewalls, in my experience, since most firewalls are configured to accept any outgoing connections.

I could see this happening if your firewall is configured to block processes, rather than just connections. In that case, it would be Node that would need to be whitelisted.

Would need more information to really understand why that's happening to you. It would probably be more effective to open a new ticket, though. This one had to do with fsevents on MacOS causing issues, and that has been addressed, which is why it's closed now. Anything not related to fsevents really should have its own issue, to get the attention it deserves.

For the record this also can happen if you have your GraphQL server running in debug mode and it's stopped at a breakpoint.

For the record: This started happening to me when I added gatsby-source-s3-image and the s3 bucket reached over 100 images. It goes through all 145 images at the source and transform nodes stage and then hangs there forever. It's still happening, I've tried the fixes above. Luckily, it works after 5-6 attempts so I am not blocked.

Weirdly resizing the terminal window worked for me

For me, limiting concurrent downloads as described here seems to help.

I added the following line to my .env file. The default is 200.
GATSBY_CONCURRENT_DOWNLOAD=50

Not sure if it solves your problem but maybe it helps someone :)

@rishabhaggarwal2 There's a similar issue that sounds like it might be what you're experiencing, where it will hang while retrieving data from an online source. Can you try using GATSBY_CONCURRENT_DOWNLOAD=10 gatsby develop?

Thank you, this fixed it for me. Since I was pulling a ton of content from a third party website it kept hanging on downloading the content. (97% - so close yet so far)