Firebase-tools: firebase.json 'functions.ignore' bloats package upload

Created on 15 Aug 2019 · 3Comments · Source: firebase/firebase-tools

Environment info

firebase-tools:

firebase --version
7.2.2

Platform:
Both MacOS (10.14.6 (18G87)) & Windows 10 development environments.

Test case

MCVE: Firebase functions template + a 'functions.ignore' section in firebase.json

Steps to reproduce

Create a firebase project.

Run firebase init functions with default options, and add it to your new project:

firebase init functions

Uncomment the helloWorld sample function in index.js and deploy:

⋊> ~/D/f/functions firebase deploy                                                                                    11:02:38

=== Deploying to 'test-function-deploy-3a957'...

i  deploying functions
i  functions: ensuring necessary APIs are enabled...
✔  functions: all necessary APIs are enabled
i  functions: preparing functions directory for uploading...
i  functions: packaged functions (22.95 KB) for uploading
✔  functions: functions folder uploaded successfully
i  functions: updating Node.js 8 function helloWorld(us-central1)...
✔  functions[helloWorld(us-central1)]: Successful update operation. 

✔  Deploy complete!

Project Console: https://console.firebase.google.com/project/test-function-deploy-3a957/overview
⋊> ~/D/f/functions

Note the deployed packaged functions size (22.95KB).

Edit firebase.json to include a 'functions.ignore' section:

{
  "functions": {
    "ignore": []
  }
}

Deploy again:

⋊> ~/D/f/functions firebase deploy                                                                                    11:03:24

=== Deploying to 'test-function-deploy-3a957'...

i  deploying functions
i  functions: ensuring necessary APIs are enabled...
✔  functions: all necessary APIs are enabled
i  functions: preparing functions directory for uploading...
i  functions: packaged functions (9.07 MB) for uploading
✔  functions: functions folder uploaded successfully
i  functions: updating Node.js 8 function helloWorld(us-central1)...
✔  functions[helloWorld(us-central1)]: Successful update operation. 

✔  Deploy complete!

Project Console: https://console.firebase.google.com/project/test-function-deploy-3a957/overview
⋊> ~/D/f/functions

Note that the deployed package has grown from 22.95KB to 9.07MB.

Expected behavior

I expect that if things are ignored, the deployed package gets smaller rather than larger.

Actual behavior

The deployed package increases in size by several orders of magnitude from 22.95KB to 9.07MB.

For reference, the size of the functions folder including local node_modules is 29MB in this case. The node_modules folder itself is 28.9MB on disk, and is ~6MB gzipped.

This also affects TypeScript projects, presumably because the issue is triggered after the tsc part of things.

Comment

I haven't yet found any good/definitive documentation for the firebase.json configuration file and specifically the functions section. There is some documentation of the hosting section here. My understanding of thefunctions.ignore structure/usage is currently based on https://github.com/firebase/firebase-tools/issues/291#issuecomment-322526710.

There is a related issue #429 where @laurenzlong indicates that the ignore section should be operational/fixed.

Open question

Where is the deployed package built to locally, before deploying? I ran with --debug and couldn't figure out any useful file paths.

Thanks!

Thanks for considering! And my phone tethering data plan thanks you for resolving this, as my current project deploy is ~55MB before I figured out what was going on :)

Source

repentsinner

👍1

Most helpful comment

The behavior that you described is the behavior that I would expect.

The documentation does not describe functions.ignore, as you pointed out; however, it is used by the CLI.

Unofficially (in that it is _undocumented_), firebase.ignore has a default of ["node_modules"]. This lets us limit the amount of stuff we have to upload during the deploy (which you can imagine could become very large, very quickly). By setting firebase.ignore in package.json, it tells the CLI that _nothing_ should be ignored. Now, the node_modules folder is included in the archive and sent to Functions.

To answer your question: at a high-level description of the process, the CLI doesn't build a package locally. It does roughly the following:

Runs any predeploy script for functions (also specified in firebase.json).
Parses the source code for functions to deploy.
Zips the source code (usually ignoring node_modules) and uploads it to Functions.

Once the source code is uploaded, the Functions infrastructure runs npm install for the package.json and the source is ready to go and serve users (at least, in this super hand-wavy explanation).

I hope that answers your questions. But since there's no CLI-related bug here, I'm going to close this issue.

bkendall on 15 Aug 2019

👍3

All 3 comments

The behavior that you described is the behavior that I would expect.

The documentation does not describe functions.ignore, as you pointed out; however, it is used by the CLI.

To answer your question: at a high-level description of the process, the CLI doesn't build a package locally. It does roughly the following:

Runs any predeploy script for functions (also specified in firebase.json).
Parses the source code for functions to deploy.
Zips the source code (usually ignoring node_modules) and uploads it to Functions.

Once the source code is uploaded, the Functions infrastructure runs npm install for the package.json and the source is ready to go and serve users (at least, in this super hand-wavy explanation).

I hope that answers your questions. But since there's no CLI-related bug here, I'm going to close this issue.

bkendall on 15 Aug 2019

👍3

Thanks for the info @bkendall.

Based on #291 I mistakenly thought that the CLI _always_ excluded node_modules, and that the stuff in functions.ignore was additive. Adding an explicit pattern for node_modules does do what you describe.

repentsinner on 15 Aug 2019

👍2

@bkendall Why not document it?