Packer: WinRM copy of chef-cookbooks is too slow

Created on 2 Mar 2018 · 3Comments · Source: hashicorp/packer

I have seen a lot of issues regarding the same. But comparing how fast test kitchen (https://github.com/test-kitchen/winrm-transport) copies files over to a Windows machine for the same runlist, this seems like an issue that could be resolved. In my case the packer copy takes nearly an hour whereas the test kitchen does that in less than 5 minutes. This seems to me like a poor implementation. I would love to help in any way possible. This is slowing down our windows build by a very significant amount and testing out small changes is wasting too much engineering hours.

Ref: https://github.com/hashicorp/packer/issues/2648

communicatowinrm enhancement

Source

frezbo

👍10

Most helpful comment

Anyone looking to speed up their WinRM Windows builds can workaround this issue by using a file provisioner to copy over all the cookbooks in a zip file before the chef-solo provisioner runs and specifying the remote_cookbook_paths. Something like this has dramatically sped up my build times:

 "provisioners": [
    {
      "type": "file",
      "source": "berks-cookbooks.zip",
      "destination": "c:\\temp\\"
    },
    {
      "type": "powershell",
      "inline": [
        "Expand-Archive -LiteralPath c:\\temp\\berks-cookbooks.zip -DestinationPath C:\\temp"
      ]
    },
    {
      "type": "chef-solo",
      "version": "15.3.14",
      "chef_license": "accept",
      "guest_os_type": "windows",
      "remote_cookbook_paths": [
        "c:/temp/berks-cookbooks"
      ],
      "run_list": "{{user `ami_run_list`}}",
      }
    }

jdebbink on 18 Oct 2019

👍3 ❤2

All 3 comments

I noticed Packer would "hang" in the chef-solo provisioner step that copies over cookbooks. It looks almost like it is copying one file every 3 seconds. I logged into the EC2 instance during this operation and verified the file count is indeed going up that slowly.

Gist of relevant log output:
https://gist.github.com/JCapriotti/4ecfe4a1e20fcbd879161086b0815861

We unfortunately depend on about 13 cookbooks, and this adds about 45 minutes to our build. Just for copying files!

JCapriotti on 26 Nov 2018

 "provisioners": [
    {
      "type": "file",
      "source": "berks-cookbooks.zip",
      "destination": "c:\\temp\\"
    },
    {
      "type": "powershell",
      "inline": [
        "Expand-Archive -LiteralPath c:\\temp\\berks-cookbooks.zip -DestinationPath C:\\temp"
      ]
    },
    {
      "type": "chef-solo",
      "version": "15.3.14",
      "chef_license": "accept",
      "guest_os_type": "windows",
      "remote_cookbook_paths": [
        "c:/temp/berks-cookbooks"
      ],
      "run_list": "{{user `ami_run_list`}}",
      }
    }

jdebbink on 18 Oct 2019

👍3 ❤2

Its OK if you de-prioritize winRM, but this should be a performance bug rather than an enhancement. The bake times are intolerable even an average-sized list of cookbooks. Ours is < 3Mb, yet the upload takes 25 minutes!