vagrant box add fails to unpack a large box

Vagrant version

Vagrant 2.0.2

Host operating system

macOS 10.11.6 and macOS 10.13.3

Guest operating system

macOS 10.11.6

Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "suran/macos1011"
  config.vm.box_url = "http://download.suran.com/developer/macos1011-0.1.0.box"
  config.vm.box_download_checksum = "b1bb86a7b56bc9a29bc6066d78ae22a76061e158"
  config.vm.box_download_checksum_type = "sha1"
end

Debug output

debug.log

Expected behavior

The machine is booted and accessible.

Actual behavior

An error occurs:

There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["import", "/Users/administrator/.vagrant.d/boxes/suran-VAGRANTSLASH-macos1011/0/virtualbox/box.ovf", "--vsys", "0", "--vmname", "macos1011_1520735715622_34133", "--vsys", "0", "--unit", "9", "--disk", "/Users/administrator/VirtualBox VMs/macos1011_1520735715622_34133/macos1011-disk001.vmdk"]

Stderr: 0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Interpreting /Users/administrator/.vagrant.d/boxes/suran-VAGRANTSLASH-macos1011/0/virtualbox/box.ovf...
OK.
0%...
Progress state: VBOX_E_FILE_ERROR
VBoxManage: error: Appliance import failed
VBoxManage: error: Could not open the medium storage unit '/Users/administrator/.vagrant.d/boxes/suran-VAGRANTSLASH-macos1011/0/virtualbox/macos1011-disk001.vmdk'.
VBoxManage: error: VMDK: error reading the magic number in '/Users/administrator/.vagrant.d/boxes/suran-VAGRANTSLASH-macos1011/0/virtualbox/macos1011-disk001.vmdk' (VERR_EOF).
VBoxManage: error: VD: error VERR_VD_VMDK_INVALID_HEADER opening image file '/Users/administrator/.vagrant.d/boxes/suran-VAGRANTSLASH-macos1011/0/virtualbox/macos1011-disk001.vmdk' (VERR_VD_VMDK_INVALID_HEADER)
VBoxManage: error: Details: code VBOX_E_FILE_ERROR (0x80bb0004), component ApplianceWrap, interface IAppliance
VBoxManage: error: Context: "RTEXITCODE handleImportAppliance(HandlerArg *)" at line 886 of file VBoxManageAppliance.cpp

Contents of /Users/administrator/.vagrant.d/boxes/suran-VAGRANTSLASH-macos1011/0/virtualbox/:

total 40
drwxr-xr-x  6 administrator  staff   204 Mar 10 19:43 .
drwxr-xr-x  3 administrator  staff   102 Mar 10 19:43 ..
-rw-r--r--  1 administrator  staff  3276 Mar 10 19:42 Vagrantfile
-rw-r--r--  1 administrator  staff  8993 Mar 10 19:42 box.ovf
-rw-r--r--  1 administrator  staff     0 Mar 10 19:42 macos1011-disk001.vmdk
-rw-r--r--  1 administrator  staff    26 Mar 10 19:43 metadata.json

Steps to reproduce

1. Create the above vagrantfile
2. vagrant up

Workaround

Use GNU tar.

1. brew install gnu-tar
2. sudo cp /usr/local/bin/gtar /opt/vagrant/embedded/bin/bsdtar

Notes

It appears the issue is related to the embedded bsdtar utility being unable to extract this box. The box was created with packer 1.2.1 on the same box using the macOS boxcutter template with a modified cmtool.sh. The only modifications to this script are to pre-install homebrew and salt.

Here is the output from using the embedded bsdtar utility to unpack the box:

$ /opt/vagrant/embedded/bin/bsdtar -v -xzf macos1011-0.1.0.box
x Vagrantfile
x box.ovf
x macos1011-disk001.vmdk
x metadata.json
$ ls -lh Vagrantfile box.ovf macos1011-disk001.vmdk metadata.json
-rw-r--r--  1 administrator  staff   3.2K Mar  6 19:48 Vagrantfile
-rw-r--r--  1 administrator  staff   8.8K Mar  6 19:48 box.ovf
-rw-r--r--  1 administrator  staff     0B Mar  6 19:48 macos1011-disk001.vmdk
-rw-r--r--  1 administrator  staff    26B Mar  6 19:48 metadata.json

Here is the same output using GNU tar:

$ /usr/local/bin/gtar -v -zxf macos1011-0.1.0.box
Vagrantfile
box.ovf
macos1011-disk001.vmdk
metadata.json
$ ls -lh Vagrantfile box.ovf macos1011-disk001.vmdk metadata.json
-rw-r--r--  1 administrator  staff   3.2K Mar  6 19:48 Vagrantfile
-rw-r--r--  1 administrator  staff   8.8K Mar  6 19:48 box.ovf
-rw-r--r--  1 administrator  staff   9.5G Mar  6 19:48 macos1011-disk001.vmdk
-rw-r--r--  1 administrator  staff    26B Mar  6 19:48 metadata.json

This is a recent problem and is not exhibited with a macOS 10.12 box built from the same templates. I suspect this error is to due to the size of the macOS 10.11 box growing from recent security updates.

This may be a packer issue, but the above workaround applies to vagrant, so I figured I'd start here.