Virtualenv: Dropping executable bit on library files in `virtualenv==20.0.14`

Created on 26 Mar 2020  ·  7Comments  ·  Source: pypa/virtualenv

Issue

virtualenv==20.0.14 appears to be losing the executable bit on some library files when installing pyspark. I have experienced this behavior on both Ubuntu and MacOS, with python3.6.9 and 3.8.1.

Virtualenv created with virtualenv==2.0.13:

(pyspark_13) ross: ~/WorkZone/VirtualEnvironments/pyspark_13
$ pip list
Package    Version
---------- -------
pip        20.0.2
py4j       0.10.7
pyspark    2.4.4
setuptools 46.0.0
wheel      0.34.2

```console
(pyspark_13) ross: ~/WorkZone/VirtualEnvironments/pyspark_13
$ ll lib/python3.8/site-packages/pyspark/bin/
total 224
-rwxr-xr-x 1 ross staff 1.1K Mar 26 09:07 beeline
-rw-r--r-- 1 ross staff 1.0K Mar 26 09:07 beeline.cmd
-rwxr-xr-x 1 ross staff 5.3K Mar 26 09:07 docker-image-tool.sh
-rwxr-xr-x 1 ross staff 1.9K Mar 26 09:07 find-spark-home
-rw-r--r-- 1 ross staff 2.6K Mar 26 09:07 find-spark-home.cmd
-rw-r--r-- 1 ross staff 1.8K Mar 26 09:07 load-spark-env.cmd
-rw-r--r-- 1 ross staff 2.0K Mar 26 09:07 load-spark-env.sh
-rwxr-xr-x 1 ross staff 2.9K Mar 26 09:07 pyspark
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:07 pyspark.cmd
-rw-r--r-- 1 ross staff 1.5K Mar 26 09:07 pyspark2.cmd
-rwxr-xr-x 1 ross staff 1.0K Mar 26 09:07 run-example
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:07 run-example.cmd
-rwxr-xr-x 1 ross staff 3.1K Mar 26 09:07 spark-class
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:07 spark-class.cmd
-rw-r--r-- 1 ross staff 2.8K Mar 26 09:07 spark-class2.cmd
-rwxr-xr-x 1 ross staff 3.0K Mar 26 09:07 spark-shell
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:07 spark-shell.cmd
-rw-r--r-- 1 ross staff 1.8K Mar 26 09:07 spark-shell2.cmd
-rwxr-xr-x 1 ross staff 1.0K Mar 26 09:07 spark-sql
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:07 spark-sql.cmd
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:07 spark-sql2.cmd
-rwxr-xr-x 1 ross staff 1.0K Mar 26 09:07 spark-submit
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:07 spark-submit.cmd
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:07 spark-submit2.cmd
-rwxr-xr-x 1 ross staff 1.0K Mar 26 09:07 sparkR
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:07 sparkR.cmd
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:07 sparkR2.cmd


Virtualenv create with `virtualenv==2.0.14`:
```console
(pyspark_14) ross: ~/WorkZone/VirtualEnvironments/pyspark_14
$ pip list
Package    Version
---------- -------
pip        20.0.2
py4j       0.10.7
pyspark    2.4.4
setuptools 46.1.1
wheel      0.34.2

```console
(pyspark_14) ross: ~/WorkZone/VirtualEnvironments/pyspark_14
$ ll lib/python3.8/site-packages/pyspark/bin/
total 224
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:13 beeline
-rw-r--r-- 1 ross staff 1.0K Mar 26 09:13 beeline.cmd
-rw-r--r-- 1 ross staff 5.3K Mar 26 09:13 docker-image-tool.sh
-rw-r--r-- 1 ross staff 1.9K Mar 26 09:13 find-spark-home
-rw-r--r-- 1 ross staff 2.6K Mar 26 09:13 find-spark-home.cmd
-rw-r--r-- 1 ross staff 1.8K Mar 26 09:13 load-spark-env.cmd
-rw-r--r-- 1 ross staff 2.0K Mar 26 09:13 load-spark-env.sh
-rw-r--r-- 1 ross staff 2.9K Mar 26 09:13 pyspark
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:13 pyspark.cmd
-rw-r--r-- 1 ross staff 1.5K Mar 26 09:13 pyspark2.cmd
-rw-r--r-- 1 ross staff 1.0K Mar 26 09:13 run-example
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:13 run-example.cmd
-rw-r--r-- 1 ross staff 3.1K Mar 26 09:13 spark-class
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:13 spark-class.cmd
-rw-r--r-- 1 ross staff 2.8K Mar 26 09:13 spark-class2.cmd
-rw-r--r-- 1 ross staff 3.0K Mar 26 09:13 spark-shell
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:13 spark-shell.cmd
-rw-r--r-- 1 ross staff 1.8K Mar 26 09:13 spark-shell2.cmd
-rw-r--r-- 1 ross staff 1.0K Mar 26 09:13 spark-sql
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:13 spark-sql.cmd
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:13 spark-sql2.cmd
-rw-r--r-- 1 ross staff 1.0K Mar 26 09:13 spark-submit
-rw-r--r-- 1 ross staff 1.2K Mar 26 09:13 spark-submit.cmd
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:13 spark-submit2.cmd
-rw-r--r-- 1 ross staff 1.0K Mar 26 09:13 sparkR
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:13 sparkR.cmd
-rw-r--r-- 1 ross staff 1.1K Mar 26 09:13 sparkR2.cmd




**Environment**

Provide at least:
- OS: MacOS 10.14.6 (also observed on Ubuntu 16.0.4)
- ``pip list`` of the host python where ``virtualenv`` is installed:

`virtualenv==20.0.13`:
```console
$ python3.8 -m pip list
Package    Version
---------- -------
appdirs    1.4.3
distlib    0.3.0
filelock   3.0.12
pip        19.3.1
setuptools 42.0.2
six        1.14.0
virtualenv 20.0.13
wheel      0.33.6

virtualenv==20.0.14:

(virtualenv_14) ross: ~/WorkZone/VirtualEnvironments
$ pip list
Package    Version
---------- -------
appdirs    1.4.3
distlib    0.3.0
filelock   3.0.12
pip        20.0.2
setuptools 45.2.0
six        1.14.0
virtualenv 20.0.14
wheel      0.34.2

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

virtualenv==20.0.13:

ross: ~/WorkZone/VirtualEnvironments
$ virtualenv pyspark_13 -vvv --with-traceback
56 setup logging to NOTSET [DEBUG report:43]
88 find interpreter for spec PythonSpec(path=/usr/local/opt/[email protected]/bin/python3.8) [INFO builtin:44]
88 proposed PythonInfo(spec=CPython3.8.1.final.0-64, exe=/usr/local/opt/[email protected]/bin/python3.8, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:50]
88 accepted PythonInfo(spec=CPython3.8.1.final.0-64, exe=/usr/local/opt/[email protected]/bin/python3.8, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:52]
96 filesystem is not case-sensitive [DEBUG info:28]
253 create virtual environment via CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_13, clear=False, global=False) [INFO session:52]
253 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin [DEBUG _sync:21]
254 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages [DEBUG _sync:21]
255 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/pyvenv.cfg [DEBUG pyenv_cfg:34]
255     home = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
255     implementation = CPython [DEBUG pyenv_cfg:38]
255     version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
255     virtualenv = 20.0.13 [DEBUG pyenv_cfg:38]
255     include-system-site-packages = false [DEBUG pyenv_cfg:38]
255     base-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
255     base-exec-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
255     base-executable = /usr/local/opt/[email protected]/bin/python3.8 [DEBUG pyenv_cfg:38]
256 symlink /usr/local/opt/[email protected]/bin/python3.8 to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python [DEBUG _sync:40]
258 create /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:78]
259 create virtualenv import hook file /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:81]
259 ============================== target debug ============================== [DEBUG session:54]
259 debug via /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python /usr/local/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:187]
259 {
  "sys": {
    "executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python",
    "_base_executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python",
    "prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13",
    "base_prefix": "/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8",
    "real_prefix": null,
    "exec_prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13",
    "base_exec_prefix": "/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8",
    "path": [
      "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python38.zip",
      "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8",
      "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload",
      "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]",
  "os": "<module 'os' from '/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py'>",
  "site": "<module 'site' from '/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/site.py'>",
  "datetime": "<module 'datetime' from '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/datetime.py'>",
  "math": "<module 'math' from '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/math.cpython-38-darwin.so'>",
  "json": "<module 'json' from '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
315 add seed packages via FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1) [INFO session:59]
316 Attempting to acquire lock 4418023584 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:270]
316 Lock 4418023584 acquired on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:274]
319 get bundled wheel /usr/local/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/pip-20.0.2-py2.py3-none-any.whl [DEBUG acquire:51]
320 get bundled wheel /usr/local/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/setuptools-46.0.0-py3-none-any.whl [DEBUG acquire:51]
320 get bundled wheel /usr/local/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/wheel-0.34.2-py2.py3-none-any.whl [DEBUG acquire:51]
325 install wheel from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
326 install setuptools from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/setuptools-46.0.0-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
326 install pip from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/pip-20.0.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
327 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/setuptools-46.0.0.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/setuptools-46.0.0.dist-info [DEBUG _sync:48]
327 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:48]
327 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pip-20.0.2.virtualenv [DEBUG _sync:48]
328 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/wheel [DEBUG _sync:48]
328 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pip [DEBUG _sync:48]
342 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/setuptools-46.0.0.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/setuptools-46.0.0.virtualenv [DEBUG _sync:48]
344 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/easy_install.py to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:48]
346 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/setuptools to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/setuptools [DEBUG _sync:48]
354 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:48]
368 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/wheel3 to 755 [INFO util:566]
369 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/wheel-3.8 to 755 [INFO util:566]
370 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/wheel to 755 [INFO util:566]
370 generated console scripts wheel-3.8 wheel3 wheel [DEBUG base:53]
457 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/pkg_resources to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:48]
481 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/easy_install-3.8 to 755 [INFO util:566]
482 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/easy_install to 755 [INFO util:566]
482 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/easy_install3 to 755 [INFO util:566]
483 generated console scripts easy_install3 easy_install easy_install-3.8 [DEBUG base:53]
789 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pip-20.0.2.dist-info [DEBUG _sync:48]
799 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip3.8 to 755 [INFO util:566]
799 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip3 to 755 [INFO util:566]
799 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip to 755 [INFO util:566]
800 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip-3.8 to 755 [INFO util:566]
800 generated console scripts pip-3.8 pip pip3.8 pip3 [DEBUG base:53]
800 Attempting to release lock 4418023584 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:315]
800 Lock 4418023584 released on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:318]
800 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
807 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/pyvenv.cfg [DEBUG pyenv_cfg:34]
807     home = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
807     implementation = CPython [DEBUG pyenv_cfg:38]
807     version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
807     virtualenv = 20.0.13 [DEBUG pyenv_cfg:38]
807     include-system-site-packages = false [DEBUG pyenv_cfg:38]
807     base-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
807     base-exec-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
807     base-executable = /usr/local/opt/[email protected]/bin/python3.8 [DEBUG pyenv_cfg:38]
808 created virtual environment CPython3.8.1.final.0-64 in 753ms
  creator CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_13, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:21]

virtualenv==20.0.14:

(virtualenv_14) ross: ~/WorkZone/VirtualEnvironments
$ virtualenv pyspark_14 -vvv --with-traceback
92 setup logging to NOTSET [DEBUG report:43]
113 find interpreter for spec PythonSpec(path=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python) [INFO builtin:44]
114 discover exe for PythonInfo(spec=CPython3.8.1.final.0-64, exe=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) in /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8 [DEBUG py_info:352]
115 filesystem is not case-sensitive [DEBUG info:28]
116 Attempting to acquire lock 4565512544 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [DEBUG filelock:270]
116 Lock 4565512544 acquired on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [INFO filelock:274]
117 get PythonInfo from /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.json for /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/bin/python3.8 [DEBUG cached_py_info:79]
117 Attempting to release lock 4565512544 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [DEBUG filelock:315]
117 Lock 4565512544 released on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [INFO filelock:318]
118 Attempting to acquire lock 4565513456 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [DEBUG filelock:270]
118 Lock 4565513456 acquired on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [INFO filelock:274]
118 get PythonInfo from /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.json for /usr/local/opt/[email protected]/bin/python3.8 [DEBUG cached_py_info:79]
119 Attempting to release lock 4565513456 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [DEBUG filelock:315]
119 Lock 4565513456 released on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [INFO filelock:318]
119 proposed PythonInfo(spec=CPython3.8.1.final.0-64, system=/usr/local/opt/[email protected]/bin/python3.8, exe=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:50]
119 accepted PythonInfo(spec=CPython3.8.1.final.0-64, system=/usr/local/opt/[email protected]/bin/python3.8, exe=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:52]
223 create virtual environment via CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_14, clear=False, global=False) [INFO session:52]
223 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin [DEBUG _sync:21]
223 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages [DEBUG _sync:21]
223 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/pyvenv.cfg [DEBUG pyenv_cfg:34]
223     home = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
223     implementation = CPython [DEBUG pyenv_cfg:38]
224     version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
224     virtualenv = 20.0.14 [DEBUG pyenv_cfg:38]
224     include-system-site-packages = false [DEBUG pyenv_cfg:38]
224     base-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
224     base-exec-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
224     base-executable = /usr/local/opt/[email protected]/bin/python3.8 [DEBUG pyenv_cfg:38]
224 symlink /usr/local/opt/[email protected]/bin/python3.8 to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python [DEBUG _sync:40]
225 create virtualenv import hook file /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:76]
226 create /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:79]
226 ============================== target debug ============================== [DEBUG session:54]
226 debug via /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:187]
226 {
  "sys": {
    "executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python",
    "_base_executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python",
    "prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14",
    "base_prefix": "/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8",
    "real_prefix": null,
    "exec_prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14",
    "base_exec_prefix": "/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8",
    "path": [
      "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python38.zip",
      "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8",
      "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload",
      "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]",
  "os": "<module 'os' from '/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py'>",
  "site": "<module 'site' from '/usr/local/opt/[email protected]/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/site.py'>",
  "datetime": "<module 'datetime' from '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/datetime.py'>",
  "math": "<module 'math' from '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/math.cpython-38-darwin.so'>",
  "json": "<module 'json' from '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
275 add seed packages via FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1) [INFO session:59]
276 Attempting to acquire lock 4570361376 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:270]
276 Lock 4570361376 acquired on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:274]
278 get bundled wheel /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/pip-20.0.2-py2.py3-none-any.whl [DEBUG acquire:51]
278 get bundled wheel /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/setuptools-46.1.1-py3-none-any.whl [DEBUG acquire:51]
278 get bundled wheel /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/wheel-0.34.2-py2.py3-none-any.whl [DEBUG acquire:51]
283 install wheel from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
284 install setuptools from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/setuptools-46.1.1-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
284 install pip from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/pip-20.0.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
285 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:48]
286 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/easy_install.py to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:48]
286 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pip-20.0.2.virtualenv [DEBUG _sync:48]
287 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/wheel [DEBUG _sync:48]
287 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pip [DEBUG _sync:48]
288 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools-46.1.1.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/setuptools-46.1.1.virtualenv [DEBUG _sync:48]
289 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/setuptools [DEBUG _sync:48]
306 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:48]
316 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/wheel-3.8 to 755 [INFO util:566]
317 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/wheel to 755 [INFO util:566]
317 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/wheel3 to 755 [INFO util:566]
318 generated console scripts wheel3 wheel wheel-3.8 [DEBUG base:53]
380 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/pkg_resources to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:48]
398 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools-46.1.1.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/setuptools-46.1.1.dist-info [DEBUG _sync:48]
410 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/easy_install to 755 [INFO util:566]
411 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/easy_install-3.8 to 755 [INFO util:566]
411 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/easy_install3 to 755 [INFO util:566]
412 generated console scripts easy_install-3.8 easy_install3 easy_install [DEBUG base:53]
645 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pip-20.0.2.dist-info [DEBUG _sync:48]
653 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip-3.8 to 755 [INFO util:566]
654 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip3.8 to 755 [INFO util:566]
654 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip3 to 755 [INFO util:566]
655 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip to 755 [INFO util:566]
655 generated console scripts pip3.8 pip3 pip-3.8 pip [DEBUG base:53]
655 Attempting to release lock 4570361376 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:315]
655 Lock 4570361376 released on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:318]
655 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
659 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/pyvenv.cfg [DEBUG pyenv_cfg:34]
659     home = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
659     implementation = CPython [DEBUG pyenv_cfg:38]
659     version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
659     virtualenv = 20.0.14 [DEBUG pyenv_cfg:38]
659     include-system-site-packages = false [DEBUG pyenv_cfg:38]
659     base-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
659     base-exec-prefix = /usr/local/Cellar/[email protected]/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
659     base-executable = /usr/local/opt/[email protected]/bin/python3.8 [DEBUG pyenv_cfg:38]
660 created virtual environment CPython3.8.1.final.0-64 in 569ms
  creator CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_14, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:21]
bug

Most helpful comment

I believe this is due to setuptools >=46.1,<46.1.3 -- in particular https://github.com/pypa/setuptools/issues/1424

it appears to be fixed here: https://github.com/pypa/setuptools/pull/2046 (46.1.3)

All 7 comments

A PR would be welcomed; we need to understand when and why this happens first.

It appears the change might be in the setuptools version bump - looking at the diff between the versions I see this: https://github.com/pypa/setuptools/commit/7843688bc33dd4e13e10130bc49da4c290fe7d7f#diff-c02383bc12d5e72a50c798501bba7984

I believe this is due to setuptools >=46.1,<46.1.3 -- in particular https://github.com/pypa/setuptools/issues/1424

it appears to be fixed here: https://github.com/pypa/setuptools/pull/2046 (46.1.3)

Yes I agree it is setuptools 46.1.1 as embedded currently.

You can work around it by running virtualenv -p python3 --seeder pip --download and you will grab the latest from PyPI; things are working for me doing that and using 46.1.3

You don't need another seeder, just pass in the --download flag will suffice 👍 or the --setuptools 46.1.3 flag

Can we have good version of setuptools 46.1.3 bundled with virtualenv instead of 46.1.1 to avoid workarounds.

Was this page helpful?
0 / 5 - 0 ratings