Virtualenv: Virtualenv cannot copy the built-in library

Created on 9 Dec 2020  ·  8Comments  ·  Source: pypa/virtualenv

Issue

I created a virtual environment,the /lib/ Python directory has no built-in libraries (only site-Package).
Because I'm going to use it.

Environment
python:3.8.1
Provide at least:

  • OS:Centos7.8
  • pip list of the host python where virtualenv is installed:

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

[root@baidu1 bin]# ./virtualenv -vvv --always-copy --clear /opt/env_test
80 setup logging to NOTSET [DEBUG report:43]
101 find interpreter for spec PythonSpec(path=/opt/python381/bin/python3.8) [INFO builtin:52]
101 proposed PythonInfo(spec=CPython3.8.1.final.0-64, exe=/opt/python381/bin/python3.8, platform=linux, version='3.8.1 (default, Dec  4 2020, 13:18:15) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:58]
101 accepted PythonInfo(spec=CPython3.8.1.final.0-64, exe=/opt/python381/bin/python3.8, platform=linux, version='3.8.1 (default, Dec  4 2020, 13:18:15) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:60]
104 filesystem is case-sensitive [DEBUG info:28]
149 create virtual environment via CPython3Posix(dest=/opt/env_test, clear=True, no_vcs_ignore=False, global=False) [INFO session:52]
149 delete /opt/env_test [DEBUG creator:168]
176 create folder /opt/env_test/bin [DEBUG _sync:25]
177 create folder /opt/env_test/lib/python3.8/site-packages [DEBUG _sync:25]
177 write /opt/env_test/pyvenv.cfg [DEBUG pyenv_cfg:34]
177     home = /opt/python381 [DEBUG pyenv_cfg:38]
177     implementation = CPython [DEBUG pyenv_cfg:38]
177     version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
177     virtualenv = 20.2.1 [DEBUG pyenv_cfg:38]
177     include-system-site-packages = false [DEBUG pyenv_cfg:38]
177     base-prefix = /opt/python381 [DEBUG pyenv_cfg:38]
177     base-exec-prefix = /opt/python381 [DEBUG pyenv_cfg:38]
177     base-executable = /opt/python381/bin/python3.8 [DEBUG pyenv_cfg:38]
177 copy /opt/python381/bin/python3.8 to /opt/env_test/bin/python [DEBUG _sync:52]
178 copy /opt/python381/bin/python3.8 to /opt/env_test/bin/python3 [DEBUG _sync:52]
178 copy /opt/python381/bin/python3.8 to /opt/env_test/bin/python3.8 [DEBUG _sync:52]
178 create virtualenv import hook file /opt/env_test/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:95]
179 create /opt/env_test/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:98]
179 ============================== target debug ============================== [DEBUG session:54]
179 debug via /opt/env_test/bin/python /opt/python381/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:223]
179 {
  "sys": {
    "executable": "/opt/env_test/bin/python",
    "_base_executable": "/opt/env_test/bin/python",
    "prefix": "/opt/env_test",
    "base_prefix": "/opt/python381",
    "real_prefix": null,
    "exec_prefix": "/opt/env_test",
    "base_exec_prefix": "/opt/python381",
    "path": [
      "/opt/python381/lib/python38.zip",
      "/opt/python381/lib/python3.8",
      "/opt/python381/lib/python3.8/lib-dynload",
      "/opt/env_test/lib/python3.8/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.8.1 (default, Dec  4 2020, 13:18:15) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]",
  "makefile_filename": "/opt/python381/lib/python3.8/config-3.8-x86_64-linux-gnu/Makefile",
  "os": "<module 'os' from '/opt/python381/lib/python3.8/os.py'>",
  "site": "<module 'site' from '/opt/python381/lib/python3.8/site.py'>",
  "datetime": "<module 'datetime' from '/opt/python381/lib/python3.8/datetime.py'>",
  "math": "<module 'math' from '/opt/python381/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'>",
  "json": "<module 'json' from '/opt/python381/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
220 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv) [INFO session:59]
223 got embed update of distribution pip from /root/.local/share/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:135]
226 got embed update of distribution pip from /root/.local/share/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:135]
227 got embed update of distribution setuptools from /root/.local/share/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:135]
228 got embed update of distribution setuptools from /root/.local/share/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:135]
229 got embed update of distribution wheel from /root/.local/share/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:135]
229 got embed update of distribution wheel from /root/.local/share/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:135]
230 install pip from wheel /opt/python381/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-20.2.4-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
230 Attempting to acquire lock 140543082325376 on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.2.4-py2.py3-none-any.lock [DEBUG filelock:270]
230 Lock 140543082325376 acquired on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.2.4-py2.py3-none-any.lock [INFO filelock:274]
230 Attempting to release lock 140543082325376 on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.2.4-py2.py3-none-any.lock [DEBUG filelock:315]
230 Lock 140543082325376 released on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.2.4-py2.py3-none-any.lock [INFO filelock:318]
231 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.2.4-py2.py3-none-any/pip to /opt/env_test/lib/python3.8/site-packages/pip [DEBUG _sync:52]
242 install setuptools from wheel /opt/python381/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/setuptools-50.3.2-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
243 Attempting to acquire lock 140542985448512 on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any.lock [DEBUG filelock:270]
243 Lock 140542985448512 acquired on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any.lock [INFO filelock:274]
243 Attempting to release lock 140542985448512 on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any.lock [DEBUG filelock:315]
243 Lock 140542985448512 released on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any.lock [INFO filelock:318]
244 copy /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any/distutils-precedence.pth to /opt/env_test/lib/python3.8/site-packages/distutils-precedence.pth [DEBUG _sync:52]
245 install wheel from wheel /opt/python381/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/wheel-0.35.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
246 Attempting to acquire lock 140542985447984 on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.35.1-py2.py3-none-any.lock [DEBUG filelock:270]
246 Lock 140542985447984 acquired on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.35.1-py2.py3-none-any.lock [INFO filelock:274]
246 Attempting to release lock 140542985447984 on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.35.1-py2.py3-none-any.lock [DEBUG filelock:315]
246 Lock 140542985447984 released on /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.35.1-py2.py3-none-any.lock [INFO filelock:318]
247 copy /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any/easy_install.py to /opt/env_test/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:52]
248 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any/_distutils_hack to /opt/env_test/lib/python3.8/site-packages/_distutils_hack [DEBUG _sync:52]
249 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any/pkg_resources to /opt/env_test/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:52]
252 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.35.1-py2.py3-none-any/wheel to /opt/env_test/lib/python3.8/site-packages/wheel [DEBUG _sync:52]
258 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.35.1-py2.py3-none-any/wheel-0.35.1.dist-info to /opt/env_test/lib/python3.8/site-packages/wheel-0.35.1.dist-info [DEBUG _sync:52]
260 copy /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.35.1-py2.py3-none-any/wheel-0.35.1.virtualenv to /opt/env_test/lib/python3.8/site-packages/wheel-0.35.1.virtualenv [DEBUG _sync:52]
262 generated console scripts wheel3.8 wheel wheel3 wheel-3.8 [DEBUG base:48]
267 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any/setuptools to /opt/env_test/lib/python3.8/site-packages/setuptools [DEBUG _sync:52]
336 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any/setuptools-50.3.2.dist-info to /opt/env_test/lib/python3.8/site-packages/setuptools-50.3.2.dist-info [DEBUG _sync:52]
338 copy /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-50.3.2-py3-none-any/setuptools-50.3.2.virtualenv to /opt/env_test/lib/python3.8/site-packages/setuptools-50.3.2.virtualenv [DEBUG _sync:52]
341 generated console scripts easy_install easy_install3.8 easy_install-3.8 easy_install3 [DEBUG base:48]
365 copy directory /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.2.4-py2.py3-none-any/pip-20.2.4.dist-info to /opt/env_test/lib/python3.8/site-packages/pip-20.2.4.dist-info [DEBUG _sync:52]
367 copy /root/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.2.4-py2.py3-none-any/pip-20.2.4.virtualenv to /opt/env_test/lib/python3.8/site-packages/pip-20.2.4.virtualenv [DEBUG _sync:52]
369 generated console scripts pip3.8 pip-3.8 pip pip3 [DEBUG base:48]
369 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
372 write /opt/env_test/pyvenv.cfg [DEBUG pyenv_cfg:34]
372     home = /opt/python381 [DEBUG pyenv_cfg:38]
372     implementation = CPython [DEBUG pyenv_cfg:38]
372     version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
373     virtualenv = 20.2.1 [DEBUG pyenv_cfg:38]
373     include-system-site-packages = false [DEBUG pyenv_cfg:38]
373     base-prefix = /opt/python381 [DEBUG pyenv_cfg:38]
373     base-exec-prefix = /opt/python381 [DEBUG pyenv_cfg:38]
373     base-executable = /opt/python381/bin/python3.8 [DEBUG pyenv_cfg:38]
373 created virtual environment CPython3.8.1.final.0-64 in 294ms
  creator CPython3Posix(dest=/opt/env_test, clear=True, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==20.2.4, setuptools==50.3.2, wheel==0.35.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:17]
[root@baidu1 bin]#
bug
Source
picture w345731923

All 8 comments

I read an article,when he creates it, he copies the.py file to lib,How do that?
https://stackoverflow.com/questions/23258029/virtualenv-doesnt-copy-all-py-files-from-the-lib-python-directory

w345731923 picture w345731923 on 9 Dec 2020

I read an article,when he creates it, he copies the.py file to lib,How do that?
stackoverflow.com/questions/23258029/virtualenv-doesnt-copy-all-py-files-from-the-lib-python-directory

It copied only part of the built-in packages, and that was for python 2 only.

I created a virtual environment,the /lib/ Python directory has no built-in libraries (only site-Package).
Because I'm going to use it.

That's by design. You'll probably need to expand more on how you want to use it and why.

gaborbernat picture gaborbernat on 9 Dec 2020

I read an article,when he creates it, he copies the.py file to lib,How do that?
stackoverflow.com/questions/23258029/virtualenv-doesnt-copy-all-py-files-from-the-lib-python-directory

It copied only part of the built-in packages, and that was for python 2 only.

I created a virtual environment,the /lib/ Python directory has no built-in libraries (only site-Package).
Because I'm going to use it.

That's by design. You'll probably need to expand more on how you want to use it and why.
I want to package the virtual environment,run it on a different machine./python3.

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = './python3'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = '/opt/env_test/bin/python3'
  sys.base_prefix = '/opt/python381'
  sys.base_exec_prefix = '/opt/python381'
  sys.executable = '/opt/env_test/bin/python3'
  sys.prefix = '/opt/python381'
  sys.exec_prefix = '/opt/python381'
  sys.path = [
    '/opt/python381/lib/python38.zip',
    '/opt/python381/lib/python3.8',
    '/opt/python381/lib/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding <
Python runtime state: core initialized \n
ModuleNotFoundError: No module named 'encodings'

Now there's a problem.

w345731923 picture w345731923 on 9 Dec 2020

I know I probably shouldn't be asking questions here, but I really don't know where to go, so I'm sorry.

w345731923 picture w345731923 on 9 Dec 2020

Please provide a detailed step-by-step guide on how you got there. Ideally a docker image.

gaborbernat picture gaborbernat on 9 Dec 2020
👀1
  1. Compile python381
  2. Install pip virtualenv
  3. Create virtual environment ./virtualenv -vvv --always-copy --clear /opt/env_test
  4. Copy libpython3.8.so.1.0 file to /opt/env_test/lib
  5. Compress/opt/env_test
  6. scp to another machine and decompress
  7. Set LD_LIBRARY_PATH to libpython3.8.so.1.0
  8. /env_test/bin/python3
w345731923 picture w345731923 on 9 Dec 2020

virtual environments are references to the global pythons, and not meant to live on their own. They are not meant to be copied to another machine. And they will not work unless you also copy the global python and create the virtual environment in copy mode. See https://www.youtube.com/watch?v=o1Vue9CWRxU

gaborbernat picture gaborbernat on 9 Dec 2020
👍1

virtual environments are references to the global pythons, and not meant to live on their own. They are not meant to be copied to another machine. And they will not work unless you also copy the global python and create the virtual environment in copy mode. See https://www.youtube.com/watch?v=o1Vue9CWRxU

Thx,video solved my problem.

w345731923 picture w345731923 on 11 Dec 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

ShuaiW picture ShuaiW  ·  22Comments
billputer picture billputer  ·  24Comments
jaraco picture jaraco  ·  27Comments
ssbarnea picture ssbarnea  ·  26Comments
JoDoX picture JoDoX  ·  26Comments