Virtualenv: Breakage in --always-copy - fails with: ImportError: No module named time

I'm hitting this issue too, but as a workaround you can add no-setuptools to not install pip in your virtualenv. This issue is related to installing pip/setuptools in the new virtualenv.

My version information:

• Fedora 19 x86_64,
• System install of Python 2.7.5
• virtualenv 1.11.4
• pip 1.5.4

Here is the full verbose output on failure:

[vagrant@localhost ~]$ virtualenv --always-copy --clear --verbose foobar
Do not need to delete foobar/lib/python2.7; already gone
Not deleting foobar/bin
Creating foobar/lib/python2.7
Cannot find file lib (bad symlink)
Symlinking Python bootstrap modules
  Copying to foobar/lib/python2.7/config
  Copying to foobar/lib/python2.7/lib-dynload
  Creating parent directories for foobar/lib64/python2.7
  Copying to foobar/lib64/python2.7/os.py
  Ignoring built-in bootstrap module: posix
  Copying to foobar/lib64/python2.7/posixpath.py
  Cannot import bootstrap module: nt
  Copying to foobar/lib64/python2.7/ntpath.py
  Copying to foobar/lib64/python2.7/genericpath.py
  Copying to foobar/lib64/python2.7/fnmatch.py
  Copying to foobar/lib64/python2.7/locale.py
  Copying to foobar/lib64/python2.7/encodings
  Copying to foobar/lib64/python2.7/codecs.py
  Copying to foobar/lib64/python2.7/stat.py
  Copying to foobar/lib64/python2.7/UserDict.py
  Creating parent directories for foobar/lib64/python2.7/lib-dynload
  Copying to foobar/lib64/python2.7/lib-dynload/readline.so
  Copying to foobar/lib64/python2.7/copy_reg.py
  Copying to foobar/lib64/python2.7/types.py
  Copying to foobar/lib64/python2.7/re.py
  Copying to foobar/lib64/python2.7/sre.py
  Copying to foobar/lib64/python2.7/sre_parse.py
  Copying to foobar/lib64/python2.7/sre_constants.py
  Copying to foobar/lib64/python2.7/sre_compile.py
  Copying to foobar/lib64/python2.7/lib-dynload/zlibmodule.so
  Copying to foobar/lib64/python2.7/warnings.py
  Copying to foobar/lib64/python2.7/linecache.py
  Copying to foobar/lib64/python2.7/_abcoll.py
  Copying to foobar/lib64/python2.7/abc.py
  Copying to foobar/lib64/python2.7/_weakrefset.py
Creating foobar/lib/python2.7/site-packages
Writing foobar/lib64/python2.7/site.py
Writing foobar/lib64/python2.7/orig-prefix.txt
Writing foobar/lib64/python2.7/no-global-site-packages.txt
Creating parent directories for foobar/include
Copying to foobar/include/python2.7
Creating foobar/bin
New python executable in foobar/bin/python
Changed mode of foobar/bin/python to 0775
Copying to foobar/bin/python2
Copying to foobar/bin/python2.7
Testing executable with foobar/bin/python -c "import sys;out=sys.stdout;getattr(out, "buffer", out).write(sys.prefix.encode("utf-8"))"
Got sys.prefix result: u'/home/vagrant/foobar'
Creating foobar/lib64/python2.7/distutils
Writing foobar/lib64/python2.7/distutils/__init__.py
Writing foobar/lib64/python2.7/distutils/distutils.cfg
Installing setuptools, pip...
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
    File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/log.py", line 6, in <module>
    File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
      import sys, os, time, cStringIO, traceback, warnings, weakref
  ImportError: No module named time
  Complete output from command /home/vagrant/foobar/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/log.py", line 6, in <module>
  File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref
ImportError: No module named time
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "/usr/bin/virtualenv", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 824, in main
    symlink=options.symlink)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 992, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 960, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 902, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/vagrant/foobar/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1

Here is the full verbose output on success:

[vagrant@localhost ~]$ virtualenv --always-copy --clear --verbose --no-setuptools foobar
Do not need to delete foobar/lib/python2.7; already gone
Not deleting foobar/bin
Creating foobar/lib/python2.7
Cannot find file lib (bad symlink)
Symlinking Python bootstrap modules
  Copying to foobar/lib/python2.7/config
  Copying to foobar/lib/python2.7/lib-dynload
  Creating parent directories for foobar/lib64/python2.7
  Copying to foobar/lib64/python2.7/os.py
  Ignoring built-in bootstrap module: posix
  Copying to foobar/lib64/python2.7/posixpath.py
  Cannot import bootstrap module: nt
  Copying to foobar/lib64/python2.7/ntpath.py
  Copying to foobar/lib64/python2.7/genericpath.py
  Copying to foobar/lib64/python2.7/fnmatch.py
  Copying to foobar/lib64/python2.7/locale.py
  Copying to foobar/lib64/python2.7/encodings
  Copying to foobar/lib64/python2.7/codecs.py
  Copying to foobar/lib64/python2.7/stat.py
  Copying to foobar/lib64/python2.7/UserDict.py
  Creating parent directories for foobar/lib64/python2.7/lib-dynload
  Copying to foobar/lib64/python2.7/lib-dynload/readline.so
  Copying to foobar/lib64/python2.7/copy_reg.py
  Copying to foobar/lib64/python2.7/types.py
  Copying to foobar/lib64/python2.7/re.py
  Copying to foobar/lib64/python2.7/sre.py
  Copying to foobar/lib64/python2.7/sre_parse.py
  Copying to foobar/lib64/python2.7/sre_constants.py
  Copying to foobar/lib64/python2.7/sre_compile.py
  Copying to foobar/lib64/python2.7/lib-dynload/zlibmodule.so
  Copying to foobar/lib64/python2.7/warnings.py
  Copying to foobar/lib64/python2.7/linecache.py
  Copying to foobar/lib64/python2.7/_abcoll.py
  Copying to foobar/lib64/python2.7/abc.py
  Copying to foobar/lib64/python2.7/_weakrefset.py
Creating foobar/lib/python2.7/site-packages
Writing foobar/lib64/python2.7/site.py
Writing foobar/lib64/python2.7/orig-prefix.txt
Writing foobar/lib64/python2.7/no-global-site-packages.txt
Creating parent directories for foobar/include
Copying to foobar/include/python2.7
Creating foobar/bin
New python executable in foobar/bin/python
Changed mode of foobar/bin/python to 0775
Copying to foobar/bin/python2
Copying to foobar/bin/python2.7
Testing executable with foobar/bin/python -c "import sys;out=sys.stdout;getattr(out, "buffer", out).write(sys.prefix.encode("utf-8"))"
Got sys.prefix result: u'/home/vagrant/foobar'
Creating foobar/lib64/python2.7/distutils
Writing foobar/lib64/python2.7/distutils/__init__.py
Writing foobar/lib64/python2.7/distutils/distutils.cfg
Writing foobar/bin/activate
Writing foobar/bin/activate.fish
Writing foobar/bin/activate_this.py
Writing foobar/bin/activate.csh

In both the failing and successful cases the first error line is:

Cannot find file lib (bad symlink)

This log isn't useful because I don't have the fullpath to the file it can't find. I hence hacked virtualenv.py to print out os.path.abspath for this file and got:

Cannot find file /home/vagrant/lib (bad symlink)

It's looking for a lib folder in the current working directory.

the same here on Centos7 with stock python 2.7

$ virtualenv --always-copy /opt/
New python executable in /opt/bin/python
Installing setuptools, pip...
  Complete output from command /opt/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-6.0.6-py2.py3-none-any.whl/pip/__init__.py", line 4, in <module>
  File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref
ImportError: No module named time
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "/usr/bin/virtualenv", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 825, in main
    symlink=options.symlink)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 993, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 961, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 903, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /opt/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1

Same here. Using --always-copy on CentOS 6.5, system Python 2.6 ends up with same ImportError: No module named time as mentioned.

the solution is to forget about the --always-copy option on EL based distros

there seems to be a typo in the fix_lib64 function that is currently running. I correct this and it works as expected. go to line:

   1579         if symlink:                              
   1580             os.symlink('lib', lib64_link)      
   1581         else:                                    
   1582             copyfile('lib', lib64_link)        

replace 'lib' with lib_dir

  1579         if symlink:                              
   1580             os.symlink(lib_dir, lib64_link)      
   1581         else:                                    
   1582             copyfile(lib_dir, lib64_link)        

Late, but hope this helps

I'm getting the same problem here as well using CentOS 7.1 with flags --no-setup-tools and --always-copy. I'd love for this to be fixed since I'm packaging up the virtualenv and shipping them to customers (that are also on CentOS7+ so it's not a major risk).

Same here on Gentoo x86_64 with --always-copy.

@jdelgad and @selurvedu did you try what i recommended above?

@ikiini Yes, your patch solves the problem.

@ikiini It worked! Thank you so much. Now to either get this into RHEL 7 or into the main branch.

+1

great. Sorry for the delayed response..but you're welcome @selurvedu and @jdelgad.

Only problem seems to be @ikiini 's patch would seem to regress back on https://github.com/pypa/virtualenv/issues/338 which is when it was actually changed from lib_dir to 'lib'

Problem still persists with CentOS 7 and python 2.7

[root@st2w-master-el7-5696a8a082 ~]# virtualenv no_copy
New python executable in no_copy/bin/python
Installing setuptools, pip, wheel...done.
[root@st2w-master-el7-5696a8a082 ~]# virtualenv --always-copy yes_copy
Cannot find file lib (bad symlink)
New python executable in yes_copy/bin/python
Installing setuptools, pip, wheel...
  Complete output from command /root/yes_copy/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip wheel:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-7.1.2-py2.py3-none-any.whl/pip/__init__.py", line 4, in <module>
  File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref
ImportError: No module named time
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "/bin/virtualenv", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 832, in main
    symlink=options.symlink)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 1004, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 969, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 910, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /root/yes_copy/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip wheel failed with error code 1
[root@st2w-master-el7-5696a8a082 ~]# virtualenv --version
13.1.2
[root@st2w-master-el7-5696a8a082 ~]# ls -rlth yes_copy/
total 0
drwxr-xr-x. 3 root root 22 Jan 13 22:04 lib
drwxr-xr-x. 3 root root 22 Jan 13 22:04 lib64
drwxr-xr-x. 3 root root 22 Jan 13 22:04 include
drwxr-xr-x. 2 root root 49 Jan 13 22:04 bin
[root@st2w-master-el7-5696a8a082 ~]#

@bwilliams42 - Your change indeed fixed the problem for symlink-ing logic. However, things are fundamentally broken when not using --always-copy. Note that copyfile has a "symlink" kwarg that defaults to True. The following logic is effectively, if symlink, call os.symlink, else call copyfile w/o the "symlink" kwarg which results in copyfile defaulting symlink=True and a symlink is created by copyfile. Not useful at all. A symlink is the result no matter what.

Shouldn't the patch be something like the following?

   1579         if symlink:                              
   1580             os.symlink(lib_dir, lib64_link)      
   1581         else:                                    
   1582             copyfile(lib_dir, lib64_link, symlink=symlink)

I had an issue whereby I did this and I had a symlink to /usr/lib64. When RH updated some of the openssl libraries this was broken (I was using ansible) with an error stating something like openssl_meth function. I fixed it by doing a deep copy of /usr/lib64. Not a complete fix @bwilliams42 but pointed me in the right direction.

Problem still persists on Centos 7.2. This ticket has been reported 2.5 years ago. It seems it's very serious problem. How can I help to fix it?

Someone needs to provide a working fix. There's one fix that's been proposed here, but it apparently has a couple of problems. I'd suggest starting there.

BTW, I'm trying to build and pack my python appliction into virtual env on special VM under Vagrant with help of fabric, so it's pretty easy to reproduce this bug with Vagrant + centos box + simple fabric script.

I don't have inner knowledge about how virtualenv is supposed to work, so I need some guidance.

I can't help there sorry, as I'm on Windows, and the whole question of symlinking vs copying (or lib64 directories, which seem involved in this) isn't relevant there.

Interesting that (I'm on Amazon Linux) when I do virtualenv --always-copy app I get this error (Cannot find file lib (bad symlink)) but when I do mkdir app; cd app; virtualenv --always-copy . I do not. And it seems I understand why - when the app is created by the virtualenv it has lib64 as a directory, but when I create the virtualenv in the current directory it becomes a link to lib.

virtualenv.py > def path_locations starts with this comment:

XXX: We'd use distutils.sysconfig.get_python_inc/lib but its
prefix arg is broken: http://bugs.python.org/issue3386

This issue has been solved about ten years ago. Maybe reconsider using it ?
Just giving another angle; writing a reliable fix for all platforms is by no means an easy task.

@kika Thank you so much.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.

This is becoming obsolete with #1366 now in full swing. We'll reopen if someone can still replicate with that.