Pip: windows in-place pip upgrades

Use python -m pip install --upgrade pip. I think @vsajip was considering some form of magic in the exe wrappers to mean that self-replacement of the exe could be made to work. But I don't know if he did anything in the version of distlib we're using. I'm not actually sure it's something that _could_ be done without help from the application (pip). And I'm pretty sure no code has been added to pip for this.

Personally, I consider this to be a minor issue - telling people to use `python -m pip`` seems to me to be a perfectly acceptable answer, given that the issue is fundamental to how Windows handles deletion of open files.

Ultimately it's going to be a question of how much complexity (with the consequent maintenance cost) and risk we want to take on in order to solve this.

Personally i'm ok with pip install --upgrade pip printing a message to use python -m pip install --upgrade pip. I just don't think we should throw an error about open file handles or whatever the exception is.

I was just about to say the same thing.

Here's a question - would you want that message to only appear on Windows, or would it be reasonable to say the same on Unix (for a consistent user experience). Personally, I'd like to be consistent across platforms (because it's easier to document that way).

Also, we should trap any command that attempts to (re-)install pip - pip install -I pip is another one, for example.

Personally I think it should be windows only. I view it as a work around for the way the Windows platform works, and something that's a target to eventually remove for something nicer down the road.

I knew you were going to say that :-(

Frankly, I don't believe we will ever get "something nicer", basically because we don't have enough Windows expertise to develop and maintain it. I know of _no_ Windows software with a satisfactory self-replacement of executables solution. I certainly don't have an answer, and I wouldn't expect any of the Unix-based developers to be able to help. Even if we did get something, it's likely to be complex and so a probable source of bit rot and obscure bugs.

Making a solution that works on all platforms the "official answer" seems to me to avoid sending confusing messages to the user. Back it up with a warning as described and we have a robust solution, even if it's not quite as perfect as getting pip install -U pip to work.

Please excuse a certain level of mild grumpiness in this reply, caused by a feeling that it's always the Windows platform that has to be the one to compromise - I don't mean to make an issue out of this. In practice, I'll go with whatever the consensus is. I don't think it's a big deal.

FWIW my goal after PEP453 is done is to get our own testing infrastructure stood up which includes Windows, so hopefully that'll solve some of the windows bitrot problems.

I can understand the grumpyness though :) If we were designing this system brand new I'd be a lot more amendable to that but I don't think it's user friendly to break pip install -U pip for everything non Windows in order to maintain consistency. If this was a brand new thing then there's no (or less at least) user epectation :]

OK, that sounds fair. You're right that historical usage is more important here.

I actually have no feel at all for what the user experience on Unix is, so I'm probably just as guilty of one-sided thinking as anyone else. I must try to see how using Python on Linux feels - is there a good "Python on Linux for beginners" resource that shows how to set up a Python (3) environment, get the various tools set up, etc? [One thing I always find doesn't get covered, is how to make "python" run Python 3 by default, how to make "virtualenv" build Python 3 virtualenvs by default, all that sort of thing - it seems to me that Python 3 is much more second-class on Unix than on Windows, oddly enough.]

Typically you don't want to make python point to python3 on *nix because the system uses Python and not all of their libraries are going to be ported to Python3. However there are some Linux distros that have Python3 as a default now... I think Arch? And Maybe the next Fedora?

Hmm, I'm not talking at a system level, but really you can't choose what to use for Python in your personal account? That definitely makes Python 3 second-class, to my way of thinking... But it does explain why Unix users don't seem to find using things like "pip3" or "pip3.3" as ugly as sin :-)

(Anyway, we're way off-topic here, so never mind).

You can setup your shell so it has ~/bin/ before the system bin and add a symlink there from python to python3, but yea offtopic. I'm happy to help though if you're actually interested in getting things set up :] IRC or Email.

My work which I believe Paul was referring to was just to implement the 'deleteme' dance for replacing files in use (i.e. exe wrappers) - that's in distlib Python code, not in the .exe wrappers. This is in the distlib.scripts module and should be exercised whenever that module is used to create a script.

So perhaps this is already solved for Wheels? And it's a setuptools problem elsewise?

Sounds possible. I'll do some tests today, if I can.

Sadly not, at least with "pip install -I pip". (It's hard to test an actual upgrade, as only the absolute latest version of pip has script wrapper support):

>pip install -f $x -I pip --use-wheel --no-index --pre
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking pip
Installing collected packages: pip
  Found existing installation: pip 1.5.dev1
    Uninstalling pip:
      Successfully uninstalled pip
Cleaning up...
Exception:
Traceback (most recent call last):
  File "C:\Work\Scratch\xxyy\lib\shutil.py", line 365, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'c:\\users\\uk03306\\appdata\\local\\temp\\pip-zhbsiy-uninstall\\work\\scratch\\xxyy\\scripts\\pip.exe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Work\Scratch\xxyy\lib\site-packages\pip\basecommand.py", line 121, in main
    status = self.run(options, args)
  File "C:\Work\Scratch\xxyy\lib\site-packages\pip\commands\install.py", line 258, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "C:\Work\Scratch\xxyy\lib\site-packages\pip\req.py", line 1330, in install
    requirement.commit_uninstall()
  File "C:\Work\Scratch\xxyy\lib\site-packages\pip\req.py", line 557, in commit_uninstall
    self.uninstalled.commit()
  File "C:\Work\Scratch\xxyy\lib\site-packages\pip\req.py", line 1743, in commit
    rmtree(self.save_dir)
  File "C:\Work\Scratch\xxyy\lib\site-packages\pip\util.py", line 42, in rmtree
    onerror=rmtree_errorhandler)
  File "C:\Work\Scratch\xxyy\lib\shutil.py", line 460, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Work\Scratch\xxyy\lib\shutil.py", line 362, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Work\Scratch\xxyy\lib\shutil.py", line 362, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Work\Scratch\xxyy\lib\shutil.py", line 362, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Work\Scratch\xxyy\lib\shutil.py", line 362, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Work\Scratch\xxyy\lib\shutil.py", line 367, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Work\Scratch\xxyy\lib\site-packages\pip\util.py", line 61, in rmtree_errorhandler
    func(path)
PermissionError: [WinError 5] Access is denied: 'c:\\users\\uk03306\\appdata\\local\\temp\\pip-zhbsiy-uninstall\\work\\scratch\\xxyy\\scripts\\pip.exe'

Storing complete log in C:\Users\uk03306\pip\pip.log

So pip's not using distlib.scripts to install scripts ... I guess it doesn't fit in the pip workflow.

I suspect it's because we do an uninstall then an install, rather than an install overwriting what's there (for obvious reasons).

Windows doesn't allow overwriting or deleting open exe files, but it does allow renaming/moving them.

Well, with some tomfoolery with the Win32 API you can close the handle to pip.exe much like you can when using Sysinternal's process explorer.

I don't think you can close all handles to the currently running exe. But if you have working code as an example, I'd love to see it, as this has long been a problem that people have debated and the consensus has always been that it's not possible (except by tricks like "rename and delete later"). Working code that does this would be extremely useful to the community.

Run into the same issue.

As a quick hack, can pip.exe detect that it is being upgraded prior to execution and restart itself with python -m pip {{ sys.argv }}?

I don't think that actually works.

But maybe a stupid thought, afaik the .exe doesn't need to be updated, can we just special case pip.exe and leave it alone? Would possibly leave a pip.exe laying around after an install though.

@dstufft correct, "restarting yourself" isn't as simple as @techtonik makes it sound on Windows, where there's no equivalent of the Unix exec call.

Technically, pip.exe may need updating if the entry point changes or if the distlib wrapper code is changed. That's not common, but it can happen (early on, the distlib code changed a fair bit till things settled down). What it may be possible to do would be to write the new pip.exe to a temporary location, then if it's identical to the current one, skip it. You wouldn't avoid errors when it did need updating, but in the 99% of cases when it didn't, you'd be fine.

BTW, we're going round in circles here. According to Vinay's earlier comments, this is solved in distlib (which is where I think it should be solved) but pip doesn't use the API in a way that allows it to work. If we could use the distlib solution, then great. Otherwise, we need someone to write a pip-specific solution (and I remain skeptical that this can be done in a robust, maintainable manner that isn't far more complex than the issue warrants).

Suggestion:

1. Someone add a section to the pip docs at http://www.pip-installer.org/en/latest/installing.html#install-or-upgrade-pip noting that the python -m pip workaround is needed on Windows when upgrading pip with itself. I might get a chance to do this at some point, but PRs from people who have contributed to this issue would be gratefully accepted :-).
2. If anyone has a viable solution, and is prepared to implement it, they should raise a PR and we can review actual code rather than proposed approaches.

I think we're at the stage where actual working code is the only thing that will move this issue forward.

I am -1 on bloating the docs without trying other solutions.

I'm +1 on documenting what works right now.

Even with doc comment it is still workaround - not a solution, and you've got command prompt for Windows wrong.

Maybe this is crazy but I always just do this:

$ curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python

of course this requires one to have libcurl, but there are windows binaries available here.

I've tried to install it on a Mac but I received this

Alexs-MacBook-Air:Pipmodules aleand$ pip install -U pip
-bash: pip: command not found
Alexs-MacBook-Air:Pipmodules aleand$ python get-pip.py
Downloading/unpacking pip
Downloading pip-1.5.5-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: pip
Cleaning up...
Exception:
Traceback (most recent call last):
File "/var/folders/8g/b9q8snrd5ln_d5hqnwh2_86r0000gn/T/tmpFTGrC3/pip.zip/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/var/folders/8g/b9q8snrd5ln_d5hqnwh2_86r0000gn/T/tmpFTGrC3/pip.zip/pip/commands/install.py", line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "/var/folders/8g/b9q8snrd5ln_d5hqnwh2_86r0000gn/T/tmpFTGrC3/pip.zip/pip/req.py", line 1435, in install
requirement.install(install_options, global_options, _args, *_kwargs)
File "/var/folders/8g/b9q8snrd5ln_d5hqnwh2_86r0000gn/T/tmpFTGrC3/pip.zip/pip/req.py", line 671, in install
self.move_wheel_files(self.source_dir, root=root)
File "/var/folders/8g/b9q8snrd5ln_d5hqnwh2_86r0000gn/T/tmpFTGrC3/pip.zip/pip/req.py", line 901, in move_wheel_files
pycompile=self.pycompile,
File "/var/folders/8g/b9q8snrd5ln_d5hqnwh2_86r0000gn/T/tmpFTGrC3/pip.zip/pip/wheel.py", line 215, in move_wheel_files
clobber(source, lib_dir, True)
File "/var/folders/8g/b9q8snrd5ln_d5hqnwh2_86r0000gn/T/tmpFTGrC3/pip.zip/pip/wheel.py", line 205, in clobber
os.makedirs(destdir)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/pip'

Storing debug log for failure in /Users/aleand/Library/Logs/pip.log

@zaducu, I can tell from your traceback that you are trying to update the pip version that is in the version of Python that came pre-installed with your mac. Since you are updating the pre-installed version, you will need to elevate your credentials by using sudo:

$ sudo curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python

To use sudo you may have to add yourself to /etc/sudousers. Google it and you will find numerous links showing how to do this.

Mac is smart, it keeps your packages in /Library/Python instead of corrupting your /System/Frameworks/Python.framework site-pacakges folder. Yay! But to access this folder you must be an administrator with root privileges, hence sudo.

An alternative to consider is to keep your personal packages and modules separated by using virtualenv and/or a .local folder. To install packages into a local folder use distutils instead of pip and follow the directions on installing python modules.

Other alternatives to using the pre-installed Python are to use official Python, homebrew, MacPorts, Fink or build it yourself.

For official Python, you will still need sudo but it will install most items in /System/Frameworks not /System/Library/Frameworks For Homebrew, you should not need sudo. Can't speak for MacPorts or Fink.

Still have the same problem on Windows, upgrading from 1.5.4 to 6.0.8:

E:\Distrib\Programming>pip install -U pip
Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-6.0.8-py2.py3-none-any.whl#md5=41e73fae2c86ba2270ff51c1d86
f7e09
Installing collected packages: pip
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Cleaning up...
Exception:
Traceback (most recent call last):
File "C:\Python\3.3-32\lib\shutil.py", line 371, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Отказано в доступе: 'c:\users\bialix\appdata\local\temppip-6i_i7a-uninstallpython\3.3-32\scripts\p
ip.exe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Python\3.3-32\lib\site-packagespip\basecommand.py", line 122, in main
write_debug_log = check_path_owner(debug_log_path)
File "C:\Python\3.3-32\lib\site-packagespip\commands\install.py", line 283, in run
build_delete = (not (options.no_clean or options.build_dir))
File "C:\Python\3.3-32\lib\site-packagespipreq.py", line 1443, in install
File "C:\Python\3.3-32\lib\site-packagespipreq.py", line 610, in commit_uninstall
File "C:\Python\3.3-32\lib\site-packagespipreq.py", line 1860, in commit
File "C:\Python\3.3-32\lib\site-packagespip\util.py", line 43, in rmtree
File "C:\Python\3.3-32\lib\shutil.py", line 478, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Python\3.3-32\lib\shutil.py", line 368, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python\3.3-32\lib\shutil.py", line 368, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python\3.3-32\lib\shutil.py", line 368, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python\3.3-32\lib\shutil.py", line 373, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Python\3.3-32\lib\site-packagespip\util.py", line 62, in rmtree_errorhandler
PermissionError: [WinError 5] Отказано в доступе: 'c:\users\bialix\appdata\local\temppip-6i_i7a-uninstallpython\3.3-32\scripts\p
ip.exe'

Any changes on this issue?

It's not fixable in any practical sense (at least, someone would need to supply a complete, working patch for assessment if they thought it was, as nobody has yet figured out a way to fix it - it's essentially a Windows limitation).

The correct way to upgrade pip on Windows is python -m pip install -U pip.

@pfmoore What about Python 2.6?

[py26] C:\Users\IEUser>python -m pip --version
C:\Users\IEUser\Anaconda\envs\py26\python.exe: pip is a package and cannot be directly executed

@dalcinl Sigh. I hate having to think about Python 2.6 :-)

If I remember, python -m pip.__main__ works there. I don't have a copy of Python 2.6 to hand to test it at the moment, sorry...

I've just tested on Python 2.6, and python -m pip.__main__ does indeed work there.

Hello!

python -m pip install -U pip does not work for me (w7, powershell, installed python via chocolatey); the install fails with permission issues:

PS C:\tools> python --version
Python 3.4.3
PS C:\tools> pip --version
pip 6.0.8 from C:\tools\python\lib\site-packages (python 3.4)
PS C:\tools> python -m pip install -U pip
You are using pip version 6.0.8, however version 7.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-7.0.1-py2.py3-none-any.whl#md5=9b403ba9b82d4a1e5fda5b6cc8952b57
  Using cached pip-7.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 6.0.8
    Uninstalling pip-6.0.8:
      Exception:
      Traceback (most recent call last):
        File "C:\tools\python\lib\shutil.py", line 523, in move
          os.rename(src, real_dst)
      PermissionError: [WinError 5] Access is denied: 'c:\\tools\\python\\lib\\site-packages\\pip-6.0.8.dist-info\\description.rst' -> 'C:\\Users\\Administrator\\AppData\\Local\\Temp\\pip-4ghq1oa9-uninstall\\tools\\python\\lib\\site-packages\\pip-6.0.8.dist-info\\description.rst'

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "C:\tools\python\lib\site-packages\pip\basecommand.py", line 232, in main
          status = self.run(options, args)
        File "C:\tools\python\lib\site-packages\pip\commands\install.py", line 347, in run
          root=options.root_path,
        File "C:\tools\python\lib\site-packages\pip\req\req_set.py", line 543, in install
          requirement.uninstall(auto_confirm=True)
        File "C:\tools\python\lib\site-packages\pip\req\req_install.py", line 667, in uninstall
          paths_to_remove.remove(auto_confirm)
        File "C:\tools\python\lib\site-packages\pip\req\req_uninstall.py", line 126, in remove
          renames(path, new_path)
        File "C:\tools\python\lib\site-packages\pip\utils\__init__.py", line 316, in renames
          shutil.move(old, new)
        File "C:\tools\python\lib\shutil.py", line 536, in move
          os.unlink(src)
      PermissionError: [WinError 5] Access is denied: 'c:\\tools\\python\\lib\\site-packages\\pip-6.0.8.dist-info\\description.rst'

Thoughts?

There's no way anything should have that file locked, or that you should not have permission to delete it. Could it be a chocolatey issue? Can you delete that file manually from a command prompt?

@AmazingHorse have you tried

curl -k https://bootstrap.pypa.io/get-pip.py | python

assuming you have cURL?

Interesting, @pfmoore, that file is locked even on an administrator command prompt.

So, sounds like a windows permissions issue rather than something with pip.

Does it have to do with the fact that my python was installed outside of the windows directory structure in C: ?

I'll be a'googling this but I'm curious if anyone can post here what they might think this would be caused by.

Cheers guys

@AmazingHorse Honestly I've no idea. The other possibility is that it may be that there's something particular about how Chocolatey installs Python - maybe report the issue on their tracker (I assume they have one?) and see what they say.

But yes, it's not a pip issue from the sound of it.

Windows allows renaming/moving memory-mapped files (mapped as image or data) to a new name on the same volume. So the rmtree error handler can try renaming the file to move it to the base temp directory. At least there it won't interfere with removing the directory. This workaround fails if the file is on a different volume than the temp directory, or for a sharing violation, or if the file's ACL prevents the operation. But none of those should apply in this case.

If the Administrators group is enabled in the process token, then the renamed file can be scheduled for deletion when the system restarts. Just use ctypes to call MoveFileExW(path, None, MOVEFILE_DELAY_UNTIL_REBOOT). This fails for regular users, however, because they aren't allowed to modify the session manager's PendingFileRenameOperations registry value.

pip on windows 7/10 working fine except for update> pip install --upgrade pip
I try to load the least possible to my C:\ (SSD) and install software to tree starting at D:\NUC\
Here's what I'm seeing:

D:\NUC\Python34>pip install --upgrade pip
You are using pip version 6.0.8, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl#md5=5ff9fec0be479e4e36df467556deed4d
  Downloading pip-7.1.2-py2.py3-none-any.whl (1.1MB)
    100% |################################| 1.1MB 204kB/s
Installing collected packages: pip
  Found existing installation: pip 6.0.8
    Uninstalling pip-6.0.8:
      Exception:
      Traceback (most recent call last):
        File "D:\NUC\Python34\lib\shutil.py", line 523, in move
          os.rename(src, real_dst)
      OSError: [WinError 17] The system cannot move the file to a different disk drive: 'd:\\nuc\\python34\\scripts\\pip.exe' -> 'C:\\Users\\David\\AppData\\Local\\Temp\\pip-247i52js-uninstall\\nuc\\python34\\scripts\\pip.exe'

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "D:\NUC\Python34\lib\site-packages\pip\basecommand.py", line 232, in main
        File "D:\NUC\Python34\lib\site-packages\pip\commands\install.py", line 347, in run
        File "D:\NUC\Python34\lib\site-packages\pip\req\req_set.py", line 543, in install
        File "D:\NUC\Python34\lib\site-packages\pip\req\req_install.py", line 667, in uninstall
        File "D:\NUC\Python34\lib\site-packages\pip\req\req_uninstall.py", line 126, in remove
        File "D:\NUC\Python34\lib\site-packages\pip\utils\__init__.py", line 316, in renames
        File "D:\NUC\Python34\lib\shutil.py", line 536, in move
          os.unlink(src)
      PermissionError: [WinError 5] Access is denied: 'd:\\nuc\\python34\\scripts\\pip.exe'

The AppData folder path is available on my D: drive, wondering how best to handle this situation for subsequent tools and usage. Thought of a junction, but it will still seem like a different drive. Maybe script the diffs of the expected location to actual location to check give the correct path? Before doing this, pip installed django just fine (but complained about the version which led to this discovery). hmmm... and now pip seems broken.

D:\NUC\Python34\Scripts>pip -h
Traceback (most recent call last):
  File "D:\NUC\Python34\lib\runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "D:\NUC\Python34\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\NUC\Python34\Scripts\pip.exe\__main__.py", line 5, in <module>
ImportError: No module named 'pip'

D:\NUC\Python34\Scripts>dir
 Volume in drive D is TOSHIBA EXT
 Volume Serial Number is E085-D5D7

 Directory of D:\NUC\Python34\Scripts

08/25/2015  09:08 AM    <DIR>          .
08/25/2015  09:08 AM    <DIR>          ..
08/25/2015  09:08 AM            92,045 django-admin.exe
08/25/2015  09:08 AM               136 django-admin.py
08/25/2015  09:01 AM               346 easy_install-3.4-script.py
08/25/2015  09:01 AM            65,536 easy_install-3.4.exe
08/25/2015  09:01 AM               651 easy_install-3.4.exe.manifest
08/25/2015  09:01 AM               338 easy_install-script.py
08/25/2015  09:01 AM            65,536 easy_install.exe
08/25/2015  09:01 AM               647 easy_install.exe.manifest
08/25/2015  08:19 AM            91,984 pip.exe
08/25/2015  08:19 AM            91,984 pip3.4.exe
08/25/2015  08:19 AM            91,984 pip3.exe
08/25/2015  09:08 AM    <DIR>          __pycache__
              11 File(s)        501,187 bytes
               3 Dir(s)  328,895,152,128 bytes free

D:\NUC\Python34\Scripts>

you have to do python.exe -m pip install --upgrade pip

On Python 2.6 you need to run python -m pip.__main__ install -U pip

I consider the current situation really bad for new Python users as well as for old Python users like me who just typically don't use Windows. I'm perfectly fine using python -m pip install --upgrade pip, though, but I really think pip install --upgrade pip should recommend that automatically on Windows. @dstufft proposed this already in 2013. Is there a good reason not to do it?

PS: Sorry for reporting #3462 as a duplicate of this issue. I didn't find this issue by searching the tracker for WindowsError. Hopefully after this comment others in the future will.

To the best of my knowledge, it is already the command that pip suggests to the user when detecting a new version of itself. The situation is not that bad.

@pekkaklarck The main reason for not doing it is that it's really hard to spot. For a start, I don't know how to reliably detect that pip is being run from pip.exe rather than from python -m pip. And even if you detect that, you need to check if pip is in the things being installed (consider pip install --upgrade -r http://some.remote/requirements.txt - by the time you've read the requirements file, it's arguably too late to issue the message).

Of course, something that's not 100% perfect is an entirely reasonable thing to implement - it's making sure we don't object to entirely valid usages, rather than making sure we catch all the corner cases, that's really important.

We'd certainly consider merging a PR to address this issue, but I don't think any of the core pip developers find the issue sufficiently problematic to attempt a fix themselves (I'm probably the heaviest Windows user in the core devs, and I find remembering to use python -m pip reasonably easy).

for the purposes of finding whether we're pip[.exe] or not, I imagine

"pip" in os.path.basename(sys.argv[0])

would work fine. Also just warning to seeing [-U|--upgrade] and pip in the invocation would probably be enough to fix 90% of invocations.

@stonebig: Could you clarify when that command is recommended? I tried pip install -U pip after a fresh Python 2.7.11 installation and that failed with this error:

WindowsError: [Error 5] Access is denied: 'c:\users\koulutus\appdata\local\temppip-65z4kx-uninstallpython27\scriptspip.exe'

@pfmoore: Upgrading pip in requirements file sounds pretty rare and could probably be ignored. Like @Ivoz I thought pip.exe would be in sys.argv. Alternatively it ought to be possible to catch the WindowsError and react to that.

@Ivoz does that work? I thought the pip.exe wrapper invoked python as a subprocess, so it wouldn't work. But a quick test (printing sys.argv[0] in pip.main()) shows:

>py -m pip
C:\Users\Gustav\AppData\Local\Programs\Python\Python35\lib\site-packages\pip\__main__.py

Usage:
...

>pip
C:\Users\Gustav\AppData\Local\Programs\Python\Python35\Scripts\pip

Usage:
...

>pip.exe
C:\Users\Gustav\AppData\Local\Programs\Python\Python35\Scripts\pip

Usage:

So it works, but you need to be careful how you spot pip in sys.argv[0] (note that the first example includes the directory pip in the path to __main__.py)

Thanks for pointing this out. I learned something today :-)

@pekkaklarck The problem with catching the WindowsError (if I recall) is that it happens quite late, and the install is part-complete (and not easy to roll back).

But I don't want to discourage anyone from trying to fix this issue, so I'll leave it at that :-)

but you need to be careful how you spot pip in sys.argv[0]

@pfmoore hence the call to os.path.basename.

@Ivoz Sigh. Too late at night here. Yes, obviously :-)

@pekkaklarck . by example when doing pip list (on windows). As far as I remember in the discussions of last year:

• this was the only solution the pypa team imagined as reliable,
• it is only shown on windows.

There could be a better solution, but I assume it's already 80% of a perfect answer from previous situation.

Maybe a better solution would be to have an "upgrade_pip.bat" delivered with python default Windows installation, not tighted to "pip", and that would do that command. (or pip_update.bat to better match conda ?)

I hit this today while diving-into the fresh Windows Python user experience. It was terrible.
Can we make -m automatic for that command? (perhaps give upgrading pip it's own command?)

No user should ever have to see this, especially after running exactly the command _pip told them to run_.

I think we already have the logging setup in more recent versions of pip than 7.0.1 to tell Windows users to use python -m pip install --upgrade pip instead of pip install --upgrade pip.

:sparkles: :cake: :sparkles:

Yes, since 7.1 cf #2830

\o/

I can't upgrade my pip on win 10, what's wrong?

@grasx
If you really want to know what's wrong then please read this thread. If you only are interested in getting this to work then please read the last line from the output you got and compare it with what you ran.

@piotr-dobrogost
Is it? The last red line appears to point to a decoding problem, rather to the access denied error that is the usual outcome of in place update on Windows. Could @grasx 's problem be related to the use of a cyrillic codepage?

I doubt the problem @grasx encountered is related to this issue. I've seen similar probles on Windows, though. See #3463.

@nmusatti
You are right. My bad.

@piotr-dobrogost @nmusatti @pekkaklarck However, thank you, I changed user's name, so now it contains only ASCII chars and successfully upgraded pip to 8.1

@grasx changed user's name？ I am confused in what you said... I got the same trouble, something difference is that I was in windows 7

random idea, how about not replacing the exe file if its not necessary

as far as i understood the exe files are very small trampolines that typically work

the more important logic is in the accompanying script files

@pzz2011 Well, actually I reinstalled my win and set new user name. It was easier in my case then to find out how to change it in all the places and routes where it is mentioned :)

@RonnyPfannschmidt Depends what is used to write the .exe - setuptools used to use foo.exe + foo-script.py (not sure what it does now), whereas distlib (if so configured) creates a single foo.exe which embeds the script.

@grasx hhh... I found an easier way to solve it. I just reinstall the python from 2.7.x to 2.7.10.
Then the success smile to me~

On Windows, use the Command Prompt windows in Administrator mode. Then type command
python -m pip install --upgrade pip

I hope it works for you. That's how I got it to work an Window 10

Good luck.

@grasx Not entirely sure why, but that bug comes up when selecting "install for all users" on the Python installer on both windows 8.1 and 10. Do a reinstall, select PATH and custom install. Make sure the top box is unchecked and select install location. Using "python -m pip install --upgrade pip" in cmd works fine after that.

Just an FYI, for Windows users. If you install Cygwin, Python2.7 is automatically installed. All you need to do then, is run:

wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py

(Yes, you need to install wget first.)

How to Solve This Problem?

@JingY0604
See To copy text from a command prompt window for an explanation on how to copy text here. This way the text you post will be searchable.

You are using pip version 7.1.0, however version 9.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' comm
and.
D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip_vendorrequ
ests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContex
t object is not available. This prevents urllib3 from configuring SSL appropriat
ely and may cause certain SSL connections to fail. For more information, see htt
ps://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
Exception:
Traceback (most recent call last):
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip\base
command.py", line 223, in main
status = self.run(options, args)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip\comm
ands\install.py", line 282, in run
requirement_set.prepare_files(finder)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpipreq\
req_set.py", line 334, in prepare_files
functools.partial(self._prepare_file, finder))
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpipreq\
req_set.py", line 321, in _walk_req_to_install
more_reqs = handler(req_to_install)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpipreq\
req_set.py", line 491, in _prepare_file
session=self.session)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip\down
load.py", line 825, in unpack_url
session,
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip\down
load.py", line 673, in unpack_http_url
from_path, content_type = _download_http_url(link, session, temp_dir)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip\down
load.py", line 886, in _download_http_url
_download_url(resp, link, content_file)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip\down
load.py", line 621, in _download_url
for chunk in progress_indicator(resp_read(4096), 4096):
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip\util
s\ui.py", line 135, in iter
self.next(n)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip_ven
dor\progress__init__.py", line 73, in next
self.update()
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip_ven
dor\progress\bar.py", line 82, in update
self.writeln(line)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip_ven
dor\progress\helpers.py", line 69, in writeln
print(line, end='', file=self.file)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip_ven
dor\colorama\ansitowin32.py", line 133, in write
self.write_and_convert(text)
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip_ven
dor\colorama\ansitowin32.py", line 161, in write_and_convert
self.write_plain_text(text, cursor, len(text))
File "D:\Program Files\Python27\lib\site-packagespip-7.1.0-py2.7.eggpip_ven
dor\colorama\ansitowin32.py", line 166, in write_plain_text
self.wrapped.write(text[start:end])
UnicodeEncodeError: 'ascii' codec can't encode character u'\u258f' in position 8
: ordinal not in range(128)

@JingY0604 See #3463.

You can upgrade your pip by uninstalling the existing installation and using get-pip.py to install the newer version.

For https://github.com/ofek/hatch I just catch all instances of updating self and disallow it. I then have an explicit --self flag to do so that Popens a python -m pip install -U hatch and exits immediately.

https://github.com/ofek/hatch/blob/22e621715fbed36c2bfab35ae9fa9649ea6dfe1e/hatch/cli.py#L509

edit: technically, instead of just python, sys.executable is used actually

@ofek #4490 made a change in pip, printing a better message, IIRC.

So for now if I've installed python without pip I will not be able to install pip module in future?

c:\Python27>python.exe get-pip.py ERROR: To modify pip, please run the following command: c:\Python27\python.exe -m pip

c:\Python27>c:\Python27\python.exe -m pip install c:\Python27\python.exe: No module named pip

Hey @ihormelnyk!

That's not the case. The conditional for this was too broad -- it blocks using any script with the word "pip" in it from modifying pip. You can try renaming get-pip.py to not contain the word "pip", to work around this issue.

A 10.0.1 release (which will likely happen this week) will fix this issue.

@pradyunsg thanks for sharing the tip. I had the same issue as @ihormelnyk and renaming the get-pip.py file to a name that does not contain the word pip helped.

I print pip in cmd inwindows,but it showed
Fatal error in launcher: Unable to create process using '""d:\program files (x86)python3.6python.exe" "D:\Program Files (x86)python3.6\Scriptspip.exe" '
I have already set environment for pip,but it doesn't work,how to figure it out?

@BJQ98 You'd want to look at #5223.

pradyunsg said 10.0.1 would fix this, but i don't know how. Thanks.

C:\Users\clarkk\AppData\Local\Programs\Python\Python36>python -m pip install -U pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
      Successfully uninstalled pip-9.0.3
  Rolling back uninstall of pip
Exception:
Traceback (most recent call last):
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\commands\install.py", line 342, in run
    prefix=options.prefix_path,
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\req\req_set.py", line 784, in install
    **kwargs
  File "C:\Users\yangg\AppData\Roaming\Python\Python36\site-packages\pip\req\req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\req\req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\wheel.py", line 462, in move_wheel_files
    generated.extend(maker.make(spec))
  File "C:\Users\yangg\AppData\Roaming\Python\Python36\site-packages\pip\_vendor\distlib\scripts.py", line 372, in make
    self._make_script(entry, filenames, options=options)
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\_vendor\distlib\scripts.py", line 276, in _make_script
    self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\_vendor\distlib\scripts.py", line 212, in _write_script
    launcher = self._get_launcher('t')
  File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\_vendor\distlib\scripts.py", line 351, in _get_launcher
    result = finder(distlib_package).find(name).bytes
AttributeError: 'NoneType' object has no attribute 'bytes'
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

That'll be the same as #5343.

Thanks pradyunsg, Someone else's solution turns out to be simpler. Just performed "python -m pip uninstall pip" first, then "python -m pip install -U pip"

Requirement already up-to-date: pip in c:\user\clarkk\appdata\local\Programs\Python\Python36\lib\site-packages (10.0.1)

https://pip.pypa.io/en/stable/installing/#upgrading-pip

      Personally I think it should be windows only. I view it as a work around for the way the Windows platform works, and something that's a target to eventually remove for something nicer down the road.

I am getting error while running above command as syntax error to the word get what might be the solution for this

@AProNoob You should be running the commands at a command prompt (cmd or powershell), not at a Python prompt.

Thanks! That worked!

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.