pip install --upgrade pip breaks pip

Created on 24 Jul 2019 · 6Comments · Source: pypa/pip

This bug looks very close to #5432 "pip install --upgrade pip breaks pip, still".

Environment

pip version: pip 19.2.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
Python version: Python 3.7.4
OS: MacOS Mojave 10.14.5 (18F132)

I use the Homebrew packet manager: Homebrew 2.1.89-07-23

Description

In the original brew install python installation, /usr/local/bin/pip3 is a symlink. It eventually reaches its target file: /usr/local/Cellar/python/3.7.4/bin/pip3.

The brew info python help recommends to use unversioned symlinks at /usr/local/opt/python/libexec/bin: /usr/local/opt/python/libexec/bin/pip is a symlink to /usr/local/Cellar/python/3.7.4/bin/pip3.

pip install --upgrade pip creates a new file /usr/local/bin/pip instead of of updating the target of the symlink, that is, /usr/local/Cellar/python/3.7.4/bin/pip3. Moreover, this latter file is not updated.

As the PATH finds /usr/local/opt/python/libexec/bin/pip before /usr/local/bin/pip, everything fails...

Expected behavior

On updating pip, /usr/local/bin/pip should be a symlink to /usr/local/opt/python/libexec/bin/pip.

How to Reproduce

Clean off all the pip's
find -E /usr/local/ -type f -iregex '.*/pip[23]?' -print -delete

Reinstall python with Homebrew
brew reinstall python
Check the status of /usr/local/bin/pip*, etc. and look for the similar i-nodes.
NB: Note there is no /usr/local/bin/pip.

ls -li /usr/local/bin/pip*
ls -li /usr/local/opt/python/libexec/bin/pip
ls -li /usr/local/opt/python/libexec/bin/../../bin/pip3
ls -li /usr/local/bin/../Cellar/python/3.7.4/bin/pip3

Upgrade pip with pip
/usr/local/opt/python/libexec/bin/pip install --upgrade pip
Redo Step 3 above.
Try out
/usr/local/bin/pip --version
Try out
/usr/local/opt/python/libexec/bin/pip --version

Output

pommeau:~ bouge$ find -E /usr/local/ -type f -iregex '.*/pip[23]?' -print -delete

/usr/local//bin/pip3
/usr/local//bin/pip
find: /usr/local//etc/FileWaveInstallers/FlatPackage-Microsoft_Office_2019_VL_Serializer.pkg: Permission denied
find: /usr/local//etc/FileWaveInstallers/FlatPackage-Microsoft_AutoUpdate_4.3.18090901_Updater.pkg: Permission denied
find: /usr/local//etc/FileWaveInstallers/FlatPackage-Microsoft_Office_16.17.18090901_Installer.pkg: Permission denied
/usr/local//Cellar/python/3.7.4/bin/pip3

pommeau:~ bouge$ brew reinstall python

==> Reinstalling python 
==> Downloading https://homebrew.bintray.com/bottles/python-3.7.4.mojave.bottle.
Already downloaded: /Users/bouge/Library/Caches/Homebrew/downloads/ffcfe300e2ad205387a2bdd27185d98d64621578d26d2f6ccbbcd091ff799fc8--python-3.7.4.mojave.bottle.tar.gz
==> Pouring python-3.7.4.mojave.bottle.tar.gz
==> /usr/local/Cellar/python/3.7.4/bin/python3 -s setup.py --no-user-cfg install
==> /usr/local/Cellar/python/3.7.4/bin/python3 -s setup.py --no-user-cfg install
==> /usr/local/Cellar/python/3.7.4/bin/python3 -s setup.py --no-user-cfg install
==> Caveats
Python has been installed as
  /usr/local/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /usr/local/opt/python/libexec/bin

If you need Homebrew's Python 2.7 run
  brew install python@2

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /usr/local/lib/python3.7/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /usr/local/Cellar/python/3.7.4: 3,869 files, 60MB

pommeau:~ bouge$ ls -li /usr/local/bin/pip*

17870036 lrwxr-xr-x  1 bouge  wheel    34B  4 mar 14:11 /usr/local/bin/pip2@ -> ../Cellar/python@2/2.7.16/bin/pip2
17870037 lrwxr-xr-x  1 bouge  wheel    36B  4 mar 14:11 /usr/local/bin/pip2.7@ -> ../Cellar/python@2/2.7.16/bin/pip2.7
36689213 lrwxr-xr-x  1 bouge  wheel    31B 24 jul 15:54 /usr/local/bin/pip3@ -> ../Cellar/python/3.7.4/bin/pip3
36689214 lrwxr-xr-x  1 bouge  wheel    33B 24 jul 15:54 /usr/local/bin/pip3.7@ -> ../Cellar/python/3.7.4/bin/pip3.7
12675428 -rwxr-xr-x@ 1 bouge  wheel   263B 16 jan  2019 /usr/local/bin/pip_upgrade_outdated*

pommeau:~ bouge$ ls -li /usr/local/opt/python/libexec/bin/pip
36689211 lrwxr-xr-x  1 bouge  staff    14B 24 jul 15:54 /usr/local/opt/python/libexec/bin/pip@ -> ../../bin/pip3

pommeau:~ bouge$ ls -li /usr/local/opt/python/libexec/bin/../../bin/pip3
36689157 -rwxr-xr-x  1 bouge  staff   386B 24 jul 15:54 /usr/local/opt/python/libexec/bin/../../bin/pip3*

pommeau:~ bouge$ ls -li /usr/local/bin/../Cellar/python/3.7.4/bin/pip3
36689157 -rwxr-xr-x  1 bouge  staff   386B 24 jul 15:54 /usr/local/bin/../Cellar/python/3.7.4/bin/pip3*

pommeau:~ bouge$ /usr/local/opt/python/libexec/bin/pip install --upgrade pip

Collecting pip
  Using cached https://files.pythonhosted.org/packages/62/ca/94d32a6516ed197a491d17d46595ce58a83cbb2fca280414e57cd86b84dc/pip-19.2.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.1.1
    Uninstalling pip-19.1.1:
      Successfully uninstalled pip-19.1.1
Successfully installed pip-19.2.1

pommeau:~ bouge$ ls -li /usr/local/bin/pip*

36690799 -rwxr-xr-x  1 bouge  wheel   234B 24 jul 15:55 /usr/local/bin/pip*
17870036 lrwxr-xr-x  1 bouge  wheel    34B  4 mar 14:11 /usr/local/bin/pip2@ -> ../Cellar/python@2/2.7.16/bin/pip2
17870037 lrwxr-xr-x  1 bouge  wheel    36B  4 mar 14:11 /usr/local/bin/pip2.7@ -> ../Cellar/python@2/2.7.16/bin/pip2.7
36690800 -rwxr-xr-x  1 bouge  wheel   234B 24 jul 15:55 /usr/local/bin/pip3*
36690801 -rwxr-xr-x  1 bouge  wheel   234B 24 jul 15:55 /usr/local/bin/pip3.7*
12675428 -rwxr-xr-x@ 1 bouge  wheel   263B 16 jan  2019 /usr/local/bin/pip_upgrade_outdated*

pommeau:~ bouge$ ls -li /usr/local/opt/python/libexec/bin/pip
36689211 lrwxr-xr-x  1 bouge  staff    14B 24 jul 15:54 /usr/local/opt/python/libexec/bin/pip@ -> ../../bin/pip3

pommeau:~ bouge$ ls -li /usr/local/opt/python/libexec/bin/../../bin/pip3
36689157 -rwxr-xr-x  1 bouge  staff   386B 24 jul 15:54 /usr/local/opt/python/libexec/bin/../../bin/pip3*

pommeau:~ bouge$ ls -li /usr/local/bin/../Cellar/python/3.7.4/bin/pip3
36689157 -rwxr-xr-x  1 bouge  staff   386B 24 jul 15:54 /usr/local/bin/../Cellar/python/3.7.4/bin/pip3*

pommeau:~ bouge$ /usr/local/bin/pip --version

pip 19.2.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)

pommeau:~ bouge$ /usr/local/opt/python/libexec/bin/pip --version

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (pip 19.2.1 (/usr/local/lib/python3.7/site-packages), Requirement.parse('pip==19.1.1'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/python/libexec/bin/pip", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3241, in <module>
    @_call_aside
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pip==19.1.1' distribution was not found and is required by the application

crash macos downstream bug

Source

lucbouge

Most helpful comment

Below steps fixed my problem. Share it out in case anyone needs it:

cp /usr/local/opt/python/libexec/bin/pip /usr/local/opt/python/libexec/bin/pip_ORIG
mv /usr/local/bin/pip /usr/local/opt/python/libexec/bin/pip
ln -s /usr/local/opt/python/libexec/bin/pip /usr/local/bin/pip
pip --version

Credit to this post.

bijoustone on 13 Aug 2019

👍9 🚀1

All 6 comments

Could you file an issue with Homebrew, linking to this one?

I think they're in a better position to address this than pip's maintainers are. We'll be happy to work with them to resolve this if there's anything they think upstream pip should do.

pradyunsg on 24 Jul 2019

I’ve always felt Homebrew should patch pip so the message recommends brew upgrade pip instead of the (built-in) pip install command. I believe Debian already does this (not sure about RPM).

pip could make that easier by e.g. make the command string a global variable (instead of buried inline in _internal.utils.outdated), but first Homebrew needs to be willing to accept the patch.

uranusjr on 26 Jul 2019

Below steps fixed my problem. Share it out in case anyone needs it:

cp /usr/local/opt/python/libexec/bin/pip /usr/local/opt/python/libexec/bin/pip_ORIG
mv /usr/local/bin/pip /usr/local/opt/python/libexec/bin/pip
ln -s /usr/local/opt/python/libexec/bin/pip /usr/local/bin/pip
pip --version

Credit to this post.

bijoustone on 13 Aug 2019

👍9 🚀1

Please consider filing an issue to Homebrew

uranusjr on 13 Aug 2019

Amateur work...