thats a debian issue

on extra note - replacing the system pip using pip is always an act of system-vandalism where the one inflicting it is responsible for the fallout

I'd suggest you should wait for a proper apt packaged copy of pip 10 from Debian - as @RonnyPfannschmidt says, you shouldn't be using pip to upgrade your system packages...

Looks like the Debian pip3 script uses pip's internals, so getting a pip10 compatible fix is definitely up to them (and I'd fully expect them to wait on releasing their pip10 package until they have that sorted).

on extra note - replacing the system pip using pip is always an act of system-vandalism where the one inflicting it is responsible for the fallout

Convince the debian/ubuntu people to not vendor and then let rot half their packages, and that'd be a valid argument.

You can use virtualenv or venv to isolate yourself from the system pip installation.

You should not be modifying the package-manager managed files (here the system pip installation) -- I think they don't expect users to modify things -- it is likely not supported by Debian. It's bound to cause issues like this.

Same issue on Fedora, too.

Perhaps there should be a "beta" channel, or some similar mechanism for people to do more testing prior to release, rather then just dumping a broken version on pypi and causing everyone's builds to explode.

@fake-name There were two pre-releases made:

• 10.0.0b1 was released on Mar 31st
• 10.0.0b2 was released on April 2nd

@fake-name additionally the general suggestion for usage on all distros is - use a virtualenv, don't vandalize the system, and that works - people just font follow it and then wonder when stuff breaks and blame pip

there has been plenty of manual and automated testing with virtualenvs which works

also in a virtualenv at least there should be no debian made pip3 command - so what the heck are you talking about wrt this breaking in virtualenvs - please provide enough data to actually verify instead of whining about breakages without providing the data needed to verify them

pip is volunteer driven, not a company with dozens of employees

~Deleted~. was confusing this with https://github.com/pypa/pip/issues/5220

I'm a derp.

Thanks, hadn't realized that replacing the system pip was a bad idea but it makes sense. However, it would be good UX if pip would not nag about upgrading in that case. Would that be possible? I reckon a lot of people (including me) would just do whatever "the thing" asks them to do.

@fake-name thanks for following up - its surprisingly common to mismatch the detail issue when a major release has multi-faceted impact and part of it tries to ruin you day

it would be good UX if pip would not nag about upgrading in that case

Distribution vendors could certainly patch pip to remove that warning (or better, replace it with a similar check against the system packages) along with the other patches they make. I'm not sure how base pip could detect that it's running from a system packaged installation without co-operation from the distribution, but if there's a way to do so that's something we could consider (but be aware that in my experience we get as much negative feedback from getting such heuristics wrong as we do from not including heuristics at all...)

Always prefer "python3 -m pip" over pip3" or even better "/usr/bin/env python3 -m pip" it is safer and allow to avoid this issue with pip10

We solved this issue by clear hash in bash:

$ hash -d pip

Or in dash (sh):

$ hash -r pip

Also hit this issue while building docker images.

@RonnyPfannschmidt says "replacing the system pip using pip is always an act of system-vandalism where the one inflicting it is responsible for the fallout", which has 6 thumbs up. I find this to be an especially obtuse comment given that I instructed to do so by pip itself:

_You are using pip version 8.1.1, however version 10.0.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command._

If there is some validity to that comment, then the creators of pip should remove this message and I would encourage @RonnyPfannschmidt to raise an issue to that effect and make his case.

@qacollective - I think the argument here is that the distro's have taken pip, modified it, and re-packaged it into their repositories. As such, it's not really Pypi's fault the message is still there.

Most of this is because a bunch of the distros try really really hard to repackage everything into their own package repositories. Mostly, the things then get left to rot.

Personally, at least for python on ubuntu, I wish they'd quit it. The version of basically every python package in apt ranges from really, really old to fossilized. Apt is basically useless for python, IMHO.

FWIW, I tend to find the best option is to never install the distro pip in the first place, but instead install it manually via get-pip.py. That way, you don't have issues with the platform package manager knowing about only some of the python packages.

Always use --user to avoid killing your system

/usr/bin/env python3 -m pip intall --user --upgrade pip

Should handle most of buggy case and result in the right version of pip being installed in ˜/.local/bin.

I can confirm @standag's solution is working.

A bit of background: After the upgrade (pip install -U pip) on a vanilla ubuntu 16.04 (AWS AMI) one gets to the following situation:
$PATH=..:/usr/local/bin:...:/usr/bin:...
/usr/bin/pip is still the old/'oem' version (broken)
/usr/local/bin/pip is the new v10 script (works fine when invoked directly)

Even though the proper pip version preceeds the broken one in the PATH, bash still remembers the old one, so when you invoke it just as 'pip', you'll get the old, broken one running. hash-d pip or hash -r solves the issue.

First, some notes about what happened here on Debian/Ubuntu (and likely a few more Linux Distros):

• pip does not support the use of it's internals by importing it. More on that in the documentation here.
• Debian (hence Ubuntu) does not support modifying their package manager managed files using something, that well, isn't their package manager.

This issue is caused by, well, both being violated in a way.

• Debian uses the pip's internal methods (which no longer work due to a reorganization of pip's internals). Debian's assuming here that the pip version in their repositories is the one that would be installed.
• Running pip install --upgrade pip, as root, without any other parameters modifies files that are supposed to managed by apt, which breaks the script by Debian.

Some general tips on Linux:

• It's a good habit to use --user whenever outside a venv.

  pip install --upgrade --user pip
  

• Never run pip with sudo unless you know what you're doing.

What's the workaround?

@standag's solution is useful when this is caused by bash's caching of executables.

hash -r pip # or hash -d pip

If you've modified your OS package manager's installation of pip (eg by using sudo pip) and python -m pip is still working, one workaround is to uninstall the pip installed version and reinstall the package manager installed version.

python -m pip uninstall pip  # this might need sudo
sudo apt install --reinstall python-pip

If you're not on Debian/Ubuntu _and_ pip broke for you, try running:

python -m pip install --force-reinstall pip

If the above this doesn't resolve your problems, please file a new issue.

[edited by @pradyunsg: make it more relevant for linking everyone with similar issues to this comment; update the suggestion to include uninstalling/reinstalling workaround]

What about solving it outside docker? It's broken in my regular system and the hash command didn't recognize pip.
thinkdigital@thinkdigital-HP-Spectre-x360-Convertible:~$ hash -d pip bash: hash: pip: not found

I found pip3, version 9.0.1 installed in a virtualenv from a project and copied it to my /usr/bin and it works again. Here are the contents of the pip3 executable for those who would like to fix it themselves as well.

# -*- coding: utf-8 -*-
import re
import sys

from pip import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

I'm sure all you have to do is save that to a file called pip3, make it executable by running sudo chmod +x ./pip3, run sudo apt remove python3-pip, and then copy it to the bin directory by running sudo cp ./pip3 /usr/bin.
Here's the raw file for those who just want to download and move it.
pip3.zip

It works for me:
curl https://bootstrap.pypa.io/get-pip.py | sudo python

I'm sorry, but I'd like to point out that running python code curl'd from some website with root access is just terrifyingly insecure.

Agreed, and I should point out that this is not an official pip recommendation. As has been stated a number of times, you should be using your system package manager to update or otherwise manage your system pip installation, not get-pip, or even pip itself via sudo.

In this case the version from the system package manager doesn't work. Even
after purging and reinstalling.

On Thu, Apr 19, 2018, 1:53 AM Paul Moore notifications@github.com wrote:

Agreed, and I should point out that this is not an official pip
recommendation. As has been stated a number of times, you should be using
your system package manager to update or otherwise manage your system pip
installation, not get-pip, or even pip itself via sudo.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pip/issues/5221#issuecomment-382660881, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AV-Hfecz8l1NEyq3vsih0DpNP7QYdxuvks5tqFCdgaJpZM4TVEq6
.

If reinstalling the system package still doesn't work, check whether you have pip10 somewhere in /usr/local/ and delete the whole folder.

Replacing the one from /usr/bin worked for me even though I THINK that's
the one that the system package manager was installing.

[@pradyunsg snipped email content]

@ThinkDigitalRepair Does the following help?

In other cases, you'll want to pass --user when installing/upgrading packages. TBH, on Linux, it's a good habit to use --user.

pip install --upgrade --user pip

Ok. Thanks. I messed it up following a Jupyter Notebook tutorial from their
site which tells you to upgrade pip directly. Copying and pasting without
knowing the consequences strikes again. :(

[@pradyunsg snipped email content]

@ThinkDigitalRepair pip 10 will hopefully be improving things -- it's printing better error messages instead of a long PermissionError when this happens.

Closing this issue now since there's nothing actionable here from pip's end.

Anyone looking for how to fix/workaround this problem, please take a look at https://github.com/pypa/pip/issues/5221#issuecomment-382069604.

@pradyunsg in many cases the solutions given in that comment won't work. Under a fresh Ubuntu 17.10, run pip install --upgrade pip: after that the pip command will be broken, and the solutions from the comment won't fix it. And they shouldn't!

Having a system installed pip 9 with a user installed pip 10, makes the system pip script try to import main() from the user pip 10, with an incorrect import path. Hash -r or -d won't fix that because the pip command will still run the system pip by default. And neither will fix it upgrading user pip, as system pip will still be 9, user pip will still be 10, so the import will keep failing.

The solution for those cases must be uninstalling one of both pips.

• python -m pip uninstall pip --user, keeping the system pip, which is older

or

• sudo apt remove python-pip and keep the user installed pip, which won't be accessible by running pip in the terminal by default. You will either need to run it with python -m pip, or add the paths to your PATH env var, etc.

All of this applies to both, pip under python 2 and 3.

On all my Ubuntu systems (16.04, 17.10. 18.04), I have the system pip to the old version and the user is with pip 10 and I never see your import error.
Are you sure you do not have a corrupted system’s pip ?

@gsemet you probably added ~/.local/bin to your PATH env var (or maybe using a different, smarter shell than the default bash), so when you run pip it uses the user installed pip 10 script, and not the system installed pip 9 script. In Ubuntu this isn't like that by default. It can be done, sure, and I wish it came like that by default. But by default the pip command will call the system installed pip, even if you have a user installed one.

How to reproduce this under a fresh Ubuntu 17.10 installation, including the evidence that the commands in comment 5221 fail to fix it, and what I proposed does fix it.

Installation of both pips (system and user), which breaks the pip command:

vfisa@vilos:~$ sudo apt install python-pip
(...)

vfisa@vilos:~$ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

vfisa@vilos:~$ which pip
/usr/bin/pip

vfisa@vilos:~$ pip install pip --upgrade --user
Collecting pip
  Downloading https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 631kB/s 
Installing collected packages: pip
Successfully installed pip-10.0.1

vfisa@vilos:~$ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

vfisa@vilos:~$ python -m pip --version
pip 10.0.1 from /home/vfisa/.local/lib/python2.7/site-packages/pip (python 2.7)

vfisa@vilos:~$ which pip
/usr/bin/pip

As you can see, the pip command points to the system pip by default, not the user installed one.

Commands from referenced comment, evidence of them not fixing the problem:

vfisa@vilos:~$ hash -r

vfisa@vilos:~$ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

vfisa@vilos:~$ hash -d
hits    command
   1    /usr/bin/pip

vfisa@vilos:~$ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

vfisa@vilos:~$ python -m pip install pip --force-reinstall --user
Collecting pip
  Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 10.0.1
    Uninstalling pip-10.0.1:
      Successfully uninstalled pip-10.0.1
Successfully installed pip-10.0.1

vfisa@vilos:~$ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

vfisa@vilos:~$ which pip
/usr/bin/pip

As you can see, as long as both pips are installed and the pip command points to the system pip (default behavior in Ubuntu), the problem will persist.

Fix option 1:

Remove system pip, keep user pip, which by default isn't accessible via the pip command (so you need to use python -m pip).

vfisa@vilos:~$ sudo apt remove python-pip
(...)

vfisa@vilos:~$ pip
bash: /usr/bin/pip: No such file or directory

vfisa@vilos:~$ python -m pip --version
pip 10.0.1 from /home/vfisa/.local/lib/python2.7/site-packages/pip (python 2.7)

You could add ~/.local/bin to your PATH env var, to be able to use the pip command with the user pip.

Fix option 2:

Remove user pip, keep the system pip, which is older but by default has a working pip command in the path.

vfisa@vilos:~$ python -m pip uninstall pip
Uninstalling pip-10.0.1:
  Would remove:
    /home/vfisa/.local/bin/pip
    /home/vfisa/.local/bin/pip2
    /home/vfisa/.local/bin/pip2.7
    /home/vfisa/.local/lib/python2.7/site-packages/pip-10.0.1.dist-info/*
    /home/vfisa/.local/lib/python2.7/site-packages/pip/*
Proceed (y/n)? y
  Successfully uninstalled pip-10.0.1
You are using pip version 9.0.1, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

vfisa@vilos:~$ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

@fisadev Sure, that's needed to support user installable 'pip install --user' package. But I think it should be told to users "if you want to force the update to pip 10 before debian/ubuntu package is updated, you need to use pip install --user --upgrade pip and ensure $HOME/.local/bin is in your path. It's simple to do.

@gsemet I agree, users should be told about the path requirement. This isn't mentioned in the comment being referenced from other threads as the solution, and in one case the discussion was locked even after a user said that he ran those commands and it didn't solve the problem :/

@fisadev Thanks a lot. Fix option1 really helps.

@RonnyPfannschmidt

replacing the system pip using pip is always an act of system-vandalism where the one inflicting it is responsible for the fallout

is a comment of mental vandalism. As if the person doing the (naive) upgrade is intentionally setting out to damage their own installation... If that were the case, then pip itself shouldn't be nagging the user to upgrade from 9.0.1 to 10.0.1 with every single pip command executed. I myself followed that recommendation and ended up in this mess.

Fortunately:
sudo python -m pip install pip==9.0.1
was an easy enough remedy.

Blaming the victim is no answer, though.

Hey @rod-app!

If that were the case, then pip itself shouldn't be nagging the user to upgrade from 9.0.1 to 10.0.1 with every single pip command executed.

We've noticed this and worked with OS vendors to avoid this in future versions of pip. -- #5346.

To address the issue I ran...

sudo geany -i /usr/bin/pip

...and edited debian's provided /usr/bin/pip to replace it with...

#!/bin/sh
# GENERATED BY CEFN
python -m pip "$@"

and the equivalent for /usr/bin/pip3 (note this invokes python3 instead).

#!/bin/sh
# GENERATED BY CEFN
python3 -m pip "$@"

...which brings back full functionality of pip despite the version 10 installed in my site-packages. I guess this will last exactly as long as debian takes to fix it, (or re-break it) by sending an updated python-pip package. Why they didn't use the package main in the first place I don't know.

Official version

The version of pip installed in .local/bin/pip shown below is a little fancier, and includes some substitution to remove -script, .py, .pyw and .exe extensions from passed arguments, but I don't know what that does or why I need it so I left it as above for simplicity.

#!/usr/bin/python

# -*- coding: utf-8 -*-
import re
import sys

from pip._internal import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

I found an unrelated cause of this pip v10 issue. When upgrading pip using a very old system pip (v1.5.6 on Debian Jessie, i.e. oldstable) for which --system is the default, I noticed that incorrect scripts were installed, e.g. /usr/local/bin/pip containing from pip import main -- which I found by looking at the files. I assume this is because the older pip (or perhaps installation packages that it uses) install the .whl file incorrectly.

python -m pip install --force-reinstall pip fixed this.

5599 provides information and provides a single location to seek help toward resolving this issue for end users.

The comments section of that issue are open for users to discuss specific problems and solutions. :)