Xgboost: ImportError: No module named numpy.distutils.core (Ubuntu xgboost installation)

Created on 11 Feb 2016  ·  17Comments  ·  Source: dmlc/xgboost

I am a new user to Ubuntu and recently downloaded Ubuntu 14.04 desktop version alongside Windows 10.

My PC configs are: 4 GB RAM, 64 bit

I installed Anaconda using the following command from the terminal (after downloading from website):
bash Anaconda3-2.5.0-Linux-x86_64.sh

Installation was successful.

Now in my ubuntu terminal, I followed steps as instructed on http://xgboost.readthedocs.org/en/latest/build.html#building-on-ubuntu-debian

  1. Installed a recent GNU C++ compiler -->successful
  2. git clone --recursive https://github.com/dmlc/xgboost cd xgboost; make -j4
    Building was also successful and I could build 'libxgboost.so'
  3. sudo apt-get install python-setuptools (Successful)
  4. cd python-package
  5. sudo python setup.py install

The 5. command returns an error after a number of lines. The error is as follows:

'ImportError: No module named numpy.distutils.core'

Can anyone suggest how to get rid of this error so that I can install xgboost ?

The full log after step 5. is as follows:

_sidvash@sidvash-HP-Pavilion-g6-Notebook-PC:~/xgboost/python-package$ sudo python setup.py install
Install libxgboost from: ['/home/sidvash/xgboost/python-package/xgboost/../../lib/libxgboost.so']
running install
running bdist_egg
running egg_info
writing requirements to xgboost.egg-info/requires.txt
writing xgboost.egg-info/PKG-INFO
writing top-level names to xgboost.egg-info/top_level.txt
writing dependency_links to xgboost.egg-info/dependency_links.txt
reading manifest file 'xgboost.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '_' under directory 'xgboost/include'
warning: no files found matching '_' under directory 'xgboost/src'
warning: no previously-included files matching 'xgboost/build/_' found anywhere in distribution
warning: no previously-included files matching 'xgboost/_.o' found anywhere in distribution
warning: no previously-included files matching '_.pyo' found anywhere in distribution
warning: no previously-included files matching '_.pyc' found anywhere in distribution
writing manifest file 'xgboost.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/libpath.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/VERSION -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/build-python.sh -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/__init__.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/training.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/compat.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/core.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/sklearn.py -> build/bdist.linux-x86_64/egg/xgboost
copying build/lib.linux-x86_64-2.7/xgboost/plotting.py -> build/bdist.linux-x86_64/egg/xgboost
byte-compiling build/bdist.linux-x86_64/egg/xgboost/libpath.py to libpath.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/training.py to training.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/compat.py to compat.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/core.py to core.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/sklearn.py to sklearn.pyc
byte-compiling build/bdist.linux-x86_64/egg/xgboost/plotting.py to plotting.pyc
installing package data to build/bdist.linux-x86_64/egg
running install_data
copying /home/sidvash/xgboost/python-package/xgboost/../../lib/libxgboost.so -> build/bdist.linux-x86_64/egg/xgboost
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xgboost.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating 'dist/xgboost-0.4-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing xgboost-0.4-py2.7.egg
removing '/usr/local/lib/python2.7/dist-packages/xgboost-0.4-py2.7.egg' (and everything under it)
creating /usr/local/lib/python2.7/dist-packages/xgboost-0.4-py2.7.egg
Extracting xgboost-0.4-py2.7.egg to /usr/local/lib/python2.7/dist-packages
xgboost 0.4 is already the active version in easy-install.pth

Installed /usr/local/lib/python2.7/dist-packages/xgboost-0.4-py2.7.egg
Processing dependencies for xgboost==0.4
Searching for scipy
Reading https://pypi.python.org/simple/scipy/
Best match: scipy 0.17.0
Downloading https://pypi.python.org/packages/source/s/scipy/scipy-0.17.0.zip#md5=28a4fe29e980804db162524f10873211
Processing scipy-0.17.0.zip
Writing /tmp/easy_install-2YPTnH/scipy-0.17.0/setup.cfg
Running scipy-0.17.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-2YPTnH/scipy-0.17.0/egg-dist-tmp-z4NE5c
Traceback (most recent call last):
File "setup.py", line 39, in
url='https://github.com/dmlc/xgboost')
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 73, in run
self.do_egg_install()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 96, in do_egg_install
cmd.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 381, in run
self.easy_install(spec, not self.no_deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 597, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 648, in install_item
self.process_distribution(spec, dist, deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 694, in process_distribution
[requirement], self.local_index, self.easy_install
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 858, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 870, in obtain
return installer(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 616, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 646, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 834, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1040, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1025, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 50, in run_setup
lambda: execfile(
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 100, in run
return func()
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 52, in
{'file':setup_script, 'name':'main'}
File "setup.py", line 265, in

File "setup.py", line 253, in setup_package

ImportError: No module named numpy.distutils`_

picture sidsvash26
👍3

Most helpful comment

If 'pip install xgboost' doesn't work for you, this issue can also be fixed by doing

sudo -s
python setup.py install

The problem is 'sudo python' will use system's Python instead of anacoda's python.

More detail: http://stackoverflow.com/questions/30626160/anaconda-python-not-available-from-sudo

picture knguyen0105 on 11 Jun 2016
👍55 😄9 ❤1 🎉1

All 17 comments

Are you sure you are using right python?

Try

which python

or

readlink -f `which python`

to determine whether you are using Anaconda or not.

brickerino picture brickerino on 12 Feb 2016

Hi,

Thanks for commenting.

which python returns the following:
/home/sidvash/anaconda3/bin/python

sidsvash26 picture sidsvash26 on 12 Feb 2016

@brickerino : I have just updated the full log after step 5 in the description. Please see if that helps.

sidsvash26 picture sidsvash26 on 12 Feb 2016

The message shows you are not having numpy, please check if your PYTHON_PATH is pointing to the anaconda package path. Also, be sure conda install numpy if not yet

phunterlau picture phunterlau on 13 Feb 2016

@phunterlau :
I checked the python_path using the follwing command:
python -c "import sys; print(sys.path)"

It returns:
['', '/home/sidvash/anaconda3/lib/python35.zip', '/home/sidvash/anaconda3/lib/python3.5', '/home/sidvash/anaconda3/lib/python3.5/plat-linux', '/home/sidvash/anaconda3/lib/python3.5/lib-dynload', '/home/sidvash/anaconda3/lib/python3.5/site-packages', '/home/sidvash/anaconda3/lib/python3.5/site-packages/Sphinx-1.3.5-py3.5.egg', '/home/sidvash/anaconda3/lib/python3.5/site-packages/cryptography-1.0.2-py3.5-linux-x86_64.egg', '/home/sidvash/anaconda3/lib/python3.5/site-packages/setuptools-19.6.2-py3.5.egg']

This looks fine to me. However, I see in my first post that the path "/usr/lib/python2.7/dist-packages" appears in a number of lines in the error and I am not sure if this should also be there in the python path or not. Could you suggest anything?

I also ran "conda install numpy" which returned the following:

Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ....
Solving package specifications: ......................
Package plan for installation in environment /home/sidvash/anaconda3:

The following packages will be downloaded:

| package | build |
| --- | --- |
| wheel-0.29.0 | py35_0 82 KB |

The following packages will be UPDATED:

wheel: 0.26.0-py35_1 --> 0.29.0-py35_0

Proceed ([y]/n)? y

Fetching packages ...
wheel-0.29.0-p 100% |################################| Time: 0:00:01 50.38 kB/s
Extracting packages ...
[ COMPLETE ]|###################################################| 100%
Unlinking packages ...
[ COMPLETE ]|###################################################| 100%
Linking packages ...
[ COMPLETE ]|###################################################| 100

Tried sudo python setup.py install after installing this but no luck, still the same error

sidsvash26 picture sidsvash26 on 13 Feb 2016

well, you have python from the system and anaconda, and python 2 3 both, and you have system's setuptool, so some env has messed up. i can suggest creating a virtualenv and install numpy scikit-learn and xgboost

phunterlau picture phunterlau on 13 Feb 2016

@phunterlau : Thanks for your comment. I really appreciate your help.

I am a novice on Ubuntu and don't want to mess up things further.

Could you please tell me a step by step process to do this?

I know that I can create a virtualenv using the following: (installing virtualenv is not recommended with conda)
conda create -n <env_name> python=3.5 anaconda

And then how should I install numpy scikit-learn and xgboost on this env ?

I can activate the env by:
source activate <env_name>

Should I just then proceed with further installations as follows:
pip install numpy
pip install scikit-learn

And for xgboost I should repeat all these steps?
git clone --recursive https://github.com/dmlc/xgboost cd xgboost; make -j4
cd python-package; sudo python setup.py install

sidsvash26 picture sidsvash26 on 14 Feb 2016

Okay, so I did the following but still getting the exact same error:
conda create -n snakes python=3.5 numpy pandas scikit-learn
sidvash:~$ source activate snakes
discarding /home/sidvash/anaconda3/bin from PATH
prepending /home/sidvash/anaconda3/envs/snakes/bin to PATH

cd xgboost/python-package
sudo python setup.py install

For some reason, the installation still keeps referring to "/usr/lib/python2.7/dist-packages"

sidsvash26 picture sidsvash26 on 14 Feb 2016

I got it sorted out by following the below steps:

  1. Remove current dir named "xgboost" which I earlier downloaded from github.
  2. pip install xgboost

This worked like a charm. I didn't try this earlier as it was mentioned on the github site that you should install from github for latest version.

I posted this question on stack overflow and someone suggested this. Here's the link:
http://stackoverflow.com/questions/35332300/importerror-no-module-named-numpy-distutils-core-ubuntu-xgboost-installation/35493714#35493714

Anyway, thanks a lot for your help guys: @brickerino @phunterlau

sidsvash26 picture sidsvash26 on 20 Feb 2016

Glad that pip install xgboost helped :-)

phunterlau picture phunterlau on 20 Feb 2016

If 'pip install xgboost' doesn't work for you, this issue can also be fixed by doing

sudo -s
python setup.py install

The problem is 'sudo python' will use system's Python instead of anacoda's python.

More detail: http://stackoverflow.com/questions/30626160/anaconda-python-not-available-from-sudo

knguyen0105 picture knguyen0105 on 11 Jun 2016
👍55 😄9 ❤1 🎉1

@knguyen0105 Good worked for me....thanks man....:)

ashish1302 picture ashish1302 on 25 Nov 2016

@knguyen0105 giving error
"/home/ashish/anaconda2/bin/../lib/libgomp.so.1: version `GOMP_4.0' not found (required by /home/ashish/anaconda2/lib/python2.7/site-packages/xgboost/./lib/libxgboost.so)"

can anybody help?

ashish1302 picture ashish1302 on 25 Nov 2016

what is PYTHON_PATH suppose to return?

brando90 picture brando90 on 11 Jan 2017

@knguyen0105 It worked just fine for me! Thanks!

TiagoMHCSantana picture TiagoMHCSantana on 6 Feb 2017

Thanks knguyen0105, Summary for Ubuntu, anaconda 2:

  1. http://xgboost.readthedocs.io/en/latest/build.html
  2. sudo -s
    python setup.py install
    instead of
    cd python-package; sudo python setup.py install
zebrabug picture zebrabug on 14 Feb 2017
👍2

＠knguyen0105， really thanks a lot

RSjakson picture RSjakson on 30 Oct 2017
Was this page helpful?
0 / 5 - 0 ratings

Related issues

tqchen picture tqchen  ·  4Comments
choushishi picture choushishi  ·  3Comments
XiaoxiaoWang87 picture XiaoxiaoWang87  ·  3Comments
vkuznet picture vkuznet  ·  3Comments
uasthana15 picture uasthana15  ·  4Comments