Xgboost: Problems while installing in OSX 10.11.6?

Created on 24 Aug 2016  ·  37Comments  ·  Source: dmlc/xgboost

I am trying to install xgboost's python wrapper in OSX 10.11.6 with python3. Firstly, I did:

user@MacBook-Pro-de-User-2:~$ brew install gcc --without-multilib
==> Using the sandbox
==> Downloading https://ftpmirror.gnu.org/gcc/gcc-6.2.0/gcc-6.2.0.tar.bz2
Already downloaded: /Users/user/Library/Caches/Homebrew/gcc-6.2.0.tar.bz2
==> Downloading https://raw.githubusercontent.com/Homebrew/formula-patches/e9e0
Already downloaded: /Users/user/Library/Caches/Homebrew/gcc--patch-863957f90a934ee8f89707980473769cff47ca0663c3906992da6afb242fb220.patch
==> Patching
==> Applying 6.1.0-jit.patch
patching file gcc/jit/Make-lang.in
==> ../configure --build=x86_64-apple-darwin15.6.0 --prefix=/usr/local/Cellar/g
==> make bootstrap

==> make install
🍺  /usr/local/Cellar/gcc/6.2.0: 1,357 files, 242.5M, built in 69 minutes 18 seconds

Then, when I proceed to install xgboost with pip3, I get the following:

user@MacBook-Pro-de-User-2:~$ pip3 install xgboost
Collecting xgboost
  Using cached xgboost-0.6a2.tar.gz
    Complete output from command python setup.py egg_info:
    rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/learner.o] Error 127
    make: *** Waiting for unfinished jobs....
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/logging.o] Error 127
    -----------------------------
    Building multi-thread xgboost failed
    Start to build single-thread xgboost
    rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/learner.o] Error 127
    make: *** Waiting for unfinished jobs....
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/logging.o] Error 127
    Successfully build single-thread xgboost
    If you want multi-threaded version
    See additional instructions in doc/build.md
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/hg/zkgs3l417b7_0l4vth4bf8l00000gn/T/pip-build-hsmd0nxh/xgboost/setup.py", line 29, in <module>
        LIB_PATH = libpath['find_lib_path']()
      File "/private/var/folders/hg/zkgs3l417b7_0l4vth4bf8l00000gn/T/pip-build-hsmd0nxh/xgboost/xgboost/libpath.py", line 45, in find_lib_path
        'List of candidates:\n' + ('\n'.join(dll_path)))
    XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
    List of candidates:
    /private/var/folders/hg/zkgs3l417b7_0l4vth4bf8l00000gn/T/pip-build-hsmd0nxh/xgboost/xgboost/libxgboost.so
    /private/var/folders/hg/zkgs3l417b7_0l4vth4bf8l00000gn/T/pip-build-hsmd0nxh/xgboost/xgboost/../../lib/libxgboost.so
    /private/var/folders/hg/zkgs3l417b7_0l4vth4bf8l00000gn/T/pip-build-hsmd0nxh/xgboost/xgboost/./lib/libxgboost.so

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/hg/zkgs3l417b7_0l4vth4bf8l00000gn/T/pip-build-hsmd0nxh/xgboost/

Then I tried to build it as the documentation states:

Last login: Tue Aug 23 20:53:31 on ttys000
user@MacBook-Pro-de-User-2:~$ git clone --recursive https://github.com/dmlc/xgboost
Cloning into 'xgboost'...
remote: Counting objects: 17651, done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 17651 (delta 9), reused 0 (delta 0), pack-reused 17611
Receiving objects: 100% (17651/17651), 5.80 MiB | 1.08 MiB/s, done.
Resolving deltas: 100% (10851/10851), done.
Checking connectivity... done.
Submodule 'dmlc-core' (https://github.com/dmlc/dmlc-core) registered for path 'dmlc-core'
Submodule 'rabit' (https://github.com/dmlc/rabit) registered for path 'rabit'
Cloning into 'dmlc-core'...
remote: Counting objects: 3710, done.
remote: Total 3710 (delta 0), reused 0 (delta 0), pack-reused 3710
Receiving objects: 100% (3710/3710), 822.86 KiB | 864.00 KiB/s, done.
Resolving deltas: 100% (2197/2197), done.
Checking connectivity... done.
Submodule path 'dmlc-core': checked out 'd8d4dccb8bd89af9f57aa19e7523f0b755563ca4'
Cloning into 'rabit'...
remote: Counting objects: 3096, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 3096 (delta 0), reused 0 (delta 0), pack-reused 3090
Receiving objects: 100% (3096/3096), 886.44 KiB | 904.00 KiB/s, done.
Resolving deltas: 100% (2007/2007), done.
Checking connectivity... done.
Submodule path 'rabit': checked out '2dd7476ad7e4afe97eb579d39aab226ff74400a6'

Then, I tried to build as follows:

user@MacBook-Pro-de-User-2:/$ cd xgboost; cp make/config.mk ./config.mk; make -j4
-bash: cd: xgboost: No such file or directory
cp: make/config.mk: No such file or directory
make: *** No targets specified and no makefile found.  Stop.

Any idea of how to install correctly xgboost?... a month ago I've installed xgboost with pip3 and I did not face the above issues.

picture alonsopg
👍3

Most helpful comment

Success! 🎉

I found this here:

clang LLVM compiler on Mac OS X from Xcode doesn't support OpenMP multi-thread. An alternative choice is installing homebrew http://brew.sh/ and brew install g++-5 which provides multi-thread OpenMP support.

Bearing that in mind, I tried this and it allows me to install from pip!

$ brew install gcc@5
$ pip install xgboost

I have no idea why this works but brew install gcc --without-multilib didn't.

I can't help but think other users will fall into this trap and waste time trying to work out what is actually going wrong. Hopefully this workaround works for them but I'd prefer to see a more reliable installation routine from the pip package 🤞

picture JoeNyland on 6 Apr 2017
👍139 🎉40 ❤26 😄16

All 37 comments

Apparently this worked for OSX:

user@MacBook-Pro-de-User-2:~$ cd xgboost/
user@MacBook-Pro-de-User-2:~/xgboost$ clear
user@MacBook-Pro-de-User-2:~/xgboost$ ls
CMakeLists.txt     R-package/         doc/               rabit/
CONTRIBUTORS.md    README.md          include/           src/
ISSUE_TEMPLATE.md  amalgamation/      jvm-packages/      tests/
LICENSE            build.sh*          make/
Makefile           demo/              plugin/
NEWS.md            dmlc-core/         python-package/
user@MacBook-Pro-de-User-2:~/xgboost$ ./build.sh
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/learner.cc -o build/learner.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/logging.cc -o build/logging.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/c_api/c_api.cc -o build/c_api/c_api.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/c_api/c_api_error.cc -o build/c_api/c_api_error.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/common/common.o src/common/common.cc >build/common/common.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/common/common.cc -o build/common/common.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/data/data.o src/data/data.cc >build/data/data.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/data/data.cc -o build/data/data.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/data/simple_csr_source.o src/data/simple_csr_source.cc >build/data/simple_csr_source.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/data/simple_csr_source.cc -o build/data/simple_csr_source.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/data/simple_dmatrix.o src/data/simple_dmatrix.cc >build/data/simple_dmatrix.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/data/simple_dmatrix.cc -o build/data/simple_dmatrix.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/data/sparse_page_dmatrix.o src/data/sparse_page_dmatrix.cc >build/data/sparse_page_dmatrix.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/data/sparse_page_dmatrix.cc -o build/data/sparse_page_dmatrix.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/data/sparse_page_raw_format.o src/data/sparse_page_raw_format.cc >build/data/sparse_page_raw_format.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/data/sparse_page_raw_format.cc -o build/data/sparse_page_raw_format.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/data/sparse_page_source.o src/data/sparse_page_source.cc >build/data/sparse_page_source.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/data/sparse_page_source.cc -o build/data/sparse_page_source.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/data/sparse_page_writer.o src/data/sparse_page_writer.cc >build/data/sparse_page_writer.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/data/sparse_page_writer.cc -o build/data/sparse_page_writer.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/gbm/gblinear.o src/gbm/gblinear.cc >build/gbm/gblinear.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/gbm/gblinear.cc -o build/gbm/gblinear.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/gbm/gbm.o src/gbm/gbm.cc >build/gbm/gbm.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/gbm/gbm.cc -o build/gbm/gbm.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/gbm/gbtree.o src/gbm/gbtree.cc >build/gbm/gbtree.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/gbm/gbtree.cc -o build/gbm/gbtree.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/metric/elementwise_metric.o src/metric/elementwise_metric.cc >build/metric/elementwise_metric.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/metric/elementwise_metric.cc -o build/metric/elementwise_metric.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/metric/metric.o src/metric/metric.cc >build/metric/metric.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/metric/metric.cc -o build/metric/metric.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/metric/multiclass_metric.o src/metric/multiclass_metric.cc >build/metric/multiclass_metric.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/metric/multiclass_metric.cc -o build/metric/multiclass_metric.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/metric/rank_metric.o src/metric/rank_metric.cc >build/metric/rank_metric.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/metric/rank_metric.cc -o build/metric/rank_metric.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/objective/multiclass_obj.o src/objective/multiclass_obj.cc >build/objective/multiclass_obj.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/objective/multiclass_obj.cc -o build/objective/multiclass_obj.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/objective/objective.o src/objective/objective.cc >build/objective/objective.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/objective/objective.cc -o build/objective/objective.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/objective/rank_obj.o src/objective/rank_obj.cc >build/objective/rank_obj.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/objective/rank_obj.cc -o build/objective/rank_obj.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/objective/regression_obj.o src/objective/regression_obj.cc >build/objective/regression_obj.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/objective/regression_obj.cc -o build/objective/regression_obj.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/tree_model.o src/tree/tree_model.cc >build/tree/tree_model.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/tree_model.cc -o build/tree/tree_model.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/tree_updater.o src/tree/tree_updater.cc >build/tree/tree_updater.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/tree_updater.cc -o build/tree/tree_updater.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/updater_colmaker.o src/tree/updater_colmaker.cc >build/tree/updater_colmaker.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/updater_colmaker.cc -o build/tree/updater_colmaker.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/updater_histmaker.o src/tree/updater_histmaker.cc >build/tree/updater_histmaker.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/updater_histmaker.cc -o build/tree/updater_histmaker.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/updater_prune.o src/tree/updater_prune.cc >build/tree/updater_prune.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/updater_prune.cc -o build/tree/updater_prune.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/updater_refresh.o src/tree/updater_refresh.cc >build/tree/updater_refresh.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/updater_refresh.cc -o build/tree/updater_refresh.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/updater_skmaker.o src/tree/updater_skmaker.cc >build/tree/updater_skmaker.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/updater_skmaker.cc -o build/tree/updater_skmaker.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/tree/updater_sync.o src/tree/updater_sync.cc >build/tree/updater_sync.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/tree/updater_sync.cc -o build/tree/updater_sync.o
cd dmlc-core; /Library/Developer/CommandLineTools/usr/bin/make libdmlc.a config=/Users/user/xgboost/make/config.mk; cd /Users/user/xgboost
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o line_split.o src/io/line_split.cc
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o recordio_split.o src/io/recordio_split.cc
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o input_split_base.o src/io/input_split_base.cc
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o io.o src/io.cc
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o local_filesys.o src/io/local_filesys.cc
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o data.o src/data.cc
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o recordio.o src/recordio.cc
g++-6 -c -O3 -Wall -msse2  -Wno-unknown-pragmas -Iinclude  -std=c++0x -fopenmp -fPIC -DDMLC_USE_HDFS=0 -DDMLC_USE_S3=0 -DDMLC_USE_AZURE=0 -o config.o src/config.cc
ar cr libdmlc.a line_split.o recordio_split.o input_split_base.o io.o local_filesys.o data.o recordio.o config.o
cd rabit; /Library/Developer/CommandLineTools/usr/bin/make lib/librabit.a; cd /Users/user/xgboost
g++-6 -c -O3 -msse2 -Wall -Wextra -Wno-unused-parameter -Wno-unknown-pragmas -std=c++0x -fPIC -o allreduce_base.o src/allreduce_base.cc
g++-6 -c -O3 -msse2 -Wall -Wextra -Wno-unused-parameter -Wno-unknown-pragmas -std=c++0x -fPIC -o allreduce_robust.o src/allreduce_robust.cc
g++-6 -c -O3 -msse2 -Wall -Wextra -Wno-unused-parameter -Wno-unknown-pragmas -std=c++0x -fPIC -o engine.o src/engine.cc
g++-6 -c -O3 -msse2 -Wall -Wextra -Wno-unused-parameter -Wno-unknown-pragmas -std=c++0x -fPIC -o c_api.o src/c_api.cc
ar cr lib/librabit.a allreduce_base.o allreduce_robust.o engine.o c_api.o
ar crv lib/libxgboost.a build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o
a - build/learner.o
a - build/logging.o
a - build/c_api/c_api.o
a - build/c_api/c_api_error.o
a - build/common/common.o
a - build/data/data.o
a - build/data/simple_csr_source.o
a - build/data/simple_dmatrix.o
a - build/data/sparse_page_dmatrix.o
a - build/data/sparse_page_raw_format.o
a - build/data/sparse_page_source.o
a - build/data/sparse_page_writer.o
a - build/gbm/gblinear.o
a - build/gbm/gbm.o
a - build/gbm/gbtree.o
a - build/metric/elementwise_metric.o
a - build/metric/metric.o
a - build/metric/multiclass_metric.o
a - build/metric/rank_metric.o
a - build/objective/multiclass_obj.o
a - build/objective/objective.o
a - build/objective/rank_obj.o
a - build/objective/regression_obj.o
a - build/tree/tree_model.o
a - build/tree/tree_updater.o
a - build/tree/updater_colmaker.o
a - build/tree/updater_histmaker.o
a - build/tree/updater_prune.o
a - build/tree/updater_refresh.o
a - build/tree/updater_skmaker.o
a - build/tree/updater_sync.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -shared -o lib/libxgboost.so build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm  -fopenmp
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/cli_main.o src/cli_main.cc >build/cli_main.d
g++-6 -c -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -c src/cli_main.cc -o build/cli_main.o
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -o xgboost  build/cli_main.o build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a  -pthread -lm  -fopenmp
Successfully build multi-thread xgboost
user@MacBook-Pro-de-User-2:~/xgboost$ cd python-package; python3 setup.py install
Install libxgboost from: ['/Users/user/xgboost/python-package/xgboost/../../lib/libxgboost.so']
running install
running bdist_egg
running egg_info
creating xgboost.egg-info
writing xgboost.egg-info/PKG-INFO
writing dependency_links to xgboost.egg-info/dependency_links.txt
writing top-level names to xgboost.egg-info/top_level.txt
writing requirements to xgboost.egg-info/requires.txt
writing manifest file 'xgboost.egg-info/SOURCES.txt'
reading manifest file 'xgboost.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'xgboost/include'
warning: no files found matching '*' under directory 'xgboost/src'
warning: no files found matching '*' under directory 'xgboost/make'
warning: no files found matching '*' under directory 'xgboost/rabit'
warning: no files found matching '*' under directory 'xgboost/lib'
warning: no files found matching '*' under directory 'xgboost/dmlc-core'
warning: no previously-included files matching '*.o' found anywhere in distribution
warning: no previously-included files matching '*.a' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'xgboost.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.11-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/xgboost
copying xgboost/__init__.py -> build/lib/xgboost
copying xgboost/callback.py -> build/lib/xgboost
copying xgboost/compat.py -> build/lib/xgboost
copying xgboost/core.py -> build/lib/xgboost
copying xgboost/libpath.py -> build/lib/xgboost
copying xgboost/plotting.py -> build/lib/xgboost
copying xgboost/rabit.py -> build/lib/xgboost
copying xgboost/sklearn.py -> build/lib/xgboost
copying xgboost/training.py -> build/lib/xgboost
copying xgboost/VERSION -> build/lib/xgboost
copying xgboost/build-python.sh -> build/lib/xgboost
creating build/bdist.macosx-10.11-x86_64
creating build/bdist.macosx-10.11-x86_64/egg
creating build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/__init__.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/build-python.sh -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/callback.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/compat.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/core.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/libpath.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/plotting.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/rabit.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/sklearn.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/training.py -> build/bdist.macosx-10.11-x86_64/egg/xgboost
copying build/lib/xgboost/VERSION -> build/bdist.macosx-10.11-x86_64/egg/xgboost
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/__init__.py to __init__.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/callback.py to callback.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/compat.py to compat.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/core.py to core.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/libpath.py to libpath.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/plotting.py to plotting.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/rabit.py to rabit.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/sklearn.py to sklearn.cpython-35.pyc
byte-compiling build/bdist.macosx-10.11-x86_64/egg/xgboost/training.py to training.cpython-35.pyc
installing package data to build/bdist.macosx-10.11-x86_64/egg
running install_data
copying /Users/user/xgboost/python-package/xgboost/../../lib/libxgboost.so -> build/bdist.macosx-10.11-x86_64/egg/xgboost
creating build/bdist.macosx-10.11-x86_64/egg/EGG-INFO
copying xgboost.egg-info/PKG-INFO -> build/bdist.macosx-10.11-x86_64/egg/EGG-INFO
copying xgboost.egg-info/SOURCES.txt -> build/bdist.macosx-10.11-x86_64/egg/EGG-INFO
copying xgboost.egg-info/dependency_links.txt -> build/bdist.macosx-10.11-x86_64/egg/EGG-INFO
copying xgboost.egg-info/not-zip-safe -> build/bdist.macosx-10.11-x86_64/egg/EGG-INFO
copying xgboost.egg-info/requires.txt -> build/bdist.macosx-10.11-x86_64/egg/EGG-INFO
copying xgboost.egg-info/top_level.txt -> build/bdist.macosx-10.11-x86_64/egg/EGG-INFO
creating dist
creating 'dist/xgboost-0.6-py3.5.egg' and adding 'build/bdist.macosx-10.11-x86_64/egg' to it
removing 'build/bdist.macosx-10.11-x86_64/egg' (and everything under it)
Processing xgboost-0.6-py3.5.egg
creating /usr/local/lib/python3.5/site-packages/xgboost-0.6-py3.5.egg
Extracting xgboost-0.6-py3.5.egg to /usr/local/lib/python3.5/site-packages
Adding xgboost 0.6 to easy-install.pth file

Installed /usr/local/lib/python3.5/site-packages/xgboost-0.6-py3.5.egg
Processing dependencies for xgboost==0.6
Searching for scipy==0.18.0
Best match: scipy 0.18.0
Adding scipy 0.18.0 to easy-install.pth file

Using /usr/local/lib/python3.5/site-packages
Searching for numpy==1.11.1
Best match: numpy 1.11.1
Adding numpy 1.11.1 to easy-install.pth file

Using /usr/local/lib/python3.5/site-packages
Finished processing dependencies for xgboost==0.6

However, I would like to know if there is an easier way to install it, any suggestion guys?.

alonsopg picture alonsopg on 24 Aug 2016
👍9 🎉1

@alonsopg the current pip installation requires gcc-5 or clang-omp which both can be installed via brew. It was done before the recent instruction update. Since major Mac OS X users now use gcc-6, I will update the next version to gcc-6, meanwhile, please install from github as you have compiled

phunterlau picture phunterlau on 24 Aug 2016
👍4

+1 also came across exactly this issue. Very tricky to figure out. Also note that clang-omp is no longer a brew package:

$ brew install clang-omp
Error: No available formula with the name "clang-omp" 
==> Searching for similarly named formulae...
Error: No similarly named formulae found.
==> Searching taps...
Error: No formulae found in taps.
samuela picture samuela on 26 Oct 2016

@samuela do you want to try brew install --with-clang llvm since clang-omp is moved

phunterlau picture phunterlau on 27 Oct 2016

@phunterlau I still cannot get pip install xgboost working on Mac 10.12.3

I'm getting:

XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?

I've tried:

$ brew install clang-omp
Error: No available formula with the name "clang-omp" 
==> Searching for similarly named formulae...
Error: No similarly named formulae found.
==> Searching taps...
Error: No formulae found in taps.

$ brew install --with-clang llvm
Warning: llvm: this formula has no --with-clang option so it will be ignored!
# snip
$ export PATH="/usr/local/opt/llvm/bin:$PATH"
$ export LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"
$ export CPPFLAGS="-I/usr/local/opt/llvm/include"
$ pip install xgboost
# snip
XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
# snip

$ brew install gcc --without-multilib
$ pip install xgboost
# snip
XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
# snip

How are we meant to install this from PyPi on Mac 10.12? Installation from source is not a solution.

JoeNyland picture JoeNyland on 21 Feb 2017

@JoeNyland maybe you need :

brew install cmake

Maybe you can get some ideas from https://github.com/dmlc/xgboost/issues/463.

GnoDGnaF picture GnoDGnaF on 11 Mar 2017

@d2Code Thanks, but I still get the same error message after brew install cmake:

$ pip install xgboost
...
XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
...
JoeNyland picture JoeNyland on 13 Mar 2017

@JoeNyland You can try to install xgboost from source , not by pip.

GnoDGnaF picture GnoDGnaF on 14 Mar 2017

@d2Code I know I can install from source, but I need to install from pip. I'm trying to get to the bottom of why I can't install from pip.

How are we meant to install this from PyPi on Mac 10.12? Installation from source is not a solution.

JoeNyland picture JoeNyland on 14 Mar 2017
👍5

@evanuk Thanks for your suggestion. Unfortunately I'm getting the same error message as before even with your command above 😕 

$ env CC=gcc CXX=gcc pip install xgboost
Collecting xgboost
  Using cached xgboost-0.6a2.tar.gz
    Complete output from command python setup.py egg_info:
    rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/learner.o] Error 127
    make: *** Waiting for unfinished jobs....
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/logging.o] Error 127
    -----------------------------
    Building multi-thread xgboost failed
    Start to build single-thread xgboost
    rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/learner.o] Error 127
    make: *** Waiting for unfinished jobs....
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/logging.o] Error 127
    Successfully build single-thread xgboost
    If you want multi-threaded version
    See additional instructions in doc/build.md
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/91/nqdcr19d0xg6sjdb4m3vcc3c0000gn/T/pip-build-ozos8fjf/xgboost/setup.py", line 29, in <module>
        LIB_PATH = libpath['find_lib_path']()
      File "/private/var/folders/91/nqdcr19d0xg6sjdb4m3vcc3c0000gn/T/pip-build-ozos8fjf/xgboost/xgboost/libpath.py", line 45, in find_lib_path
        'List of candidates:\n' + ('\n'.join(dll_path)))
    XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
    List of candidates:
    /private/var/folders/91/nqdcr19d0xg6sjdb4m3vcc3c0000gn/T/pip-build-ozos8fjf/xgboost/xgboost/libxgboost.so
    /private/var/folders/91/nqdcr19d0xg6sjdb4m3vcc3c0000gn/T/pip-build-ozos8fjf/xgboost/xgboost/../../lib/libxgboost.so
    /private/var/folders/91/nqdcr19d0xg6sjdb4m3vcc3c0000gn/T/pip-build-ozos8fjf/xgboost/xgboost/./lib/libxgboost.so

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/91/nqdcr19d0xg6sjdb4m3vcc3c0000gn/T/pip-build-ozos8fjf/xgboost/

$ brew info gcc | head -n 5
gcc: stable 6.3.0 (bottled), HEAD
GNU compiler collection
https://gcc.gnu.org/
/usr/local/Cellar/gcc/6.3.0_1 (1,358 files, 237.8MB) *
  Built from source on 2017-04-06 at 11:53:19 with: --without-multilib
JoeNyland picture JoeNyland on 6 Apr 2017

Success! 🎉

I found this here:

clang LLVM compiler on Mac OS X from Xcode doesn't support OpenMP multi-thread. An alternative choice is installing homebrew http://brew.sh/ and brew install g++-5 which provides multi-thread OpenMP support.

Bearing that in mind, I tried this and it allows me to install from pip!

$ brew install gcc@5
$ pip install xgboost

I have no idea why this works but brew install gcc --without-multilib didn't.

I can't help but think other users will fall into this trap and waste time trying to work out what is actually going wrong. Hopefully this workaround works for them but I'd prefer to see a more reliable installation routine from the pip package 🤞

JoeNyland picture JoeNyland on 6 Apr 2017
👍139 🎉40 ❤26 😄16

@JoeNyland's solution worked for me also. Thanks!!

isuda picture isuda on 7 Apr 2017
❤3

thank for your solution!@JoeNyland

paulranp picture paulranp on 10 May 2017
❤2

I did not use pip for it. There were some errors in the documentation, but if you follow the link below there are some ways to work through it.

http://stackoverflow.com/questions/39315156/how-to-install-xgboost-in-python-on-mac/43101972?noredirect=1#comment75102985_43101972

bgbutler picture bgbutler on 19 May 2017

I installed gcc7.1.0 with brew install gcc --without-multilib, then cloned xgboost, cd-ed into the folder, and edited make/config.mk to have 

# choice of compiler, by default use system preference.
export CC = gcc-7
export CXX = g++-7 
# export MPICXX = mpicxx

and finally executed cp make/minimum.mk ./config.mk; make -j4

then cd-ed into python-package and executed sudo python3 setup.py install

okomarov picture okomarov on 29 Jul 2017
👍5

Joe You ara amazing!

erdemalpkaya picture erdemalpkaya on 5 Aug 2017

@JoeNyland ran into the exact same problem and your solution was perfect. thank you!

carletonc picture carletonc on 25 Aug 2017

@JoeNyland You are my hero! Sometimes we just need the stubborn attitude otherwise no problem could be solved. Thank you!

fengerzh picture fengerzh on 30 Aug 2017

JoeNyland picture JoeNyland on 30 Aug 2017

It's not the best solution; it would be nice to not have to depend on an older version of GCC. But it works for now, at least 🙂

JoeNyland picture JoeNyland on 30 Aug 2017

However I still get the following error message after running
$ brew install gcc@5
$ pip install xgboost

Any idea why am I getting this? Much appreciate. 

make: *** [build/data/simple_dmatrix.o] Error 1
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/private/var/folders/60/6w9_z9yd24n7nwv1cg23zz340000gn/T/pip-build-B7gSEL/xgboost/setup.py", line 29, in <module>
    LIB_PATH = libpath['find_lib_path']()
  File "/private/var/folders/60/6w9_z9yd24n7nwv1cg23zz340000gn/T/pip-build-B7gSEL/xgboost/xgboost/libpath.py", line 45, in find_lib_path
    'List of candidates:\n' + ('\n'.join(dll_path)))
__builtin__.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
List of candidates:
/private/var/folders/60/6w9_z9yd24n7nwv1cg23zz340000gn/T/pip-build-B7gSEL/xgboost/xgboost/libxgboost.so
/private/var/folders/60/6w9_z9yd24n7nwv1cg23zz340000gn/T/pip-build-B7gSEL/xgboost/xgboost/../../lib/libxgboost.so
/private/var/folders/60/6w9_z9yd24n7nwv1cg23zz340000gn/T/pip-build-B7gSEL/xgboost/xgboost/./lib/libxgboost.so

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/60/6w9_z9yd24n7nwv1cg23zz340000gn/T/pip-build-B7gSEL/xgboost/

cacoderquan picture cacoderquan on 10 Sep 2017
👍2

@cacoderquan Hey, not sure why you're seeing that issue still. Have you tried pip install xgboost --no-cache-dir.

If you still see the error, please can you post the output of:

  • python --version
  • pip --version
  • brew info gcc@5
JoeNyland picture JoeNyland on 11 Sep 2017

GitHub's really gotta get comment-tldrs, I sifted through this for like 10 minutes, trying random junk along the way until finally seeing the one comment with over 100 likes and thumbs ups and etc (and it worked for me too! so, yay!).

Is there a particular reason it works only with gcc@5 and not whatever gcc people currently/typically have (I've got gcc 7.2.0 at the moment)?

anderson-dan-w picture anderson-dan-w on 12 Oct 2017

Hi guys, I have no hope left, but only you.
I have succeeded to install xgboost with help of okomarov.

```msk-wire-darkstar-m:xgboost m.keremet$ cd python-package
msk-wire-darkstar-m:python-package m.keremet$ sudo python3 setup.py install
Password:
Install libxgboost from: ['../lib/libxgboost.dylib']
running install
running bdist_egg
running egg_info
creating xgboost.egg-info
writing xgboost.egg-info/PKG-INFO
writing dependency_links to xgboost.egg-info/dependency_links.txt
writing requirements to xgboost.egg-info/requires.txt
writing top-level names to xgboost.egg-info/top_level.txt
writing manifest file 'xgboost.egg-info/SOURCES.txt'
reading manifest file 'xgboost.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '' under directory 'xgboost/include'
warning: no files found matching '' under directory 'xgboost/src'
warning: no files found matching '' under directory 'xgboost/make'
warning: no files found matching '' under directory 'xgboost/rabit'
warning: no files found matching '' under directory 'xgboost/lib'
warning: no files found matching '' under directory 'xgboost/dmlc-core'
warning: no previously-included files matching '.o' found anywhere in distribution
warning: no previously-included files matching '.a' found anywhere in distribution
warning: no previously-included files matching '.pyo' found anywhere in distribution
warning: no previously-included files matching '.pyc' found anywhere in distribution
writing manifest file 'xgboost.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.7-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/xgboost
copying xgboost/__init__.py -> build/lib/xgboost
copying xgboost/callback.py -> build/lib/xgboost
copying xgboost/compat.py -> build/lib/xgboost
copying xgboost/core.py -> build/lib/xgboost
copying xgboost/libpath.py -> build/lib/xgboost
copying xgboost/plotting.py -> build/lib/xgboost
copying xgboost/rabit.py -> build/lib/xgboost
copying xgboost/sklearn.py -> build/lib/xgboost
copying xgboost/training.py -> build/lib/xgboost
copying xgboost/VERSION -> build/lib/xgboost
copying xgboost/build-python.sh -> build/lib/xgboost
creating build/bdist.macosx-10.7-x86_64
creating build/bdist.macosx-10.7-x86_64/egg
creating build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/__init__.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/build-python.sh -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/callback.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/compat.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/core.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/libpath.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/plotting.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/rabit.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/sklearn.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/training.py -> build/bdist.macosx-10.7-x86_64/egg/xgboost
copying build/lib/xgboost/VERSION -> build/bdist.macosx-10.7-x86_64/egg/xgboost
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/callback.py to callback.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/compat.py to compat.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/core.py to core.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/libpath.py to libpath.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/plotting.py to plotting.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/rabit.py to rabit.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/sklearn.py to sklearn.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/xgboost/training.py to training.cpython-36.pyc
installing package data to build/bdist.macosx-10.7-x86_64/egg
running install_data
copying ../lib/libxgboost.dylib -> build/bdist.macosx-10.7-x86_64/egg/xgboost
creating build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying xgboost.egg-info/PKG-INFO -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying xgboost.egg-info/SOURCES.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying xgboost.egg-info/dependency_links.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying xgboost.egg-info/not-zip-safe -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying xgboost.egg-info/requires.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying xgboost.egg-info/top_level.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
creating dist
creating 'dist/xgboost-0.6-py3.6.egg' and adding 'build/bdist.macosx-10.7-x86_64/egg' to it
removing 'build/bdist.macosx-10.7-x86_64/egg' (and everything under it)
Processing xgboost-0.6-py3.6.egg
removing '/anaconda/lib/python3.6/site-packages/xgboost-0.6-py3.6.egg' (and everything under it)
creating /anaconda/lib/python3.6/site-packages/xgboost-0.6-py3.6.egg
Extracting xgboost-0.6-py3.6.egg to /anaconda/lib/python3.6/site-packages
xgboost 0.6 is already the active version in easy-install.pth

Installed /anaconda/lib/python3.6/site-packages/xgboost-0.6-py3.6.egg
Processing dependencies for xgboost==0.6
Searching for scipy==0.19.1
Best match: scipy 0.19.1
Adding scipy 0.19.1 to easy-install.pth file

Using /anaconda/lib/python3.6/site-packages
Searching for numpy==1.13.1
Best match: numpy 1.13.1
Adding numpy 1.13.1 to easy-install.pth file

Using /anaconda/lib/python3.6/site-packages
Finished processing dependencies for xgboost==0.6
msk-wire-darkstar-m:python-package m.keremet$ python
Python 3.6.1 |Anaconda custom (x86_64)| (default, May 11 2017, 13:04:09)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import xgboost 

Seems to work since no errors. However, when I try to import xgboost in jupyter it shoots this error:
```---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-1-1feb2e63ea38> in <module>()
      3 import numpy as np
      4 import pandas as pd
----> 5 import xgboost as xgb

/anaconda/lib/python3.6/site-packages/xgboost/__init__.py in <module>()
      9 import os
     10 
---> 11 from .core import DMatrix, Booster
     12 from .training import train, cv
     13 from . import rabit                   # noqa

/anaconda/lib/python3.6/site-packages/xgboost/core.py in <module>()
    110 
    111 # load the XGBoost library globally
--> 112 _LIB = _load_lib()
    113 
    114 

/anaconda/lib/python3.6/site-packages/xgboost/core.py in _load_lib()
    104     if len(lib_path) == 0:
    105         return None
--> 106     lib = ctypes.cdll.LoadLibrary(lib_path[0])
    107     lib.XGBGetLastError.restype = ctypes.c_char_p
    108     return lib

/anaconda/lib/python3.6/ctypes/__init__.py in LoadLibrary(self, name)
    424 
    425     def LoadLibrary(self, name):
--> 426         return self._dlltype(name)
    427 
    428 cdll = LibraryLoader(CDLL)

/anaconda/lib/python3.6/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
    346 
    347         if handle is None:
--> 348             self._handle = _dlopen(self._name, mode)
    349         else:
    350             self._handle = handle

OSError: dlopen(/anaconda/lib/python3.6/site-packages/xgboost/./lib/libxgboost.so, 6): Library not loaded: /usr/local/opt/gcc/lib/gcc/5/libgomp.1.dylib
  Referenced from: /anaconda/lib/python3.6/site-packages/xgboost/./lib/libxgboost.so
  Reason: image not found

I haven't found much on this error. Does anyone has come across it?

maximkeremet picture maximkeremet on 30 Oct 2017

@cacoderquan I met similar issue as you and find that actually the gcc is not linked.
You may want to check it as well but I bet your problem has solved already.
For others interested, the unlinked gcc may result from homebrew permission issues.

@JoeNyland many thanks for saving my day.

nickyongzhang picture nickyongzhang on 22 Nov 2017

@cacoderquan I have similar issues when trying to pip install xgboost
My version of gcc is

gcc: stable 7.2.0 (bottled), HEAD
GNU compiler collection
https://gcc.gnu.org/
/usr/local/Cellar/gcc/7.2.0 (1,487 files, 285.2MB) *
  Built from source on 2017-12-01 at 17:22:53

I have tried installing gcc many times but nothing works. Every time I tried pip install xgboost the following errors appear:

Collecting xgboost
  Using cached xgboost-0.6a2.tar.gz
    Complete output from command python setup.py egg_info:
    rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/learner.o] Error 127
    make: *** Waiting for unfinished jobs....
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/logging.o] Error 127
    -----------------------------
    Building multi-thread xgboost failed
    Start to build single-thread xgboost
    rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/learner.o] Error 127
    make: *** Waiting for unfinished jobs....
    clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
    /bin/sh: clang-omp++: command not found
    make: *** [build/logging.o] Error 127
    Successfully build single-thread xgboost
    If you want multi-threaded version
    See additional instructions in doc/build.md
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/19/1pcpc2r57_g6gjl_2v5_4kgc0000gn/T/pip-build-q8wmsvem/xgboost/setup.py", line 29, in <module>
        LIB_PATH = libpath['find_lib_path']()
      File "/private/var/folders/19/1pcpc2r57_g6gjl_2v5_4kgc0000gn/T/pip-build-q8wmsvem/xgboost/xgboost/libpath.py", line 45, in find_lib_path
        'List of candidates:\n' + ('\n'.join(dll_path)))
    XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
    List of candidates:
    /private/var/folders/19/1pcpc2r57_g6gjl_2v5_4kgc0000gn/T/pip-build-q8wmsvem/xgboost/xgboost/libxgboost.so
    /private/var/folders/19/1pcpc2r57_g6gjl_2v5_4kgc0000gn/T/pip-build-q8wmsvem/xgboost/xgboost/../../lib/libxgboost.so
    /private/var/folders/19/1pcpc2r57_g6gjl_2v5_4kgc0000gn/T/pip-build-q8wmsvem/xgboost/xgboost/./lib/libxgboost.so

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/19/1pcpc2r57_g6gjl_2v5_4kgc0000gn/T/pip-build-q8wmsvem/xgboost/

PLZ HELP

schen57 picture schen57 on 2 Dec 2017

@schen57 It's documented in several places that the pip install xgboost method of installation only works with GCC 5.

Please brew install gcc@5 and retry the installation.

https://github.com/dmlc/xgboost/blob/master/python-package/README.rst
https://xgboost.readthedocs.io/en/latest/build.html#building-on-macos
https://github.com/dmlc/xgboost/issues/1501#issuecomment-292209578

@alonsopg @phunterlau I wonder if this issue can be closed now? It's getting pretty big and out of control as there's a solution to the original problem that the OP reported. The long term fix for this is to allow the pip install xgboost method to work with up to date GCC versions, but I feel this is better handled in a separate issue focussed specifically on that. What do you think?

JoeNyland picture JoeNyland on 2 Dec 2017

Yes, agreed. The pip installation was made for the golden stable version of xgboost so a stable version of gcc was chosen. The pip installation will soon use the official release and get precompiled binary as planned.

phunterlau picture phunterlau on 3 Dec 2017
👍2

Although I installed gcc 5 (with brew), the xgboost build still used the LLVM version of gcc (7 in my case). To get the build to use gcc 5 I did:

brew install gcc@5
env CC=gcc-5 CXX=g++-5 pip install xgboost

and this worked for me.

davidread picture davidread on 8 Jan 2018
👍30 ❤8 🎉8

The following worked for me to get the newest version of XGBoost into Anaconda painlessly:

  • install brew (if not already there)
  • follow @davidread 's instructions: (thank you @davidread !!!)
brew install gcc@5
env CC=gcc-5 CXX=g++-5 pip install xgboost
mweber-ovt picture mweber-ovt on 15 Jan 2018
❤2

Thank you @davidread your solution worked after struggling for hours. Godsend!!!

sagar-m picture sagar-m on 16 Jan 2018

In addition to @davidread 's solution, if pip install doesn't work for you, try the second line as:
env CC=gcc-5 CXX=g++-5 pip3 install xgboost
This should work!

JingyuanXing picture JingyuanXing on 27 Jan 2018
👍4

Thank you @davidread. After struggling for hours with gcc-7 and --without-multilib option (which by the way has been disabled in gcc), I struggled some more with gcc-5 and pip until I finally found your solution! Thanks again.

chetankshetty picture chetankshetty on 8 Feb 2018

@cacoderquan I had the same issue. Don't know if you've had this sorted yet (didn't read through all the comments) but I was able to install by first running the build.sh file in the terminal and then running setup.py in the python-package folder.

ssr48 picture ssr48 on 18 Feb 2018

Thank you for all the help above, and especially to @davidread, your method worked for me as well, how do we make sure that we have multi-threading? Thanks in advance guys

thinslicer picture thinslicer on 19 Feb 2018

@okomarov Your solution for installing xgboost worked for me. Thanks a bunch!

ecampana picture ecampana on 8 Mar 2018

@davidread Thanks a lot, your method worked for me!

drewszurko picture drewszurko on 12 Mar 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

RanaivosonHerimanitra picture RanaivosonHerimanitra  ·  3Comments
tqchen picture tqchen  ·  4Comments
mhnamaki picture mhnamaki  ·  3Comments
choushishi picture choushishi  ·  3Comments
nicoJiang picture nicoJiang  ·  4Comments