Xgboost: libpath['find_lib_path']() returns absolute path for libxgboost.dll on Windows; setup install fails

Created on 7 Dec 2016 · 3Comments · Source: dmlc/xgboost

Environment info

Operating System: Windows 7

Compiler: MSVC++ 12 (Visual Studio 2013)

Package used (python/R/jvm/C++): Python 3.4.4

xgboost version used: 0.6

If installing from source, please provide

The commit hash (git rev-parse HEAD): e7fbc8591fa7277ee4c474b7371c48c11b34cbde
Logs:

Install libxgboost from: ['C:\\Python34\\XGBoost\\python-package\\xgboost\\../..
/lib/libxgboost.dll']
running install
running bdist_egg
running egg_info
writing requirements to xgboost.egg-info\requires.txt
writing xgboost.egg-info\PKG-INFO
writing dependency_links to xgboost.egg-info\dependency_links.txt
writing top-level names to xgboost.egg-info\top_level.txt
reading manifest file 'xgboost.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'xgboost\include'
warning: no files found matching '*' under directory 'xgboost\src'
warning: no files found matching '*' under directory 'xgboost\make'
warning: no files found matching '*' under directory 'xgboost\rabit'
warning: no files found matching '*' under directory 'xgboost\lib'
warning: no files found matching '*' under directory 'xgboost\dmlc-core'
warning: no previously-included files matching '*.o' found anywhere in distribution
warning: no previously-included files matching '*.a' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'xgboost.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
error: Error: setup script specifies an absolute path:

    C:\Python34\XGBoost\python-package\xgboost\..\..\lib\libxgboost.dll

setup() arguments must *always* be /-separated paths relative to the
setup.py directory, *never* absolute paths.

If you are using python package, please provide

The python version and distribution - 3.4.4 official build for Win64
The command to install xgboost if you are not installing from source - python.exe setup.py install (via setuptools)

Steps to reproduce

Download source code of xgboost from github
mkdir build && cd build && cmake .. -G"Visual Studio 12 2013 Win64"
Open xgboost.sln with VS2013 and build all solution with Release configuration, for x64 platform. You'll get lib/libxgboost.dll library.
cd ..python-package && python.exe setup.py install

What have you tried?

Replace file path in SOURCES.txt file - this has no effect, cuz this file is overwritten every time
Change python-package/MANIFEST.in and exclude lib directory - this has no effect, cuz there is no rule for .dll file
Replace python-package/setup.py, line 19
LIB_PATH = libpath['find_lib_path']()
to
LIB_PATH = ['../lib/libxgboost.dll']
This was the solution.
By the way, this is an issue, and XGBoost team maybe will solve this in a proper way.

Source

Str1ker17

Most helpful comment

I have also met this errror in centos 6.5 64. I search solutions and decide to edit the python-package/setup.py.
Just set 38th "include_package_data=False" then I execute "python setup.py install" succussfully. Finally "import xgboost as xgb" is correct. However I run the demo, I met errors again.

Finally I clean the files then rebuild the xgboost again. To execute python install and now it's ok!

12-31-2016:
when I install xgboost in centos 7 x86_64, I met this error again. However I recompiled the xgboost and failed to install python.
I try make or cmake to compile, they both ok. But when I install python and meet the same error.
I just wonder why it's not good in centos 7, just because of the version.

anddelu on 27 Dec 2016

🎉2 👍1

All 3 comments

Finally I clean the files then rebuild the xgboost again. To execute python install and now it's ok!

anddelu on 27 Dec 2016

🎉2 👍1

@anddelu include_package_data=False is a must since it uses MANIFEST.in for package maintenance and this only affects the installation, not the path. I think your last step of rebuilding xgboost is the right solution where include_package_data=False should remain as it is.

phunterlau on 27 Dec 2016

I was running into the same error on Linux running Anaconda Python 3.5.

I think I fixed it by making the following modifications to setup.py:

diff --git a/python-package/setup.py b/python-package/setup.py
index 27fc212..ec9b806 100644
--- a/python-package/setup.py
+++ b/python-package/setup.py
@@ -7,7 +7,7 @@ from setuptools import setup, find_packages
 # import subprocess
 sys.path.insert(0, '.')

-CURRENT_DIR = os.path.dirname(__file__)
+CURRENT_DIR = os.path.abspath(os.path.dirname(__file__))

 # We can not import `xgboost.libpath` in setup.py directly since xgboost/__init__.py
 # import `xgboost.core` and finally will import `numpy` and `scipy` which are setup
@@ -17,6 +17,7 @@ libpath = {'__file__': libpath_py}
 exec(compile(open(libpath_py, "rb").read(), libpath_py, 'exec'), libpath, libpath)

 LIB_PATH = libpath['find_lib_path']()
+LIB_PATH = [os.path.relpath(LIB_PATH[0], CURRENT_DIR)]
 print("Install libxgboost from: %s" % LIB_PATH)
 # Please use setup_pip.py for generating and deploying pip installation
 # detailed instruction in setup_pip.py