numpy installation thru install_requires directive issue (Trac #1841)

Created on 20 Oct 2012  ·  39Comments  ·  Source: numpy/numpy

_Original ticket http://projects.scipy.org/numpy/ticket/1841 on 2011-05-25 by trac user ohe, assigned to unknown._

Hello there,

In a virtualenv, with python v2.7.1, i can't install numpy thru the install_requires directive of setuptools/distribute.

from setuptools import setup
setup(...
      install_requires=['numpy'],
      ...)

A python setup.py install gives me the following traceback :

running install
running bdist_egg
running egg_info
writing requirements to UNKNOWN.egg-info/requires.txt
writing UNKNOWN.egg-info/PKG-INFO
writing top-level names to UNKNOWN.egg-info/top_level.txt
writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.6-intel/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

creating build/bdist.macosx-10.6-intel
creating build/bdist.macosx-10.6-intel/egg
creating build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/PKG-INFO -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/SOURCES.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/dependency_links.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/requires.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/top_level.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/UNKNOWN-0.0.0-py2.7.egg' and adding 'build/bdist.macosx-10.6-intel/egg' to it
removing 'build/bdist.macosx-10.6-intel/egg' (and everything under it)
Processing UNKNOWN-0.0.0-py2.7.egg
Copying UNKNOWN-0.0.0-py2.7.egg to /Users/olivier/tc/karmatest/lib/python2.7/site-packages
Adding UNKNOWN 0.0.0 to easy-install.pth file

Installed /Users/olivier/tc/karmatest/lib/python2.7/site-packages/UNKNOWN-0.0.0-py2.7.egg
Processing dependencies for UNKNOWN==0.0.0
Searching for numpy
Reading http://pypi.python.org/simple/numpy/
Reading http://numpy.scipy.org
Reading http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=175103
Reading http://numeric.scipy.org
Best match: numpy 1.6.0
Downloading http://sourceforge.net/projects/numpy/files/NumPy/1.6.0/numpy-1.6.0.tar.gz/download
Processing download
Running numpy-1.6.0/setup.py -q bdist_egg --dist-dir /var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/egg-dist-tmp-Zr2lMr
Running from numpy source directory.Warning: distutils distribution has been initialized, it may be too late to add a subpackage commandWarning: distutils distribution has been initialized, it may be too late to add a subpackage fcompilernon-existing path in '/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils': 'site.cfg'
Warning: distutils distribution has been initialized, it may be too late to add a subpackage distutilsWarning: distutils distribution has been initialized, it may be too late to add a subpackage testingWarning: distutils distribution has been initialized, it may be too late to add a subpackage f2pyWarning: distutils distribution has been initialized, it may be too late to add an extension _sortWarning: distutils distribution has been initialized, it may be too late to add an extension multiarrayWarning: distutils distribution has been initialized, it may be too late to add an extension umathWarning: distutils distribution has been initialized, it may be too late to add an extension scalarmathWarning: distutils distribution has been initialized, it may be too late to add an extension _dotblasWarning: distutils distribution has been initialized, it may be too late to add an extension umath_testsWarning: distutils distribution has been initialized, it may be too late to add an extension multiarray_testsWarning: distutils distribution has been initialized, it may be too late to add a subpackage coreWarning: distutils distribution has been initialized, it may be too late to add an extension _compiled_baseWarning: distutils distribution has been initialized, it may be too late to add a subpackage libWarning: distutils distribution has been initialized, it may be too late to add a subpackage oldnumericWarning: distutils distribution has been initialized, it may be too late to add an extension _capiWarning: distutils distribution has been initialized, it may be too late to add a subpackage numarrayWarning: distutils distribution has been initialized, it may be too late to add an extension fftpack_liteWarning: distutils distribution has been initialized, it may be too late to add a subpackage fftWarning: distutils distribution has been initialized, it may be too late to add an extension lapack_liteWarning: distutils distribution has been initialized, it may be too late to add a subpackage linalgWarning: distutils distribution has been initialized, it may be too late to add an extension mtrandWarning: distutils distribution has been initialized, it may be too late to add a subpackage randomWarning: distutils distribution has been initialized, it may be too late to add a subpackage maWarning: distutils distribution has been initialized, it may be too late to add a subpackage matrixlibWarning: distutils distribution has been initialized, it may be too late to add a subpackage compatWarning: distutils distribution has been initialized, it may be too late to add a subpackage polynomialWarning: distutils distribution has been initialized, it may be too late to add a subpackage docWarning: distutils distribution has been initialized, it may be too late to add a subpackage numpyCould not locate executable f95
Could not locate executable f90
Could not locate executable f77
Could not locate executable xlf90
Could not locate executable xlf
Could not locate executable ifort
Could not locate executable ifc
Could not locate executable g77
Found executable /usr/local/bin/gfortran
_configtest.c:1: warning: conflicting types for built-in function ‘exp’
_configtest.c:1: warning: conflicting types for built-in function ‘exp’
_configtest.c:1:20: error: Python.h: No such file or directory
_configtest.c:1:20: error: Python.h: No such file or directory
lipo: can't figure out the architecture type of: /var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-//ccLgEqLk.out
_configtest.c:1:20: error: Python.h: No such file or directory
_configtest.c:1:20: error: Python.h: No such file or directory
lipo: can't figure out the architecture type of: /var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-//ccLgEqLk.out
Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    install_requires=['numpy']
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 76, in run
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 104, in do_egg_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 211, in run
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 427, in easy_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 478, in install_item
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 519, in process_distribution
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 563, in resolve
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 799, in best_match
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 811, in obtain
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 446, in easy_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 476, in install_item
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 655, in install_eggs
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 930, in build_and_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 919, in run_setup
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", line 62, in run_setup
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", line 105, in run
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", line 64, in <lambda>
  File "setup.py", line 196, in <module>

  File "setup.py", line 189, in setup_package

  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/core.py", line 186, in setup
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/bdist_egg.py", line 167, in run
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/egg_info.py", line 8, in run
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 152, in run
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 169, in build_sources
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 328, in build_extension_sources
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 385, in generate_sources
  File "/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/core/setup.py", line 410, in generate_config_h

  File "/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/core/setup.py", line 41, in check_types

  File "/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/core/setup.py", line 271, in check_types

SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.
/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/misc_util.py:251: RuntimeWarning: Parent module 'numpy.distutils' not found while handling absolute import
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/misc_util.py", line 251, in clean_up_temporary_directory
ImportError: No module named numpy.distutils
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/misc_util.py", line 251, in clean_up_temporary_directory
ImportError: No module named numpy.distutils

I have no probleme to install numpy with pip install numpy or easy_install numpy inside or outside any virtualenv.

OS: Mac OS 10.6.7

00 - Bug Other

Most helpful comment

As an FYI for anyone experiencing this issue, with the latest version of setuptools, a workaround is to add NumPy to setup_requires. I still kept it in install_requires, so my setup.py looks like:

setup(
...
    setup_requires=["numpy"],
    install_requires=["numpy"],
)

This works, and only installs NumPy once, on both Mac OS X and Linux (not tested on Windows).

As for why I have no clue.

All 39 comments

[email protected] wrote on 2011-05-27_

I'm not sure exactly what the problem is, but can you get around it by just running an install script that first installs numpy and then your package? Relying on these badly-documented setuptools features is not a good idea if you can avoid it.

If I'd have to guess, I'd say calling setuptools.setup() for the first time invokes a bunch of magic that screws up something for numpy.distutils, hence:

Running from numpy source directory.Warning: distutils distribution has been
initialized, it may be too late to add a subpackage commandWarning: distutils
distribution has been initialized, it may be too late to add a subpackage

That warning comes from distutils/misc_util.py.

_trac user ijstokes wrote on 2011-06-08_

I just had this same error and for me it seems it is caused by some dependency problems. If I re-try the easy_install with the other modules, they work the second time 'round. My best guess is that some dependency is only "visible" after the easy_install session ends.

Ian

Is it possible to get this fixed at some point? This causes workarounds where there should be non IHMO. I'd really like to get to a point where numpy can be specified in the install_requires option of a setup.py.

It's possible of course, if someone invests time to really understand the issue and submits a clean fix. It's not a prio for me personally - install_requires should be avoided imho (at least until pip and easy_install stop upgrading dependencies when upgrading a package at least).

Can you confirm that the same problem still occurs with modern distutils/setuptools/pip?

It does.

Sent from my iPad

On 29 Nov 2013, at 7:22, njsmith [email protected] wrote:

Can you confirm that the same problem still occurs with modern
distutils/setuptools/pip?


Reply to this email directly or view it on
GitHubhttps://github.com/numpy/numpy/issues/2434#issuecomment-29485806
.

Just ran across this issue so can confirm it's there. The option install_requires should be preferred over having to write a custom installation script when it's not really needed.

OS: Mac 10.8.3

Hitting this issue to. Not a show stopped, but definitely annoying. Especially because it's not in any way obvious that it's a bug rather than some user/env issue.

I've just tried reproducing this by installing pandas (which uses install_requires) in a 2.7 virtualenv created like:

$ virtualenv --no-site-packages nonumpy
$ source nonumpy/bin/activate
$ cd $PANDAS_DIR
$ pip install cython # needed for pandas
$ python setup.py install

If someone can give an easy way to reproduce this, that would help.

I also have this issue in a 2.7 virtual env. os ubuntu wheezy

@zethraeus could you provide a set of commands to reproduce this (like in my comment above)? It's hard to fix if we can't reproduce the issue.

Hi! Sorry it took so long, but here's a reproducer.

See: https://gist.github.com/kejbaly2/71517b08536776399198

Thanks for the example @kejbaly2, I can reproduce it now.

Having a very similar issue with trying to use install_requires and numpy

Here is how I can reproduce the error on Red Hat 6.5
https://gist.github.com/necrolyte2/530e9b39dcb0259b69d4

As an FYI for anyone experiencing this issue, with the latest version of setuptools, a workaround is to add NumPy to setup_requires. I still kept it in install_requires, so my setup.py looks like:

setup(
...
    setup_requires=["numpy"],
    install_requires=["numpy"],
)

This works, and only installs NumPy once, on both Mac OS X and Linux (not tested on Windows).

As for why I have no clue.

I've been trying to track down a similar issue that can occur sometimes when Numpy's build is being run by way of setup_requires=['Numpy'] in Astropy, when Astropy itself is being installed as a setup_requires of another pacakge (I call this 'setupception').

I've tracked it down to this ancient (but in principle still relevant, at least slightly?) bit of code: https://github.com/numpy/numpy/blob/9fc077332b34a7b2a90126f5ae6cc50d57162db5/numpy/distutils/core.py#L92

In most cases when running Numpy's setup.py, it shoves a function called configuration into the kwargs for numpy.distutils.core.setup. numpy.distutils.core.setup in turn pops the 'configuration' key off the setup kwargs, then sets up some "magic" global variables in distutils.core, and then calls _itself_ recursively. The purpose of this recursive call is to use the distutils machinery to check the command-line arguments for errors/--help.

It then calls the configuration function which kicks off the process of configuring all the Numpy subpackages, before once again calling back into the distutils setup() function with all the packages now fully configured.

Where this all runs afoul is that the Configuration manager occasionally needs to call the get_distribution utility in order to attach stuff to the Distribution object. The need for this is debatable. In fact, while there are some parts of the code in Configuration that think it should be able to do this, there are other parts that think that if there is already a Distribution instance attached to distutils.core._setup_distribution then something is probably wrong (hence the "distutils distribution has been initialized, it may be too late to add a subpackage" warnings that some people have reported in this thread).

To back up a bit and hopefully clarify--when running the setup() function from distutils (or setuptools) it initializes a Distribution object from the setup() kwargs, and then assigns the resulting Distribution to the global variable distutils.core._setup_distribution. At no point does it actually clean up this variable because it assumes it is the _only_ setup that's going to run in the current Python interpreter. It was not designed for setupception. (pip gets around this and similar issues when installing multiple packages by installing each package from its own subprocess). However, in other cases, such as setup_requires, there can already be a Distribution instance from the root package hanging around in distutils.core._setup_distribution. So when Numpy's Configuration class tries to attach stuff to the Distribution object for _Numpy itself_ it can end up messing around with the wrong Distribution in some cases. This results in a mangled Numpy build since some bits are missing.

There are a number of possible workarounds to this, both from setuptools end and/or the Numpy end. I haven't decided what the best approach would be, but the whole thing is definitely a mess :)

For what it's worth, one possible workaround from the setuptools end would be for setuptools.sandbox.run_setup, the function that setuptools uses to achieve setupception, to save/restore all distutils.core global variables before/after running the setup script in question.

As it happens, I believe the current version of setuptools may already fix that by hiding away the original distutils.core (removing it from sys.modules) before calling the child setup.py script. That _should_ take care of it. But I haven't double-checked this.

Update: Confirmed, the issue I described above is worked around (somewhat by happy accident) in setuptools >= 10.0.

@embray that's a nice bit of detective work! (and it hurts my head a little:) )

You're right that easy_install - and hence get_distribution() I assume - is still relevant enough that we shouldn't be breaking it.

@rgommers The thing is, that workaround for easy_install was added a very long time ago, and I don't believe it to be entirely necessary anymore (not only for the fact that easy_install itself is effectively deprecated, though I know there are still a handful of good reasons people have to use it). I'll see if I can come up with a patch that addresses the original issue while not being a problem in other contexts.

@embray sounds good. If you get a patch ready and it doesn't get reviewed within ~2 weeks, please do kick me (gently)

setup_requires=["numpy"] didn't work for me. I got this error message:

AttributeError: 'module' object has no attribute 'get_include'.

But running "python setup.py install" a second time fixed the problem. See also http://stackoverflow.com/questions/21605927/why-doesnt-setup-requires-work-properly-for-numpy

I just ran into the issue reported by @asmaier.

For what it's worth, I never forgot my ideas for fixing this issue, but just have had to focus on other things right now. If anyone else wants to take a stab at it, I made some high-level recommendations for where they might go looking, a few comments up from here.

so, this issue is due to the fact that __builtins__.__NUMPY_SETUP__ = True is being set
and when you install a dependency which needs numpy as a dependency, the __NUMPY_SETUP__ is not being reset by setuptools.
it looks like you are trying to set __NUMPY_SETUP__ = False here
https://github.com/numpy/numpy/blob/055056a3108ab19883e50ac125ab9974823498c7/numpy/distutils/misc_util.py#L2195
but this code is never reached, because the return on the previous line...

actually, I believe the entire isbootstrapping function is misusing/abusing __builtins__ here, in a way that breaks when you have numpy as a dependency, and have a dependency which has numpy as a dependency.

@JensTimmerman I suspect this is also related to https://github.com/matplotlib/matplotlib/issues/6928,

Any suggestions of the proper way to fix this are welcome.

@rgommers Thoughts?

it looks like you are trying to set NUMPY_SETUP = False here
https://github.com/numpy/numpy/blob/055056a3108ab19883e50ac125ab9974823498c7/numpy/distutils/misc_util.py#L2195
but this code is never reached, because the return on the previous line...

That is_bootstrapping function isn't used anywhere it looks like, so is irrelevant.

actually, I believe the entire isbootstrapping function is misusing/abusing builtins here, in a way that breaks when you have numpy as a dependency, and have a dependency which has numpy as a dependency.

The only use of __builtins__ is in the main setup.py here.

It is ugly, but it's also hard to write that in a more robust way. My guess at the moment is that if there's an issue with this, it will only show up for setup_requires='numpy' under some circumstances and not with install_requires. The difference between them is that the former invokes setuptools because pip doesn't yet know how to handle it. In Scipy we do use setup_requires='numpy' though, and have never had an issue with this. So it may be that plus the funkiness in matplotlib's setup scripts.

Reading again through https://github.com/numpy/numpy/issues/2434#issuecomment-77898051, that seems to describe the cause of the problem well, I don't think it has anything to do with __NUMPY_SETUP__.

EDIT: or maybe it does in some cases, but it's hard to deal with that another way. Normally there's no issue here because pip manages to build/install stuff in a way that's well-isolated.

Shouldn't be too hard to prove this theory right or wrong, by usingdel builtins.__NUMPY_SETUP__ at the end of setup.py.

gave that a try in gh-7956

Just in case the gist in https://github.com/numpy/numpy/issues/2434#issuecomment-49156521 disappears at some point, here's a copy of how to reproduce this issue:

cd /tmp

cat > ./setup.py <<DELIM
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Author: "Chris Ward" <[email protected]>

from setuptools import setup

default_setup = dict(
    name='numpy_install_test',
    description='https://github.com/numpy/numpy/issues/2434',
    provides=['numpy_install_test'],
    install_requires=['pandas'],
    requires=['pandas'],
    license='GPLv3',
    author='Chris Ward',
    author_email='[email protected]',
)   

setup(**default_setup)

DELIM


virtualenv --no-site-packages ~/virtenvs/nonumpy
source ~/virtenvs/nonumpy/bin/activate
python setup.py install

having hard time installing pandas in python2.7.16 using pip install -R requirements.txt

https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz (11.8MB)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-WznqHc/pandas/setup.py", line 746, in
setuptools_kwargs)
File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup
_install_setup_requires(attrs)
File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 724, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
replace_conflicting=replace_conflicting
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
return installer(requirement)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 791, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 705, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
run_setup(setup_script, args)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
raise
File "/usr/local/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
**File "/tmp/easy_install-2M8B7z/numpy-1.17.0rc1/setup.py", line 31, in

**RuntimeError: Python version >= 3.5 required.**

Not sure why it's look for numpy-1.17.0rc1 which is not support for python2

Any help would be appreciated, thanks

Please share your requirements.txt

Requirement file contains:

flask==1.0.2
flask-jwt==0.3.2
flask-script==2.0.6
flask-scrypt==0.1.3.6
PyJWT<1.5.0
blinker==1.4
phonenumbers==8.9.12
flanker==0.9.0
bugsnag==3.4.3
pickledb==0.7.2
requests==2.19.1
python-dateutil==2.8.0
gunicorn==19.9.0
gevent==1.3.6
webargs==4.0.0
ply==3.10
bson==0.5.7
celery==4.2.1
redis==3.2.1
hiredis
numpy==1.16.4

Finally manage it to work keeping the numpy inside requirement file and later installing pandas

Run command in docker:

RUN pip install https://github.com/pydata/pandas/releases/download/v0.24.2/pandas-0.24.2.tar.gz

However, now there is another issue in compiling it's say you need to install cython to compile pandas.

Error:

Exception: Cython-generated file 'pandas/io/sas/sas.c' not found.
Cython is required to compile pandas from a development branch.
Please install Cython or download a release package of pandas.

Any idea?

Not connected to this issue which concerns using install_requires inside setup.py. Please open a new issue. Do you really need to install pandas from source? Why not pip install pandas==0.24.2 ?

Yes I will open this issue in pandas library. However, the post that I mentioned earlier thats related to numPy since you can not install pandas directly numPy is properly installed on Python2.7 under docker env.
Why not pip install pandas==0.24.2 ? Because when I run it under docker it says.

Exception: Cython-generated file 'pandas/msgpack.c' not found.
Cython is required to compile pandas from a development branch.
Please install Cython or download a release package of pandas.

So I need to use release version of pandas to be installed from tarball

Closing. I think the python world is moving away from setup_requires and more towards requirements.txt or pyproject.toml. The work-around is below for those who still want to use setup_requires.

setup(
...
    setup_requires=["numpy"],
    install_requires=["numpy"],
)

As for @omeryounus trying to install numpy on python2, that is a separate issue not connected to this one. Some users have reported that upgrading pip and setuptools to the most recent version helped with that problem.

Please reopen if there is more to discuss around setup_requires.

Was this page helpful?
0 / 5 - 0 ratings