Pipenv: incorrect dependencies (versions) locked by pip

Created on 13 Aug 2018  ·  3Comments  ·  Source: pypa/pipenv

tried to add flower to the project, but pipenv lock is picking newest tornado version (5.1) instead one specified in requirements (tornado>=4.2.0,<5.0.0) (tested on 2.7.15 & 3.7)

Expected result

proper tornado version to be picked

Steps to replicate

$ mkdir tmp1
$ cd tmp1
$ pipenv install flower

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/Users/jerzyk/.pyenv/versions/3.7.0/lib/python3.7/site-packages/pipenv'

Python location: '/Users/jerzyk/.pyenv/versions/3.7.0/bin/python3.7'

Other Python installations in PATH:

  • 2.7: /usr/local/bin/python2.7
  • 2.7: /Users/jerzyk/.pyenv/shims/python2.7
  • 2.7: /usr/local/bin/python2.7
  • 2.7: /usr/local/bin/python2.7
  • 2.7: /usr/bin/python2.7
  • 3.6: /Users/jerzyk/.pyenv/shims/python3.6
  • 3.7: /Users/jerzyk/.pyenv/versions/3.7.0/bin/python3.7m
  • 3.7: /Users/jerzyk/.pyenv/versions/3.7.0/bin/python3.7
  • 3.7: /Users/jerzyk/.pyenv/shims/python3.7
  • 3.7: /usr/local/bin/python3.7

  • 3.7: /usr/local/bin/python3.7

  • 3.7.0: /Users/jerzyk/.pyenv/versions/3.7.0/bin/python

  • 3.7.0: /Users/jerzyk/.pyenv/shims/python
  • 2.7.15: /usr/local/bin/python
  • 2.7.15: /usr/local/bin/python
  • 2.7.10: /usr/bin/python
  • None: /Users/jerzyk/.pyenv/shims/python2
  • 2.7.15: /usr/local/bin/python2
  • 2.7.15: /usr/local/bin/python2
  • 3.7.0: /Users/jerzyk/.pyenv/versions/3.7.0/bin/python3
  • 3.7.0: /Users/jerzyk/.pyenv/shims/python3
  • 3.7.0: /usr/local/bin/python3
  • 3.7.0: /usr/local/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT '
                     '2018; root:xnu-4570.71.2~1/RELEASE_X86_64',
 'python_full_version': '3.7.0',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

  • TERM_PROGRAM
  • PYENV_ROOT
  • TERM
  • SHELL
  • TMPDIR
  • Apple_PubSub_Socket_Render
  • TERM_PROGRAM_VERSION
  • TERM_SESSION_ID
  • PYENV_VERSION
  • USER
  • SSH_AUTH_SOCK
  • PYENV_DIR
  • __CF_USER_TEXT_ENCODING
  • PATH
  • PWD
  • LANG
  • PGHOST
  • ITERM_PROFILE
  • PYENV_HOOK_PATH
  • XPC_FLAGS
  • XPC_SERVICE_NAME
  • SHLVL
  • HOME
  • COLORFGBG
  • ITERM_SESSION_ID
  • LOGNAME
  • PGDATA
  • DISPLAY
  • COLORTERM
  • PYTHONDONTWRITEBYTECODE
  • PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /Users/jerzyk/.pyenv/versions/3.7.0/bin:/usr/local/Cellar/pyenv/1.2.6/libexec:/Users/jerzyk/.pyenv/shims:/usr/local/bin:/usr/local/sbin:/Applications/Xcode.app/Contents/Developer/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin
  • SHELL: /bin/bash
  • LANG: en_US.UTF-8
  • PWD: /Users/jerzyk/tmp

Contents of Pipfile ('/Users/jerzyk/tmp/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flower = "*"

[dev-packages]

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/Users/jerzyk/tmp/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "e224f38cf9f41ab210a6546986368cfee3a9087dc1c0d756024e2f6aea1f7e9a"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "amqp": {
            "hashes": [
                "sha256:073dd02fdd73041bffc913b767866015147b61f2a9bc104daef172fc1a0066eb",
                "sha256:eed41946890cd43e8dee44a316b85cf6fee5a1a34bb4a562b660a358eb529e1b"
            ],
            "version": "==2.3.2"
        },
        "babel": {
            "hashes": [
                "sha256:6778d85147d5d85345c14a26aada5e478ab04e39b078b0745ee6870c2b5cf669",
                "sha256:8cba50f48c529ca3fa18cf81fa9403be176d374ac4d60738b839122dfaaa3d23"
            ],
            "version": "==2.6.0"
        },
        "billiard": {
            "hashes": [
                "sha256:ed65448da5877b5558f19d2f7f11f8355ea76b3e63e1c0a6059f47cfae5f1c84"
            ],
            "version": "==3.5.0.4"
        },
        "celery": {
            "hashes": [
                "sha256:77dab4677e24dc654d42dfbdfed65fa760455b6bb563a0877ecc35f4cfcfc678",
                "sha256:ad7a7411772b80a4d6c64f2f7f723200e39fb66cf614a7fdfab76d345acc7b13"
            ],
            "version": "==4.2.1"
        },
        "flower": {
            "hashes": [
                "sha256:a7a828c2dbea7e9cff1c86d63626f0eeb047b1b1e9a0ee5daad30771fb51e6d0"
            ],
            "index": "pypi",
            "version": "==0.9.2"
        },
        "kombu": {
            "hashes": [
                "sha256:86adec6c60f63124e2082ea8481bbe4ebe04fde8ebed32c177c7f0cd2c1c9082",
                "sha256:b274db3a4eacc4789aeb24e1de3e460586db7c4fc8610f7adcc7a3a1709a60af"
            ],
            "version": "==4.2.1"
        },
        "pytz": {
            "hashes": [
                "sha256:a061aa0a9e06881eb8b3b2b43f05b9439d6583c206d0a6c340ff72a7b6669053",
                "sha256:ffb9ef1de172603304d9d2819af6f5ece76f2e85ec10692a524dd876e72bf277"
            ],
            "version": "==2018.5"
        },
        "tornado": {
            "hashes": [
                "sha256:1c0816fc32b7d31b98781bd8ebc7a9726d7dce67407dc353a2e66e697e138448",
                "sha256:4f66a2172cb947387193ca4c2c3e19131f1c70fa8be470ddbbd9317fd0801582",
                "sha256:5327ba1a6c694e0149e7d9126426b3704b1d9d520852a3e4aa9fc8fe989e4046",
                "sha256:6a7e8657618268bb007646b9eae7661d0b57f13efc94faa33cd2588eae5912c9",
                "sha256:a9b14804783a1d77c0bd6c66f7a9b1196cbddfbdf8bceb64683c5ae60bd1ec6f",
                "sha256:c58757e37c4a3172949c99099d4d5106e4d7b63aa0617f9bb24bfbff712c7866",
                "sha256:d8984742ce86c0855cccecd5c6f54a9f7532c983947cff06f3a0e2115b47f85c"
            ],
            "markers": "python_version != '3.2.*' and python_version != '3.1.*' and python_version != '3.3.*' and python_version != '3.0.*' and python_version >= '2.7'",
            "version": "==5.1"
        },
        "vine": {
            "hashes": [
                "sha256:52116d59bc45392af9fdd3b75ed98ae48a93e822cee21e5fda249105c59a7a72",
                "sha256:6849544be74ec3638e84d90bc1cf2e1e9224cc10d96cd4383ec3f69e9bce077b"
            ],
            "version": "==1.1.4"
        }
    },
    "develop": {}
}

picture jerzyk
👍3

Most helpful comment

```[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flower = "*"

[dev-packages]
tornado = "<5"

[requires]
python_version = "3.7"
```

This file reproduces the problem. pipenv lock seems to lock packages and dev-packages in different stages. When it finds out that flower depends on tornado>=4.2.0 is ignores the constraint defined in dev-packages and installs tornado 5. Running pipenv lock -d doesn't help either.

picture r-darwish on 14 Aug 2018
👍3

All 3 comments

```[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flower = "*"

[dev-packages]
tornado = "<5"

[requires]
python_version = "3.7"
```

This file reproduces the problem. pipenv lock seems to lock packages and dev-packages in different stages. When it finds out that flower depends on tornado>=4.2.0 is ignores the constraint defined in dev-packages and installs tornado 5. Running pipenv lock -d doesn't help either.

r-darwish picture r-darwish on 14 Aug 2018
👍3

As a workaround, I had to move all [dev-packages] requirements to [packages]. Or I guess I could've duplicated all [packages] entries in [dev-packages].

mizi picture mizi on 16 Aug 2018

Sorry for the delay getting back to you all, to answer at least one part of the inquiry, dev-packages take a back seat to packages to prevent developing accidentally in an environment that won't mirror whatever you deploy.

Since you have a top level package here that depends on tornado (flower), and a strict constraint on what version of tornado is OK to install, it is pretty likely you would want to pin the tornado entry in packages since it's being installed either way. If the version pin is conditional, you should consider adding an environment marker.

Roughly speaking, this Pipfile says you will be using tornado 5.x in production, but <5 in development, which is something we designed around (i.e. this was a design decision -- unless you use environment markers your local dependencies will match the ones in production no matter what).

Closing for now as this particular dependency resolution issue is a design decision, you'll have to move your requirement into the packages section and use environment markers

techalchemy picture techalchemy on 4 Sep 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

jacebrowning picture jacebrowning  ·  3Comments
FooBarQuaxx picture FooBarQuaxx  ·  3Comments
jacek-jablonski picture jacek-jablonski  ·  3Comments
hynek picture hynek  ·  3Comments
AkiraSama picture AkiraSama  ·  3Comments