Requests: urllib3 v1.25 support

Created on 23 Apr 2019 · 15Comments · Source: psf/requests

Support for urllib3 v1.25 (released today/yesterday). The usual yada-yada "my CI is broken".

Multiple packages that we depend on (eg google.cloud.storage, azure.storage) have strict version-check on requests using pkg_resources.require('requests >= 2.18.0'). Any unmet dependencies in the chain below and including requests will cause the check to fail and raise an exception. urllib3<1.25 is currently one such dependency.

Expected Result

No exception raised on import

Actual Result

pkg_resources.ContextualVersionConflict: (urllib3 1.25 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('urllib3<1.25,>=1.21.1'), {'requests'})

The above exception was the direct cause of the following exception:
ImportError: ``requests >= 2.18.0`` is required by the ``google.resumable_media.requests`` subpackage.
It can be installed via
    pip install google-resumable-media[requests].

Reproduction Steps

pip3 install sentry-sdk google-cloud-storage

>>> from google.cloud import storage



md5-cbc1943a1d569f335059802806d81765



/home/laurie/env-tmp/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.7"
  },
  "platform": {
    "release": "4.18.0-17-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "1010100f"
  },
  "urllib3": {
    "version": "1.25"
  },
  "using_pyopenssl": false
}

Source

EpicWink

👍20

Most helpful comment

Here's a possibly dumb question: why does requests restrict urllib3 to be < 1.25? Since many other pip dependencies are going to continue to increase their minimum allowed urllib3 version, it seems like this is going to guarantee two things:

Every minor version bump in urllib3 will break many projects which depend on requests (e.g. anyone who is using the major Google or Microsoft Azure packages, among many others).
Every minor version bump in urllib3 will then require a new release of requests to allow the new version of urllib3.

Does urllib3 make breaking changes that often that we're worried about just trusting their versioning? Or is there another concern?

plowman on 9 May 2019

👍13

All 15 comments

Note that a pull-request fixing this should close #5065 and #4961

EpicWink on 23 Apr 2019

Also note that a solution to the above reproduction example is:

pip3 install sentry-sdk google-cloud-storage 'urllib3<1.25'

EpicWink on 23 Apr 2019

There’s already a PR in the works: https://github.com/kennethreitz/requests/pull/5063

Freso on 24 Apr 2019

👍6

To those watching, as mentioned, we're blocked on urllib3 v1.25.2

EpicWink on 29 Apr 2019

@nateprewitt FYI 1.25.2 will be released very soon, I would recommend in addition to disallowing 1.25.0 also disallow 1.25.1?

sethmlarson on 29 Apr 2019

Our CI pipeline starts to get an error [integration_py3_docker_metric] 19-04-29:10:26:57 ERROR [clipper_metric_docker.py:127] Failed to parse: http://localhost:44328/api/v1/series?match[]=clipper_mc_parse_time_ms_sum from April 22nd, from when urllib3 version 1.25 was released. Do you think this is related to this issue?

rkooo567 on 29 Apr 2019

@rkooo567, have you tried running the CI with the additional requirement urllib3<1.25?

EpicWink on 30 Apr 2019

@EpicWink Yes. And it seems like it resolves an issue. I also found out urllib3==1.25.2 also resolves the problem.

rkooo567 on 30 Apr 2019

urllib3 1.24.3 has been released which fixes only the CRLF injection issue for 1.24.X users.

sethmlarson on 2 May 2019

👍5 🎉4

Every minor version bump in urllib3 will break many projects which depend on requests (e.g. anyone who is using the major Google or Microsoft Azure packages, among many others).
Every minor version bump in urllib3 will then require a new release of requests to allow the new version of urllib3.

Does urllib3 make breaking changes that often that we're worried about just trusting their versioning? Or is there another concern?

plowman on 9 May 2019

👍13

@plowman https://github.com/kennethreitz/requests/issues/5080 Looks like the update to version 1.25 would make requests start up much slower.

yjqiang on 10 May 2019

Based on the number of references to this issue in other projects’ trackers, I’m sure there are a lot of other project maintainers that would appreciate a resolution to this sooner rather than later… :) (Pointing it out since issue references don’t trigger e‐mail notifications.)

Freso on 15 May 2019

👍10

+1 because it aggravates problems caused by Pip's lack of a dependency resolver in some cases: https://github.com/pradyunsg/zazo/issues/14