Requests: https:// does not work with Python 3.6

This sounds like this is a bug in the standard library, actually: it seems like they introduced an infinite recursion when setting options directly. @tiran?

I can't reproduce the issue on my machine (Python from latest hg 3.6 branch)

$ ./python 
Python 3.6.0+ (3.6:c4f39b6f3176, Dec  7 2016, 21:55:50) 
[GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
>>> ctx.options |= 5
>>> class MySSLContext(ssl.SSLContext):
...     pass
... 
>>> ctx = MySSLContext(ssl.PROTOCOL_TLS)
>>> ctx.options |= 5

I just had a try to reproduce it from the requests end; I can’t hit it on 3.6.0b4 or 3.6.0rc1:

$ python
Python 3.6.0b4 (v3.6.0b4:18496abdb3d5, Nov 21 2016, 20:44:47)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('https://github.com/kennethreitz/requests/issues/3752')
<Response [200]>

$ python
Python 3.6.0rc1 (v3.6.0rc1:29a273eee9a5, Dec  6 2016, 16:24:13)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('https://github.com/kennethreitz/requests/issues/3752')
<Response [200]>

Could it be something that’s platform or URL-specific?

(OS X 10.12.1, 16B2659, for my part)

Requests works with Python 3.6.0 from hg, too:

$ ./python -m venv venv
$ venv/bin/pip install requests
Collecting requests
  Using cached requests-2.12.3-py2.py3-none-any.whl
Installing collected packages: requests
Successfully installed requests-2.12.3
$ venv/bin/python
Python 3.6.0+ (3.6:c4f39b6f3176, Dec  7 2016, 21:55:50) 
[GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('https://www.python.org')
<Response [200]>

Ok, so right now this looks like a "can't repro" situation. Might be worth looking at your install @juokaz, seems to be misbehaving a bit.

After further debugging, I was able to reproduce it as a gevent issue (which was the context I was using this in).

I'm guessing I'm going to have to take it to gevent folks to have a look? Thanks for looking into this, everyone.

root@8042e3f57981:/app# python
Python 3.6.0b4 (default, Nov 23 2016, 21:34:29)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gevent.monkey
>>> gevent.monkey.patch_all()
>>>
>>> from requests.packages.urllib3.util.ssl_ import create_urllib3_context
>>> create_urllib3_context()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/requests/packages/urllib3/util/ssl_.py", line 268, in create_urllib3_context
    context.options |= options
  File "/usr/local/lib/python3.6/ssl.py", line 459, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/local/lib/python3.6/ssl.py", line 459, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/local/lib/python3.6/ssl.py", line 459, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 329 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object
>>>

Yup, this is a gevent issue. I'd assume gevent doesn't yet work with Python 3.6.

The super() call looks weird but it's correct. It's the generic way to fetch the attribute descriptor of the parent class. It's a necessary evil to change a property in a subclass. Ask Google for "David Beazley subclass property" and it will show you the same, https://www.safaribooksonline.com/library/view/python-cookbook-3rd/9781449357337/ch08s08.html

>>> from ssl import SSLContext
>>> super(SSLContext, SSLContext).options
<attribute 'options' of '_ssl._SSLContext' objects>
>>> import _ssl
>>> super(SSLContext, SSLContext).options is _ssl._SSLContext.options
True

The call super(SSLContext, SSLContext).options.__set__(self, value) is the same as setting options on an instance of the parent class.

Here's the reference to the gevent ticket for the same issue: https://github.com/gevent/gevent/issues/903
Gevent has now been bumped to >1.2 by default. If you're still having this issue, it can be fixed with a simple pip install --upgrade gevent.

@pirate that doesn't fix the issue for me. Any other ideas? The other thread is locked.

@AeroNotix please don't use this issue or this project's tracker to discuss gevent problems.

@pirate can you unlock that ticket, then? I'm still having the same issue on python 3.6 with upgraded versions of everything.

Nope @AeroNotix , I'm not a contributor on either project, perhaps you can email/irc one of the people in the other thread directly?

@pirata This issue still exists with python 3.6.1, gevent 1.2.1, requests 2.13.0. Steps to reproduce:

import requests
import gevent.monkey
gevent.monkey.patch_ssl()
requests.get("https://google.com")

If I import requests after the patch, everything works. It used to be the case that it won't matter and I'm wondering if something can be changed to keep the old behavior. What do you think @Lukasa ?

Requests has some problems with gevent and monkey patching because it performs feature detection on the select module at import time. A future release will bring it up to date with a newer urllib3 that contains fixes for this problem, but in the meantime gevent's monkey patch should be applied before importing requests.

@Lukasa are you referring to this release?