requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:645)

Then, some of my own scripts crashed which was normal.

What does a crash like that look like?

After that, when I try to run my script again, I got following exceptions

Are you completely unable to run them at all after that?

@sigmavirus24 It's just some bug in my own code without anything to do with networking.
I'm not sure whether this is the real cause or just coincidence.

But one thing is for sure, after some point, I'm completely unable to make request to https://www.telegram.org, which I can do right after install request.

Just FYI: #2906

So I'd like to point out that you're installing requests[security] which means we should be using pyOpenSSL but your stacktrace shows that we aren't. That's intriguing.

@sigmavirus24 So this there anything I can do to help you?

Any idea? @Lukasa

So what matters most here is: why does your code stop using PyOpenSSL? When you encounter your crash, can you open a python console in your virtual environment and then run import urllib3.contrib.pyopenssl, to see if that works?

Couldn't reproduce it now. I will close this first, and if I encounter the problem again, I will paste the result and reopen it.

@Lukasa I think since requests is shipped with its own urllib3, I could not import urllib3 alone. And the following result confirms it.

Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib3.contrib.pyopenssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'urllib3'
>>> ^D




pip list
cffi (1.5.0)
cryptography (1.2.2)
idna (2.0)
ndg-httpsclient (0.4.0)
pip (8.0.2)
pyasn1 (0.1.9)
pycparser (2.14)
pyOpenSSL (0.15.1)
requests (2.9.1)
setuptools (20.0)
six (1.10.0)
wheel (0.26.0)

I'm sorry, try importing requests.packages.urllib3.contrib.pyopenssl

@Lukasa

Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests.packages.urllib3.contrib.pyopenssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "VirtualenvPath/lib/python3.5/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 57, in <module>
    from socket import _fileobject, timeout, error as SocketError
ImportError: cannot import name '_fileobject'
>>> 

I bet that is a bug on 3.5 with pyOpenSSL that we didn't know about @lukasa (with respect to _fileobject not existing.

Nope, I know about it, and have proposed a fix upstream in urllib3 when we started testing PyOpenSSL. The reality is that requests[security] does not (and has never) worked on Python 3. That'll be fixed in an upcoming version of Requests.

So that doesn't solve our puzzle: why did this work for a bit and then stop?

tangent comment based on original report, the built in ssl.py in python has an option to suppress ragged EOFs as there are plenty of sites that will uncleanly shutdown SSL connections. sometimes unreliably as if it were a matter of timing or coincidence. the following is an extract from ssl.py

class SSLSocket(socket):
    [...]
    def read(self, len=0, buffer=None):
        """Read up to LEN bytes and return them.
        Return zero-length string on EOF."""

        self._checkClosed()
        if not self._sslobj:
            raise ValueError("Read on closed or unwrapped SSL socket.")
        try:
            return self._sslobj.read(len, buffer)
        except SSLError as x:
            if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
                if buffer is not None:
                    return 0
                else:
                    return b''
            else:
                raise

https://github.com/shazow/urllib3/pull/795
https://github.com/shazow/urllib3/issues/791

Having the same exception on Python 2.7.11/OSX when pounding a server with requests under gevent. Could this be related?

@the-efi could you be more specific about which exception you're seeing?

    r = requests.post(self.MY_URL, data=parameters)
  File "/Users/me/Envs/my_env/lib/python2.7/site-packages/requests/api.py", line 109, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/Users/me/Envs/my_env/lib/python2.7/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/Users/me/Envs/my_env/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/me/Envs/my_env/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/Users/me/Envs/my_env/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
SSLError: EOF occurred in violation of protocol (_ssl.c:590)

@the-efi Do you have pyopenssl, pyasn1, and ndg-httpsclient installed?

pyopenssl: negative
pyasn1: pyasn1==0.1.8 (transient, I guess)
ndg-httpsclient: negative

Ok. You're also running Python 2.7, so you and the poster seem to be having different problems.

Do you know if you hit your problem during connection setup, or on a long-running connection?

I would imagine this is during connection setup, but if requests reuses previously opened connections with Connection: keep-alive I might be wrong.

Requests does indeed re-use previously opened connections where possible, which is why I asked the question. ;)

It would be very useful if we could get a packet capture of this problem in your case, though that may be tricky given that it occurs under heavy load.

No problem, I will see what I can do about it after the weekend. Would you like me to submit that as a new issue?

Yes please. =)

@Lukasa Any progress with urllib3?

@caizixian We're getting there, but we have some problems with our CI testing because Travis CI has a fairly old PyPy image that doesn't behave well with PyOpenSSL at the moment. I'll see if I can get this to work sometime this weekend.

@Lukasa @shazow urllib3 always has a home at http://ci.kennethreitz.org, if desired!

Any update on this one? It happens quite a lot when working with aws.

@mindw As best as I know we don't have a good understanding of exactly where and when it's happening. In the backtrace above it's happening during the handshake, which is usually a problem with negotitation: when are you encountering this error?

OS X, Python 3.5.1, requests 2.10.0 + security.
I will try and provide any additional info upom request :)

Traceback (most recent call last):
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 351, in _make_request
    self._validate_conn(conn)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 814, in _validate_conn
    conn.connect()
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 289, in connect
    ssl_version=resolved_ssl_version)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/packages/urllib3/util/ssl_.py", line 308, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 376, in wrap_socket
    _context=self)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 747, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 983, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 628, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/adapters.py", line 403, in send
    timeout=timeout
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 604, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/gabdav01/.virtualenvs/splatt/bin/splatt", line 9, in <module>
    load_entry_point('splatt', 'console_scripts', 'splatt')()
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/gabdav01/work/paas/splatt/splatt/cmd.py", line 108, in create
    url, json=json, headers=headers)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/sessions.py", line 518, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/Users/gabdav01/.virtualenvs/splatt/lib/python3.5/site-packages/requests/adapters.py", line 477, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:645)

This strongly suggests a lack of SNI or no cipher overlap. Can you print the cryptography command line in the OP and tell me what it says?

how would I do that?

@mindw Look at OP (the original post) where @caizixian imports the backend from cryptography and prints the OpenSSL version.

Also getting this issue. What information can I provide to help resolve?

@LukeNZ Lots of things. Python version, installed packages, requests version, OpenSSL version, operating system, and the website you're trying to contact.

Hi, I just started using Requests and I am getting this same thing too. I am on OS X 10.11.5, Python 3.5.1 in a virtual env, newest version of requests. Running the open_ssl version as in the OP gives me

OpenSSL 1.0.2h 3 May 2016

I am trying to connect to https://api.marketcircle.net/v1/ .

It looks as if Requests is the only game in town without writing a bunch of code - Unirest no longer installs for Python 3, so boy, I hope this gets figured out soon..

@eoco Ok, that's interesting. I can't reproduce this on OS X 10.11.6 Beta, Python 3.5.1 with Requests 2.10.0. Do you encounter the issue consistently or intermittently?

Consistently - I haven't been able to connect successfully yet.

My tracebacks are pretty close to mindw's above, will post at the bottom here. But just as FYI, I installed requests for Python 2.7 to test that as well and did get an SNIMissingWarning: Though the same warning said it may have to do with it being an older version of Python, but I am not sure if that is the case. Regardless, no connection luck there either.

SNIMissingWarning /Library/Python/2.7/site-packages/requests-2.10.0-py2.7.egg/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

I also went back and installed Unirest in Python 2.7 and get another EOF error, but this time in urllib2:

urllib2.URLError: <urlopen error [Errno 8] _ssl.c:510: EOF occurred in violation of protocol>

So- this really may not have anything to do with Requests.py at all, but to be honest, I don't know where to go from here, other than not using Python. Here is the traceback from requests/Python 3.5.1:
`
import requests
url = 'https://api.marketcircle.net/v1/'
r = requests.get(url) # not adding any authorization headers, just checking for connection

Traceback (most recent call last):
File "/Volumes/Harvey_740/Flask_Daylite_02/myvenv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 578, in urlopen
chunked=chunked)
File "/Volumes/Harvey_740/Flask_Daylite_02/myvenv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 351, in _make_request
self._validate_conn(conn)
File "/Volumes/Harvey_740/Flask_Daylite_02/myvenv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 814, in _validate_conn
conn.connect()
File "/Volumes/Harvey_740/Flask_Daylite_02/myvenv/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 289, in connect
ssl_version=resolved_ssl_version)
File "/Volumes/Harvey_740/Flask_Daylite_02/myvenv/lib/python3.5/site-packages/requests/packages/urllib3/util/ssl_.py", line 308, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 376, in wrap_socket
_context=self)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 747, in __init__
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 983, in do_handshake
self._sslobj.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 628, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Volumes/Harvey_740/Flask_Daylite_02/myvenv/lib/python3.5/site-packages/requests/adapters.py", line 403, in send
timeout=timeout
File "/Volumes/Harvey_740/Flask_Daylite_02/myvenv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 604, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:645)
`
So, my apologies if this is not helpful. Thank you!

Can you please run this and show me the output? python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"

So this is the result: (Different version than the command run in OP)

OpenSSL 0.9.8zh 14 Jan 2016

Thank you!

@eoco Ok, cool.

So the problem here is that your Python 3 is linked against the system OpenSSL, which is ancient. What's in your pip environment? Can you run python3 -m pip freeze for me?

Not too much there..

Mozaica-iMac:~ eoc$ python3 -m pip freeze

py3minepi==0.0.1
requests==2.10.0
You are using pip version 8.1.1, however version 8.1.2 is available.

Thank you for going above and beyond on this. It is obviously not an issue with Requests so I really appreciate your time.

Ok, so the easiest way for you to fix this up is to run python3 -m pip install pyopenssl pyasn1 ndg-httpsclient. That should fix your problem, but if it doesn't we can pursue more aggressive options.

Hmm. Still the same error afterwards-

python3.5 -m pip freeze

cffi==1.6.0
cryptography==1.4
idna==2.1
ndg-httpsclient==0.4.1
py3minepi==0.0.1
pyasn1==0.1.9
pycparser==2.14
pyOpenSSL==16.0.0
requests==2.10.0
six==1.10.0

@eoco Hrm. Can you run this for me? python3 -c "from requests.packages.urllib3.contrib import pyopenssl; pyopenssl.inject_into_urllib3()" and tell me what that outputs? Also run python3 -c "from cryptography.hazmat.backends.openssl.backend import backend; print(backend.openssl_version_text())".

Whoa, that certainly points to the problem. Upon first running I got the following traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 63, in <module>
    from urllib3.packages.backports.makefile import backport_makefile
ImportError: No module named 'urllib3'

Isn't urllib part of the standard library? In any case, I pip installed urllib and now I am up and running.

I can't tell you how much I appreciate your help with this, you went way above and beyond. I wonder if there is an issue with the Python 3.5 download for Mac on python.org? This was pretty much a clean install for this project. In any case, maybe this will help someone else if they run into something like this.

Again, thank you so much!

Ah, this looks like a proper requests bug. Urllib3 has a non-relative import in its codebase that breaks when we vendorize it! We should aim to fix that.

@eoco So while we're running this fix through the pipeline (see shazow/urllib3#901), your easiest fix will be to install Python 3 using something like Homebrew. This will link your Python 3 against a much newer OpenSSL, which will solve this problem for you both in requests and in any other application that uses the system TLS bindings.

Got it. Just installing urllib3 seems to have solved my issue, but I may just do that. Apologies for any confusions this may have caused.

Thank you!

I came across the same bug AFTER switching to aws and putting the app behind an elastic loadbalancer, never occured before.

I have urllib3 installed as well.

Is there hope for installing pyopenssl? I tried and there were lot's of c-level errors (which may be cause of some missing parts in my docker image).

[2016-06-29 02:27:59,932: CRITICAL/MainProcess] Task ventures.tasks.cache_warmup[ae2c97eb-3b28-4896-8d0f-42f712115707] INTERNAL ERROR: SSLError(SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:645)'),),)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 253, in trace_task
    I, R, state, retval = on_error(task_request, exc, uuid)
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 201, in on_error
    R = I.handle_error_state(task, eager=eager)
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 85, in handle_error_state
    }[self.state](task, store_errors=store_errors)
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 125, in handle_failure
    einfo=einfo)
  File "/usr/local/lib/python3.5/dist-packages/celery/utils/dispatch/signal.py", line 166, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/srv/core/celery.py", line 31, in notify_task_failure
    """.format(sender, exception, args, kwargs))
  File "/usr/lib/python3.5/logging/__init__.py", line 1308, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/usr/lib/python3.5/logging/__init__.py", line 1415, in _log
    self.handle(record)
  File "/usr/lib/python3.5/logging/__init__.py", line 1425, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.5/logging/__init__.py", line 855, in handle
    self.emit(record)
  File "/srv/core/management/logger/slack.py", line 28, in emit
    notify_slack(message, channel)
  File "/usr/local/lib/python3.5/dist-packages/celery/local.py", line 188, in __call__
    return self._get_current_object()(*a, **kw)
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 439, in __protected_call__
    return orig(self, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/celery/app/task.py", line 420, in __call__
    return self.run(*args, **kwargs)
  File "/srv/core/tasks.py", line 66, in notify_slack
    data=json.dumps(payload), headers={'Content-Type': 'application/json'}
  File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 107, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/adapters.py", line 447, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:645)

pip freeze:

amqp==1.4.9
analytics-python==1.1.0
anyjson==0.3.3
beautifulsoup4==4.4.1
billiard==3.3.0.23
bleach==1.4.2
boto==2.39.0
celery==3.1.23
Django==1.8.13
django-appconf==1.0.2
django-compressor==2.0
django-countries==3.4.1
django-filter==0.13.0
django-haystack==2.5.dev0
django-htmlmin==0.9.0
django-modelcluster==1.1
django-overextends==0.4.1
django-redis==4.3.0
django-rest-swagger==0.3.5
django-storages-redux==1.3.2
django-taggit==0.18.3
django-treebeard==4.0.1
djangorestframework==3.3.3
docopt==0.4.0
docutils==0.12
drf-extensions==0.2.8
dropbox==4.0
elasticsearch==1.8.0
et-xmlfile==1.0.1
geopy==1.11.0
google-api-python-client==1.5.0
gunicorn==19.4.5
hiredis==0.2.0
html5lib==0.9999999
httplib2==0.9.2
jdcal==1.2
jsonfield==1.0.3
kombu==3.0.35
mandrill==1.0.57
Markdown==2.6.5
mock==1.0.1
numpy==1.11.0
oauth2client==2.2.0
openpyxl==2.3.4
Pillow==3.1.1
psycopg2==2.6.1
pyasn1==0.1.9
pyasn1-modules==0.0.8
pycurl==7.43.0
pygobject==3.20.0
PyMySQL==0.7.2
python-apt==1.1.0b1
python-dateutil==2.5.3
pytz==2016.4
PyYAML==3.11
rcssmin==1.0.6
redis==2.10.5
requests==2.9.1
rjsmin==1.0.12
rsa==3.4.2
simplejson==3.8.2
six==1.10.0
sphinx-me==0.3
unattended-upgrades==0.1
Unidecode==0.4.19
uritemplate==0.6
urllib3==1.16
wagtail==1.4.5
Willow==0.3.1

python3 -c "from requests.packages.urllib3.contrib import pyopenssl; pyopenssl.inject_into_urllib3()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 49, in <module>
    from ndg.httpsclient.ssl_peer_verification import SUBJ_ALT_NAME_SUPPORT
ImportError: No module named 'ndg'

@shredding Try installing pyopenssl, ndg-httpsclient, and pyasn1. You need all three at this time.

Okay, i'll give it a shot (I can only reproduce that on aws and report back).

For those who run python3 within an ubuntu:latest package on docker, you need to add

RUN apt-get install build-essentials python-dev libffi

before adding

pyOpenSSL==16.0.0
ndg-httpsclient==0.4.1
pyasn1==0.1.9

... to your requirements.txt

Got the same issue here.
Fixed by installing pyopenssl, ndg-httpsclient, pyasn1 and urllib3 ofcourse.

I din't occur again on my side as well.

I was trying to connect to twitter API so at first I was getting the below error
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:645)

I used monkey patch and but letter money patch didn't work, I installed pyOpenSSL ,ndg-httpsclient, pyasn1 and somehow this worked.
But after this I got this
Traceback (most recent call last): File "twitter_friend.py", line 32, in <module> stream.statuses.filter(follow='sajjkum') File "C:\Python27\lib\site-packages\twython\streaming\types.py", line 66, in filter self.streamer._request(url, 'POST', params=params) File "C:\Python27\lib\site-packages\twython\streaming\api.py", line 139, in _request response = _send(retry_counter) File "C:\Python27\lib\site-packages\twython\streaming\api.py", line 123, in _send response = func(url, **requests_args) File "C:\Python27\lib\site-packages\requests\sessions.py", line 511, in post return self.request('POST', url, data=data, json=json, **kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request resp = self.send(prep, **send_kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send r = adapter.send(request, **kwargs) File "C:\Python27\lib\site-packages\requests\adapters.py", line 426, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

and this
Traceback (most recent call last): File "twitter_friend.py", line 32, in <module> stream.statuses.filter(follow='sajjkum') File "C:\Python27\lib\site-packages\twython\streaming\types.py", line 66, in filter self.streamer._request(url, 'POST', params=params) File "C:\Python27\lib\site-packages\twython\streaming\api.py", line 139, in _request response = _send(retry_counter) File "C:\Python27\lib\site-packages\twython\streaming\api.py", line 123, in _send response = func(url, **requests_args) File "C:\Python27\lib\site-packages\requests\sessions.py", line 511, in post return self.request('POST', url, data=data, json=json, **kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request resp = self.send(prep, **send_kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send r = adapter.send(request, **kwargs) File "C:\Python27\lib\site-packages\requests\adapters.py", line 447, in send raise SSLError(e, request=request) requests.exceptions.SSLError: bad handshake: SysCallError(-1, 'Unexpected EOF')

I am using python 2.7 , urllib3(1.4), requests(2.9.1).

Both of these errors suggest that the remote server is not accepting your request. However, it is not really possible for us to diagnose that error. I recommend you upgrade your OpenSSL and try again.

pip install --force-reinstall requests[security] solved it for me.

I am getting the same exception after trying every solution mentioned above. Error trace:
File "/home/prateek/.local/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, *send_kwargs)
File "/home/prateek/.local/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, *kwargs)
File "/home/prateek/.local/lib/python2.7/site-packages/requests/adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

System Package Information:
cffi==1.9.1
cryptography==1.6
idna==2.1
ndg-httpsclient==0.4.2
pkg-resources==0.0.0
pyasn1==0.1.9
pycparser==2.17
pyOpenSSL==16.2.0
requests==2.12.3
six==1.10.0

Python: Python 3.5.2 (default, Jul 5 2016, 12:43:10)
OS: Ubuntu 16.04
OpenSSL 1.0.2g 1 Mar 2016

Appreciate your help in this matter.

This almost certainly means the ciphers being offered to the remote peer are not acceptable. If you use openssl s_client -connect <host>:<port> to the website you're trying to contact, can you tell me what the output is? (Please make sure to do so with the OpenSSL 1.0.2g you're using).

@Lukasa Thanks for your quick response. The output as you asked is as follows:

Also I am adding the response from OpenSSL 1.0.1f 6 Jan 2014
It appears to be working on this version.

@> openssl version
OpenSSL 1.0.2g  1 Mar 2016
@> openssl s_client -connect <host>:<port>
CONNECTED(00000003)
140401805448856:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
no peer certificate available
No client certificate CA names sent
SSL handshake has read 0 bytes and written 305 bytes
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1480703355
    Timeout   : 300 (sec)

Also I tried with earlier version of openssl and it seems to be working:

@> openssl version
OpenSSL 1.0.1f 6 Jan 2014
@> openssl s_client -connect <server>:<host>
CONNECTED(00000003)
depth=0 C = US, ST = California, O = Internet Widgits Pty Ltd
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = California, O = Internet Widgits Pty Ltd
verify error:num=27:certificate not trusted
verify return:1
depth=0 C = US, ST = California, O = Internet Widgits Pty Ltd
verify error:num=21:unable to verify the first certificate
verify return:1
Certificate chain
 0 s:/C=US/ST=California/O=Internet Widgits Pty Ltd
   i:/C=US/ST=California/O=Internet Widgits Pty Ltd/CN=John Smith
Server certificate
BEGIN CERTIFICATE
<certificate>
END CERTIFICATE
subject=/C=US/ST=California/O=Internet Widgits Pty Ltd
issuer=/C=US/ST=California/O=Internet Widgits Pty Ltd/CN=<Name>

No client certificate CA names sent
SSL handshake has read 2120 bytes and written 477 bytes

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-SHA384
    Session-ID: C7AEA448B9E21C30D90D1377904426A9D1A21971785D547378CA07ACDAC00161
    Session-ID-ctx: 
    Master-Key: C5B31E9D7A59EF4E6A2657E9F55A64B89F4AE3BBDFD864ADD1087449AA927D9C1655A76E44C3E30FF2301DB00C1CB2F7
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1480703425
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)

So this looks like your server has a pretty strenuous objection to something in your OpenSSL 1.0.2 handshake. Are you familiar with tcpdump or wireshark? If you are, it'd be really helpful if you could take a packet capture of the two handshakes and make those available to me.

Same problem here with www.celestrak.com

pip freeze --local

[...] Omitting other packages
cffi==1.9.1
cryptography==1.7.1
idna==2.2
ipaddress==1.0.17
ndg-httpsclient==0.4.2
numpy==1.11.3
pyasn1==0.1.9
pyOpenSSL==16.2.0
requests==2.12.4
scandir==1.4

openssl s_client -connect www.celestrak.com:443

CONNECTED(00000003)
140736264172552:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 308 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1483987800
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

openssl version

OpenSSL 1.0.2j  26 Sep 2016

I'm using the fix (that was working before) suggested here.

class CustomAdapter(requests.adapters.HTTPAdapter):
    ''' See: http://stackoverflow.com/a/14146031/1334711 '''
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = requests.packages.urllib3.poolmanager.PoolManager(
            num_pools=connections,
            maxsize=maxsize,
            block=block,
            ssl_version=ssl.PROTOCOL_TLSv1)

# Other code

        if url_parse.scheme in ('http', 'https'):
            # It's a URL, get from Internet
            try:
                tle_file = requests.get(source).text

            except requests.exceptions.SSLError:
                pytest.set_trace()
                s = requests.Session()
                s.mount('https://', CustomAdapter())
                tle_file = s.get(source).text

Python v.2.7.13
OS: macOS 10.12.2

The error I'm getting is:
SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

@rubendibattista www.celestrack.com has a terrible TLS configuration, as you can see here. This means that modern versions of Requests do not support any of the cipher suites that the server does, because all of them are weak and broken.

In the first instance, I recommend not contacting this server at all. If that's not possible, you can amend the default cipher suites by doing something like this.

@Lukasa Thanks for your support. I contacted the admin of the site to report what you say and try to find a better solution than using the workaround you suggest.

Hi guys! Got this errors too. Using python3 and installing the combo (pyopenssl ndg-httpsclient pyasn1 urllib3) did the trick. But i go curious how python2.7 behaved on OS X and Ubuntu:

Scenario 001

• python: 2.7
• requests: 2.3.0
• pyopenssl ndg-httpsclient pyasn1 urllib3: Not installed
• virtualenv: 1.11.6
• OS: OS X El Captain 10.11.6 (15G1217)
• OpenSSL: 0.9.8zh 14 Jan 2016

Error

Traceback (most recent call last):
  File "loanpro_doc_uploader.py", line 113, in <module>
    run()
  File "loanpro_doc_uploader.py", line 109, in run
    loanpro = LoanPro(args.env, args.folder)
  File "loanpro_doc_uploader.py", line 36, in __init__
    self.get_all_customforms_data()
  File "loanpro_doc_uploader.py", line 69, in get_all_customforms_data
    resp = requests.get(url, headers=headers)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env/lib/python2.7/site-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env/lib/python2.7/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env/lib/python2.7/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env/lib/python2.7/site-packages/requests/adapters.py", line 382, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)

Scenario 002

• python: 3.4
• requests: 2.12.5
• pyopenssl ndg-httpsclient pyasn1 urllib3: Not installed
• virtualenv: 1.11.6
• OS: OS X El Captain 10.11.6 (15G1217)
• OpenSSL: 0.9.8zh 14 Jan 2016

Error

Traceback (most recent call last):
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 350, in _make_request
    self._validate_conn(conn)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 835, in _validate_conn
    conn.connect()
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 323, in connect
    ssl_context=context)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/packages/urllib3/util/ssl_.py", line 324, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 365, in wrap_socket
    _context=self)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 583, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 810, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 624, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "loanpro_doc_uploader.py", line 113, in <module>
    run()
  File "loanpro_doc_uploader.py", line 109, in run
    loanpro = LoanPro(args.env, args.folder)
  File "loanpro_doc_uploader.py", line 36, in __init__
    self.get_all_customforms_data()
  File "loanpro_doc_uploader.py", line 69, in get_all_customforms_data
    resp = requests.get(url, headers=headers)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/Users/vyscond/Projects/razorvision/loan-documents/env3/lib/python3.4/site-packages/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:600)

Scenario 004

• python: 2.7
• requests: 2.12.5
• pyopenssl ndg-httpsclient pyasn1 urllib3: Not Installed
• OS: Ubuntu 14.4 (Dockerized)
• OpenSSL: not installed

Error is gone

Scenario 003

• python: 3.4
• requests: 2.12.5
• pyopenssl ndg-httpsclient pyasn1 urllib3: Installed
• virtualenv: 1.11.6
• OS: OS X El Captain 10.11.6 (15G1217)
• OpenSSL: 0.9.8zh 14 Jan 2016

Error is gone

The error is unclear, but most likely it is a result of your OpenSSL version. Your claim that Ubuntu 14.04 has no OpenSSL simply cannot be true if you are making HTTPS requests. My best guess is that the server is sending EOF because none of your ciphers are acceptable. Ubuntu works because it has a newer OpenSSL with newer ciphers. Installing PyOpenSSL works because it provides a vendored copy of OpenSSL 1.0.2, also with newer ciphers.

Hello!

I have a problem with some https-sites and request library.
Site: https://ndmc.gov.in/vacancy.aspx

installed:
appdirs==1.4.0
cffi==1.9.1
cryptography==1.7.2
idna==2.2
ndg-httpsclient==0.4.2
packaging==16.8
pyasn1==0.1.9
pycparser==2.17
pyOpenSSL==16.2.0
pyparsing==2.1.10
requests==2.13.0
six==1.10.0
urllib3==1.20
OpenSSL 1.0.2k 26 Jan 2017

get an error: requests.exceptions.SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

OS: Archbang linux, latest
Python 3.6 & Python 2.7 - same error

You're another person who is winning our competition of who can attempt to contact the worst HTTPS server in the world. This server is awfully configured, it is entirely insecure. The reason you're encountering this problem is because Requests no longer supports any of the cipher suites this server supports because they're all either weak or insecure. In the first instance I recommend you attempt to pressure someone to fix the server, but if you really must contact it you can re-add 3DES like this.

@vyscond saved my life. Never mind what I did on my OSX, it went away only when installing pyopenssl ndg-httpsclient pyasn1 urllib3. Maybe it should be added as dependencies for OSX installation of requests?

It is only useful on some configurations of OS X and Python. The end result is that they remain optional.

I'm wondering how much time people spend before they (maybe) find the solution to that problem. I had to through a lot to find this thread and go through it. Is it worth it?

Depends. Problems can be introduced by requiring these dependencies too, not least that it adds a new OpenSSL to your system that is versioned and managed separately.

@Lukasa for some reasons I got my setup working, seems pyopenssl (installed via requests[security] is messing up the certificates attached to some requests.
I had to uninstall the pyopenssl package in order to force requests to use OpenSSL in urllib3.
This happens when using adal (https://github.com/AzureAD/azure-activedirectory-library-for-python) which basically does not work with some old versions of requests (the ones supporting the certificates).
Seems a bit voodoo to me, though.

I was having this issue on Mac OSX Sierra v 10.12.0. Tried everything on this thread, didn't work. Finally upgraded my OS to v10.12.6 and the issue went away.

The error may also manifest itself when using python-requests to make an api call to a site that is behind a vpn, and that vpn tunnel happens to be down at that moment.

Hi guys!

I am a bit new to requests myself and found this error while doing a POC with Requests to see if it can fulfill my requirements for common criteria compliance. I tried to connect my Requests POC to this tool, I am specifically looking at the test cases for FIA-X509 Ext 1.1 but this is what i got:

Traceback (most recent call last):
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
conn.connect()
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
ssl_context=context)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='10.0.0.221', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:777)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "req.py", line 91, in
r = session.request('GET', 'https://10.0.0.221', verify=True)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, *send_kwargs)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, *kwargs)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='10.0.0.221', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:777)'),))

this is the POC code that i am using:

import ssl
import requests

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
from requests.packages.urllib3.util import ssl_

CIPHERS = (
    'RSA+AES'
)

class TlsAdapter(HTTPAdapter):
    def __init__(self, ssl_options=0, **kwargs):
        self.ssl_options = ssl_options
        super(TlsAdapter, self).__init__(**kwargs)

    def init_poolmanager(self, *pool_args, **pool_kwargs):
        ctx = ssl_.create_urllib3_context(ciphers=CIPHERS, cert_reqs=ssl.CERT_REQUIRED, options=self.ssl_options)

        self.poolmanager = PoolManager(*pool_args,
                                       ssl_context=ctx,
                                       **pool_kwargs)

session = requests.session()

adapter = TlsAdapter(ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1)
session.mount("https://", adapter)
r = session.request('GET', 'https://10.0.0.221', verify=True)
print(r)

where 10.0.0.221 is the place where i have placed the tool for verification.

I am not exactly sure as to why this error is occurring. I tried going through this thread as well as multiple other questions on stackoverflow as well but could not find a suitable solution.

I will admit that i do not have a clear understanding of the TLS protocol as well at this point in time. So if someone could guide me as to what's the problem here, i'd be grateful.

Please let me know if i missed anything or need to provide any sort of additional information. Thanks!

Just as an FYI, this error also occurs when SSL isn't enabled on the server you're connecting to.