Requests: Bad error handling in requests in python 3 [During handling of the above exception, another exception occurred]

Created on 16 Dec 2019  ·  3Comments  ·  Source: psf/requests

request library with Python3 handles exceptions in a pretty bad way. In python2 this works fine. I tried to request GitHub using requests.get('https://api.github.com') using both python2 installed requests library and python3 installed requests library. They perform differently when the requests are made during network is down. (any exception is handled differently). Try installing requests in both python2 and python3 and you can recreate the issue by removing internet connection when making the requests. Code I used
import requests
requests.get('https://api.github.com')

I have installed requests==2.5.3 in my system and I tried upgrading the version to requests==2.22.0 but still problem persists.

Expected Result

In python2 it gives a cleaner exception like the following
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 65, in get return request('get', url, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 49, in request response = session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 461, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 415, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', gaierror(-2, 'Name or service not known'))

Actual Result

`
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 157, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 334, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, *kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, *
kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, *send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, *
kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',))

`

Reproduction Steps

import requests

System Information

$ python -m requests.help
Python 2.7.15+ (Python2 version)
requests==2.5.3 (Requests version in python2)
=============================
Python 3.6.9 (Python3 version)
requests==2.5.3(Requests version in python3)
=============================
OS: Ubuntu 18.04.3 LTS

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Most helpful comment

I just stumbled upon the same issue, is there any way with this workaround to continue after the error?
My program should not exit after the raised exception but I didn't find any way to do that.

All 3 comments

I ran into this problem too, but I found a work-around for prettier output. Notice the "from None" in the script here:

import requests
url = 'http://10.0.0.12/example/'
try:
    requests.get(url, timeout=2)
except requests.exceptions.RequestException:
    raise Exception('Failed to connect to %s' % url) from None

If you run the above script the output is much cleaner:

$ python example.py
Traceback (most recent call last):
  File "example.py", line 6, in <module>
    raise Exception('Failed to connect to %s' % url) from None
Exception: Failed to connect to http://10.0.0.12/example/

You can only do the from None in Python3, and I think it comes from this PEP: https://www.python.org/dev/peps/pep-0409/

I ran into this problem too, but I found a work-around for prettier output. Notice the "from None" in the script here:

import requests
url = 'http://10.0.0.12/example/'
try:
    requests.get(url, timeout=2)
except requests.exceptions.RequestException:
    raise Exception('Failed to connect to %s' % url) from None

If you run the above script the output is much cleaner:

$ python example.py
Traceback (most recent call last):
  File "example.py", line 6, in <module>
    raise Exception('Failed to connect to %s' % url) from None
Exception: Failed to connect to http://10.0.0.12/example/

You can only do the from None in Python3, and I think it comes from this PEP: https://www.python.org/dev/peps/pep-0409/

I just stumbled upon the same issue, is there any way with this workaround to continue after the error?
My program should not exit after the raised exception but I didn't find any way to do that.

Was this page helpful?
0 / 5 - 0 ratings