Requests: Requests 2.11: check_header_validity failed on header with integer value
Hi,
Since requests 2.11, all my calls using requests for my app are broken. After debugging, it seems that this version does not accept header with integer value, like it was before.
2.10:
In [1]: import requests
In [2]: requests.get('http://bing.com', headers={'Content-Length': 42})
Out[2]: <Response [200]>
2.11
In [1]: import requests
In [2]: requests.get('http://bing.com', headers={'Content-Length': 42})
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\utils.py in check_header_validity(header)
751 try:
--> 752 if not pat.match(value):
753 raise InvalidHeader("Invalid return character or leading space in header: %s" % name)
TypeError: expected string or bytes-like object
During handling of the above exception, another exception occurred:
InvalidHeader Traceback (most recent call last)
<ipython-input-2-ae7ec2933e34> in <module>()
----> 1 requests.get('http://bing.com', headers={'Content-Length': 42})
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\api.py in get(url, params, **kwargs)
68
69 kwargs.setdefault('allow_redirects', True)
---> 70 return request('get', url, params=params, **kwargs)
71
72
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\api.py in request(method, url, **kwargs)
54 # cases, and look like a memory leak in others.
55 with sessions.Session() as session:
---> 56 return session.request(method=method, url=url, **kwargs)
57
58
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
455 hooks = hooks,
456 )
--> 457 prep = self.prepare_request(req)
458
459 proxies = proxies or {}
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\sessions.py in prepare_request(self, request)
388 auth=merge_setting(auth, self.auth),
389 cookies=merged_cookies,
--> 390 hooks=merge_hooks(request.hooks, self.hooks),
391 )
392 return p
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\models.py in prepare(self, method, url, headers, files, data, params, auth, cookies, hooks, json)
293 self.prepare_method(method)
294 self.prepare_url(url, params)
--> 295 self.prepare_headers(headers)
296 self.prepare_cookies(cookies)
297 self.prepare_body(data, files, json)
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\models.py in prepare_headers(self, headers)
407 for header in headers.items():
408 # Raise exception on invalid header value.
--> 409 check_header_validity(header)
410 name, value = header
411 self.headers[to_native_string(name)] = value
D:\VSProjects\azure-sdk-for-python\env3.5\Lib\site-packages\requests\utils.py in check_header_validity(header)
754 except TypeError:
755 raise InvalidHeader("Header value %s must be of type str or bytes, "
--> 756 "not %s" % (value, type(value)))
757
758
InvalidHeader: Header value 42 must be of type str or bytes, not <class 'int'>
We define 'Content-Length' in each request. Anyway, using an integer for an header which is semantically an integer makes sense no?
lmazuel
All 23 comments
Non-strings for header values was never an accepted way to use Requests, sadly, and while it was allowed in prior versions we have since made changes that disallow it. This is primarily because headers are _really_ string-string mappings, but also because the general approach of calling str on things that are passed to Requests has a tendency to misbehave in unexpected ways that surprise users.
The TL;DR of this is: yes, this was done on purpose, and no, we don't think it's a bug.
Lukasa
on 8 Aug 2016
See also: #865.
Lukasa
on 8 Aug 2016
Seems like a breaking change for a dot-release, no?
clarkbreyman-yammer
on 8 Aug 2016
And you didn't describe this huge modification in the ChangeLog :(
lmazuel
on 8 Aug 2016
This is not a breaking API change — the API is clearly documented as being string-based, throughout the documentation. The fact that you were using it with integers, instead of its intended input, is a bug in your code, not with in this codebase or a change in the API.
Adding a note in the changelog about this non-breaking change may be a good idea.
kennethreitz42
on 8 Aug 2016
@clarkbreyman-yammer To be clear, this is a bit of a borderline case. You can see the decision making most recently in #3386 and #3388, but the basic argument is: non-string headers were never _intended_ to work, so the fact that they stopped working is acceptable. In fact, that have previously not worked in the past.
@lmazuel You're right that this got missed from the changelog though, and that's 100% my fault: the breakage here was incidental of our tighter verification of header values, and as a result I didn't see it when I was compiling the changelog. I'd welcome a PR that updates the changelog if you'd like to make one.
Lukasa
on 8 Aug 2016
And to be clear, all three of us are in agreement on this. The fact that sending an integer there _ever_ worked was purely luck. It used to not work at all and we decided it should never work long ago. We never had anything that _enforced_ that though.
sigmavirus24
on 8 Aug 2016
That being said, adding that as a supported feature is potentially a great idea, and would be a welcome API change in my book (if the implementation was made well). However, the current status of things is likely to remain for the extended future.
kennethreitz42
on 8 Aug 2016
Added a note in the changelog.
kennethreitz42
on 8 Aug 2016
Thanks @kennethreitz.
Lukasa
on 8 Aug 2016
Ok, thanks for the clarification. I will update my code accordingly.
@kennethreitz if you could add your note in PyPI too, would be amazing.
lmazuel
on 8 Aug 2016
@lmazuel on it!
kennethreitz42
on 8 Aug 2016
@lmazuel done ✨🍰✨
kennethreitz42
on 8 Aug 2016
@lmazuel p.s. thank you for caring :P
kennethreitz42
on 8 Aug 2016
I fixed my code, everything else is working great. Thanks for your ultra quick response time!
lmazuel
on 9 Aug 2016
in the official documentations there is no mention about the major change. It is really misleading.
COLDMOUNT
on 26 Sep 2016
It's not considered a major change by the Requests team, it's considered a bug fix that fits within the boundaries of the documented library behaviour.
Lukasa
on 26 Sep 2016
Hey @COLDMOUNT, you can find the documentation here in the last paragraph from the quickstart section for custom headers. We also published the change in HISTORY.rst for 2.11.0 which is Requests' changelog.
nateprewitt
on 26 Sep 2016
It is better to offer an example of the headers in str type, or how to
translate a headers in dict type to str type, thanks.
2016-09-27 1:30 GMT+08:00 Nate Prewitt [email protected]:
Hey @COLDMOUNT https://github.com/COLDMOUNT, you can find the
documentation here
http://docs.python-requests.org/en/master/user/quickstart/#custom-headers
in the last paragraph from the quickstart section for custom headers. We
also published the change in (HISTORY.rst)[https://github.
com/kennethreitz/requests/blob/master/HISTORY.rst] for 2.11.0 which is
Requests' changelog.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kennethreitz/requests/issues/3477#issuecomment-249638650,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARIa89wU7tKL8Br2WWU6lJy8HEbNr72Vks5quAE3gaJpZM4JffFG
.
COLDMOUNT
on 27 Sep 2016
or how to translate a headers in dict type to str type
Wait, what? How were you sending headers before?
Lukasa
on 27 Sep 2016
Just as the guide on the page --
http://docs.python-requests.org/en/master/user/quickstart/ -- "If you'd
like to add HTTP headers to a request, simply pass in a dict to the headers
parameter." But now dict type is no longer accepted, and what does the str
type look like?
2016-09-27 17:51 GMT+08:00 Cory Benfield [email protected]:
or how to translate a headers in dict type to str type
Wait, what? How were you sending headers before?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kennethreitz/requests/issues/3477#issuecomment-249819028,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARIa8_ZPJmzxH2lWmpTvSJ3PvHE7jlpvks5quOcrgaJpZM4JffFG
.
COLDMOUNT
on 27 Sep 2016
@COLDMOUNT dict is _absolutely_ accepted, we haven't changed that at all. What we have changed is that the keys and values of that dict must now be strings. Previously it was possible for them to be a few other types by accident, which has now been resolved. The documentation does not need changing.
Lukasa
on 27 Sep 2016
Got it, cool! Thanks! :)
2016-09-27 18:23 GMT+08:00 Cory Benfield [email protected]:
@COLDMOUNT https://github.com/COLDMOUNT dict is _absolutely_ accepted,
we haven't changed that at all. What we have changed is that the keys and
values of that dict must now be strings. Previously it was possible for
them to be a few other types by accident, which has now been resolved. The
documentation does not need changing.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kennethreitz/requests/issues/3477#issuecomment-249825918,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARIa8z0jZ-ovBtvFWl4hTXnkk_kJXobrks5quO6ggaJpZM4JffFG
.
COLDMOUNT
on 27 Sep 2016
Related issues
JimHokanson
·
3Comments
Gonzalliz
·
3Comments
xsren
·
3Comments
8key
·
3Comments
jakul
·
3Comments
Most helpful comment
Seems like a breaking change for a dot-release, no?