BOM not stripped from response body when body filled with buffer
request({
url: url,
json: true,
strictSSL: false,
}, (error, response, data) => {
if (error) {
throw error;
}
if (response.statusCode === 200) {
// data is string instead of json deserialized object
}
})
BOM char stripped and json response deserialized
JSON.parse fails and a string is returned
if (bufferLength) {
debug('has body', self.uri.href, bufferLength)
response.body = Buffer.concat(buffers, bufferLength)
if (self.encoding !== null) {
response.body = response.body.toString(self.encoding)
}
// `buffer` is defined in the parent scope and used in a closure it exists for the life of the Request.
// This can lead to leaky behavior if the user retains a reference to the request object.
buffers = []
bufferLength = 0
} else if (strings.length) {
// The UTF8 BOM [0xEF,0xBB,0xBF] is converted to [0xFE,0xFF] in the JS UTC16/UCS2 representation.
// Strip this value out when the encoding is set to 'utf8', as upstream consumers won't expect it and it breaks JSON.parse().
if (self.encoding === 'utf8' && strings[0].length > 0 && strings[0][0] === '\uFEFF') {
strings[0] = strings[0].substring(1)
}
response.body = strings.join('')
}
As we can see the BOM char is stripped when strings are used instead of buffer
Extract the code outside of the else if (strings.length) to always clean the response body.
Reading json response from a company custom swagger based on swashbuckle
| software | version
| ---------------- | -------
| request | 2.83.0
| node | 9.3.0
| npm | 5.5.1
| Operating System | Windows 10
If we set encoding: 'utf8' in the request's options, the response is parsed
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I am seeing this issue too - thank you for posting the workaround - would be nice if the library handled this though.
Most helpful comment
If we set encoding: 'utf8' in the request's options, the response is parsed