request.get crashes with "The header content contains invalid characters"

Also - I found no way to catch that exception. No matter if there's a try-catch, or .on('error', ...

I'm getting the same error using Node version 0.12.12 and greater. The same code works with 0.12.9:

_http_outgoing.js:355
      throw new TypeError('The header content contains invalid characters');
            ^
TypeError: The header content contains invalid characters
    at ClientRequest.OutgoingMessage.setHeader (_http_outgoing.js:355:13)
    at new ClientRequest (_http_client.js:101:14)
    at Object.exports.request (http.js:49:10)
    at Object.exports.request (https.js:136:15)
    at Request.start (//application/node_modules/request/request.js:747:30)
    at Request.end (/application/node_modules/request/request.js:1381:10)
    at end (/application/node_modules/request/request.js:575:14)
    at Immediate._onImmediate (/application/node_modules/request/request.js:589:7)
    at processImmediate [as _immediateCallback] (timers.js:367:17)

Turns out that in my case I was sending a line break on a cookie. I had to remove it.

No, my case is a simple one-liner, no cookies, no additional headers.

I get the same issue , using https://github.com/matt-major/do-wrapper when trying to access my account information.

_http_outgoing.js:348
    throw new TypeError('The header content contains invalid characters');
    ^

TypeError: The header content contains invalid characters
    at ClientRequest.OutgoingMessage.setHeader (_http_outgoing.js:348:11)
    at new ClientRequest (_http_client.js:85:14)
    at Object.exports.request (http.js:31:10)
    at Object.exports.request (https.js:197:15)
    at Request.start (/home/pierre/Documents/freelance/upwork/dosh/node_modules/do-wrapper/node_modules/request/request.js:746:30)
    at Request.write (/home/pierre/Documents/freelance/upwork/dosh/node_modules/do-wrapper/node_modules/request/request.js:1345:10)
    at end (/home/pierre/Documents/freelance/upwork/dosh/node_modules/do-wrapper/node_modules/request/request.js:560:16)
    at Immediate._onImmediate (/home/pierre/Documents/freelance/upwork/dosh/node_modules/do-wrapper/node_modules/request/request.js:588:7)
    at processImmediate [as _immediateCallback] (timers.js:383:17)

Thing is, while it's easy to fix this, request should catch the error and propagate it through the error handlers. Currently it's thrown as an uncaught exception which can cause pretty bad server crashes that you can never recover from because you cannot manually try/catch this.

So I'd say this is a pretty severe bug in request. The error needs to be propagated.

I've invested a good 2h now trying to fix this issue, for which writing a reproducing test case was pretty trivial.
I've come to the conclusion that with my current knowledge of the codebase it's unfixable unless the validation would be added to https://github.com/request/caseless, which btw cost me a solid 20-30 minutes to discover is actually self.setHeader. That behavior is neither documented on caseless, nor is it obvious from the name that it deals with setting and getting headers, and nor is it expected that caseless.httpify(self, self.headers) would monkey patch the object with setHeader.

try/catch at the point where the request object is created is pretty much impossible to recover from because there is no simple way to abort execution then. self.req.end will still be called, and if the call is removed none of the subsequent behavior is triggered (again, for entirely non obvious reasons).

To be blunt, it scares me that my production services rely on a library with a 1.4k line main file in which the logic is so convoluted that something as easy as propagating an error from a try/catch is practically impossible without deeply studying the entire rube goldberg machine of events. Time to move away from this.

This also affects Node 5.6.0 and above which has more strict header validation: https://github.com/nodejs/node/blob/v5.6.0/CHANGELOG.md

Fixed here https://github.com/request/request/pull/2164

Take a look at the test on how to handle the error in your code.

Seems like handling the error is as usual, with an .on('error', ... event. Good!

This will make the request more bulletproof, without crashing the app on stupid things in the future.

Well, now I feel like an incredibly silly prick.
I actually had exactly the same solution ready and my tests would constantly fail.
Turns out in my 'trivial test case' I forgot to call t.end().

My apologies, next time I'll just publish my branch and ask for feedback.

@TimBeyer no problem, version 2.71 is published with the fix :+1:

@simov The version 2.71 has fixed the problem partially for me, Here is my scenario I am sending a post request and getting data as chunks,

request({
              method: 'POST',
              headers: {
                 'test': 'אבגד'
              },
              uri: apiUrl,
              qs: req.query,
              json: req.body,
              gzip: true
            }
            , function(error, response, body) {
               console.log('callback')
            })
            .on('response', function (response) {
             console.log(response)
            })
            .on('error', function (err) {
              console.log(err);
            });

When I run this I get an error

return self.req.write.apply(self.req, arguments)
                 ^

TypeError: Cannot read property 'write' of undefined
    at Request.write (/Users/muhammadkhurrumqureshi/Workspace/www/node_modules/request/request.js:1385:18)
    at end (/Users/muhammadkhurrumqureshi/Workspace/www/node_modules/request/request.js:565:18)
    at Immediate._onImmediate (/Users/muhammadkhurrumqureshi/Workspace/www/node_modules/request/request.js:594:7)
    at processImmediate [as _immediateCallback] (timers.js:383:17)

Can you kindly help on this?

@khurrumqureshi fixed here https://github.com/request/request/pull/2165 :+1:

Basically the same check as in the end method. The reason why is because start is being called on the previous line, and up until now it was not expected to throw. When your request have a body write is called before end, hence the error.

@simov :+1:

Same here.

TypeError: The header content contains invalid characters
    at ClientRequest.OutgoingMessage.setHeader (_http_outgoing.js:348:11)
    at new ClientRequest (_http_client.js:85:14)
    at Object.exports.request (http.js:31:10)
    at Object.exports.request (https.js:199:15)
    at Request.start (/home/mymodule/node_modules/request/request.js:744:32)
    at Request.end (/home/mymodule/node_modules/request/request.js:1433:10)
    at end (/home/mymodule/node_modules/request/request.js:566:14)
    at Immediate.<anonymous> (/home/mymodule/node_modules/request/request.js:580:7)
    at runCallback (timers.js:570:20)
    at tryOnImmediate (timers.js:550:5)

Response headers are:

headers:
      { 'x-backside-transport': 'OK OK,FAIL FAIL',
        connection: 'close',
        'transfer-encoding': 'chunked',
        'x-dp-local-file': 'true',
        'x-client-ip': '127.0.0.1,176.31.126.162',
        'x-global-transaction-id': '145630106',
        date: 'Tue, 08 Nov 2016 01:03:59 GMT',
        'www-authenticate': 'Basic realm="Gateway(Log-in)"',
        'x-archived-client-ip': '127.0.0.1',
        'content-type': '',
        'x-dp-tran-id': 'gateway' },
     rawHeaders:
      [ 'X-Backside-Transport',
        'OK OK,FAIL FAIL',
        'Connection',
        'close',
        'Transfer-Encoding',
        'chunked',
        'x-dp-local-file',
        'true',
        'X-Client-IP',
        '127.0.0.1,176.31.126.162',
        'X-Global-Transaction-ID',
        '145630106',
        'Date',
        'Tue, 08 Nov 2016 01:03:59 GMT',
        'WWW-Authenticate',
        'Basic realm="Gateway(Log-in)"',
        'X-Archived-Client-IP',
        '127.0.0.1',
        'Content-Type',
        '',
        'X-DP-Watson-Tran-ID',
        'gateway' ]

Headers of request are:

{ accept: 'application/json',
     authorization: 'Basic ...g==',
     referer: 'https://hostname.com/classify?text=%20So%20happy%20with%20this%20👍👍👍👍👍',
     host: 'gateway.myclass.net' }

As you can see the request contains 3 times a special character <U+1F44D>.

Might it be that the problem?

It looks like node.js https module does not support it.

Just in case someone else had the same issue as I did - I had this error with the aws-sdk and it was driving me crazy for hours. Turned out to be a PrvScan character \u001b[5~ that for some reason was lurking about in my ~/.aws/credentials file, which propagated into the Authorization header used by AWS requests. The error only appeared on my Windows 7 machine, so it was probably something with how the credentials file was retrieved from my AWS account and some Windows character encoding. It wasn't visible when cating the credentials file either, but became visible once I opened the file with vi.

Yeah, just had the same issue. Took me a few hours to figure it out I must admit.
was using MINGW64 (Git Bash) on a windows machine. Tried running the same script using Node in command prompt.... and hello.... it worked????

I just hit the same error and cannot find a way to catch it. I'm building an image proxy so I cannot really controll what the other server is returning. How can I avoid carshing the whole server in production ?!

I had the same issue, it started to work when i exchange the "set" for "auth". For example:

it('ERROR, Wrong GET request', (done)=>{
    request(server).get("/api/")
    .set('Authorization', 'Bearer ' + tokenKey)
    .then(res=>{
        expect(res.statusCode).to.equal(404);
        done()
    }).catch(err=>done(err))
})

CONSOLE RES: ERROR, Wrong GET request:
TypeError [ERR_INVALID_CHAR]: Invalid character in header content ["Authorization"]

After changing the code to:

it('ERROR, Wrong GET request', (done)=>{
    request(server).get("/api/")
    .auth('Authorization', 'Bearer ' + tokenKey) //**<---here is the change**
    .then(res=>{
        expect(res.statusCode).to.equal(404);
        done()
    }).catch(err=>done(err))
})

CONSOLE RES: OKI ;p