Sendgrid-nodejs: Send bulk mail get error 401 Unauthorized

Hi,

I have a very similiar issues with the SDK here (Using NodeJS SDK 6.4.0), and customer support sent me here as they think it's an issue from the SDK itself. I often send emails at the same time, from the same server, but got different answer from the API calls.

Exemple :

• email sent to [email protected], with API key YYY ==> email accepted
• email sent to [email protected], with API key YYY ==> rejected with error 401, with the same error message than above
• email sent to [email protected], with API key YYY ==> email accepted

Here are some figures about one day, when we tried to send multiple email using the SDK to call SendGrid :

• we tried to send 70 emails (with the same API key)
• each of this email had 1 email address in "to" field and 2 emails adresses in "bcc" field
• These API calls were made from 18 differents servers (in fact 18 different Heroku dynos, some of them might be on the same serveur)
• 52 calls were accepted by the API, and 18 were rejected with "401 Unauthorized Error"
  
  
  
  • In those 18 rejected with 401 error, 7 were rejected with the Error body being "undefined", and 11 were rejected with Error body being "[ { message: 'Could not authenticate', field: null, help: null } ] }"
  
  
• Everything was executed between 06:00:00 AM and 06:00:02 AM
• The source code using the uses the SDK method sgMail.setApiKey(SENDGRID_API_KEY) to set up the key (which is well set up for each of the calls, I saw that reading the field sgMail.client.apiKey) and we use the SDK method sgMail.sendMultiple(email) to send the emails.

Can there be concurrency issues with the SDK when trying multiple calls with the same API Key ?

Here is the code used :

const sgMail = require('@sendgrid/mail');

const SENDGRID_API_KEY = process.env.SENDGRID_API_KEY;
const SENDGRID_MY_TEMPLATE_ID = process.env.SENDGRID_MY_TEMPLATE_ID;

sgMail.setApiKey(SENDGRID_API_KEY);

class Mail {
  static async sendReport(recipientEmails, data) {
    const email = {
      to: recipientEmails,
      bcc: ['[email protected]', '[email protected]'],
      from: {
        email: '[email protected]',
        name: 'MyCompany',
      },
      replyTo: '[email protected]',
      template_id: SENDGRID_MY_TEMPLATE_ID,
      asm: {
        group_id: 1234,
      },
      dynamic_template_data: data,
    };

    try {
      await sgMail.sendMultiple(email);
    } catch (e) {
      console.log('ERROR Sending Report', e);
      console.log('ERROR Sending Report', e.toString());
      console.log('ERROR Sending Report', e.response.body);
      throw e;
    }
  }
}

I confirm that it's the same for me. Some request are accepted, but others get 401 unauthorized.
But it's really impossible to understand the reason of the 401, because it's seems random and it not depends on the code for me.

I tried multiple changes :

• using sgMail.send instead of sgMail.sendMultiple
• adding a small delay of half a second between 2 calls
• hardocde the API key

const sgMail = require('@sendgrid/mail');
const SENDGRID_MY_TEMPLATE_ID = process.env.SENDGRID_MY_TEMPLATE_ID;
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

sgMail.setApiKey('HARDCODED_API_KEY');

class Mail {
  static async sendReport(recipientEmails, data) {
    const email = {
      to: recipientEmails,
      bcc: ['[email protected]', '[email protected]'],
      from: {
        email: '[email protected]',
        name: 'MyCompany',
      },
      replyTo: '[email protected]',
      template_id: SENDGRID_MY_TEMPLATE_ID,
      asm: {
        group_id: 1234,
      },
      dynamic_template_data: data,
    };

    try {
      await sgMail.send(email);
      await sleep(500);

    } catch (e) {
      console.log('ERROR Sending Report', e);
      console.log('ERROR Sending Report', e.toString());
      console.log('ERROR Sending Report', e.response.body);
      throw e;
    }
  }
}

But same issue ...

What is strange is that the corresponding source code seems clean, so kind of difficult to troubleshoot without the help of SendGrid devs accessing the API code ...

@yoagauthier Did you manage to fix the issue?
I don't think the developer team will help us.

@francesco-clementi-bip No, and we'll probably switch mail provider very soon, this repo seems quite dead and support is not very helpful understanding the issue.

I'm not sure anymore the issue comes from the SDK though : I've tried to code directly against the REST API, and I still have API calls to /mail/send that are rejected with 401 (around 10%), while most of them with the same config (around 90%) and same API key are accepted. For my case, I suspect it's because these calls are done simultaneously from different servers and the auth server probably cannot handle it.

My code :

const axios = require('axios');
const {
  SENDGRID_MY_TEMPLATE_ID,
  SENDGRID_API_KEY,
} = process.env;

class SendGridClient {
  constructor() {
    this.axiosInstance = axios.create({
      baseURL: 'https://api.sendgrid.com/v3',
      headers: {
        Authorization: `Bearer ${SENDGRID_API_KEY}`,
      },
    });
  }

  async sendMail(toEmail, data) {
    const email = {
      personalizations: [
        {
          to: [
            {
              email: toEmail,
            },
          ],
          dynamic_template_data: data,
        },
      ],
      from: {
        email: '[email protected]',
        name: 'MyCompany',
      },
      reply_to: {
        email: '[email protected]',
        name: 'MyCompany',
      },
      template_id: SENDGRID_MY_TEMPLATE_ID,
      asm: {
        group_id: 1234,
      },
    };

    try {
        await this.axiosInstance.post('/mail/send', email);
    } catch (e) {
      console.log('ERROR Sending Report', e);
      console.log('ERROR Sending Report', e.response.data);
      throw e;
    }
  }
}

module.exports = SendGridClient;

Thanks @yoagauthier ,
I tried to use directly the api and get the same result as you.
We are going to change provider next week, because sendgrid is not ready for a production use.
Have a nice day

We're also having this issue randomly (maybe twice a month or so): 401 Unauthorized for no reason.

Same problem. I used Sendgrid for couple of years. And recently started getting 401 error. Customer service is useless.

Same here, just started getting these errors randomly while in dev-mode (low volume testing). Nothing changed just randomly started getting these errors.

I got same problem, and I removed sendgrid.env and declare SENDGRID_API_KEY as variable

const sgMail = require('@sendgrid/mail');
const SENDGRID_API_KEY='{api_key};
sgMail.setApiKey(SENDGRID_API_KEY);

If you're seeing similar behaviour when accessing the API directly, then this library won't be able resolve the issue for you.

That said, it's possible the API key is not being loaded properly based on your environment. Sticking the API key in a .env file and sourcing that is just one way of getting it loaded, but there are other ways and it just depends on your environment how best to configure env vars.

What we _can_ do in this library is update the underlying client to throw an exception if you're setting the API key to an empty string which would indicate an issue with your environment and fail-fast to help debug the issue.

I'll leave this issue open for anyone wanting to implement that last piece.

Also, I've confirmed that API keys always start with the prefix SG. so that can also be used to further sanity-check the passed in API key when calling setApiKey().

+1 have the same issue when sending several emails to different receivers in a short time period.
Using heroku and pass key via env variable, not sure why the env variable should be not available if its working most of the time and having no other issues with env variables.

@childish-sambino is there a sure-fire way to avoid this issue? Perhaps there is a max number of emails that can be sent in the same MailData object? Thanks!

@dockleryxk If you're seeing seemingly random Unauthorized 401s and your API key is being loaded properly, I don't know what the root cause is and it's probably not something that should be handled in this lib. SendGrid Support has the tools to help debug these kinds of issues.

Changing my previous comment about leaving this open to track handling empty/undefined/null/blank API keys. Let's use https://github.com/sendgrid/sendgrid-nodejs/issues/1068 to track that work.

@childish-sambino I cannot agree with your conclusion to close this issue. you asuming something, but do you have proof? more likely its an issue with this library.

@crascher To help debug the issue you're seeing, have you verified (say, through logging) that you're sending the correct API key when you're seeing the 401s? Does retying the same request result in the same error? Do you see the same issue when calling the API directly?

@childish-sambino thanks for your response. To provide some more context, my API key is hardcoded (it doesn't matter in my case).
I'll get this error on the same sendMail call. Specifically, the email will successfully send to some of the recipients, but one or more will not send and get this error instead. It's always fine in the to field (I only ever have one email here), but the cc field with multiple emails is where the issue arises for me.

I have tried to recreate the error but it never happens. It's only in a prod environment where I can't simply resend them because it would result in some annoyed customers.

@childish-sambino I added inside the callback method of sendMail logic to execute another sendMail if an error is occured. The mail is sent to me and that is working.
I have no cc field field. Maybe the returned code 401 is wrong or this library is loosing the token or something with the api triggers the problem.

Are you planing to open this ticket again? For me its not solved yet.

More context: We've been using this package for almost two years. February 25 was the first time this ever happened for us. Then March 15 and March 17 and now twice today. This issue isn't resolved and seems to be getting worse.

@dockleryxk @crascher Sounds like the same issue: seemingly random 401s even though nothing else in your environment has changed. I highly doubt it's an issue with this library itself having reviewed the code and not spotted much complexity with regards to API keys.

My recommendation to open a support ticket still stands (support doesn't monitor issues here, engineers do). If you can provide them with details about when the request failed and any other details will help to track it down. If that doesn't help or you don't hear back in a timely manner, let me know. We're working to be more responsive in these forums.

Lastly, the library will soon be migrating the underlying HTTP client (https://github.com/sendgrid/sendgrid-nodejs/issues/1042) which we'll do a major version roll for. It's possible there was a bug in the current client so may be worth waiting for the changes to roll out to see if it resolves the issue.

@childish-sambino I'll open a ticket then, thank you! If I end up resolving it with them I will post back here about it

_fwiw adding my comment here in case someone might hit it on different key words_

We're running several dynamic template tests using jest. We started running into the same random 401 issue this afternoon, after running the same suite of tests all morning without issue. I reduced the test group to 10 tests, and we're running them in jest sequentially (so there's not multiple send requests being made at the same time). Of those 10 tests, no number of tests and no single test will fail on subsequent runs, reliably. Tests in the suite of 10 fail at random returning the same 401. We've gone so far as to log what the client is using for request data when using request internally - the Authorization header is correct for each failing request.

Given the simplistic nature of the client's code, this looks more like a network issue or network limitation to me.

the problem ist not solved yet; I am using latest version 7 and get sometimes:

Unauthorized (401) Error: Unauthorized
at ResponseError (node_modules/@sendgrid/helpers/classes/response-error.js:19:5)
at axios.then.catch.error (node_modules/@sendgrid/client/src/classes/client.js:105:29)
at process._tickCallback (internal/process/next_tick.js:109:7)

code: 401,
message: 'Unauthorized',
response:
{ headers:
{ server: 'nginx',
date: 'Fri, 24 Apr 2020 12:45:01 GMT',
'content-type': 'application/json',
'content-length': '74',
connection: 'close',
'access-control-allow-origin': 'https://sendgrid.api-docs.io',
'access-control-allow-methods': 'POST',
'access-control-allow-headers': 'Authorization, Content-Type, On-behalf-of, x-sg-elas-acl',
'access-control-max-age': '600',
'x-no-cors-reason': 'https://sendgrid.com/docs/Classroom/Basics/API/cors.html' },
body: '' } }

Support says:

Mar 24, 12:33 PM PDT
Hi,

Thanks for your reply and additional insight. I checked with our team regarding this issue. As it turns out, making rapid Mail Send calls in a row can intermittently lead to an odd 401 error. I sincerely apologize for any inconvenience caused. In this instance, we recommend implementing some form of retry logic within your code when you receive the 401 error as a workaround.

Hopefully, this information was helpful. Please let me know if I was unable to answer your questions, or if you have any other questions or concerns.

Best,
...

When does it get fixed and why the issue ist not open if it is still an issue?

It happened to me too, but i think my problem was different.

Because I had already reached limit to of 100 mail/day nothing else.

Bulk email is now sent perfectly without any errors.

idk if it helps just wanted to share.