Sentry-javascript: Google Cloud Functions - Sentry Client Set Up

Hey, thanks! :)

Is that still accurate? Can I get Sentry JS library working with Google Cloud Functions?

According to https://github.com/googleapis/nodejs-error-reporting#catching-and-reporting-application-wide-uncaught-errors and https://github.com/googleapis/nodejs-error-reporting#unhandled-rejections it's possible to do so now, however, I haven't tested it myself yet.

I can do so in a few days, but you can give it a spin if you have some spare time :)

Thanks @kamilogorek. Interesting!

I'm not sure how that interacts w/ Google Cloud Functions. I'm in the middle of technical planning now so won't get to testing that soon. But I'll let you know if / when I do. :)

Right now we are just wrapping all of our Cloud Functions in:

try () {
...
} catch (e) {
  Sentry.captureException(e);
}

But we aren't seeing the source code or other context in Sentry.

Is there a better way of doing this integration?

@vpontis just tested it myself and context resolution seems to work just fine. What's your function/config?

Hi @kamilogorek thanks for helping!

I have replicated what you get with Hello World, so that's great.

But there are no context variables. Like you can't see the value of the variables in scope? Or is that only a Python feature?

Also, it looks like the more complicated errors I am getting are when I am making writes / reads to the database and there is some error on a stream / event.

For example, in the screenshot below, I can't even tell what function it was called from.

What is your recommendation? Is there a way of getting a longer StackTrace that includes the call from the originating function handler? Or do I just need to add more breadcrumbs?

Also, any idea what's going on with these Source Code not found errors?

But there are no context variables. Like you can't see the value of the variables in scope? Or is that only a Python feature?

It's Python only feature. JS doesn't provide a mechanism to implement this feature, unfortunately.

For example, in the screenshot below, I can't even tell what function it was called from. What is your recommendation? Is there a way of getting a longer StackTrace that includes the call from the originating function handler? Or do I just need to add more breadcrumbs?

There's no way to tell what called this specific function, as it was triggered by a socket. And as you mentioned, the best way to track this is to use breadcrumbs when you perform db queries, proxy calls or external service requests.

And because serverless instances are long-lived, you can call:

Sentry.configureScope(scope => scope.clear())
// or
Sentry.configureScope(scope => scope.clearBreadcrumbs())

to get a clean slate.

Also, any idea what's going on with these Source Code not found errors?

Turn off Enable JavaScript source fetching in your projects settings, eg. https://sentry.io/settings/kamil-ogorek/projects/testing-project/
It's serverless node app, so there's no point in doing that.

It's Python only feature. JS doesn't provide a mechanism to implement this feature, unfortunately.

Damn. I loved this about Python!

Super helpful, thank you Kamil. I'll implement those changes and get back to you here if I have anymore questions. I'm sure this issue will be helpful to other people using Sentry on JS serverless.

Awesome! I'll close the issue for triaging sake, but feel free to ping me anytime and I'll reopen it if necessary! :)

@kamilogorek I added Breadcrumbs on Google Cloud Functions and I think I am seeing breadcrumbs from different function calls.

Does that make sense? How can I limit the breadcrumbs to be only from one HTTP request w/ Google Cloud Functions?

It makes sense, however can you show me example code that you use in your cloud functions? This way it'd be easier to understand everything.

@kamilogorek you've been really helpful!

We switched to using Koa with Docker / Node. So we have the Koa integration set up like this: https://docs.sentry.io/platforms/node/koa/

It's working pretty great except for breadcrumbs, we are seeing breadcrumbs for all requests on the server, not just breadcrumbs tied to the current request.

Is there a way to fix that?

Is there a way to fix that?

There is, but I need to test it before providing the code here :)
I'll try to get back to you in ~1-2 days.

@kamilogorek you are an absolute legend my friend

@vpontis so the only thing you really need to do is to create a domain instance in one of your middlewares. Once it's there, SDK will detect it and use to separate the contexts. You can also move parseRequest to there as well. (example based on the https://github.com/koajs/examples/blob/master/errors/app.js )

const Sentry = require("@sentry/node");
const Koa = require("koa");
const app = (module.exports = new Koa());
const domain = require("domain");

Sentry.init({
  // ...
});

app.use(async function(ctx, next) {
  const local = domain.create();
  local.add(ctx);
  local.on("error", next);
  local.run(() => {
    Sentry.configureScope(scope => {
      scope.addEventProcessor(event => Sentry.Handlers.parseRequest(event, ctx.request));
    });
    next();
  });
});

app.use(async function(ctx, next) {
  try {
    await next();
  } catch (err) {
    // some errors will have .status
    // however this is not a guarantee
    ctx.status = err.status || 500;
    ctx.type = "html";
    ctx.body = "<p>Something <em>exploded</em>, please contact Maru.</p>";

    // since we handled this manually we'll
    // want to delegate to the regular app
    // level error handling as well so that
    // centralized still functions correctly.
    ctx.app.emit("error", err, ctx);
  }
});

// response

app.use(async function() {
  throw new Error("boom boom");
});

// error handler

app.on("error", function(err) {
  Sentry.captureException(err);
});

if (!module.parent) app.listen(3000);

parseRequest accepts some options that you may want to use to pick what request data you want to extract - https://github.com/getsentry/sentry-javascript/blob/f71c17426c7053d46fe3e2e35e77c564749d0eb7/packages/node/src/handlers.ts#L177

Thanks @kamilogorek!

Some thoughts:

1. You can bind the actual Node req and res to the domain which are stored on ctx.req and `ctx.res

2. You can pass in the Node req to parseRequest

3. Why do you call ctx.app.emit("error", err, ctx); after manually catching the error?

    // since we handled this manually we'll
    // want to delegate to the regular app
    // level error handling as well so that
    // centralized still functions correctly.
    ctx.app.emit("error", err, ctx);

Wouldn't that be a case where you want to just return the response and not pass the error up to the centralized error handler?

1. In Koa next is async. Will that cause any issues inside of a Node domain.run(...)

Let me know if that makes sense. This is already be super, super helpful. I'll test it out later this week.

Ah re 3 I see you are just copying that from the example so I will ignore that part :). Testing it now...

Hmm, I'm having trouble getting this to work. I think it's due to the mixing of domain callbacks and async functions in Koa.

Also it looks like domains don't even work in Node 12 (I'm planning on upgrading soon to get async stack trace support).

Either way, I don't understand how this code with domain works so I'm hesitant to put it in a crucial part of the app.

Is there another way of putting the current "hub" on ctx and calling addBreadcrumb somehow associated with the current request? I'm not quite sure how hubs and message handling works in Sentry...

I got this code working (these are two successive middleware functions) but I don't feel comfortable with _why_ it works...

export const setSentryDomain = async (ctx, next) => {
  await new Promise(async (resolve, reject) => {
    const local = domain.create();

    local.add(ctx.req);
    local.add(ctx.res);

    local.run(async () => {
      Sentry.configureScope((scope) => {
        scope.addEventProcessor((event) => Sentry.Handlers.parseRequest(event, ctx.req));
      });

      await next();
      resolve();
    });
  });
};

export const catchErrors = async (ctx, next) => {
  try {
    await next();
  } catch (err) {
    console.log('got error');
    ctx.app.emit('error', err, ctx);
  }
};

I'm calling it a night. I didn't get a working solution that I was happy with.

I also realized that most of my breadcrumbs are from console.log statements which are automatically captured by Sentry. So I need to figure out how to get these breadcrumbs to be on the right scope...

I poked around the code and domains sounds like it solves this problem. But domains don't seem reliable with async functions and they are going away soon...

Is there another way of putting the current "hub" on ctx and calling addBreadcrumb somehow associated with the current request?

You can assign the hub directly to the ctx, however it won't work for automatically captured breadcrumbs, as those require Sentry.getCurrentHub() to return the hub, the breadcrumb should be assigned to. And one of the ways it detects it, is by using domain.active.

But domains don't seem reliable with async functions and they are going away soon...

Unfortunately, they are going away since December 2014, with no clear replacement (there are async hooks, but they are not perfect as well) in sight, as well as they have been not removed from the node's core in those 5 years.

We are playing around with zone.js now, as it'd help tremendously, but it's still a huge PoC for now and can't tell when or even if we'll ever use it to replace domains.

Hi @kamilogorek,

I am trying to capture unhandled errors into Sentry, and tried out what you had mentioned in https://github.com/getsentry/sentry-javascript/issues/2122#issuecomment-503440087

But I guess google doesn't let you hook up to the process.on('uncaughtException'), I wasn't able to log any errors to Sentry using that approach.

Is there any other way that you would recommend trying out, wrapping every function body in a try catch block doesn't seem like the ideal way to go.

We provide a wrap method, but I don't think it's much better than try/catch tbh.

exports.helloBackground = (data, context) => {
  return `Hello ${data.name || 'World'}!`;
};

// becomes

exports.helloBackground = (data, context) => {
  return Sentry.wrap(() => {
    return `Hello ${data.name || 'World'}!`;
  })
};

It seems to me that this issue might be worth keeping open as a feature request to support firebase/google cloud functions in a simpler manner.

I was incredibly impressed with the ease of set up on the client-side and disappointed when I realized setting up the server-side would be much more complex. Is there any plan to improve the experience on GCP (functions specifically)?

@goleary Would you mind opening a new issue exactly describing what you would like to see?
This thread is already quite large and hard to follow.

Thanks

Is there still no simpler way to set this up? Ideally being able to set it once globally, without having to set it for every function?

@marcospgp no, and there won't be, unfortunately, as Google itself doesn't provide a mechanism that'd allow that. See their own reporter - https://cloud.google.com/error-reporting/docs/setup/nodejs it uses manual calls as well.

Hmm interesting, I set up Sentry on Firebase cloud functions (which uses google cloud functions behind the scenes) and I just got an error report - so it seems to work!

Here's my index.js, where all Sentry code is:

const admin = require("firebase-admin");
const functions = require("firebase-functions");
const Sentry = require("@sentry/node");

/**
 * Set up Sentry for error reporting
 */
if (functions.config().sentry && functions.config().sentry.dsn) {
  Sentry.init({ dsn: functions.config().sentry.dsn });
} else {
  console.warn(
    "/!\\ sentry.dsn environment variable not found. Skipping setting up Sentry..."
  );
}

admin.initializeApp();

const { function1 } = require("./cloud-functions/function1");
const { function2 } = require("./cloud-functions/function2");

module.exports = {
  function1,
  function2
};

@marcospgp was the above index.js able to send uncaught exceptions within ./cloud-functions/function1 and ./cloud-functions/function2 to Sentry, or did you have to actively log to Sentry within those files?

I just tried @marcospgp solution, but it doesn't seem to be logging uncaught exceptions, he must be manually logging them (using sentry.captureException()?)

Same as @goleary here, nothing gets logged.

Also it seems that .wrap() isn't available anymore.

Does it mean that we should manually wrap all the code inside try/catches?

@Dinduks that's what I'm doing. The overhead is a little annoying but being able to use sentry is worth the effort over the firebase cloud function logging (not great).

@Dinduks wrap is still there, see: https://github.com/getsentry/sentry-javascript/blob/master/packages/browser/src/exports.ts#L40
However, keep in mind that it executes the function immediately and gives you the return value back. So I'm not sure whether that's any more helpful than regular try/catch which allows you to do some user fallback action as well.

const myHandler = (req, res) => Sentry.wrap(() => {
  someFunctionThatCanBreak(req);
  return res.send(200);
});

I don't think it's a good idea, but I've created a wrapper that looks like this.

import * as Sentry from "@sentry/node";

export const sentryWrapper = (f) => {
  return async function () {
    try {
      // eslint-disable-next-line prefer-rest-params
      return await f.apply(this, arguments);
    } catch (e) {
      Sentry.captureException(e);
      await Sentry.flush(2000);
      throw new Error(e);
    }
  };
};

It will be used as follows.

export const getChannelVideoTest = functions.https.onRequest(
  sentryWrapper(async (req, res) => {
    someFunctionThatCanBreak(req);
    return res.send(200);
  }),
);

I'd like to know if there is a better way.

@kamilogorek
I'm struggling with this as well.

I've successfully Sentry.init() and I've got my issues when I wrap all my code in a try {} catch {} statement and manual call to Sentry.captureException and Sentry.flush().
However, I'm not able to get anything reported if I remove the try/catch statement.
Same goes for the performance monitoring where I'm not getting anything unless I manually create a transaction with Sentry.startTransaction() at the beginning of the function.

Is this expected?
Is there any way to get unhandled errors sent to Sentry?
If not, does this mean the performance tab will always set the failure rate at 0% ? (because we catch the error and manually report it, the transaction is being closed properly, thus has a ok status ?)

@axelvaindal we don't support performance monitoring for serverless just yet. As for this question:

Is there any way to get unhandled errors sent to Sentry?

Then no, not really, as GCF doesn't provide a way to hook into unhandled exceptions/rejections, so we are not able to intercept that. You need to wrap your handlers (see the comment above yours) in order to manually catch it.

You can also read our AWSLambda handler implementation to get some ideas how to improve on that snippet - https://github.com/getsentry/sentry-javascript/blob/master/packages/serverless/src/awslambda.ts