Skip to content

unhandled promise crahing server with sentry middleware #6750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
modestaspruckus opened this issue Jan 12, 2023 · 13 comments
Closed
3 tasks done

unhandled promise crahing server with sentry middleware #6750

modestaspruckus opened this issue Jan 12, 2023 · 13 comments
Assignees
Labels
Package: node Issues related to the Sentry Node SDK

Comments

@modestaspruckus
Copy link

Is there an existing issue for this?

How do you use Sentry?

Sentry Saas (sentry.io)

Which package are you using?

@sentry/node

SDK Version

7.30.0

Framework Version

No response

Link to Sentry event

No response

Steps to Reproduce

Sentry request handler middlware is changing behaviour of unhandledPromise. Without this middleware below

app.use(Sentry.Handlers.requestHandler());

Error with unhandled promise is thrown with unhandledRejection event which doesn't force node instance to restart.

With middleware above the error is thrown with uncaughtException event which crashing the node server.

Expected Result

Sentry.Handlers.requestHandler() middleware should not change behavior of promise errors.

Actual Result

Sentry.Handlers.requestHandler() middleware is changing promise error behavior

@Lms24
Copy link
Member

Lms24 commented Jan 13, 2023

Hi @modestaspruckus thanks for writing in. Can you post your Sentry.init config? Are you adding or removing any integrations? The reason I'm asking is because by default, our OnUnhandledRejection integration should capture unhandled promise rejections but only emit a warning. If you set it to strict mode, it will exit the process.

@Lms24 Lms24 added Status: Needs Information Package: node Issues related to the Sentry Node SDK and removed Status: Untriaged labels Jan 13, 2023
@modestaspruckus
Copy link
Author

modestaspruckus commented Jan 15, 2023

Hey @Lms24, here is the config.

Sentry.init({
    dsn: process.env.SENTRY_DSN,
    environment: process.env.ENV_NAME || 'local',
    release: `backend-${process.env.ENV_NAME}@${process.env.BUILD_NUMBER}`,
});

I'm not using any integrations. Additionally, there is sentry error middleware

app.use(Sentry.Handlers.errorHandler());

@modestaspruckus
Copy link
Author

modestaspruckus commented Jan 18, 2023

Hey, any news for this? Various API calls sometimes fails due network issues and this causes node instances to restart. This wasn't the case on 7.6.0 version

This can simply be reproduced by rejecting promise

app.get('/test', async (req, res, next) => {
	const promise = new Promise((resolve, reject) => {
		reject(new Error('Something bad hapened'));
	});

	res.json();
});

By commenting out sentry middleware it starts working as expected

// app.use(Sentry.Handlers.requestHandler());

@Lms24
Copy link
Member

Lms24 commented Jan 18, 2023

@modestaspruckus no news yet. I quickly tried reproducing this and so far I couldn't. Here's what I did:

  1. yarn add express @sentry/node

  2. index.js:

const express = require("express");
const Sentry = require("@sentry/node");

const app = express();

Sentry.init({
  dsn: "___DSN___",
  environment: "local",
  release: `backend-test@$1`,
});

app.get("/test", async (req, res, next) => {
  const promise = new Promise((resolve, reject) => {
    reject(new Error("Something bad hapened"));
  });

  res.json();
});

// app.use(Sentry.Handlers.errorHandler());
app.listen(3000);
console.log("Listening on port 3000");
  1. curl localhost:3000/test

Regardless of using the errorHandler or not, the app doesn't crash for me.

Can you provide a full reproduction so that we can investigate this?

Oh, one more thing: The order in which you add middleware matters. According to our docs, you should add the errorhandler "before any other error middleware and after all controllers".

@modestaspruckus
Copy link
Author

modestaspruckus commented Jan 18, 2023

You need to add request middleware also

app.use(Sentry.Handlers.requestHandler());

@modestaspruckus
Copy link
Author

I've created repository to debug - https://github.com/modestaspruckus/sentry-debug

  1. run npm ci
  2. run node index.js
  3. Go to http://localhost:3000/test
  4. Node crashed.

Comment out app.use(Sentry.Handlers.requestHandler()); and do same steps again.
You will notice that node will not crash and will show error in console This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:

@Lms24
Copy link
Member

Lms24 commented Jan 18, 2023

Thanks for the repro (and my bad about the request handler, I misread your description, sorry). I can reproduce it now. Will try to find a fix.

@Lms24
Copy link
Member

Lms24 commented Jan 18, 2023

Update: It seems #5627 is the culprit here. Let's see how to fix this

@Lms24
Copy link
Member

Lms24 commented Jan 20, 2023

Hi @modestaspruckus, I thought a little about this and did some research: We can't revert #5627 as it would be a breaking change (I know ironic, as introducing it was also kind of an accidental breaking change - really sorry about that).

After some research, I think the problem is more related to the fact that your handler is async and express can't handle these errors. Hence, neither our OnUnhandledRejection integration nor our errorHandler catch this error at all and the process exits. The fact that this only happens when requestHandler is used is probably caused by us using domains in that handler.
So my advice here would be to try/catch async errors and use await as follows:

app.get("/test", async (req, res, next) => {
  try {
    const promise = new Promise((resolve, reject) => {
      reject(new Error("Something bad hapened"));
    });
    await promise;
  } catch (error) {
    next(error);
  }

  res.json();
});

More information about async handlers: https://zellwk.com/blog/async-await-express/
Also, this library helps abstracting the try/catch away: https://www.npmjs.com/package/express-async-handler

Hope this helps. Let me know if this issue is resolved or you have additional questions :)

@modestaspruckus
Copy link
Author

Thanks @Lms24. Adding try/catch is a solution.

I think the problem is more related to the fact that your handler is async and express can't handle these errors - your state here is incorrect because node.js emits event unhandledRejection when error occurs on unhandled promise. This behavior is changed with sentry middleware, instead of emitting unhandledRejection event, it emits uncaughtException

https://github.com/modestaspruckus/sentry-debug/blob/main/index.js#L7

@KFoxder
Copy link

KFoxder commented Feb 7, 2023

We are running into the same issue preventing us from being able to upgrade to the latest sentry. We will see if this works for us as well...

@jcasner
Copy link

jcasner commented Feb 10, 2023

We're running into this also. Unhandled async errors work fine when using Sentry 7.11.1, but start breaking in 7.12.0 when running express 4.18.2.

  • Upgrading to Express 5 beta "fixes" this; we're not ready to be on the beta channel for express
  • Using an error wrapper, as described in this SO post, works well https://stackoverflow.com/a/49417798

We're going to stick with Sentry 7.11.1 until we get things fixed up. Ultimately, this seems to come down to an anti-pattern of allowing unhandled async errors / promise rejections in Express. According to the express documentation:

For errors returned from asynchronous functions invoked by route handlers and middleware, you must pass them to the next() function, where Express will catch and process them.

It seems like something in older versions of Sentry actually captures these unhandled promise rejections and masks the true problem.

@Lms24
Copy link
Member

Lms24 commented Feb 13, 2023

It seems like something in older versions of Sentry actually captures these unhandled promise rejections and masks the true problem.

Yes, after researching and discussing this internally, we came to the same conclusion. I'm going to close this issue, as it seems most users here were able to resolve this with properly handling async errors. Everyone, feel free to leave are reply if things are still unclear.

@Lms24 Lms24 closed this as completed Feb 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Package: node Issues related to the Sentry Node SDK
Projects
None yet
Development

No branches or pull requests

4 participants