AWS Lambda failing on Sentry failures

[mariomnts]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which package are you using?

@sentry/serverless

SDK Version

50 (Lambda layer eu-west-1)

Framework Version

No response

Link to Sentry event

No response

Steps to Reproduce

1. Enable AWS Lambda Sentry layer on a AWS Lambda function.
2. When sending an event to Sentry this fails with an error (for example, could be 429 because we ran out of Sentry tracing quota).
3. Lambda is marked as failed with and invoke error.

Expected Result

Lambda is NOT marked as failed or we can configure it we want it to be marked as failed or not like we can do when using the AWS.wrapHandler option in the library.

Actual Result

---

[mariomnts]

I brought this in this closed issue but it's still with no response: #4582 (comment)

[lforst]

I recently snooped around and found a potential issue of unhandled rejections when receiving a quota response while flushing the promise buffer: #4809 (comment)

This might be related because the AWS Lambda SDK's ignoreSentryErrors option only ignores SentryErrors and the errors that bubble up from flush should be native errors. @AbhiPrasad wdyt?

[mariomnts]

This might be related because the AWS Lambda SDK's ignoreSentryErrors option only ignores SentryErrors and the errors that bubble up from flush should be native errors. @AbhiPrasad wdyt?

Adding more to this, this option is only available at that wrapper level, but no in the Lambda Layer integration. I would expect something like SENTRY_IGNORE_ERRORS environment variable that I can use to configure it.

[merrickfox]

I'm also experiencing this issue, ignoreSentryErrors is set to true, my business logic executes fine with no errors, then it seems like Sentry tries to send contextual/breadcrumb data, because I've exceeded my quota the SDK gets HTTP 429 and then doesn't seem to handle it correctly. Resulting in an invoke error.

[smeubank]

maybe my mind is playing tricks on me but did we have a similar issue come up already @sl0thentr0py or @antonpirker ?

[bmatto]

I'm seeing something similar, though the Sentry error is a 404 and sometimes a 403 - even though when I throw inside my function it is being reported to Sentry. Also ignoreSentryErrors does not seem to help me.

[jenni]

Same as above, getting a 403 from Sentry but the events are posted to Sentry.

The Lambda invocation fails with the Sentry error instead of the reported error, and the CloudWatch logs show the Sentry error.

Lambda error and CloudWatch logs:

{
    "errorType": "SentryError",
    "errorMessage": "HTTP Error (403)",
    "name": "SentryError",
    "stack": [
        "SentryError: HTTP Error (403)",
        "    at new SentryError (/var/task/node_modules/@sentry/utils/dist/error.js:9:28)",
        "    at ClientRequest.<anonymous> (/var/task/node_modules/@sentry/node/dist/transports/base/index.js:212:44)",
        "    at Object.onceWrapper (events.js:520:26)",
        "    at ClientRequest.emit (events.js:400:28)",
        "    at ClientRequest.emit (domain.js:475:12)",
        "    at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:647:27)",
        "    at HTTPParser.parserOnHeadersComplete (_http_common.js:127:17)",
        "    at TLSSocket.socketOnData (_http_client.js:515:22)",
        "    at TLSSocket.emit (events.js:400:28)",
        "    at TLSSocket.emit (domain.js:475:12)"
    ]
}

ignoreSentryErrors: true did solve the issue in my case, now the Lambda fails with the expected error, and the error is also reported to Sentry:

{
  "errorType": "Error",
  "errorMessage": "[ jen ] another one.",
  "trace": [
    "Error: [ jen ] another one: ignoreSentryErrors: true .",
    "    at exports.handler.Sentry.AWSLambda.wrapHandler.ignoreSentryErrors (/var/task/index.js:15:11)",
    "    at Runtime.handler (/var/task/node_modules/@sentry/serverless/dist/awslambda.js:219:24)",
    "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
  ]
}

using "@sentry/serverless": "^6.19.7"

[GuilhermeMReis]

Sentry is currently in an ongoing outage which is throwing 502s. Unfortunately our lambdas are throwing 502s too. I would try/catch it if I could but Sentry SDK has abstracted that.

[lforst]

Hi everyone, we just merged #5090 which will hopefully get rid of this problem entirely.

The SDK should not throw anymore when transmitting errors to Sentry fails. In case of an error, the SDK will simply log the error (if you set debug: true in the Sentry.init() options). This change also made the ignoreSentryErrors option redundant so we removed it.

When will this be released? We are currently amidst preparing for the v7 release of the SDK - meaning this change will become available with the next beta or release candidate version of v7. Additionally (and unfortunately), we are currently rethinking how we publish our lambda layers (due to the nature of lambda layer versioning) - that might add a bit to the release timeline of this change. We will keep you posted.

[github-actions]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀