Skip to content

chore(logger): clear prev request buffers in manual mode #6314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions aws_lambda_powertools/logging/logger.py
Original file line number Diff line number Diff line change
Expand Up @@ -1127,6 +1127,12 @@ def _add_log_record_to_buffer(
tracer_id = get_tracer_id()

if tracer_id and self._buffer_config:
if not self._buffer_cache.get(tracer_id):
# Detect new Lambda invocation context and reset buffer to maintain log isolation
# Ensures logs from previous invocations do not leak into current execution
# Prevent memory excessive usage
self._buffer_cache.clear()

log_record: dict[str, Any] = _create_buffer_record(
level=level,
msg=msg,
Expand Down
11 changes: 7 additions & 4 deletions docs/core/logger.md
Original file line number Diff line number Diff line change
Expand Up @@ -681,13 +681,16 @@ sequenceDiagram

4. **What happens if the log buffer reaches its maximum size?** Older logs are removed from the buffer to make room for new logs. This means that if the buffer is full, you may lose some logs if they are not flushed before the buffer reaches its maximum size. When this happens, we emit a warning when flushing the buffer to indicate that some logs have been dropped.

5. **What timestamp is used when I flush the logs?** The timestamp preserves the original time when the log record was created. If you create a log record at 11:00:10 and flush it at 11:00:25, the log line will retain its original timestamp of 11:00:10.
5. **How is the log size of a log line calculated?**
The log size is calculated based on the size of the log line in bytes. This includes the size of the log message, any exception (if present), the log line location, additional keys, and the timestamp.

6. **What happens if I try to add a log line that is bigger than max buffer size?** The log will be emitted directly to standard output and not buffered. When this happens, we emit a warning to indicate that the log line was too big to be buffered.
6. **What timestamp is used when I flush the logs?** The timestamp preserves the original time when the log record was created. If you create a log record at 11:00:10 and flush it at 11:00:25, the log line will retain its original timestamp of 11:00:10.

7. **What happens if Lambda times out without flushing the buffer?** Logs that are still in the buffer will be lost.
7. **What happens if I try to add a log line that is bigger than max buffer size?** The log will be emitted directly to standard output and not buffered. When this happens, we emit a warning to indicate that the log line was too big to be buffered.

8. **Do child loggers inherit the buffer?** No, child loggers do not inherit the buffer from their parent logger but only the buffer configuration. This means that if you create a child logger, it will have its own buffer and will not share the buffer with the parent logger.
8. **What happens if Lambda times out without flushing the buffer?** Logs that are still in the buffer will be lost.

9. **Do child loggers inherit the buffer?** No, child loggers do not inherit the buffer from their parent logger but only the buffer configuration. This means that if you create a child logger, it will have its own buffer and will not share the buffer with the parent logger.

### Built-in Correlation ID expressions

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -470,7 +470,12 @@ def test_buffer_configuration_propagation_across_child_logger_instances(stdout,
assert primary_logger._buffer_cache != secondary_logger._buffer_cache


def test_logger_buffer_is_cleared_between_lambda_invocations(stdout, service_name, monkeypatch, lambda_context):
def test_logger_buffer_is_cleared_between_lambda_invocations_with_decorator(
stdout,
service_name,
monkeypatch,
lambda_context,
):
# Set initial trace ID for first Lambda invocation
monkeypatch.setenv(constants.XRAY_TRACE_ID_ENV, "1-67c39786-5908a82a246fb67f3089263f")

Expand All @@ -491,3 +496,30 @@ def handler(event, context):

# THEN Verify buffer for the original trace ID is cleared
assert not logger._buffer_cache.get("1-67c39786-5908a82a246fb67f3089263f")


def test_logger_buffer_is_cleared_between_lambda_invocations_without_decoration(
stdout,
service_name,
monkeypatch,
lambda_context,
):
# Set initial trace ID for first Lambda invocation
monkeypatch.setenv(constants.XRAY_TRACE_ID_ENV, "1-67c39786-5908a82a246fb67f3089263f")

# GIVEN A logger configured with specific buffer parameters
logger_buffer_config = LoggerBufferConfig(max_bytes=10240)
logger = Logger(level="DEBUG", service=service_name, stream=stdout, buffer_config=logger_buffer_config)

def handler(event, context):
logger.debug("debug line")

# WHEN First Lambda invocation with initial trace ID
handler({}, lambda_context)

# WHEN New Lambda invocation arrives with different trace ID
monkeypatch.setenv(constants.XRAY_TRACE_ID_ENV, "2-ABC39786-5908a82a246fb67f3089263f")
handler({}, lambda_context)

# THEN Verify buffer for the original trace ID is cleared
assert not logger._buffer_cache.get("1-67c39786-5908a82a246fb67f3089263f")
Loading