Spec service.environment for logs

specs/agents/log-correlation.md

@@ -9,7 +9,33 @@ By adding trace context to log records, users will be able to move between the A
 Logging frameworks and libraries may provide an MDC (Message Diagnostic Context) that allow to also
 reuse those fields in log message formats (for example in plain text).
 
-Log correlation is implemented by adding the following fields to a log document:
+Log correlation relies on two sets of fields:
+- [metadata fields](#metadata-fields)
+  - They allow to build the per-service logs view in UI.
+  - They are implicitly provided when using log-sending by the agent metadata.
+  - When using ECS logging, they might be set by the application.
+- [per-log-event fields](#per-log-event-fields): `trace.id`, `transaction.id` and `error.id`
+  - They allow to build the per-trace/transaction/error logs view in UI.
+  - They are added to the MDC
+  - They must be written in each log event document
+
+The values for those fields can be set in two places:
+- when using [ecs-logging](https://github.com/elastic/ecs-logging) directly in the application
+- when the agent reformats a log event
+
+The values set at the application level have higher priority than the values set by agents.
+Agents must provide fallback values if they are not explicitly set by the application.
+
+In case the values set in the application and agent configuration differ, the resulting log
+messages won't correlate to the expected service in UI. In order to prevent such inconsistencies
+agents may issue a warning when there is a mis-configuration.
+
+### Metadata fields
+
+They allow to build the per-service logs view in UI.
+They are implicitly provided when using log-sending by the agent metadata.
+When using ECS logging, they might be set by the application in ECS logging configuration.
+
 - `service.name`:
   - used to filter/link log messages to a given service.
   - must be provided even if there is no active transaction
@@ -19,27 +45,18 @@ Log correlation is implemented by adding the following fields to a log document:
 - `service.environment`:
   - allows to filter/link log messages to a given service/environment.
   - must be provided even if there is no active transaction
-- `trace.id` and `transaction.id`
-  - allows to correlate with the APM trace/transaction.
-  - should be added to the MDC
-- `error.id`:
-  - allows to correlate with an Error.
-  - should be added to the MDC
 
 In addition, the `container.id` can be used as a fallback when `service.name` is not avaiable on the log documents.
 However, the APM agents are not expected to set it. It is expected to be set by filebeat when ingesting log
 documents through auto-discover feature (which captures logs from containers and provides the value).
 
-The values for those fields can be set in two places:
-- when using [ecs-logging](https://github.com/elastic/ecs-logging) directly in the application
-- when the agent reformats a log event
-
-The values set at the application level have higher priority than the values set by agents.
-Agents must provide fallback values if they are not explicitly set by the application.
-
-In case the values set in the application and agent configuration differ, the resulting log
-messages won't correlate to the expected service in UI. In order to prevent such inconsistencies
-agents may issue a warning when there is a mis-configuration.
+### Per log event fields
 
-See also the [log-reformatting](log-reformatting.md) spec.
+They allow to build the per-trace/transaction/error logs view in UI.
+They allow to navigate from the log event to the trace/transaction/error in UI.
+They should added to the MDC.
+They must be written in each log event document they relate to, either reformatted or sent by the agent.
 
+- `trace.id`
+- `transaction.id`
+- `error.id`

specs/agents/logging.md

@@ -43,7 +43,7 @@ Also, the documentation should not mention the old log levels going forward.
 
 Agents may provide the following log-related features:
 
-- [Log correlation](log-correlation.md): inject current service/trace/transaction/error ID into logs.
+- [Log correlation](log-correlation.md): inject service metadata, trace IDs and error IDs in log events.
 - [Log reformatting](log-reformatting.md): reformat plain-text logs to ECS, equivalent to using [ecs logging](https://github.com/elastic/ecs-logging) 
 without modifying the application nor its dependencies.
 - [Log sending](log-sending.md): send logs directly to APM server.