feat(trace_to_log transform): introduces transform that converts traces to logs #22287
Conversation
|
|
github-actions
bot
added
domain: transforms
| fn schema_definition(log_namespace: LogNamespace) -> Definition { | ||
| let mut schema_definition = Definition::default_for_namespace(&BTreeSet::from([log_namespace])); | ||
|
|
||
| match log_namespace { | ||
| LogNamespace::Vector => { | ||
| schema_definition = schema_definition.with_event_field( | ||
| &owned_value_path!("timestamp"), | ||
| Kind::bytes().or_undefined(), | ||
| None, | ||
| ); | ||
|
|
||
| schema_definition = schema_definition.with_metadata_field( | ||
| &owned_value_path!("vector"), | ||
| Kind::object(Collection::empty()), | ||
| None, | ||
| ); | ||
| } | ||
| LogNamespace::Legacy => { | ||
| if let Some(timestamp_key) = log_schema().timestamp_key() { | ||
| schema_definition = | ||
| schema_definition.with_event_field(timestamp_key, Kind::timestamp(), None); | ||
| } | ||
| } | ||
| } | ||
| schema_definition | ||
| } |
There was a problem hiding this comment.
How do we decide which attributes are available in a trace and how do we map them to a log?
This is related to #20170. IMO this is a simplistic view but we can document here that this is just a naive implementation and once we have a finalized trace model we need to revisit.
| fn transform(&mut self, output: &mut OutputBuffer, event: Event) { | ||
| let log = match event { | ||
| Event::Trace(trace) => LogEvent::from(trace), | ||
| _ => return, | ||
| }; | ||
| output.push(Event::Log(log)); | ||
| } |
There was a problem hiding this comment.
| fn transform(&mut self, output: &mut OutputBuffer, event: Event) { | |
| let log = match event { | |
| Event::Trace(trace) => LogEvent::from(trace), | |
| _ => return, | |
| }; | |
| output.push(Event::Log(log)); | |
| } | |
| impl FunctionTransform for TraceToLog { | |
| fn transform(&mut self, output: &mut OutputBuffer, event: Event) { | |
| if let Event::Trace(trace) = event { | |
| output.push(Event::Log(LogEvent::from(trace))); | |
| } | |
| } | |
| } |
| } | ||
|
|
||
| impl InternalEvent for TraceToLogConversionError { | ||
| fn emit(self) { |
There was a problem hiding this comment.
Did you plan to emit this in fn transform()?
| } | ||
|
|
||
| #[tokio::test] | ||
| async fn transform_trace() { |
There was a problem hiding this comment.
Let's expand the tests. You can use let (actual_map, actual_metadata) = trace_event.into_parts(); and then assert for both.
|
Let's also add a new scope to |
pront
changed the title
pront
added
the
meta: awaiting author
|
Any update on this PR @huevosabio |
|
Thank you for working on this PR. I'm currently blocked on this issue and would love to help to speed up this PR submission and enable the feature. |
Hi @kusum18, you are very welcome to work on this PR (or create a new PR based on this one). There are a few open review comments from my previous review. You can address those and take it from there 🚀 |
|
Hi! Sorry, yea I don't have time to address the issues and I am currently just using vector at this commit for now. @satellite-no , @kusum18 or @pront if you want to take it from here in this PR or another, please do! |
pront
force-pushed
the
master
branch
4 times, most recently
from
1720078 to
ffe54be
Compare
Summary
Change Type
Is this a breaking change?
How did you test this PR?
Does this PR include user facing changes?
Checklist
make check-allis a good command to run locally. This check isdefined here. Some of these
checks might not be relevant to your PR. For Rust changes, at the very least you should run:
cargo fmt --allcargo clippy --workspace --all-targets -- -D warningscargo nextest run --workspace(alternatively, you can runcargo test --all)Cargo.lock), pleaserun
dd-rust-license-tool writeto regenerate the license inventory and commit the changes (if any). More details here.References