Fluentd service start has error in the fluentd's log

[EricRyu945]

(check apply)

• read the contribution guideline
• (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.

Problem

When I used the flunetd, and start the service. The fluentd log has some error.
Here is the part of the log

Config

Here is my config for fluentd
<match rem.track.apache.access>
@type elasticsearch
@log_level debug
suppress_type_name true
logstash_format true
logstash_prefix apache.log
logstash_dateformat %Y.%m.%d
hosts host1:9200,host2:9200,host3:9200
utc_index false
<buffer>
@type file
path /var/log/fluent/buffer/td
chunk_limit_size 32m
queue_limit_length 64
flush_interval 60s
retry_max_times 18
retry_wait 1s
</buffer>
</match>

Using Fluentd and ES plugin versions

• OS version -> Apache (2.4.57) rocky 9
• fluentd -> 1.16.6
• fluent-plugin-elasticsearch -> 5.4.3
• elasticsearch -> 7.17.11
• fluent-plugin-elasticsearch-timestamp-check -> 0.3.1

[EricRyu945]

@cosmo0920 さん
お疲れ様です。
Could you help me find the reason and how can I fix it for this issue?

[daipom]

@EricRyu945
I'm not familiar with this plugin, but it appears that you can ignore those debug log messages.
They appear to be merely debugging messages that occur during the process of prechecking something (template??).

template関連の何かをチェックする過程で発生しているdebugメッセージに過ぎないので、無視してよさそうに見えます。
メッセージは「プレースホルダーにタグが含まれていない」というものですが、含めなくても良さそうに見えますし、実際に必要なプレースホルダーの設定が漏れていた場合は出力時などに正式な警告ログが発生するはずです。

[EricRyu945]

@daipom san

@EricRyu945 I'm not familiar with this plugin, but it appears that you can ignore those debug log messages. They appear to be merely debugging messages that occur during the process of prechecking something (template??).

template関連の何かをチェックする過程で発生しているdebugメッセージに過ぎないので、無視してよさそうに見えます。 メッセージは「プレースホルダーにタグが含まれていない」というものですが、含めなくても良さそうに見えますし、実際に必要なプレースホルダーの設定が漏れていた場合は出力時などに正式な警告ログが発生するはずです。

Thanks for your support.
Yes, I also think so. When I set the log_level to info. The error message will not display.
In my opinion, there is no any impact to my logic of collection logs and create the index to the ES on my product env.
What I want to confirm was is that correct?

[daipom]

In my opinion, there is no any impact to my logic of collection logs and create the index to the ES on my product env.
What I want to confirm was is that correct?

Yes.
Looking briefly at the use of placeholders, they do not appear to impact operation unless warning or error logs occur.

[EricRyu945]

In my opinion, there is no any impact to my logic of collection logs and create the index to the ES on my product env.
What I want to confirm was is that correct?

Yes. Looking briefly at the use of placeholders, they do not appear to impact operation unless warning or error logs occur.

Thank you very much.

BTW, Such as the error as below.
2025-01-09 14:22:45 +0900 [debug]: #0 'logstash_prefix apache.log' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: apache.log' doesn't have tag placeholder

When I was start the service. Check which part this error will happen.
Because my config has already setting the logstash_prefix apache.log I don't understand the real reason why the error was occur.

[cosmo0920]

This debug level messages are just printed for debugging purpose. Fluentd's template mechanism is only working for runtime.
I just added them for confirming how template system works. You can ignore those messages.

[cosmo0920]

In most cases, using debug level on this plugin is too verbose to get log messages. So, you should raise the log level to info or warn to reduce unimportant logs.

[EricRyu945]

@cosmo0920 san
Thank you very much for your response. It was very helpful to me.

[daipom]

@EricRyu945

BTW, Such as the error as below. 2025-01-09 14:22:45 +0900 [debug]: #0 'logstash_prefix apache.log' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: apache.log' doesn't have tag placeholder

When I was start the service. Check which part this error will happen. Because my config has already setting the logstash_prefix apache.log I don't understand the real reason why the error was occur.

I don't know the purpose, but there is a process to validate the existence of a tag placeholder.
You don't set the placeholder, so this message occurs.
(You don't need to set the placeholder.)

ref: placeholder : https://docs.fluentd.org/plugin-development/api-plugin-output#understanding-chunking-and-metadata

[EricRyu945]

@EricRyu945

BTW, Such as the error as below. 2025-01-09 14:22:45 +0900 [debug]: #0 'logstash_prefix apache.log' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: apache.log' doesn't have tag placeholder
When I was start the service. Check which part this error will happen. Because my config has already setting the logstash_prefix apache.log I don't understand the real reason why the error was occur.

I don't know the purpose, but there is a process to validate the existence of a tag placeholder. You don't set the placeholder, so this message occurs. (You don't need to set the placeholder.)

ref: placeholder : https://docs.fluentd.org/plugin-development/api-plugin-output#understanding-chunking-and-metadata

Thank you very much for your response.
This document was very useful for me. I can learn more detail from this.
And I think the response from you and @cosmo0920 san has resolve my concern.
Thanks again for your kindly support. I appreciate it。