Skip to content

Hostagent logs are too noisy #2577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
nirs opened this issue Sep 1, 2024 · 1 comment
Open

Hostagent logs are too noisy #2577

nirs opened this issue Sep 1, 2024 · 1 comment
Labels
area/cli limactl CLI user experience enhancement New feature or request

Comments

@nirs
Copy link
Member

nirs commented Sep 1, 2024 •
edited
Loading

Description

INFO level logs from "[hostagent"] contain many messages that could be debug level or duplicate messages.

Example log:

msg="[hostagent] Waiting for the essential requirement 1 of 2: \"ssh\""
msg="[hostagent] [VZ] - vm state change: running"
msg="[hostagent] Waiting for the essential requirement 1 of 2: \"ssh\""
msg="[hostagent] The essential requirement 1 of 2 is satisfied"
msg="[hostagent] Waiting for the essential requirement 2 of 2: \"user session is ready for ssh\""
msg="[hostagent] The essential requirement 2 of 2 is satisfied"
msg="[hostagent] Waiting for the optional requirement 1 of 6: \"systemd must be available\""
msg="[hostagent] Guest agent is running"
msg="[hostagent] Not forwarding TCP 127.0.0.53:53"
msg="[hostagent] Not forwarding TCP 127.0.0.54:53"
msg="[hostagent] The optional requirement 1 of 6 is satisfied"
msg="[hostagent] Not forwarding TCP [::]:22"
msg="[hostagent] Waiting for the optional requirement 2 of 6: \"containerd binaries to be installed\""
msg="[hostagent] Not forwarding TCP 127.0.0.1:41899"
msg="[hostagent] The optional requirement 2 of 6 is satisfied"
msg="[hostagent] Waiting for the optional requirement 3 of 6: \"kubeadm to be installed\""
msg="[hostagent] The optional requirement 3 of 6 is satisfied"
msg="[hostagent] Waiting for the optional requirement 4 of 6: \"kubeadm to be completed\""
msg="[hostagent] Not forwarding TCP 127.0.0.1:45865"
msg="[hostagent] Not forwarding TCP 127.0.0.1:10248"
msg="[hostagent] Not forwarding TCP [::]:10250"
msg="[hostagent] The optional requirement 4 of 6 is satisfied"
msg="[hostagent] Waiting for the optional requirement 5 of 6: \"kubernetes cluster to be running\""
msg="[hostagent] Not forwarding TCP 192.168.105.23:2380"
msg="[hostagent] Not forwarding TCP 192.168.105.23:2379"
msg="[hostagent] Not forwarding TCP 127.0.0.1:2379"
msg="[hostagent] Not forwarding TCP 127.0.0.1:2381"
msg="[hostagent] Not forwarding TCP 127.0.0.1:10257"
msg="[hostagent] Not forwarding TCP 127.0.0.1:10259"
msg="[hostagent] Not forwarding TCP [::]:6443"
msg="[hostagent] The optional requirement 5 of 6 is satisfied"
msg="[hostagent] Waiting for the optional requirement 6 of 6: \"coredns deployment to be running\""
msg="[hostagent] Not forwarding TCP 127.0.0.1:10249"
msg="[hostagent] Not forwarding TCP [::]:10256"
msg="[hostagent] The optional requirement 6 of 6 is satisfied"
msg="[hostagent] Waiting for the guest agent to be running"
msg="[hostagent] Waiting for the final requirement 1 of 1: \"boot scripts must have finished\""
msg="[hostagent] The final requirement 1 of 1 is satisfied"
msg="[hostagent] Copying config from /etc/kubernetes/admin.conf to /Users/nsoffer/.lima/cluster/copied-from-guest/kubeconfig.yaml"

Duplicate messages:

msg="[hostagent] Waiting for the essential requirement 1 of 2: \"ssh\""
msg="[hostagent] Waiting for the essential requirement 1 of 2: \"ssh\""

Debug level messages:

msg="[hostagent] Not forwarding TCP 127.0.0.53:53"
msg="[hostagent] Not forwarding TCP 127.0.0.54:53"
msg="[hostagent] Not forwarding TCP [::]:22"
msg="[hostagent] Not forwarding TCP 127.0.0.1:41899"
msg="[hostagent] Not forwarding TCP 127.0.0.1:45865"
msg="[hostagent] Not forwarding TCP 127.0.0.1:10248"
msg="[hostagent] Not forwarding TCP [::]:10250"
msg="[hostagent] Not forwarding TCP 192.168.105.23:2380"
msg="[hostagent] Not forwarding TCP 192.168.105.23:2379"
msg="[hostagent] Not forwarding TCP 127.0.0.1:2379"
msg="[hostagent] Not forwarding TCP 127.0.0.1:2381"
msg="[hostagent] Not forwarding TCP 127.0.0.1:10257"
msg="[hostagent] Not forwarding TCP 127.0.0.1:10259"
msg="[hostagent] Not forwarding TCP [::]:6443"
msg="[hostagent] Not forwarding TCP 127.0.0.1:10249"
msg="[hostagent] Not forwarding TCP [::]:10256"

Another logs that I see from time to time is:

msg="[hostagent] r.CreateEndpoint() = connection was refused"

I have working clusters (using complicated setup and running DR flows) so I guess this is not really an error and can reported as debug message, or maybe as a warning if this can help users to debug this issue.

If this is a real issue it need much more context in the error message (e.g. connection to which address? why the connection is needed?)
@jandubois
Copy link
Member

Another logs that I see from time to time is:

msg="[hostagent] r.CreateEndpoint() = connection was refused"

That is #1160.

@AkihiroSuda AkihiroSuda added enhancement New feature or request area/cli limactl CLI user experience labels Sep 2, 2024
nirs added a commit to nirs/lima that referenced this issue Sep 8, 2024
@nirs
Change "Not forwarding" messages to debug level
2bb0552 
Info level should be used only for important events or changes to the
system, for example when we forward a port from the guest. Not
forwarding a port can be helpful when debugging port forwarding, so
keeping it as debug message.

Part-of: lima-vm#2577
Signed-off-by: Nir Soffer <[email protected]>
@nirs nirs mentioned this issue Sep 8, 2024
Closed
nirs added a commit to nirs/lima that referenced this issue Sep 8, 2024
@nirs
Change "Not forwarding" messages to debug level
7019984 
Info level should be used only for important events or changes to the
system, for example when we forward a port from the guest. Not
forwarding a port can be helpful when debugging port forwarding, so
keeping it as debug message.

Part-of: lima-vm#2577
Signed-off-by: Nir Soffer <[email protected]>
nirs added a commit to nirs/ramen that referenced this issue Sep 22, 2024
@nirs
Simplify port forwarding rules
0f4b049 
We can ignore all ports for all protocols for all addresses using a
simpler rule.

With this change we see this log in --verbose mode:

    2024-09-23 01:12:18,130 DEBUG   [cluster] [hostagent] TCP (except for SSH) and UDP port forwarding is disabled

And no "Not forwarding port ..." message. Previously we had lot of these
messages[1]. This requires lima commit[2].  pull current master if you
run older version.

[1] lima-vm/lima#2577
[2] lima-vm/lima@9a09350

Signed-off-by: Nir Soffer <[email protected]>
nirs added a commit to nirs/ramen that referenced this issue Sep 23, 2024
@nirs
Simplify port forwarding rules
ac3e018 
We can ignore all ports for all protocols for all addresses using a
simpler rule.

With this change we see this log in --verbose mode:

    2024-09-23 01:12:18,130 DEBUG   [cluster] [hostagent] TCP (except for SSH) and UDP port forwarding is disabled

And no "Not forwarding port ..." message. Previously we had lot of these
messages[1]. This requires lima commit[2].  pull current master if you
run older version.

[1] lima-vm/lima#2577
[2] lima-vm/lima@9a09350

Signed-off-by: Nir Soffer <[email protected]>
nirs added a commit to nirs/ramen that referenced this issue Sep 30, 2024
@nirs
Simplify port forwarding rules
c6aa744 
We can ignore all ports for all protocols for all addresses using a
simpler rule.

With this change we see this log in --verbose mode:

    2024-09-23 01:12:18,130 DEBUG   [cluster] [hostagent] TCP (except for SSH) and UDP port forwarding is disabled

And no "Not forwarding port ..." message. Previously we had lot of these
messages[1]. This requires lima commit[2].  pull current master if you
run older version.

[1] lima-vm/lima#2577
[2] lima-vm/lima@9a09350

Signed-off-by: Nir Soffer <[email protected]>
raghavendra-talur pushed a commit to RamenDR/ramen that referenced this issue Oct 7, 2024
@nirs @raghavendra-talur
Simplify port forwarding rules
c5af216 
We can ignore all ports for all protocols for all addresses using a
simpler rule.

With this change we see this log in --verbose mode:

    2024-09-23 01:12:18,130 DEBUG   [cluster] [hostagent] TCP (except for SSH) and UDP port forwarding is disabled

And no "Not forwarding port ..." message. Previously we had lot of these
messages[1]. This requires lima commit[2].  pull current master if you
run older version.

[1] lima-vm/lima#2577
[2] lima-vm/lima@9a09350

Signed-off-by: Nir Soffer <[email protected]>
asn1809 pushed a commit to asn1809/ramen that referenced this issue Oct 8, 2024
@nirs @asn1809
Simplify port forwarding rules
3692dc5 
We can ignore all ports for all protocols for all addresses using a
simpler rule.

With this change we see this log in --verbose mode:

    2024-09-23 01:12:18,130 DEBUG   [cluster] [hostagent] TCP (except for SSH) and UDP port forwarding is disabled

And no "Not forwarding port ..." message. Previously we had lot of these
messages[1]. This requires lima commit[2].  pull current master if you
run older version.

[1] lima-vm/lima#2577
[2] lima-vm/lima@9a09350

Signed-off-by: Nir Soffer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/cli limactl CLI user experience enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jandubois @nirs @AkihiroSuda