Skip to content

chore: make grpc compression configurable for store gateway and alertmanager client #4889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 20, 2022

Conversation

yeya24
Copy link
Contributor

@yeya24 yeya24 commented Sep 30, 2022

Signed-off-by: Ben Ye [email protected]

What this PR does:

Enable snappy compression for store gateway and alertmanager gRPC client.
Now I just make it a default, but we can go with flag, too.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@alanprot
Copy link
Member

alanprot commented Oct 1, 2022

I think this should be a config as in the other clients , no?

@yeya24
Copy link
Contributor Author

yeya24 commented Oct 3, 2022

// We prefer sane defaults instead of exposing further config options.

This seems like an old comment. I am okay with exposing this flag. @alvinlin123 @alanprot WDYT?

@alanprot
Copy link
Member

alanprot commented Oct 4, 2022

I think all the other clients expose this as a config..

It weird just this one hard code to snappy. I know in theory snappy should not use lots of resources but who knows...

@yeya24
Copy link
Contributor Author

yeya24 commented Oct 4, 2022

@alanprot If I am going to expose the compressor as a flag then I might need to expose other fields like max send receive size, etc. If I am going to use https://github.com/cortexproject/cortex/blob/master/pkg/util/grpcclient/grpcclient.go#L40, then seems the default value of grpc-max-recv-msg-size and grpc-max-send-msg-size for alertmanager client got changed. Are we okay with this? But for the store gateway client, that remain the same.

@pull-request-size pull-request-size bot added size/M and removed size/XS labels Oct 6, 2022
@yeya24 yeya24 changed the title chore: enable snappy grpc compression for store gateway and alertmanager client chore: make grpc compression configurable for store gateway and alertmanager client Oct 6, 2022
@yeya24
Copy link
Contributor Author

yeya24 commented Oct 6, 2022

I updated the code and exposed all grpc configurations as flags.
This also introduces a compatiblity issue for the flags.
The store gateway TLS enabled flag changes from querier.store-gateway-client.tls_enabled to querier.store-gateway-client.grpc-client-config.tls_enabled.
The AM client TLS enabled flag changes from alertmanager.alertmanager-client.tls_enabled to alertmanager.alertmanager-client.grpc-client-config.tls_enabled.

Copy link
Member

@friedrichg friedrichg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the TLS configuration is not part of the guarantees.
https://cortexmetrics.io/docs/configuration/v1guarantees/
We can change them as needed.

This looks good but needs a CHANGELOG entry and running make doc.

@alanprot
Copy link
Member

@yeya24 Thanks...

Can you fix the lint?

@pull-request-size pull-request-size bot added size/L and removed size/M labels Oct 16, 2022
@yeya24
Copy link
Contributor Author

yeya24 commented Oct 16, 2022

PR updated. PTAL again @alanprot @friedrichg

@alanprot
Copy link
Member

Thanks!!

I have 2 comments though.

  • Can we avoid this change to be a breaking change? Otherwise people will need to change their config when updating cortex.
  • Do we need all those extra configs? If for now we just wanna make the compression configurable can we add just that field? We are trying to make the cortex configuration less confusing and i guess adding lots of configs that customer would never use it not going in that direction.

WDYT?

@yeya24 yeya24 force-pushed the enable-grpc-compress branch from aa99eb3 to 2194731 Compare October 20, 2022 00:42
@pull-request-size pull-request-size bot added size/S and removed size/L labels Oct 20, 2022
@yeya24 yeya24 force-pushed the enable-grpc-compress branch from 2194731 to 7e0fed4 Compare October 20, 2022 00:45
@pull-request-size pull-request-size bot added size/M and removed size/S labels Oct 20, 2022
@yeya24
Copy link
Contributor Author

yeya24 commented Oct 20, 2022

@alanprot @friedrichg
Based on Alan's suggestion, I expose the flags for grpc compression only for now. PTAL

@alanprot alanprot merged commit 9a622ac into cortexproject:master Oct 20, 2022
@yeya24 yeya24 deleted the enable-grpc-compress branch October 20, 2022 02:24
t00350320 added a commit to t00350320/cortex that referenced this pull request Nov 25, 2022
…llowing to configure the size of the in-memory queue used before flushing chunks to the disk . cortexproject#4889
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants