Skip to content

Add socketKeepAlive configuration property for Elasticsearch #32051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

puppylpg
Copy link
Contributor

Keepalive property for the connection between client and elasticsearch should be considered as configurable. Futher more, i do think it's good to keep it alive by default.

Here is the issue:

When read/write elasticsearch in high concurrency, threads have chances to stuck on obsolete connection between client and elasticsearch. The more threads involves, the quicker stucking on connection happens. The reason is described here:

Worse, if the connection is dropped silently while a request is in flight then the client may block indefinitely since it will send no more data on this connection and will therefore never find out that it's been closed.

And here is the situation i run into. Logs can also be found here:

similar issue:

However, spring boot can't config keepalive through current properties. We have to write some config codes like:

    @Bean
    public RestClientBuilderCustomizer keepalive() {
        return new RestClientBuilderCustomizer() {
            @Override
            public void customize(RestClientBuilder builder) {

            }

            @Override
            public void customize(HttpAsyncClientBuilder builder) {
                builder.setDefaultIOReactorConfig(IOReactorConfig.custom().setSoKeepAlive(true).build());
                log.info("es tcp keep-alive must be enabled!");
            }
        };
    }

So i add some support for spring.elasticsearch.keepalive=true to make it easier to config.

@pivotal-cla
Copy link

@puppylpg Please sign the Contributor License Agreement!

Click here to manually synchronize the status of this Pull Request.

See the FAQ for frequently asked questions.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Aug 10, 2022
@pivotal-cla
Copy link

@puppylpg Thank you for signing the Contributor License Agreement!

Copy link
Member

@wilkinsona wilkinsona left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the proposal, @puppylpg. I've left a couple of comments for you consideration. In addition to those, it would be good to have a test that verifies that our default for the new property aligns with Elasticsearch's default. If the Elastic default is changed in the future, this will ensure that we notice and can update our default to match.

@wilkinsona wilkinsona added the status: waiting-for-feedback We need additional information before we can continue label Aug 10, 2022
Copy link
Contributor Author

@puppylpg puppylpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved

Copy link
Member

@wilkinsona wilkinsona left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. I've left a comment about the testing of Elastic's default.

@puppylpg puppylpg requested a review from wilkinsona August 10, 2022 16:28
@snicoll snicoll changed the title Add keepalive configure for elasticsearch autoconfig. Add socketKeepAlive configuration poperty for Elasticsearch Aug 17, 2022
@snicoll snicoll changed the title Add socketKeepAlive configuration poperty for Elasticsearch Add socketKeepAlive configuration property for Elasticsearch Aug 17, 2022
@snicoll snicoll self-assigned this Aug 17, 2022
@snicoll snicoll added type: enhancement A general enhancement and removed status: waiting-for-feedback We need additional information before we can continue status: waiting-for-triage An issue we've not yet triaged labels Aug 17, 2022
@snicoll snicoll added this to the 3.0.0-M5 milestone Aug 17, 2022
@snicoll snicoll closed this in bb07eec Aug 17, 2022
@snicoll
Copy link
Member

snicoll commented Aug 17, 2022

@puppylpg thank you for making your first contribution to Spring Boot.

@puppylpg
Copy link
Contributor Author

puppylpg commented Sep 1, 2022

Will this mr be merged into 2.x.x besides 3.0.0-M5?

@wilkinsona
Copy link
Member

Sorry, it won't. It's an enhancement so it only belongs in a new major or minor release. We don't have any plans at the moment for a 2.8 release so 3.0 is the next available release in which it can be included.

@puppylpg
Copy link
Contributor Author

puppylpg commented Sep 1, 2022

ok, thanks for your reply (sad, for still using jdk8 in our production environment 😅😅😅)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants