Skip to content

SQL: concurrent access to RestSqlQueryAction is not thread safe #69866

Closed
@astefan

Description

@astefan

RestSqlQueryAction has a TextFormat variable defined at instance level. This one can be overriden by two threads running in parallel servicing REST requests with prepareRequest calls where the textFormat value is changed.

This means that one call looking like /_sql?format=txt (textFormat here gets the value TXT) and another like /_sql (textFormat here remains null since xContentType gets a value of JSON) running at the same time makes the textFormat for the first call to be set to null and an exception like the following is getting logged and the call fails:

[WARN ][r.suppressed             ] [runTask-0] path: /_sql, params: {format=txt}
java.lang.NullPointerException: Cannot invoke "org.elasticsearch.xpack.sql.plugin.TextFormat.format(org.elasticsearch.rest.RestRequest, org.elasticsearch.xpack.sql.action.SqlQueryResponse)" because "this.this$0.textFormat" is null
        at org.elasticsearch.xpack.sql.plugin.RestSqlQueryAction$1.buildResponse(RestSqlQueryAction.java:130) ~[?:?]
        at org.elasticsearch.xpack.sql.plugin.RestSqlQueryAction$1.buildResponse(RestSqlQueryAction.java:117) ~[?:?]
        at org.elasticsearch.rest.action.RestResponseListener.processResponse(RestResponseListener.java:26) ~[elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.rest.action.RestActionListener.onResponse(RestActionListener.java:36) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:83) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:77) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.xpack.sql.plugin.TransportSqlQueryAction.lambda$operation$0(TransportSqlQueryAction.java:97) [x-pack-sql-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:134) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:134) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:134) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.xpack.sql.execution.search.CompositeAggCursor.handle(CompositeAggCursor.java:196) [x-pack-sql-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.xpack.sql.execution.search.Querier$CompositeActionListener.handleResponse(Querier.java:389) [x-pack-sql-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.xpack.sql.execution.search.Querier$BaseActionListener.onResponse(Querier.java:558) [x-pack-sql-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.xpack.sql.execution.search.Querier$BaseActionListener.onResponse(Querier.java:532) [x-pack-sql-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:83) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:77) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.ActionListener$RunAfterActionListener.onResponse(ActionListener.java:313) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.sendSearchResponse(AbstractSearchAsyncAction.java:630) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.ExpandSearchPhase.run(ExpandSearchPhase.java:109) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:397) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:391) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.FetchSearchPhase.moveToNextPhase(FetchSearchPhase.java:219) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.FetchSearchPhase.lambda$innerRun$1(FetchSearchPhase.java:101) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.FetchSearchPhase.innerRun(FetchSearchPhase.java:107) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.FetchSearchPhase.access$000(FetchSearchPhase.java:36) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.action.search.FetchSearchPhase$1.doRun(FetchSearchPhase.java:84) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.13.0-SNAPSHOT.jar:7.13.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
        at java.lang.Thread.run(Thread.java:832) [?:?]

Also, at the time this bug report was created, the code in master and the one in 7.x differ significantly when it comes to the parts relevant for this bug, so a fix should address 7.x code only.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions