TASK_LOCK is not cleared when using single-instance-enabled option. #640

cow630 · 2019-12-13T20:48:03Z

I have followed the steps in the documentation to implement the single-instance-enabled option for my Spring Cloud Task Application.

I added the following to my application.yml
single-instance-enabled: true

I added the following dependencies to my pom.xml

org.springframework.integration
spring-integration-core

	<dependency>
		<groupId>org.springframework.integration</groupId>
		<artifactId>spring-integration-jdbc</artifactId>
	</dependency>

Behavior:
The first task execution will run, record an entry in the task_lock table, task will complete normally. The record is not removed from task_lock even when the task has completed.

The second task execution will fail, because task_lock record still exists from the first run.

The text was updated successfully, but these errors were encountered:

cppwfs · 2019-12-13T21:07:42Z

What version of Task are you using.
What database and version are you using.

cow630 · 2019-12-13T21:35:00Z

Hello, please see below for versions.

spring-cloud-task-core 2.1.3.RELEASE
spring-cloud-starter-task 2.1.3.RELEASE
Oracle 12.1.0.2.0

cppwfs · 2019-12-13T22:41:46Z

This looks to be an issue for Oracle databases. I've isolated it to Spring Integration and will talk with them and verify. An addition issue was discussed in Data Flow here: spring-attic/spring-cloud-dataflow#3631

cow630 · 2019-12-13T23:01:55Z

Very good. Thanks for the update.

cppwfs · 2019-12-18T23:19:51Z

The issue seems to be with DefaultLockRepository in SI. Created issue spring-projects/spring-integration#3129

cppwfs · 2019-12-19T22:36:02Z

We've identified the issue and will submit a PR to resolve it. But in the meantime you can change the LOCK_KEY and CLIENT_ID from CHAR2 to VARCHAR2.
You can alter the table TASK_LOCK table using the following commands:

ALTER TABLE TASK_LOCK MODIFY LOCK_KEY VARCHAR2(36);
ALTER TABLE TASK_LOCK MODIFY CLIENT_ID VARCHAR2(36);

or Drop the table and recreate it with the DDL below:

CREATE TABLE TASK_LOCK  (
	LOCK_KEY VARCHAR2(36) NOT NULL,
	REGION VARCHAR2(100) NOT NULL,
	CLIENT_ID VARCHAR2(36),
	CREATED_DATE TIMESTAMP NOT NULL,
	constraint LOCK_PK primary key (LOCK_KEY, REGION)
);

cow630 · 2019-12-20T16:30:21Z

Fantastic. This is great. I've tested the workaround in my development and it is working fine.

So an official fix will be provided in future release?

cppwfs · 2019-12-20T16:45:40Z

Yes, the schemas will be updated in the 2.3 release of Task as well as the next maintenance release of 2.2.x.

chenmaolin928 · 2024-04-15T07:00:53Z

I have encountered the same issue. However, I am using MariaDB version 3.1.4. When I run task A successfully once and then try to run task A again, it reports that the task is already running. I even modified the LOCK_KEY and CLIENT_ID columns in the task_lock table from char(36) to varchar(36), but the situation did not improve. Could you please advise me on how to resolve this?

ilopezv · 2024-12-09T10:05:56Z

Same problem here.

It seems that Spring close context without waiting to spring task to complete the task_lock deletion.

We try following options without success:

spring:
  cloud:
    task:
      single-instance-enabled: true
      closecontext-enabled: true
  task:
    execution:
      shutdown:
        await-termination: true
        await-termination-period: 30s
    scheduling:
      shutdown:
        await-termination: true
        await-termination-period: 30s

And configure TaskExecutor with

@Configuration(proxyBeanMethods = false)
public class AsyncConfiguration implements TaskExecutorCustomizer {

 @Override
 public void customize(ThreadPoolTaskExecutor taskExecutor) {
	log.trace("Customization of ThreadPoolTaskExecutor");
	taskExecutor.setWaitForTasksToCompleteOnShutdown(true);
	
 }
}

2024-12-05 10:03:35.670  INFO 1 --- [ck-leadership-1] oracle.jdbc                              : null
 
java.sql.SQLRecoverableException: ORA-18730: Error de E/S interrumpida.: Socket read interrupted
https://docs.oracle.com/error-help/db/ora-18730/
        at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1460) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.OracleStatement.executeSQLStatement(OracleStatement.java:1958) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1594) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3754) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.OraclePreparedStatement.executeLargeUpdate(OraclePreparedStatement.java:4100) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:4071) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:997) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) ~[HikariCP-4.0.3.jar:na]
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) ~[HikariCP-4.0.3.jar:na]
        at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:967) ~[spring-jdbc-5.3.31.jar:5.3.31]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:650) ~[spring-jdbc-5.3.31.jar:5.3.31]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:962) ~[spring-jdbc-5.3.31.jar:5.3.31]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1017) ~[spring-jdbc-5.3.31.jar:5.3.31]
        at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1027) ~[spring-jdbc-5.3.31.jar:5.3.31]
        at org.springframework.integration.jdbc.lock.DefaultLockRepository.acquire(DefaultLockRepository.java:168) ~[spring-integration-jdbc-5.5.20.jar:5.5.20]
        at org.springframework.integration.jdbc.lock.JdbcLockRegistry$JdbcLock.doLock(JdbcLockRegistry.java:268) ~[spring-integration-jdbc-5.5.20.jar:5.5.20]
        at org.springframework.integration.jdbc.lock.JdbcLockRegistry$JdbcLock.tryLock(JdbcLockRegistry.java:249) ~[spring-integration-jdbc-5.5.20.jar:5.5.20]
        at org.springframework.integration.support.leader.LockRegistryLeaderInitiator$LeaderSelector.tryAcquireLock(LockRegistryLeaderInitiator.java:388) ~[spring-integration-core-5.5.20.jar:5.5.20]
        at org.springframework.integration.support.leader.LockRegistryLeaderInitiator$LeaderSelector.call(LockRegistryLeaderInitiator.java:357) ~[spring-integration-core-5.5.20.jar:5.5.20]
        at org.springframework.integration.support.leader.LockRegistryLeaderInitiator$LeaderSelector.call(LockRegistryLeaderInitiator.java:337) ~[spring-integration-core-5.5.20.jar:5.5.20]
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
        at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
Caused by: java.io.InterruptedIOException: Socket read interrupted
        at oracle.net.nt.TimeoutSocketChannel.doSelect(TimeoutSocketChannel.java:706) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.nt.TimeoutSocketChannel.doSelectForRead(TimeoutSocketChannel.java:599) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.nt.TimeoutSocketChannel.read(TimeoutSocketChannel.java:501) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.ns.NSProtocolNIO.doSocketRead(NSProtocolNIO.java:1228) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.ns.NIOPacket.readHeader(NIOPacket.java:271) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.ns.NIOPacket.readPacketFromSocketChannel(NIOPacket.java:204) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:147) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:120) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.net.ns.NIONSDataChannel.readDataFromSocketChannel(NIONSDataChannel.java:105) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.T4CMAREngineNIO.prepareForUnmarshall(T4CMAREngineNIO.java:893) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:464) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:949) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:456) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:498) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:272) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1451) ~[ojdbc11-23.3.0.23.09.jar:23.3.0.23.09]
        ... 23 common frames omitted

cppwfs mentioned this issue Dec 13, 2019

Restricting concurrent task execution on Spring Cloud Data Flow spring-attic/spring-cloud-dataflow#3631

Closed

cppwfs self-assigned this Dec 13, 2019

sabbyanandan added this to the 2.3.0.M1 milestone Dec 18, 2019

sabbyanandan added the backport-2.2.x label Dec 18, 2019

cppwfs mentioned this issue Dec 18, 2019

DefaultLockRepository fails to acquire lock on Oracle. spring-projects/spring-integration#3129

Closed

cppwfs closed this as completed Feb 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

TASK_LOCK is not cleared when using single-instance-enabled option. #640

TASK_LOCK is not cleared when using single-instance-enabled option. #640

cow630 commented Dec 13, 2019

cppwfs commented Dec 13, 2019

Uh oh!

cow630 commented Dec 13, 2019

Uh oh!

cppwfs commented Dec 13, 2019

Uh oh!

cow630 commented Dec 13, 2019

Uh oh!

cppwfs commented Dec 18, 2019

Uh oh!

cppwfs commented Dec 19, 2019

Uh oh!

cow630 commented Dec 20, 2019

Uh oh!

cppwfs commented Dec 20, 2019 •

edited

Loading

Uh oh!

chenmaolin928 commented Apr 15, 2024

Uh oh!

ilopezv commented Dec 9, 2024 •

edited

Loading

Uh oh!

TASK_LOCK is not cleared when using single-instance-enabled option. #640

TASK_LOCK is not cleared when using single-instance-enabled option. #640

Comments

cow630 commented Dec 13, 2019

cppwfs commented Dec 13, 2019

Uh oh!

cow630 commented Dec 13, 2019

Uh oh!

cppwfs commented Dec 13, 2019

Uh oh!

cow630 commented Dec 13, 2019

Uh oh!

cppwfs commented Dec 18, 2019

Uh oh!

cppwfs commented Dec 19, 2019

Uh oh!

cow630 commented Dec 20, 2019

Uh oh!

cppwfs commented Dec 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chenmaolin928 commented Apr 15, 2024

Uh oh!

ilopezv commented Dec 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cppwfs commented Dec 20, 2019 •

edited

Loading

ilopezv commented Dec 9, 2024 •

edited

Loading