Skip to content

Hung Build #3061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
garyrussell opened this issue Sep 24, 2019 · 4 comments
Closed

Hung Build #3061

garyrussell opened this issue Sep 24, 2019 · 4 comments
Assignees
@artembilan
Labels
type: bug
Milestone
5.2.GA

Comments

@garyrussell
Copy link
Contributor 

Found one Java-level deadlock:
=============================
"SpringContextShutdownHook":
  waiting to lock monitor 0x00007fc021fbba98 (object 0x00000007b5dcfe58, a org.h2.engine.Session),
  which is held by "SpringContextShutdownHook"
"SpringContextShutdownHook":
  waiting to lock monitor 0x00007fc021fbd4b8 (object 0x00000007b5f19690, a org.h2.engine.Session),
  which is held by "SpringContextShutdownHook"

Java stack information for the threads listed above:
===================================================
"SpringContextShutdownHook":
	at org.h2.command.dml.TransactionCommand.update(TransactionCommand.java:97)
	- waiting to lock <0x00000007b5dcfe58> (a org.h2.engine.Session)
	at org.h2.command.CommandContainer.update(CommandContainer.java:133)
	at org.h2.command.Command.executeUpdate(Command.java:267)
	- locked <0x00000007b5f19690> (a org.h2.engine.Session)
	at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:233)
	- locked <0x00000007b5f19690> (a org.h2.engine.Session)
	at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:205)
	at org.springframework.jdbc.datasource.embedded.AbstractEmbeddedDatabaseConfigurer.shutdown(AbstractEmbeddedDatabaseConfigurer.java:48)
	at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory.shutdownDatabase(EmbeddedDatabaseFactory.java:229)
	at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactoryBean.destroy(EmbeddedDatabaseFactoryBean.java:90)
	at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:258)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:571)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:543)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1071)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:504)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1064)
	at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1060)
	at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1029)
	at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:948)
	- locked <0x00000007a0d7b6e8> (a java.lang.Object)
"SpringContextShutdownHook":
	at org.h2.command.dml.TransactionCommand.update(TransactionCommand.java:97)
	- waiting to lock <0x00000007b5f19690> (a org.h2.engine.Session)
	at org.h2.command.CommandContainer.update(CommandContainer.java:133)
	at org.h2.command.Command.executeUpdate(Command.java:267)
	- locked <0x00000007b5dcfe58> (a org.h2.engine.Session)
	at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:233)
	- locked <0x00000007b5dcfe58> (a org.h2.engine.Session)
	at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:205)
	at org.springframework.jdbc.datasource.embedded.AbstractEmbeddedDatabaseConfigurer.shutdown(AbstractEmbeddedDatabaseConfigurer.java:48)
	at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory.shutdownDatabase(EmbeddedDatabaseFactory.java:229)
	at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactoryBean.destroy(EmbeddedDatabaseFactoryBean.java:90)
	at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:258)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:571)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:543)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1071)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:504)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1064)
	at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1060)
	at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1029)
	at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:948)
	- locked <0x00000007a053a060> (a java.lang.Object)

Found 1 deadlock.
@garyrussell garyrussell added the status: waiting-for-triage The issue need to be evaluated and its future decided label Sep 24, 2019
@artembilan
Copy link
Member

According that stack trace it looks like the story is only about the cases where we have a <jdbc:embedded-database id="dataSource" type="H2"> definition.

@garyrussell
Copy link
Contributor Author

I still have the full thread dump; I'll find out which threads own the locks.

@garyrussell
Copy link
Contributor Author

Oh - it's a mutual deadlock between those two threads.

@artembilan artembilan self-assigned this Oct 1, 2019
@artembilan artembilan removed the status: waiting-for-triage The issue need to be evaluated and its future decided label Oct 1, 2019
@artembilan artembilan added this to the 5.2.GA milestone Oct 1, 2019
@artembilan
Copy link
Member

A couple tests with an embedded DB don't have a @DirtiesContext on them to close a DB in the end of test suite.
This way when context cache cleanup comes to the scene, it is a fact that several cleaners try to destroy the same in-memory DB instance cause the mentioned dead lock.

garyrussell pushed a commit that referenced this issue Nov 4, 2019
@artembilan @garyrussell
GH-3061: Add @DirtiesContext to JPA tests
5c6f65f 
Fixes #3061

Without a `@DirtiesContext` we keep not only a ctx in the cache, but
also an embedded DB instance, which is shared between contexts.
When ctx cache clean up happens, several ctxes would like to destroy
their associated embedded DB which, essentially, is the same in-memory
instance

* The `@DirtiesContext` give us a chance to destroy embedded DB for
the particular ctx in isolation
This was referenced Nov 7, 2019
Closed
Open
This was referenced Nov 26, 2019
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@artembilan artembilan

Labels
type: bug
Projects
None yet
Milestone
5.2.GA
Development

No branches or pull requests
2 participants
@garyrussell @artembilan