[GR-59864] GraalVM JDK behaves differently when run -agentlib:native-image-agent and -agentpath:$GRAALVM_HOME/lib/libnative-image-agent.so

[ziyilin]

Describe the issue
When native-image-agent is running along with opentelemetry-agent.jar, it behaves differently with option -agentlib:native-image-agent and -agentpath:$GRAALVM_HOME/lib/libnative-image-agent.so.

This happens with Labs-openJDK21 and Labs-openJDK17, while openJDK21 and openJDK17 are good.
I encountered this issue when developing the #8077.
Steps to reproduce the issue
Use the ot-agent-demo.zip.

Unzip the file and cd into the directory.

1. Run ./build.sh to build the project.
2. Assume GraalVM CE 21 has been installed. Run the following script, the program shall crash

JAVA="<path>/graalvm-community-openjdk-21.0.1+12.1"
NATIVE_IMAGE_AGENT="-agentlib:native-image-agent=config-output-dir=native-configs"
agentJar="ot-native-support-libs/opentelemetry-javaagent-1.32.0.jar"
$JAVA/bin/java $DEBUG $NATIVE_IMAGE_AGENT -javaagent:$agentJar -jar target/demo-0.0.1-SNAPSHOT.jar

3. Change -agentlib to -agentpath, and related contents. It will be ok.

Describe GraalVM and your environment:

• GraalVM version: Master build version, GraalVM CE 21.0.1+12.1 (build 21.0.1+12-jvmci-23.1-b19)
• Good JDK version: GraalVM 21.0.1+12.1 (build 21.0.1+12-jvmci-23.1-b19), oepnJDK 21.0.1
• Architecture: AMD64

More details
Add any other information about the problem here. Especially important are stack traces or log output. Feel free to link to gists or to screenshots if necessary.

 OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[otel.javaagent 2023-12-27 09:35:14:631 +0800] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.32.0
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.2.1)

Exception in thread "main" Fatal error: guarantee failed

Printing instructions (ip=0x00007f3f350bd560):
  0x00007f3f350bd460: 0xe8 0xab 0x2f 0x05 0x00 0x90 0xeb 0xda 0x48 0x89 0x44 0x24 0x08 0x66 0x66 0x90
  0x00007f3f350bd470: 0xe9 0x55 0xfd 0xff 0xff 0x48 0x8b 0x7c 0x24 0x10 0xe8 0x21 0x9f 0x03 0x00 0x90
  0x00007f3f350bd480: 0xe9 0x86 0xfd 0xff 0xff 0x48 0x8b 0xc8 0xe9 0x78 0xfe 0xff 0xff 0x48 0x8b 0x44
  0x00007f3f350bd490: 0x24 0x28 0x48 0x8b 0xf0 0xe9 0xd9 0xfe 0xff 0xff 0xe8 0x31 0xe5 0x08 0x00 0x90
  0x00007f3f350bd4a0: 0xeb 0xab 0xbf 0x01 0x00 0x00 0x00 0xbe 0x01 0x00 0x00 0x00 0x0f 0x1f 0x40 0x00
  0x00007f3f350bd4b0: 0xe8 0xbb 0xe7 0x08 0x00 0x90 0xeb 0x8a 0xe9 0xf0 0xfe 0xff 0xff 0xe9 0xeb 0xfe
  0x00007f3f350bd4c0: 0xff 0xff 0xe9 0xe6 0xfe 0xff 0xff 0xe9 0xe1 0xfe 0xff 0xff 0x0f 0x1f 0x40 0x00
  0x00007f3f350bd4d0: 0xe9 0xd8 0xfe 0xff 0xff 0xe9 0xd3 0xfe 0xff 0xff 0xe9 0xce 0xfe 0xff 0xff 0xe9
  0x00007f3f350bd4e0: 0xc9 0xfe 0xff 0xff 0xe9 0xc4 0xfe 0xff 0xff 0xe9 0xbf 0xfe 0xff 0xff 0x66 0x90
  0x00007f3f350bd4f0: 0xe9 0xb8 0xfe 0xff 0xff 0xe9 0xb3 0xfe 0xff 0xff 0xe9 0xae 0xfe 0xff 0xff 0xe9
  0x00007f3f350bd500: 0xa9 0xfe 0xff 0xff 0xe9 0xa4 0xfe 0xff 0xff 0xe8 0x92 0xa0 0xfe 0xff 0x90 0xcc
  0x00007f3f350bd510: 0x48 0x83 0xec 0x28 0x48 0x89 0x54 0x24 0x18 0x48 0x89 0x74 0x24 0x10 0x48 0x89
  0x00007f3f350bd520: 0x7c 0x24 0x20 0xe8 0x48 0x8a 0x08 0x00 0x90 0x41 0xc7 0x87 0x18 0x01 0x00 0x00
  0x00007f3f350bd530: 0x01 0x00 0x00 0x00 0xf0 0x83 0x04 0x24 0x00 0x49 0xc7 0x47 0x08 0x01 0x00 0x00
  0x00007f3f350bd540: 0x00 0x41 0xc7 0x87 0x14 0x01 0x00 0x00 0xfe 0xfe 0xfe 0x7e 0x48 0x8b 0x7c 0x24
  0x00007f3f350bd550: 0x20 0x48 0x8b 0x74 0x24 0x10 0x48 0x8b 0x54 0x24 0x18 0xe8 0x90 0xfb 0xff 0xff
> 0x00007f3f350bd560: 0x90 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc
  0x00007f3f350bd570: 0x48 0x83 0xec 0x18 0x48 0xb8 0xd0 0x1f 0x9a 0x00 0x00 0x00 0x00 0x00 0x49 0x8d
  0x00007f3f350bd580: 0x04 0x06 0x49 0x3b 0x67 0x08 0x0f 0x86 0xb1 0x00 0x00 0x00 0x0f 0x1f 0x40 0x00
  0x00007f3f350bd590: 0x49 0x3b 0xf6 0x0f 0x84 0xaf 0x00 0x00 0x00 0x48 0x89 0x44 0x24 0x10 0x48 0xbf
  0x00007f3f350bd5a0: 0x48 0x65 0x7b 0x00 0x00 0x00 0x00 0x00 0x49 0x8d 0x3c 0x3e 0x48 0xc7 0xc1 0xf8
  0x00007f3f350bd5b0: 0xff 0xff 0xff 0x48 0x23 0x0e 0x49 0x8d 0x0c 0x0e 0x48 0x89 0x74 0x24 0x08 0x48
  0x00007f3f350bd5c0: 0x8b 0xf1 0xe8 0x79 0xf0 0x5a 0x00 0x90 0x0f 0x1f 0x84 0x00 0x00 0x00 0x00 0x00
  0x00007f3f350bd5d0: 0x49 0x3b 0xc6 0x0f 0x84 0x23 0x00 0x00 0x00 0x48 0xc7 0xc7 0xf8 0xff 0xff 0xff
  0x00007f3f350bd5e0: 0x48 0x23 0x38 0xbe 0xff 0xff 0x00 0x00 0x41 0x23 0x74 0x3e 0x12 0x66 0x66 0x90
  0x00007f3f350bd5f0: 0x81 0xfe 0xac 0x02 0x00 0x00 0x0f 0x85 0x65 0x00 0x00 0x00 0x49 0x3b 0xc6 0x0f
  0x00007f3f350bd600: 0x84 0x49 0x00 0x00 0x00 0x48 0xc7 0xc7 0xf8 0xff 0xff 0xff 0x48 0x23 0x38 0x49
  0x00007f3f350bd610: 0x8b 0x8c 0x3e 0xe8 0x00 0x00 0x00 0x48 0x8b 0xf8 0x48 0x8b 0x74 0x24 0x08 0x48
  0x00007f3f350bd620: 0x8b 0xc1 0xff 0xd0 0x90 0x41 0x83 0x6f 0x10 0x01 0x0f 0x8e 0x05 0x00 0x00 0x00
  0x00007f3f350bd630: 0x48 0x83 0xc4 0x18 0xc3 0xe8 0x96 0xe3 0x08 0x00 0x90 0xeb 0xf3 0x48 0x89 0x74
  0x00007f3f350bd640: 0x24 0x08 0xe8 0x89 0xa0 0xfe 0xff 0x90 0xe8 0x73 0x6a 0x08 0x00 0x90 0x48 0x8b
  0x00007f3f350bd650: 0x44 0x24 0x10 0x48 0x8b 0xf8 0x48 0x8b 0x74 0x24 0x08 0xe8 0x20 0x00 0x00 0x00

Top of stack (sp=0x00007f3f38cbe270):
  0x00007f3f38cbe250: 0x00007f3f3002ba10 0x00007f3f3002bde8 0x00000000000003d8 0x00007f3f350bd560
> 0x00007f3f38cbe270: 0x00007f3f38cbe2a0 0x0000000000000008 0x00007f3f344a5e28 0x00007f3f34100000
  0x00007f3f38cbe290: 0x00007f3f34fa56c1 0x00007f3f35153484 0x00007f3f344a7828 0x00007f3f34fa56c1
  0x00007f3f38cbe2b0: 0x00007f3f38175ec0 0x00007f3f3514ed0a 0x00007f3e75604220 0x00007f3f34406980
  0x00007f3f38cbe2d0: 0x00007f3f34b14e60 0x00007f3f34fa5672 0x00007f3f38cbe2b0 0x0000000000000000
  0x00007f3f38cbe2f0: 0x00087f3f30003210 0x00007f3f3021f970 0x00007f3f3002b6a8 0x00007f3f3021f970
  0x00007f3f38cbe310: 0x00007f3f302aad58 0x00007f3f3505529a 0x00007f3f38cbe360 0x00007f3f3002b2f0
  0x00007f3f38cbe330: 0x0000000000000000 0x00007f3f302aad58 0x00007f3f3021f970 0x00007f3f3002b6a8
  0x00007f3f38cbe350: 0x00007f3f300199e0 0x00007f3f38cbe4c0 0x00007f3e845265b0 0x00007f3f38cbe420
  0x00007f3f38cbe370: 0x00007f3f302a8a80 0x00007f3e845265b0 0x00007f3f3002b2f0 0x00007f3f37820f35
  0x00007f3f38cbe390: 0x00000001000001cc 0x00007f3f38cbe3e0 0x00007f3f3021f970 0x00007f3f302aad58
  0x00007f3f38cbe3b0: 0x00007f3f300199e0 0x0000000214002670 0x00007f3e00000000 0x00007f3f3002b2f0
  0x00007f3f38cbe3d0: 0x00007f3e84526628 0x00007f3f3002b2f0 0x00007f3f3002b2f0 0x00007f3f3002a140
  0x00007f3f38cbe3f0: 0x00007f3f3002a180 0x00007f3f3002a1b0 0x00007f3f3002a268 0x00000000000000d8
  0x00007f3f38cbe410: 0x00007f3f38da44a0 0x00007f3f38bd0ef2 0x0000000000000003 0x00007f3f3002a5e0
  0x00007f3f38cbe430: 0x00007f3f3002ba00 0x00007f3f3002ba10 0x00007f3f3002bde8 0x00000000000003d8
  0x00007f3f38cbe450: 0x00007f3f3002b2f0 0x00007f3f3002b2f0 0x00007f3f3002a140 0x00007f3f3002a180

VM thread locals for the failing thread 0x00007f3f30019640:
  0 (8 bytes): JNIThreadLocalEnvironment.jniFunctions = (bytes)
    0x00007f3f30019640: 0x00007f3f34624010
  8 (8 bytes): StackOverflowCheckImpl.stackBoundaryTL = (Word) 1 (0x0000000000000001)
  16 (4 bytes): Safepoint.safepointRequested = (int) 2036949931 (0x796963ab)
  20 (4 bytes): StatusSupport.statusTL = (int) 1 (0x00000001)
  24 (32 bytes): ThreadLocalAllocation.regularTLAB = (bytes)
    0x00007f3f30019658: 0x00007f3e32500000 0x00007f3e32580000
    0x00007f3f30019668: 0x00007f3e3256f650 0x0000000000000000
  56 (8 bytes): PlatformThreads.currentThread = (Object) java.lang.Thread (0x00007f3f3487ff40)
  64 (8 bytes): JavaFrameAnchors.lastAnchor = (Word) 0 (0x0000000000000000)
  72 (8 bytes): PlatformThreads.currentVThreadId = (long) 1 (0x0000000000000001)
  80 (8 bytes): AccessControlContextStack = (Object) java.util.ArrayDeque (0x00007f3e75600b10)
  88 (8 bytes): ExceptionUnwind.currentException = (Object) null
  96 (8 bytes): IdentityHashCodeSupport.hashCodeGeneratorTL = (Object) null
  104 (8 bytes): IsolatedCompileClient.currentClient = (Object) null
  112 (8 bytes): IsolatedCompileContext.currentContext = (Object) null
  120 (8 bytes): JNIObjectHandles.handles = (Object) com.oracle.svm.core.handles.ThreadLocalHandles (0x00007f3e75600b38)
  128 (8 bytes): JNIThreadLocalPendingException.pendingException = (Object) null
  136 (8 bytes): JNIThreadLocalReferencedObjects.referencedObjectsListHead = (Object) null
  144 (8 bytes): JNIThreadOwnedMonitors.ownedMonitors = (Object) null
  152 (8 bytes): NoAllocationVerifier.openVerifiers = (Object) null
  160 (8 bytes): ThreadingSupportImpl.activeTimer = (Object) null
  168 (8 bytes): ActionOnExitSafepointSupport.returnIP = (Word) 0 (0x0000000000000000)
  176 (8 bytes): ActionOnExitSafepointSupport.returnSP = (Word) 0 (0x0000000000000000)
  184 (8 bytes): SubstrateDiagnostics.threadOnlyAttachedForCrashHandler = (bytes)
    0x00007f3f300196f8: 0x0000000000000000
  192 (8 bytes): ThreadLocalAllocation.allocatedBytes = (Word) 253771016 (0x000000000f203d08)
  200 (8 bytes): VMThreads.IsolateTL = (Word) 139909433131008 (0x00007f3f34100000)
  208 (8 bytes): VMThreads.OSThreadHandleTL = (Word) 139909513504512 (0x00007f3f38da6700)
  216 (8 bytes): VMThreads.OSThreadIdTL = (Word) 3851 (0x0000000000000f0b)
  224 (8 bytes): VMThreads.StackBase = (Word) 139909513506816 (0x00007f3f38da7000)
  232 (8 bytes): VMThreads.StackEnd = (Word) 139909512454144 (0x00007f3f38ca6000)
  240 (8 bytes): VMThreads.StartedByCurrentIsolate = (bytes)
    0x00007f3f30019730: 0x0000000000000000
  248 (8 bytes): VMThreads.nextTL = (Word) 0 (0x0000000000000000)
  256 (8 bytes): VMThreads.unalignedIsolateThreadMemoryTL = (Word) 139909365077520 (0x00007f3f30019610)
  264 (4 bytes): ActionOnExitSafepointSupport.actionTL = (int) 0 (0x00000000)
  268 (4 bytes): ActionOnTransitionToJavaSupport.actionTL = (int) 0 (0x00000000)
  272 (4 bytes): ImplicitExceptions.implicitExceptionsAreFatal = (int) 0 (0x00000000)
  276 (4 bytes): StackOverflowCheckImpl.yellowZoneStateTL = (int) 2130640638 (0x7efefefe)
  280 (4 bytes): StatusSupport.safepointBehaviorTL = (int) 1 (0x00000001)
  284 (4 bytes): ThreadingSupportImpl.currentPauseDepth = (int) 0 (0x00000000)

Java frame anchors for the failing thread 0x00007f3f30019640:
  No anchors

Stacktrace for the failing thread 0x00007f3f30019640 (A=AOT compiled, J=JIT compiled, D=deoptimized, i=inlined):
  i  SP 0x00007f3f38cbe270 IP 0x00007f3f350bd560 size=48    com.oracle.svm.core.jdk.VMErrorSubstitutions.shutdown(VMErrorSubstitutions.java:148)
  A  SP 0x00007f3f38cbe270 IP 0x00007f3f350bd560 size=48    com.oracle.svm.core.jdk.VMErrorSubstitutions.shouldNotReachHere(VMErrorSubstitutions.java:141)
  A  SP 0x00007f3f38cbe2a0 IP 0x00007f3f35153484 size=16    com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:90)
  i  SP 0x00007f3f38cbe2b0 IP 0x00007f3f34fa56c1 size=112   com.oracle.svm.core.util.VMError.guarantee(VMError.java:115)
  A  SP 0x00007f3f38cbe2b0 IP 0x00007f3f34fa56c1 size=112   com.oracle.svm.agent.BreakpointInterceptor.onBreakpoint(BreakpointInterceptor.java:1113)
  A  SP 0x00007f3f38cbe320 IP 0x00007f3f3505529a size=112   com.oracle.svm.core.code.IsolateEnterStub.BreakpointInterceptor_onBreakpoint_0173e8123278a797c40ae318dca97fed56f24ff9(IsolateEnterStub.java:0)

Threads:
  0x00007f3d00001080 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-7" - 0x00007f3e28500830, daemon, stack(0x00007f3d056ff000,0x00007f3d05800000)
  0x00007f3d2400f240 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-6" - 0x00007f3f34000830, daemon, stack(0x00007f3e3f1ff000,0x00007f3e3f400000)
  0x00007f3d10001180 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-5" - 0x00007f3e2e000830, daemon, stack(0x00007f3e297ff000,0x00007f3e29900000)
  0x00007f3d20007500 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-4" - 0x00007f3e756008d0, daemon, stack(0x00007f3e3eafe000,0x00007f3e3ecff000)
  0x00007f3e14001080 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-3" - 0x00007f3e75600990, daemon, stack(0x00007f3e2e4ff000,0x00007f3e2e600000)
  0x00007f3e58023cc0 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-0" - 0x00007f3e75600a50, daemon, stack(0x00007f3ed6acc000,0x00007f3ed6ccd000)
  0x00007f3f200008c0 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "Reference Handler" - 0x00007f3f348871d8, daemon, stack(0x00007f3f25eea000,0x00007f3f266e9000)
  0x00007f3f30019640 STATUS_IN_JAVA (PREVENT_VM_FROM_REACHING_SAFEPOINT) "main" - 0x00007f3f3487ff40, stack(0x00007f3f38ca6000,0x00007f3f38da7000)

No VMOperation in progress

The 30 most recent VM operation status changes:
  2.555s - Started Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 1)
  2.556s - Finished Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 1)
  3.981s - Started Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 2)
  3.981s - Finished Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 2)
  5.338s - Started Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 3)
  5.338s - Finished Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 3)
  6.696s - Started Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 4)
  6.697s - Finished Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 4)
  8.888s - Started Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 5)
  8.890s - Finished Garbage collection (safepoint: true, queueingThread: 0x00007f3f30019640, executingThread: 0x00007f3f30019640, safepointId: 5)

VM mutexes:
  mutex "RealLog.backTracePrinterMutex" is unlocked.
  mutex "mainVMOperationControlWorkQueue" is unlocked.
  mutex "referencePendingList" is unlocked.
  mutex "thread" is unlocked.

General information:
  VM version: 21.0.1+12, linux/amd64
  Current timestamp: 1703640926782
  VM uptime: 12.393s
  AOT compiled code: 0x00007f3f34f91000 - 0x00007f3f35abd84f
  CPU features used for AOT compiled code: CX8, CMOV, FXSR, MMX, SSE, SSE2

Heap settings and statistics:
  Supports isolates: true
  Heap base: 0x00007f3f34100000
  Object reference size: 8
  Aligned chunk size: 524288
  Large array threshold: 131072
  Incremental collections: 5
  Complete collections: 4

Heap usage:
  Eden: 0.00M (0.00M in 0 aligned chunks, 0.00M in 0 unaligned chunks)
  Old: 1.50M (1.50M in 3 aligned chunks, 0.00M in 0 unaligned chunks)

Native image heap boundaries:
  ReadOnly Primitives: 0x00007f3f34180830 - 0x00007f3f3440a4e0
  ReadOnly References: 0x00007f3f3440a4e0 - 0x00007f3f34623040
  ReadOnly Relocatables: 0x00007f3f34624000 - 0x00007f3f3471c040
  Writable Primitives: 0x00007f3f3471d000 - 0x00007f3f34810f68
  Writable References: 0x00007f3f34810f68 - 0x00007f3f34b4adb0
  Writable Huge: 0x00007f3f34b80038 - 0x00007f3f34bc11c0
  ReadOnly Huge: 0x00007f3f34bc2038 - 0x00007f3f34ecb938

Heap chunks: E=eden, S=survivor, O=old, F=free; A=aligned chunk, U=unaligned chunk; T=to space
  |0x00007f3e75600000|0x00007f3e75600830, 0x00007f3e7567fff0, 0x00007f3e75680000| 99%|  O|A|
  |0x00007f3e75700000|0x00007f3e75700830, 0x00007f3e7577ffb8, 0x00007f3e75780000| 99%|  O|A|
  |0x00007f3e2f800000|0x00007f3e2f800830, 0x00007f3e2f8426c0, 0x00007f3e2f880000| 51%|  O|A|
  |0x00007f3e2db00000|0x00007f3e2db00830, 0x00007f3e2db00830, 0x00007f3e2db80000|  0%|  F|A|
  |0x00007f3e3d600000|0x00007f3e3d600830, 0x00007f3e3d600830, 0x00007f3e3d680000|  0%|  F|A|
  |0x00007f3e3c500000|0x00007f3e3c500830, 0x00007f3e3c500830, 0x00007f3e3c580000|  0%|  F|A|
  |0x00007f3e33000000|0x00007f3e33000830, 0x00007f3e33000830, 0x00007f3e33080000|  0%|  F|A|
  |0x00007f3e80300000|0x00007f3e80300830, 0x00007f3e80300830, 0x00007f3e80380000|  0%|  F|A|
  |0x00007f3e75300000|0x00007f3e75300830, 0x00007f3e75300830, 0x00007f3e75380000|  0%|  F|A|
  |0x00007f3e74a00000|0x00007f3e74a00830, 0x00007f3e74a00830, 0x00007f3e74a80000|  0%|  F|A|
  |0x00007f3e74900000|0x00007f3e74900830, 0x00007f3e74900830, 0x00007f3e74980000|  0%|  F|A|
  |0x00007f3e74700000|0x00007f3e74700830, 0x00007f3e74700830, 0x00007f3e74780000|  0%|  F|A|
  |0x00007f3e3e500000|0x00007f3e3e500830, 0x00007f3e3e500830, 0x00007f3e3e580000|  0%|  F|A|
  |0x00007f3e3e400000|0x00007f3e3e400830, 0x00007f3e3e400830, 0x00007f3e3e480000|  0%|  F|A|
  |0x00007f3e3e200000|0x00007f3e3e200830, 0x00007f3e3e200830, 0x00007f3e3e280000|  0%|  F|A|
  |0x00007f3e3e100000|0x00007f3e3e100830, 0x00007f3e3e100830, 0x00007f3e3e180000|  0%|  F|A|
  |0x00007f3e3de00000|0x00007f3e3de00830, 0x00007f3e3de00830, 0x00007f3e3de80000|  0%|  F|A|
  |0x00007f3e3dd00000|0x00007f3e3dd00830, 0x00007f3e3dd00830, 0x00007f3e3dd80000|  0%|  F|A|
  |0x00007f3e3dc00000|0x00007f3e3dc00830, 0x00007f3e3dc00830, 0x00007f3e3dc80000|  0%|  F|A|
  |0x00007f3e3d500000|0x00007f3e3d500830, 0x00007f3e3d500830, 0x00007f3e3d580000|  0%|  F|A|
  |0x00007f3e3cd00000|0x00007f3e3cd00830, 0x00007f3e3cd00830, 0x00007f3e3cd80000|  0%|  F|A|
  |0x00007f3e3cc00000|0x00007f3e3cc00830, 0x00007f3e3cc00830, 0x00007f3e3cc80000|  0%|  F|A|
  |0x00007f3e3cb00000|0x00007f3e3cb00830, 0x00007f3e3cb00830, 0x00007f3e3cb80000|  0%|  F|A|
  |0x00007f3e3c900000|0x00007f3e3c900830, 0x00007f3e3c900830, 0x00007f3e3c980000|  0%|  F|A|
  |0x00007f3e3c700000|0x00007f3e3c700830, 0x00007f3e3c700830, 0x00007f3e3c780000|  0%|  F|A|
  |0x00007f3e3c600000|0x00007f3e3c600830, 0x00007f3e3c600830, 0x00007f3e3c680000|  0%|  F|A|
  |0x00007f3e3c400000|0x00007f3e3c400830, 0x00007f3e3c400830, 0x00007f3e3c480000|  0%|  F|A|
  |0x00007f3e3c200000|0x00007f3e3c200830, 0x00007f3e3c200830, 0x00007f3e3c280000|  0%|  F|A|
  |0x00007f3e3c100000|0x00007f3e3c100830, 0x00007f3e3c100830, 0x00007f3e3c180000|  0%|  F|A|
  |0x00007f3e3c000000|0x00007f3e3c000830, 0x00007f3e3c000830, 0x00007f3e3c080000|  0%|  F|A|
  |0x00007f3e33f00000|0x00007f3e33f00830, 0x00007f3e33f00830, 0x00007f3e33f80000|  0%|  F|A|
  |0x00007f3e33d00000|0x00007f3e33d00830, 0x00007f3e33d00830, 0x00007f3e33d80000|  0%|  F|A|
  |0x00007f3e33a00000|0x00007f3e33a00830, 0x00007f3e33a00830, 0x00007f3e33a80000|  0%|  F|A|
  |0x00007f3e33900000|0x00007f3e33900830, 0x00007f3e33900830, 0x00007f3e33980000|  0%|  F|A|
  |0x00007f3e33700000|0x00007f3e33700830, 0x00007f3e33700830, 0x00007f3e33780000|  0%|  F|A|
  |0x00007f3e33600000|0x00007f3e33600830, 0x00007f3e33600830, 0x00007f3e33680000|  0%|  F|A|
  |0x00007f3e33500000|0x00007f3e33500830, 0x00007f3e33500830, 0x00007f3e33580000|  0%|  F|A|
  |0x00007f3e33300000|0x00007f3e33300830, 0x00007f3e33300830, 0x00007f3e33380000|  0%|  F|A|
  |0x00007f3e33100000|0x00007f3e33100830, 0x00007f3e33100830, 0x00007f3e33180000|  0%|  F|A|
  |0x00007f3e32f00000|0x00007f3e32f00830, 0x00007f3e32f00830, 0x00007f3e32f80000|  0%|  F|A|
  |0x00007f3e32e00000|0x00007f3e32e00830, 0x00007f3e32e00830, 0x00007f3e32e80000|  0%|  F|A|
  |0x00007f3e32c00000|0x00007f3e32c00830, 0x00007f3e32c00830, 0x00007f3e32c80000|  0%|  F|A|
  |0x00007f3e32900000|0x00007f3e32900830, 0x00007f3e32900830, 0x00007f3e32980000|  0%|  F|A|
  |0x00007f3e32800000|0x00007f3e32800830, 0x00007f3e32800830, 0x00007f3e32880000|  0%|  F|A|
  |0x00007f3e32400000|0x00007f3e32400830, 0x00007f3e32400830, 0x00007f3e32480000|  0%|  F|A|
  |0x00007f3e32200000|0x00007f3e32200830, 0x00007f3e32200830, 0x00007f3e32280000|  0%|  F|A|
  |0x00007f3e32100000|0x00007f3e32100830, 0x00007f3e32100830, 0x00007f3e32180000|  0%|  F|A|
  |0x00007f3e32000000|0x00007f3e32000830, 0x00007f3e32000830, 0x00007f3e32080000|  0%|  F|A|
  |0x00007f3e31f00000|0x00007f3e31f00830, 0x00007f3e31f00830, 0x00007f3e31f80000|  0%|  F|A|
  |0x00007f3e31d00000|0x00007f3e31d00830, 0x00007f3e31d00830, 0x00007f3e31d80000|  0%|  F|A|
  |0x00007f3e31c00000|0x00007f3e31c00830, 0x00007f3e31c00830, 0x00007f3e31c80000|  0%|  F|A|
  |0x00007f3e31b00000|0x00007f3e31b00830, 0x00007f3e31b00830, 0x00007f3e31b80000|  0%|  F|A|
  |0x00007f3e31a00000|0x00007f3e31a00830, 0x00007f3e31a00830, 0x00007f3e31a80000|  0%|  F|A|
  |0x00007f3e31800000|0x00007f3e31800830, 0x00007f3e31800830, 0x00007f3e31880000|  0%|  F|A|
  |0x00007f3e31600000|0x00007f3e31600830, 0x00007f3e31600830, 0x00007f3e31680000|  0%|  F|A|
  |0x00007f3e31500000|0x00007f3e31500830, 0x00007f3e31500830, 0x00007f3e31580000|  0%|  F|A|
  |0x00007f3e31400000|0x00007f3e31400830, 0x00007f3e31400830, 0x00007f3e31480000|  0%|  F|A|
  |0x00007f3e31300000|0x00007f3e31300830, 0x00007f3e31300830, 0x00007f3e31380000|  0%|  F|A|
  |0x00007f3e31200000|0x00007f3e31200830, 0x00007f3e31200830, 0x00007f3e31280000|  0%|  F|A|
  |0x00007f3e31000000|0x00007f3e31000830, 0x00007f3e31000830, 0x00007f3e31080000|  0%|  F|A|
  |0x00007f3e30f00000|0x00007f3e30f00830, 0x00007f3e30f00830, 0x00007f3e30f80000|  0%|  F|A|
  |0x00007f3e30e00000|0x00007f3e30e00830, 0x00007f3e30e00830, 0x00007f3e30e80000|  0%|  F|A|
  |0x00007f3e30d00000|0x00007f3e30d00830, 0x00007f3e30d00830, 0x00007f3e30d80000|  0%|  F|A|
  |0x00007f3e30b00000|0x00007f3e30b00830, 0x00007f3e30b00830, 0x00007f3e30b80000|  0%|  F|A|
  |0x00007f3e30a00000|0x00007f3e30a00830, 0x00007f3e30a00830, 0x00007f3e30a80000|  0%|  F|A|
  |0x00007f3e30900000|0x00007f3e30900830, 0x00007f3e30900830, 0x00007f3e30980000|  0%|  F|A|
  |0x00007f3e30800000|0x00007f3e30800830, 0x00007f3e30800830, 0x00007f3e30880000|  0%|  F|A|
  |0x00007f3e30700000|0x00007f3e30700830, 0x00007f3e30700830, 0x00007f3e30780000|  0%|  F|A|
  |0x00007f3e30600000|0x00007f3e30600830, 0x00007f3e30600830, 0x00007f3e30680000|  0%|  F|A|
  |0x00007f3e30400000|0x00007f3e30400830, 0x00007f3e30400830, 0x00007f3e30480000|  0%|  F|A|
  |0x00007f3e30200000|0x00007f3e30200830, 0x00007f3e30200830, 0x00007f3e30280000|  0%|  F|A|
  |0x00007f3e30100000|0x00007f3e30100830, 0x00007f3e30100830, 0x00007f3e30180000|  0%|  F|A|
  |0x00007f3e30000000|0x00007f3e30000830, 0x00007f3e30000830, 0x00007f3e30080000|  0%|  F|A|
  |0x00007f3e2ff00000|0x00007f3e2ff00830, 0x00007f3e2ff00830, 0x00007f3e2ff80000|  0%|  F|A|
  |0x00007f3e2fe00000|0x00007f3e2fe00830, 0x00007f3e2fe00830, 0x00007f3e2fe80000|  0%|  F|A|
  |0x00007f3e2fc00000|0x00007f3e2fc00830, 0x00007f3e2fc00830, 0x00007f3e2fc80000|  0%|  F|A|
  |0x00007f3e2fa00000|0x00007f3e2fa00830, 0x00007f3e2fa00830, 0x00007f3e2fa80000|  0%|  F|A|
  |0x00007f3e2f900000|0x00007f3e2f900830, 0x00007f3e2f900830, 0x00007f3e2f980000|  0%|  F|A|
  |0x00007f3f25e00000|0x00007f3f25e00830, 0x00007f3f25e00830, 0x00007f3f25e80000|  0%|  F|A|
  |0x00007f3ed4b00000|0x00007f3ed4b00830, 0x00007f3ed4b00830, 0x00007f3ed4b80000|  0%|  F|A|

Fatal error: guarantee failed

[medoussboug]

Hello, Thanks for sharing your issue.

I have tried to reproduce your issue, but it seems that the commands you provided aren't suitable for the reproducer you shared. (no target/demo-0.0.1-SNAPSHOT.jar generate, no ot-native-support-libs/opentelemetry-javaagent-1.32.0.jar). I checked the step you are referring to and I found that there is mismatch of commands in the collectInstrument.sh you talked about and the steps you shared here, which is confusing. Can you please clarify in order for me to investigate

[ziyilin]

There are indeed a few mistakes in my demo. I have updated the description and the demo project. Thank you

[nreid260]

We're seeing a very similar thing when combining native-image-agent from Graal 22 with Java 24.

We tried updating to Graal 23 for unrelated reasons, but saw NPEs coming from Points-To analysis, so had to revert. It would still be useful to verify if Graal 23 solves this problem, since then we could focus on a single solution.

Fatal error: guarantee failed

Printing instructions (ip=0x00007f6ddb373df0):
  0x00007f6ddb373cf0: 0x48 0xc3 0x48 0xbf 0x88 0xd9 0x9b 0x00 0x00 0x00 0x00 0x00 0x49 0x8d 0x3c 0x3e
  0x00007f6ddb373d00: 0xe8 0x5b 0xd9 0x04 0x00 0x90 0xeb 0xda 0x48 0x89 0x44 0x24 0x08 0x66 0x66 0x90
  0x00007f6ddb373d10: 0xe9 0x55 0xfd 0xff 0xff 0x48 0x8b 0xc8 0xe9 0x89 0xfe 0xff 0xff 0x48 0x8b 0x44
  0x00007f6ddb373d20: 0x24 0x28 0x48 0x8b 0xf0 0xe9 0xe9 0xfe 0xff 0xff 0xe8 0x01 0x88 0x08 0x00 0x90
  0x00007f6ddb373d30: 0xeb 0xbb 0xbf 0x01 0x00 0x00 0x00 0xbe 0x01 0x00 0x00 0x00 0xe8 0x2f 0x8a 0x08
  0x00007f6ddb373d40: 0x00 0x90 0xeb 0x9e 0xe9 0x04 0xff 0xff 0xff 0xe9 0xff 0xfe 0xff 0xff 0x66 0x90
  0x00007f6ddb373d50: 0xe9 0xf8 0xfe 0xff 0xff 0xe9 0xf3 0xfe 0xff 0xff 0xe9 0xee 0xfe 0xff 0xff 0xe9
  0x00007f6ddb373d60: 0xe9 0xfe 0xff 0xff 0xe9 0xe4 0xfe 0xff 0xff 0xe9 0xdf 0xfe 0xff 0xff 0x66 0x90
  0x00007f6ddb373d70: 0xe9 0xd8 0xfe 0xff 0xff 0xe9 0xd3 0xfe 0xff 0xff 0xe9 0xce 0xfe 0xff 0xff 0xe9
  0x00007f6ddb373d80: 0xc9 0xfe 0xff 0xff 0xe9 0xc4 0xfe 0xff 0xff 0xe9 0xbf 0xfe 0xff 0xff 0x66 0x90
  0x00007f6ddb373d90: 0xe9 0xb8 0xfe 0xff 0xff 0xe8 0x26 0xa4 0xfe 0xff 0x90 0xcc 0xcc 0xcc 0xcc 0xcc
  0x00007f6ddb373da0: 0x48 0x83 0xec 0x28 0x48 0x89 0x54 0x24 0x18 0x48 0x89 0x74 0x24 0x10 0x48 0x89
  0x00007f6ddb373db0: 0x7c 0x24 0x20 0xe8 0xd8 0x2f 0x08 0x00 0x90 0x41 0xc7 0x87 0x04 0x01 0x00 0x00
  0x00007f6ddb373dc0: 0x01 0x00 0x00 0x00 0xf0 0x83 0x04 0x24 0x00 0x49 0xc7 0x47 0x08 0x01 0x00 0x00
  0x00007f6ddb373dd0: 0x00 0x41 0xc7 0x87 0x00 0x01 0x00 0x00 0xfe 0xfe 0xfe 0x7e 0x48 0x8b 0x7c 0x24
  0x00007f6ddb373de0: 0x20 0x48 0x8b 0x74 0x24 0x10 0x48 0x8b 0x54 0x24 0x18 0xe8 0xa0 0xfb 0xff 0xff
> 0x00007f6ddb373df0: 0x90 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc
  0x00007f6ddb373e00: 0x48 0x83 0xec 0x18 0x48 0xb8 0xa8 0xa0 0x92 0x00 0x00 0x00 0x00 0x00 0x49 0x8d
  0x00007f6ddb373e10: 0x04 0x06 0x49 0x3b 0x67 0x08 0x0f 0x86 0xb1 0x00 0x00 0x00 0x0f 0x1f 0x40 0x00
  0x00007f6ddb373e20: 0x49 0x3b 0xf6 0x0f 0x84 0xaf 0x00 0x00 0x00 0x48 0x89 0x44 0x24 0x10 0x48 0xbf
  0x00007f6ddb373e30: 0x80 0xcb 0x75 0x00 0x00 0x00 0x00 0x00 0x49 0x8d 0x3c 0x3e 0x48 0xc7 0xc1 0xf8
  0x00007f6ddb373e40: 0xff 0xff 0xff 0x48 0x23 0x0e 0x49 0x8d 0x0c 0x0e 0x48 0x89 0x74 0x24 0x08 0x48
  0x00007f6ddb373e50: 0x8b 0xf1 0xe8 0x29 0x22 0x45 0x00 0x90 0x0f 0x1f 0x84 0x00 0x00 0x00 0x00 0x00
  0x00007f6ddb373e60: 0x49 0x3b 0xc6 0x0f 0x84 0x23 0x00 0x00 0x00 0x48 0xc7 0xc7 0xf8 0xff 0xff 0xff
  0x00007f6ddb373e70: 0x48 0x23 0x38 0xbe 0xff 0xff 0x00 0x00 0x41 0x23 0x74 0x3e 0x12 0x66 0x66 0x90
  0x00007f6ddb373e80: 0x81 0xfe 0x91 0x02 0x00 0x00 0x0f 0x85 0x65 0x00 0x00 0x00 0x49 0x3b 0xc6 0x0f
  0x00007f6ddb373e90: 0x84 0x49 0x00 0x00 0x00 0x48 0xc7 0xc7 0xf8 0xff 0xff 0xff 0x48 0x23 0x38 0x49
  0x00007f6ddb373ea0: 0x8b 0x8c 0x3e 0xf0 0x00 0x00 0x00 0x48 0x8b 0xf8 0x48 0x8b 0x74 0x24 0x08 0x48
  0x00007f6ddb373eb0: 0x8b 0xc1 0xff 0xd0 0x90 0x41 0x83 0x6f 0x10 0x01 0x0f 0x8e 0x05 0x00 0x00 0x00
  0x00007f6ddb373ec0: 0x48 0x83 0xc4 0x18 0xc3 0xe8 0x66 0x86 0x08 0x00 0x90 0xeb 0xf3 0x48 0x89 0x74
  0x00007f6ddb373ed0: 0x24 0x08 0xe8 0x19 0xa4 0xfe 0xff 0x90 0xe8 0x63 0x0f 0x08 0x00 0x90 0x48 0x8b
  0x00007f6ddb373ee0: 0x44 0x24 0x10 0x48 0x8b 0xf8 0x48 0x8b 0x74 0x24 0x08 0xe8 0x20 0x00 0x00 0x00

Top of stack (sp=0x00007f6de5bfc450):
  0x00007f6de5bfc430: 0x00007f6de54505d0 0x00007f6ddb411758 0x00007f6de54504f8 0x00007f6ddb373df0
> 0x00007f6de5bfc450: 0x00007f6de5450540 0x00007f6ddb411745 0x00007f6dda3c74b0 0x00007f6dda080000
  0x00007f6de5bfc470: 0x00007f6ddb41151e 0x00007f6ddb403dd4 0x00005381ffc383d0 0x00007f6ddb41151e
  0x00007f6de5bfc490: 0x00005381ffc383d0 0x00007f6ddb277de7 0x00007f6d30180000 0x00007f6dda3dacd0
  0x00007f6de5bfc4b0: 0x00007f6dda3cff70 0x00007f6dda3e2b30 0x00007f6de544f238 0x00005381ffe34e98
  0x00007f6de5bfc4d0: 0x00005381ffc383d0 0x00007f6ddb41333a 0x00007f6dda080000 0x00007f6dda8544a8
  0x00007f6de5bfc4f0: 0x00007f6de544f238 0x00005381ffe51e00 0x00005381ffc383d0 0x00007f6ddb314ee7
  0x00007f6de5bfc510: 0x00007f6de5bfe6a0 0x00007f6de74c2040 0x00000000ffffffff 0x00005381ffc383d0
  0x00007f6de5bfc530: 0x00005381ffe51e00 0x00007f6de5bfc640 0x00005381ffe34e70 0x00007f6de74049c8
  0x00007f6de5bfc550: 0x00005381ffc38000 0x00005381ffe34a80 0x00005381ffe51e00 0x00007f6de6b6183f
  0x00007f6de5bfc570: 0x00000000ffffffff 0x0000000000000000 0x00007f6de5bfc640 0x00007f6de715cbdd
  0x00007f6de5bfc590: 0x00005381ffe49e40 0x00005381ffc2cc00 0x00005381ffc2cc10 0x00005381ffc2cfe8
  0x00007f6de5bfc5b0: 0x00000000000003d8 0x00005381ffc38000 0x00005381ffc38000 0x00005381ffe49e80
  0x00007f6de5bfc5d0: 0x00005381ffe3e5a0 0x00005381ffe3e688 0x00005381ffe3e688 0x00000000000000d8
  0x00007f6de5bfc5f0: 0x00007f6de5bfe6a0 0x00000009e5bfc5e1 0x00007f6de74c2040 0x00005381ffc38000
  0x00007f6de5bfc610: 0x00007f6d00000000 0x00005381ffc38000 0x00007f6de5bfe6a0 0x00007f6de74c2040
  0x00007f6de5bfc630: 0x00000000ffffffff 0x0000000000000000 0x00007f6de5bfe700 0x00007f6de715c330

VM thread locals for the failing thread 0x00005381ffe346c0:
  0: JNIThreadLocalEnvironment.jniFunctions = (bytes) 0x00007f6dda521010
  8: StackOverflowCheckImpl.stackBoundaryTL = (Word) 0x0000000000000001 (1)
  16: Safepoint.safepointRequested = (int) 0x7fff2b2f (2147429167)
  20: StatusSupport.statusTL = (int) 0x00000001 (1)
  24: ThreadLocalAllocation.regularTLAB = (bytes) 
    0x00005381ffe346d8: 0x00007f6de5400000 0x00007f6de5480000
    0x00005381ffe346e8: 0x00007f6de5450618 0x0000000000000000
  56: PlatformThreads.currentThread = (Object) 0x00007f6dda7aeeb0
    is an object of type java.lang.Thread
  64: JavaFrameAnchors.lastAnchor = (Word) 0x0000000000000000 (0)
  72: JavaThreads.currentVThreadId = (long) 0x0000000000000001 (1)
  80: ExceptionUnwind.currentException = (Object) 0x0000000000000000
  88: IdentityHashCodeSupport.hashCodeGeneratorTL = (Object) 0x0000000000000000
  96: IsolatedCompileClient.currentClient = (Object) 0x0000000000000000
  104: IsolatedCompileContext.currentContext = (Object) 0x0000000000000000
  112: JNIObjectHandles.handles = (Object) 0x00007f6de5400928
    is an object of type com.oracle.svm.core.handles.ThreadLocalHandles
  120: JNIThreadLocalPendingException.pendingException = (Object) 0x0000000000000000
  128: JNIThreadLocalReferencedObjects.referencedObjectsListHead = (Object) 0x0000000000000000
  136: JNIThreadOwnedMonitors.ownedMonitors = (Object) 0x0000000000000000
  144: NoAllocationVerifier.openVerifiers = (Object) 0x0000000000000000
  152: RecurringCallbackTimer.exception = (Object) 0x0000000000000000
  160: ThreadingSupportImpl.activeTimer = (Object) 0x0000000000000000
  168: SubstrateDiagnostics.threadOnlyAttachedForCrashHandler = (bytes) 0x0000000000000000
  176: ThreadLocalAllocation.allocatedBytes = (Word) 0x0000000000000000 (0)
  184: VMThreads.IsolateTL = (Word) 0x00007f6dda080000 (140109786120192)
  192: VMThreads.OSThreadHandleTL = (Word) 0x00007f6de5bff700 (140109982725888)
  200: VMThreads.OSThreadIdTL = (Word) 0x0000000000000fe1 (4065)
  208: VMThreads.StackBase = (Word) 0x00007f6de5c00000 (140109982728192)
  216: VMThreads.StackEnd = (Word) 0x00007f6de5500000 (140109975388160)
  224: VMThreads.StartedByCurrentIsolate = (bytes) 0x0000000000000000
  232: VMThreads.nextTL = (Word) 0x0000000000000000 (0)
  240: VMThreads.unalignedIsolateThreadMemoryTL = (Word) 0x00005381ffe34690 (91817808971408)
  248: ActionOnTransitionToJavaSupport.actionTL = (int) 0x00000000 (0)
  252: ImplicitExceptions.implicitExceptionsAreFatal = (int) 0x00000000 (0)
  256: StackOverflowCheckImpl.yellowZoneStateTL = (int) 0x7efefefe (2130640638)
  260: StatusSupport.safepointBehaviorTL = (int) 0x00000001 (1)
  264: ThreadingSupportImpl.currentPauseDepth = (int) 0x00000000 (0)
  
Java frame anchors for the failing thread 0x00005381ffe346c0:
  No anchors
  
Stacktrace for the failing thread 0x00005381ffe346c0 (A=AOT compiled, J=JIT compiled, D=deoptimized, i=inlined):
  i  SP 0x00007f6de5bfc450 IP 0x00007f6ddb373df0 size=48    com.oracle.svm.core.jdk.VMErrorSubstitutions.shutdown(VMErrorSubstitutions.java:148)
  A  SP 0x00007f6de5bfc450 IP 0x00007f6ddb373df0 size=48    com.oracle.svm.core.jdk.VMErrorSubstitutions.shouldNotReachHere(VMErrorSubstitutions.java:141)
  A  SP 0x00007f6de5bfc480 IP 0x00007f6ddb403dd4 size=16    com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:90)
  i  SP 0x00007f6de5bfc490 IP 0x00007f6ddb41151e size=16    com.oracle.svm.core.util.VMError.guarantee(VMError.java:115)
  A  SP 0x00007f6de5bfc490 IP 0x00007f6ddb41151e size=16    com.oracle.svm.jvmtiagentbase.JNIHandleSet.findClass(JNIHandleSet.java:74)
  A  SP 0x00007f6de5bfc4a0 IP 0x00007f6ddb277de7 size=64    com.oracle.svm.agent.NativeImageAgentJNIHandleSet.<init>(NativeImageAgentJNIHandleSet.java:114)
  i  SP 0x00007f6de5bfc4e0 IP 0x00007f6ddb41333a size=48    com.oracle.svm.agent.NativeImageAgent.constructJavaHandles(NativeImageAgent.java:126)
  A  SP 0x00007f6de5bfc4e0 IP 0x00007f6ddb41333a size=48    com.oracle.svm.jvmtiagentbase.JvmtiAgentBase.onVMStart(JvmtiAgentBase.java:275)
  A  SP 0x00007f6de5bfc510 IP 0x00007f6ddb314ee7 size=96    com.oracle.svm.core.code.IsolateEnterStub.JvmtiAgentBase_onVMStart_b239adacb81763565e6898e3884190995c759d91(IsolateEnterStub.java:0)
  
Threads:
  0x00005381ffe357c0 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "Reference Handler" - 0x00007f6dda7aef70, daemon, stack(0x00007f6dd9881000,0x00007f6dda080000)
  0x00005381ffe346c0 STATUS_IN_JAVA (PREVENT_VM_FROM_REACHING_SAFEPOINT) "main" - 0x00007f6dda7aeeb0, stack(0x00007f6de5500000,0x00007f6de5c00000)
  
No VMOperation in progress

The 30 most recent VM operation status changes:
  
VM mutexes:
  mutex "RealLog.backTracePrinterMutex" is unlocked.
  mutex "mainVMOperationControlWorkQueue" is unlocked.
  mutex "referencePendingList" is unlocked.
  mutex "thread" is unlocked.
  
Build time information:
  Version: 22.0.1+8, serial gc
  Platform: linux/amd64
  Page size: 4096
  Container support: true
  CPU features used for AOT compiled code: CX8, CMOV, FXSR, MMX, SSE, SSE2
  
Runtime information:
  CPU cores (container): unknown
  CPU cores (OS): 33
  Memory: 12288M
  Page size: 4096
  VM uptime: 0.051s
  Current timestamp: 1731712929696
  AOT compiled code: 0x00007f6ddb246000 - 0x00007f6ddbbc4a9f
  
OS information:
  Max threads: 8250967
  Max memory mappings: 4194304
  Max PID: 4048576
  
Heap settings and statistics:
  Supports isolates: true
  Heap base: 0x00007f6dda080000
  Object reference size: 8
  Reserved object header bits: 0b111
  Aligned chunk size: 524288
  Large array threshold: 131072
  Incremental collections: 0
  Complete collections: 0
  
Heap usage:
  Eden: 1.00M (0.00M in 0 aligned chunks, 0.00M in 0 unaligned chunks)
  Old: 0.00M (0.00M in 0 aligned chunks, 0.00M in 0 unaligned chunks)
  
Native image heap boundaries:
  ReadOnly Primitives: 0x00007f6dda100830 - 0x00007f6dda332cc0
  ReadOnly References: 0x00007f6dda332cc0 - 0x00007f6dda520f50
  ReadOnly Relocatables: 0x00007f6dda521000 - 0x00007f6dda602db0
  Writable Primitives: 0x00007f6dda603000 - 0x00007f6dda73eb88
  Writable References: 0x00007f6dda73eb88 - 0x00007f6ddaa48aa0
  Writable Huge: 0x0000000000000000 - 0x0000000000000000
  ReadOnly Huge: 0x00007f6ddaa80038 - 0x00007f6ddad21068
  
Heap chunks: E=eden, S=survivor, O=old, F=free; A=aligned chunk, U=unaligned chunk; T=to space

[wirthi]

@nreid260 thanks for your log. Can you clarify whether this is from GraalVM 22 or 23? Please post the exact version you are using. Also, what exactly do you mean with combining the agent from one version with a different version of GraalVM (that seems quite unsupported).

Graal 22.x is EOL already, we would not look into that. NPEs on GraalVM 23 do seem relevant, though.

[nreid260]

We're using the agent from Graal with our own (slightly modified) JDK. This is same JDK we use for our non-graalified Java apps. This produces the following compat table:

	Graal 22	Graal 23
JDK 21	GOOD	GOOD
JDK 24	BAD	BAD

[wirthi]

@nreid260 moving the Agent between versions is not supported from our side; I believe we have never tested that so far, and certainly have not put any effort into that to work. This likely explains the crash.

We do have a JDK21 version, and our latest is on 24 as well. We believe you should be using an agent from the right Java version and not try to transplant that across versions. CC @DSouzaM

[DSouzaM]

@ziyilin for the --agentpath setup, is the JVM the same GraalVM in which the agent is defined, or is it an OpenJDK JVM? As Christian mentioned above, it is recommended to use the same JVM, since the agent is configured to trace JDK internals differently, and these can vary between runs. We will add some checks to the agent to assert the versions match.

I could not reproduce this on master. Is it still an issue on your end (and if so, is it possible to upgrade to a newer release?).

[nreid260]

We ultimately reworked our builds to run the agent using the Graal JDK. It requires us to package JNI code differently than is done for our production apps, but it seems to work.