[lldb][test][win][x86_64] Fix XFAIL and XPASS on LLDB API tests #100477
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-lldb Author: Kendal Harland (kendalharland) ChangesI'm currently working on getting the LLDB test suites to pass on Windows x86_64 which is not currently included in LLVM CI. These tests are currently failing in this configuration. See #100474 See also #75936 Patch is 26.00 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/100477.diff 30 Files Affected:
diff --git a/lldb/test/API/commands/apropos/with-process/TestAproposWithProcess.py b/lldb/test/API/commands/apropos/with-process/TestAproposWithProcess.py
index 268317a4bf212..adeb7b4f906c5 100644
--- a/lldb/test/API/commands/apropos/with-process/TestAproposWithProcess.py
+++ b/lldb/test/API/commands/apropos/with-process/TestAproposWithProcess.py
@@ -2,8 +2,8 @@
Test that apropos env doesn't crash trying to touch the process plugin command
"""
-
import lldb
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
import lldbsuite.test.lldbutil as lldbutil
@@ -17,6 +17,7 @@ def setUp(self):
# Find the line number to break inside main().
self.line = line_number("main.cpp", "// break here")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_apropos_with_process(self):
"""Test that apropos env doesn't crash trying to touch the process plugin command."""
self.build()
diff --git a/lldb/test/API/commands/command/nested_alias/TestNestedAlias.py b/lldb/test/API/commands/command/nested_alias/TestNestedAlias.py
index 0919caa7d0056..315576afde703 100644
--- a/lldb/test/API/commands/command/nested_alias/TestNestedAlias.py
+++ b/lldb/test/API/commands/command/nested_alias/TestNestedAlias.py
@@ -2,8 +2,8 @@
Test that an alias can reference other aliases without crashing.
"""
-
import lldb
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
import lldbsuite.test.lldbutil as lldbutil
@@ -17,6 +17,7 @@ def setUp(self):
# Find the line number to break inside main().
self.line = line_number("main.cpp", "// break here")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_nested_alias(self):
"""Test that an alias can reference other aliases without crashing."""
self.build()
diff --git a/lldb/test/API/commands/expression/entry-bp/TestExprEntryBP.py b/lldb/test/API/commands/expression/entry-bp/TestExprEntryBP.py
index 1e7882b4d0236..0ee7e46d73cd1 100644
--- a/lldb/test/API/commands/expression/entry-bp/TestExprEntryBP.py
+++ b/lldb/test/API/commands/expression/entry-bp/TestExprEntryBP.py
@@ -4,12 +4,14 @@
import lldb
import lldbsuite.test.lldbutil as lldbutil
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
class ExprEntryBPTestCase(TestBase):
NO_DEBUG_INFO_TESTCASE = True
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr_entry_bp(self):
"""Tests expressions evaluation when the breakpoint on module's entry is set."""
self.build()
diff --git a/lldb/test/API/commands/memory/write/TestMemoryWrite.py b/lldb/test/API/commands/memory/write/TestMemoryWrite.py
index 45787243a614d..2924f1b97fa1a 100644
--- a/lldb/test/API/commands/memory/write/TestMemoryWrite.py
+++ b/lldb/test/API/commands/memory/write/TestMemoryWrite.py
@@ -39,6 +39,7 @@ def build_run_stop(self):
lldbutil.check_breakpoint(self, bpno=1, expected_hit_count=1)
@no_debug_info_test
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_memory_write(self):
"""Test the 'memory write' command for writing values and file contents."""
self.build_run_stop()
diff --git a/lldb/test/API/commands/settings/use_source_cache/TestUseSourceCache.py b/lldb/test/API/commands/settings/use_source_cache/TestUseSourceCache.py
index 421599080a9e5..cb58fab3304b6 100644
--- a/lldb/test/API/commands/settings/use_source_cache/TestUseSourceCache.py
+++ b/lldb/test/API/commands/settings/use_source_cache/TestUseSourceCache.py
@@ -18,7 +18,7 @@ def test_set_use_source_cache_false(self):
self.set_use_source_cache_and_test(False)
@skipIf(hostoslist=no_match(["windows"]))
- @skipIf(oslist=["windows"]) # Fails on windows 11
+ @expectedFailureAll(oslist=["windows"])
def test_set_use_source_cache_true(self):
"""Test that after 'set use-source-cache false', files are locked."""
self.set_use_source_cache_and_test(True)
diff --git a/lldb/test/API/functionalities/breakpoint/address_breakpoints/TestAddressBreakpoints.py b/lldb/test/API/functionalities/breakpoint/address_breakpoints/TestAddressBreakpoints.py
index 3ceccc6e6d641..6696880f8aa4a 100644
--- a/lldb/test/API/functionalities/breakpoint/address_breakpoints/TestAddressBreakpoints.py
+++ b/lldb/test/API/functionalities/breakpoint/address_breakpoints/TestAddressBreakpoints.py
@@ -2,9 +2,9 @@
Test address breakpoints set with shared library of SBAddress work correctly.
"""
-
import lldb
import lldbsuite.test.lldbutil as lldbutil
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -16,6 +16,7 @@ def test_address_breakpoints(self):
self.build()
self.address_breakpoints()
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def address_breakpoints(self):
"""Test address breakpoints set with shared library of SBAddress work correctly."""
target = self.createTestTarget()
diff --git a/lldb/test/API/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py b/lldb/test/API/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py
index 47cbc4139dc02..5cc30e7e7f103 100644
--- a/lldb/test/API/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py
+++ b/lldb/test/API/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py
@@ -2,9 +2,9 @@
Test that the breakpoint auto-continue flag works correctly.
"""
-
import lldb
import lldbsuite.test.lldbutil as lldbutil
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -21,6 +21,7 @@ def test_auto_continue_with_command(self):
self.build()
self.auto_continue_with_command()
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_auto_continue_on_location(self):
"""Set auto-continue on a location and make sure only that location continues"""
self.build()
diff --git a/lldb/test/API/functionalities/breakpoint/breakpoint_command/TestBreakpointCommandsFromPython.py b/lldb/test/API/functionalities/breakpoint/breakpoint_command/TestBreakpointCommandsFromPython.py
index 7889f08ba8dbb..bf245da05152c 100644
--- a/lldb/test/API/functionalities/breakpoint/breakpoint_command/TestBreakpointCommandsFromPython.py
+++ b/lldb/test/API/functionalities/breakpoint/breakpoint_command/TestBreakpointCommandsFromPython.py
@@ -13,6 +13,7 @@ class PythonBreakpointCommandSettingTestCase(TestBase):
NO_DEBUG_INFO_TESTCASE = True
@add_test_categories(["pyapi"])
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_step_out_python(self):
"""Test stepping out using a python breakpoint command."""
self.build()
diff --git a/lldb/test/API/functionalities/breakpoint/breakpoint_options/TestBreakpointOptions.py b/lldb/test/API/functionalities/breakpoint/breakpoint_options/TestBreakpointOptions.py
index d262b627195bc..28e18795e8291 100644
--- a/lldb/test/API/functionalities/breakpoint/breakpoint_options/TestBreakpointOptions.py
+++ b/lldb/test/API/functionalities/breakpoint/breakpoint_options/TestBreakpointOptions.py
@@ -2,13 +2,14 @@
Test breakpoint command for different options.
"""
-
import lldb
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
import lldbsuite.test.lldbutil as lldbutil
class BreakpointOptionsTestCase(TestBase):
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test(self):
"""Test breakpoint command for different options."""
self.build()
diff --git a/lldb/test/API/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py b/lldb/test/API/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py
index 3a7440a31677a..26406abb6d302 100644
--- a/lldb/test/API/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py
+++ b/lldb/test/API/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py
@@ -5,7 +5,6 @@
and eStopReasonPlanComplete when breakpoint's condition fails.
"""
-
import lldb
from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -56,6 +55,7 @@ def setUp(self):
)
self.assertIsNotNone(self.thread, "Didn't stop at breakpoint 1.")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_step_instruction(self):
# Count instructions between breakpoint_1 and breakpoint_4
contextList = self.target.FindFunctions("main", lldb.eFunctionNameTypeAuto)
diff --git a/lldb/test/API/functionalities/conditional_break/TestConditionalBreak.py b/lldb/test/API/functionalities/conditional_break/TestConditionalBreak.py
index 725e5d4722dd1..63e3d766ded1d 100644
--- a/lldb/test/API/functionalities/conditional_break/TestConditionalBreak.py
+++ b/lldb/test/API/functionalities/conditional_break/TestConditionalBreak.py
@@ -15,6 +15,7 @@
class ConditionalBreakTestCase(TestBase):
@add_test_categories(["pyapi"])
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_with_python(self):
"""Exercise some thread and frame APIs to break if c() is called by a()."""
self.build()
diff --git a/lldb/test/API/functionalities/memory/find/TestMemoryFind.py b/lldb/test/API/functionalities/memory/find/TestMemoryFind.py
index 09611cc808777..4f1cfe75a2066 100644
--- a/lldb/test/API/functionalities/memory/find/TestMemoryFind.py
+++ b/lldb/test/API/functionalities/memory/find/TestMemoryFind.py
@@ -2,7 +2,6 @@
Test the 'memory find' command.
"""
-
import lldb
from lldbsuite.test.lldbtest import *
import lldbsuite.test.lldbutil as lldbutil
@@ -16,6 +15,7 @@ def setUp(self):
# Find the line number to break inside main().
self.line = line_number("main.cpp", "// break here")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_memory_find(self):
"""Test the 'memory find' command."""
self.build()
diff --git a/lldb/test/API/functionalities/multiple-slides/TestMultipleSlides.py b/lldb/test/API/functionalities/multiple-slides/TestMultipleSlides.py
index 3d6b27fe68a1b..d27eed9c1b9bd 100644
--- a/lldb/test/API/functionalities/multiple-slides/TestMultipleSlides.py
+++ b/lldb/test/API/functionalities/multiple-slides/TestMultipleSlides.py
@@ -2,7 +2,6 @@
Test that a binary can be slid to different load addresses correctly
"""
-
import lldb
from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -12,6 +11,7 @@
class MultipleSlidesTestCase(TestBase):
NO_DEBUG_INFO_TESTCASE = True
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_mulitple_slides(self):
"""Test that a binary can be slid multiple times correctly."""
self.build()
diff --git a/lldb/test/API/functionalities/var_path/TestVarPath.py b/lldb/test/API/functionalities/var_path/TestVarPath.py
index 53c45f57f517d..40daf6174d2a6 100644
--- a/lldb/test/API/functionalities/var_path/TestVarPath.py
+++ b/lldb/test/API/functionalities/var_path/TestVarPath.py
@@ -2,10 +2,8 @@
Make sure the getting a variable path works and doesn't crash.
"""
-
import lldb
import lldbsuite.test.lldbutil as lldbutil
-from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
diff --git a/lldb/test/API/lang/c/anonymous/TestAnonymous.py b/lldb/test/API/lang/c/anonymous/TestAnonymous.py
index da342aaa8d8c4..a15476f47ba6f 100644
--- a/lldb/test/API/lang/c/anonymous/TestAnonymous.py
+++ b/lldb/test/API/lang/c/anonymous/TestAnonymous.py
@@ -1,6 +1,5 @@
"""Test that anonymous structs/unions are transparent to member access"""
-
import lldb
from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -12,6 +11,7 @@ class AnonymousTestCase(TestBase):
compiler="icc",
bugnumber="llvm.org/pr15036: LLDB generates an incorrect AST layout for an anonymous struct when DWARF is generated by ICC",
)
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr_nest(self):
self.build()
self.common_setup(self.line0)
@@ -23,6 +23,7 @@ def test_expr_nest(self):
self.expect("expression n->b", VARIABLES_DISPLAYED_CORRECTLY, substrs=["= 2"])
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr_child(self):
self.build()
self.common_setup(self.line1)
@@ -40,6 +41,7 @@ def test_expr_child(self):
compiler="icc",
bugnumber="llvm.org/pr15036: This particular regression was introduced by r181498",
)
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr_grandchild(self):
self.build()
self.common_setup(self.line2)
@@ -53,6 +55,7 @@ def test_expr_grandchild(self):
"expression g.child.b", VARIABLES_DISPLAYED_CORRECTLY, substrs=["= 2"]
)
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr_parent(self):
self.build()
if "clang" in self.getCompiler() and "3.4" in self.getCompilerVersion():
@@ -74,6 +77,7 @@ def test_expr_parent(self):
substrs=["(type_y) $", "dummy = 2"],
)
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr_null(self):
self.build()
self.common_setup(self.line2)
@@ -126,6 +130,7 @@ def test_child_by_name(self):
if not error.Success() or value != 0:
self.fail("failed to get the correct value for element a in n")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_nest_flat(self):
self.build()
self.common_setup(self.line2)
diff --git a/lldb/test/API/lang/c/array_types/TestArrayTypes.py b/lldb/test/API/lang/c/array_types/TestArrayTypes.py
index fc234f5842508..a1bad791f0dda 100644
--- a/lldb/test/API/lang/c/array_types/TestArrayTypes.py
+++ b/lldb/test/API/lang/c/array_types/TestArrayTypes.py
@@ -1,6 +1,5 @@
"""Test breakpoint by file/line number; and list variables with array types."""
-
import lldb
from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -14,6 +13,7 @@ def setUp(self):
# Find the line number to break inside main().
self.line = line_number("main.c", "// Set break point at this line.")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_and_run_command(self):
"""Test 'frame variable var_name' on some variables with array types."""
self.build()
@@ -80,6 +80,7 @@ def test_and_run_command(self):
)
@expectedFailureNetBSD
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
@add_test_categories(["pyapi"])
def test_and_python_api(self):
"""Use Python APIs to inspect variables with array types."""
diff --git a/lldb/test/API/lang/c/enum_types/TestEnumTypes.py b/lldb/test/API/lang/c/enum_types/TestEnumTypes.py
index 0015c8f478578..5093b9d69700d 100644
--- a/lldb/test/API/lang/c/enum_types/TestEnumTypes.py
+++ b/lldb/test/API/lang/c/enum_types/TestEnumTypes.py
@@ -1,6 +1,5 @@
"""Look up enum type information and check for correct display."""
-
import lldb
from lldbsuite.test.lldbtest import *
import lldbsuite.test.lldbutil as lldbutil
@@ -14,6 +13,7 @@ def setUp(self):
# Find the line number to break inside main().
self.line = line_number("main.c", "// Set break point at this line.")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_command_line(self):
"""Test 'image lookup -t enum_test_days' and check for correct display and enum value printing."""
self.build()
diff --git a/lldb/test/API/lang/c/forward/TestForwardDeclaration.py b/lldb/test/API/lang/c/forward/TestForwardDeclaration.py
index 5d05f255623f4..fb17928e5d57f 100644
--- a/lldb/test/API/lang/c/forward/TestForwardDeclaration.py
+++ b/lldb/test/API/lang/c/forward/TestForwardDeclaration.py
@@ -1,6 +1,5 @@
"""Test that forward declaration of a data structure gets resolved correctly."""
-
import lldb
from lldbsuite.test.lldbtest import *
from lldbsuite.test.decorators import *
@@ -46,6 +45,7 @@ def do_test(self, dictionary=None):
substrs=["(bar)", "(int) a = 1", "(int) b = 2"],
)
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test(self):
self.do_test()
diff --git a/lldb/test/API/lang/c/function_types/TestFunctionTypes.py b/lldb/test/API/lang/c/function_types/TestFunctionTypes.py
index 2f6bb7fc0bb16..84691b2a383fd 100644
--- a/lldb/test/API/lang/c/function_types/TestFunctionTypes.py
+++ b/lldb/test/API/lang/c/function_types/TestFunctionTypes.py
@@ -1,6 +1,5 @@
"""Test variable with function ptr type and that break on the function works."""
-
import lldb
from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -14,6 +13,7 @@ def setUp(self):
# Find the line number to break inside main().
self.line = line_number("main.c", "// Set break point at this line.")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test(self):
"""Test 'callback' has function ptr type, then break on the function."""
self.build()
diff --git a/lldb/test/API/lang/c/non-mangled/TestCNonMangled.py b/lldb/test/API/lang/c/non-mangled/TestCNonMangled.py
index 6f7ef247b063a..f7aa1f01ccf82 100644
--- a/lldb/test/API/lang/c/non-mangled/TestCNonMangled.py
+++ b/lldb/test/API/lang/c/non-mangled/TestCNonMangled.py
@@ -1,8 +1,10 @@
import lldbsuite.test.lldbutil as lldbutil
+from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
class TestCase(TestBase):
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_functions_having_dlang_mangling_prefix(self):
"""
Ensure C functions with a '_D' prefix alone are not mistakenly treated
diff --git a/lldb/test/API/lang/c/register_variables/TestRegisterVariables.py b/lldb/test/API/lang/c/register_variables/TestRegisterVariables.py
index 0e51d4f7ff468..cfe2d994de59c 100644
--- a/lldb/test/API/lang/c/register_variables/TestRegisterVariables.py
+++ b/lldb/test/API/lang/c/register_variables/TestRegisterVariables.py
@@ -18,6 +18,7 @@ class RegisterVariableTestCase(TestBase):
compiler="gcc", compiler_version=[">=", "4.8.2"], archs=["i386"]
)
@expectedFailureAll(compiler="gcc", compiler_version=["<", "4.9"], archs=["x86_64"])
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_and_run_command(self):
"""Test expressions on register values."""
diff --git a/lldb/test/API/lang/c/set_values/TestSetValues.py b/lldb/test/API/lang/c/set_values/TestSetValues.py
index 0d697d6719d5f..251c7abc54780 100644
--- a/lldb/test/API/lang/c/set_values/TestSetValues.py
+++ b/lldb/test/API/lang/c/set_values/TestSetValues.py
@@ -1,6 +1,5 @@
"""Test settings and readings of program variables."""
-
import lldb
from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
@@ -18,6 +17,7 @@ def setUp(self):
self.line4 = line_number("main.c", "// Set break point #4.")
self.line5 = line_number("main.c", "// Set break point #5.")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test(self):
"""Test settings and readings of program variables."""
self.build()
diff --git a/lldb/test/API/lang/c/shared_lib/TestSharedLib.py b/lldb/test/API/lang/c/shared_lib/TestSharedLib.py
index e0994aae76169..9da93730f1d24 100644
--- a/lldb/test/API/lang/c/shared_lib/TestSharedLib.py
+++ b/lldb/test/API/lang/c/shared_lib/TestSharedLib.py
@@ -1,8 +1,7 @@
"""Test that types defined in shared libraries work correctly."""
-
-import unittest
import lldb
+import unittest
from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
import lldbsuite.test.lldbutil as lldbutil
@@ -27,10 +26,12 @@ def common_test_expr(self, preload_symbols):
self.expect("expression GetMeASubFoo(my_foo_ptr)", startstr="(sub_foo *) $")
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr(self):
"""Test that types work when defined in a shared library and forward-declared in the main executable"""
self.common_test_expr(True)
+ @expectedFailureAll(triple="x86_64-.*-windows.*")
def test_expr_no_preload(self):
"""Test that types work when defined in a shared library and forward-declared in the main executable, but wit...
[truncated]
|
|
✅ With the latest revision this PR passed the Python code formatter. |
|
Rather than a regex for the triple, can you use a combination of |
Yes. I will fix this in each of the PRs I recently mailed you. Will fix the dark formatter issue above too. |
kendalharland
force-pushed
the
swiftlang/kendal/xfail-failing-lldb-api-tests
branch
from
f8e40b9 to
3a7e979
Compare
Done |
labath
reviewed
Jul 25, 2024
There was a problem hiding this comment.
I'm somewhat surprised to see this many x86 xfails, given that we already have an arm64 windows bot, and that none of the tests I see here are very architecture-specific.
I wonder if this could be due to how you've configured your build. @DavidSpickett, do you have any insight here?
lldb/test/API/commands/settings/use_source_cache/TestUseSourceCache.py
Outdated
Show resolved
Hide resolved
Notable things about the Windows on Arm buildbot:
I'd show a full test log but for whatever reason, it's not being verbose despite being asked to, working on that. |
kendalharland
force-pushed
the
swiftlang/kendal/xfail-failing-lldb-api-tests
branch
from
3a7e979 to
5a32aac
Compare
These test are passing locally on win x86_64 using the following configuration (some options ellided for brevity) CMAKE_C_COMPILER clang-cl.exe CMAKE_CXX_COMPILER clang-cl.exe CMAKE_BUILD_TYPE Release CMAKE_LINKER lld-link.exe LLVM_HOST_TRIPLE="x86_64-unknown-windows-msvc" LLDB_TEST_USER_ARGS=--skip-category=watchpoint LLDB_ENFORCE_STRICT_TEST_REQUIREMENTS=1
kendalharland
force-pushed
the
swiftlang/kendal/xfail-failing-lldb-api-tests
branch
from
5a32aac to
ac737ef
Compare
|
Thanks for the assistance @labath and @DavidSpickett. After fixing my cmake configuration I'm actually seeing a few tests unexpectedly pass with different options: Updated in the latest commit |
kendalharland
changed the title
kendalharland
commented
Jul 25, 2024
| @@ -52,6 +52,7 @@ def test_negative_indexing(self): | |||
| self.build() | |||
| self.validate_negative_indexing() | |||
|
|
|||
| @expectedFailureAll(oslist=["windows"], archs=["x86_64"]) | |||
There was a problem hiding this comment.
Without this line, I am still seeing this test fail with:
======================================================================
FAIL: test_StepInstruction_dwarf (TestThreadAPI.ThreadAPITestCase.test_StepInstruction_dwarf)
Test that StepInstruction preserves the plan stack.
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\workspace\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 1756, in test_method
return attrvalue(self)
^^^^^^^^^^^^^^^
File "C:\workspace\llvm-project\lldb\test\API\python_api\thread\TestThreadAPI.py", line 58, in test_StepInstruction
self.step_instruction_in_called_function()
File "C:\workspace\llvm-project\lldb\test\API\python_api\thread\TestThreadAPI.py", line 334, in step_instruction_in_called_function
self.assertRegex(
AssertionError: Regex didn't match: '.*call_me\\(bool\\)$' not found in 'main' : Stopped in call_me(bool)
Config=x86_64-C:\workspace\llvm-project\build\bin\clang.exe
----------------------------------------------------------------------
There was a problem hiding this comment.
I recently updated this for Windows on Arm 3e06392, that's where that regex came from.
We need to know if it's a cosmetic difference or whether it's stopped at a completely different place. Running the test with lldb-dotest and the -t flag will hopefully show you what it was doing.
There was a problem hiding this comment.
Thanks! I'll try that and update here.
There was a problem hiding this comment.
Here's the full output with 3e06392 cherry-picked. https://pastebin.com/LQ9QReBr
There was a problem hiding this comment.
If I change this line:
self.runCmd("expr -i 0 -- call_me(true)", check=False)
to this:
self.runCmd("expr -i 0 -- call_me(true)", check=True)
I see this error:
======================================================================
FAIL: test_StepInstruction_dwarf (TestThreadAPI.ThreadAPITestCase.test_StepInstruction_dwarf)
Test that StepInstruction preserves the plan stack.
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\workspace\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 1761, in test_method
return attrvalue(self)
^^^^^^^^^^^^^^^
File "C:\workspace\llvm-project\lldb\test\API\python_api\thread\TestThreadAPI.py", line 16, in test_StepInstruction
self.step_instruction_in_called_function()
File "C:\workspace\llvm-project\lldb\test\API\python_api\thread\TestThreadAPI.py", line 291, in step_instruction_in_called_function
self.runCmd("expr -i 0 -- call_me(true)", check=True)
File "C:\workspace\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 1006, in runCmd
self.assertTrue(self.res.Succeeded(), msg if (msg) else CMD_MSG(cmd))
AssertionError: False is not true : Command 'expr -i 0 -- call_me(true)
Error output:
error: Couldn't look up symbols:
void call_me(bool)
Hint: The expression tried to call a function that is not present in the target, perhaps because it was optimized out by the compiler.
' did not return successfully
Config=x86_64-C:\workspace\llvm-project\build\bin\clang.exe
----------------------------------------------------------------------
There was a problem hiding this comment.
Seems like my Clang invocation is generating different debug information than expected:
PS C:\workspace\llvm-project> llvm-pdbutil.exe pretty -all C:\workspace\llvm-project\build\lldb-test-build.noindex\python_api\thread\TestThreadAPI.test_StepInstruction_dwarf\a.pdb | Select-String -Pattern call_me
func [0x00001000+ 0 - 0x00001025-37 | sizeof= 37] (FPO) void __cdecl call_me(bool should_spin)
func [0x00001000+ 0 - 0x00001025-37 | sizeof= 37] (FPO) void __cdecl call_me(bool should_spin)
public [0x00001000] ?call_me@@YAX_N@Z
Is the issue here that debug symbols have the additional should_spin parameter name, thus LLDB is not able to look them up since it is search for void call_me(bool)?
There was a problem hiding this comment.
I see the same function name/signature.
Btw, I'd recommend adding --env;LLDB_USE_LLDB_SERVER=1 to LLDB_TEST_USER_ARGS cmake argument. Without it, lldb can't even stop on a breakpoint on my machine. It may be helpful if you haven't set it yet.
kendalharland
changed the title
kendalharland
changed the title
I've fixed this, you can get a full test list by downloading the log file e.g. https://lab.llvm.org/buildbot/#/builders/141/builds/1139/steps/6/logs/stdio. |
labath
reviewed
Jul 26, 2024
There was a problem hiding this comment.
This looks much better, and it looks like @dzhidzhoev is just about to remove some of the same XFAILS as well (in #100628). I suggest you two collaborate on the remaining ones.
dzhidzhoev
reviewed
Jul 26, 2024
| @@ -12,6 +12,7 @@ | |||
| class MultipleSlidesTestCase(TestBase): | |||
| NO_DEBUG_INFO_TESTCASE = True | |||
|
|
|||
| @expectedFailureAll(oslist=["windows"], archs=["x86_64"]) | |||
There was a problem hiding this comment.
Could you provide a failure log for this? It passes on our CI (Windows native x86_64).
********************
PASS: lldb-api :: functionalities/multiple-slides/TestMultipleSlides.py (530 of 1191)
Script:
--
C:/Python312/python.exe C:/lldb\test\API\dotest.py -u CXXFLAGS -u CFLAGS --env OBJCOPY=C:/build-lldb-native/./bin/llvm-objcopy.exe --env STRIP=C:/build-lldb-native/./bin/llvm-strip.exe --env LLVM_LIBS_DIR=C:/build-lldb-native/./lib --env LLVM_INCLUDE_DIR=C:/build-lldb-native/include --env LLVM_TOOLS_DIR=C:/build-lldb-native/./bin --arch x86_64 --build-dir C:/build-lldb-native/lldb-test-build.noindex --lldb-module-cache-dir C:/build-lldb-native/lldb-test-build.noindex/module-cache-lldb\lldb-api --clang-module-cache-dir C:/build-lldb-native/lldb-test-build.noindex/module-cache-clang\lldb-api --executable C:/build-lldb-native/./bin/lldb.exe --make c:/gl/b1/3rdpaty/bin/make-wfix.exe --compiler C:/build-lldb-native/./bin/clang.exe --dsymutil C:/build-lldb-native/./bin/dsymutil.exe --llvm-tools-dir C:/build-lldb-native/./bin --lldb-obj-root C:/build-lldb-native/tools/lldb --lldb-libs-dir C:/build-lldb-native/./lib --skip-category=watchpoint C:\lldb\test\API\functionalities\multiple-slides -p TestMultipleSlides.py
--
Exit Code: 0
Command Output (stdout):
--
lldb version 20.0.0git (https://gitlab-ci-token:[email protected]/accesssoftek/lldb-test-scripts.git revision cbd39658b6aed4dbb66a2b775096974c9087e93d)
clang revision cbd39658b6aed4dbb66a2b775096974c9087e93d
llvm revision cbd39658b6aed4dbb66a2b775096974c9087e93d
Skipping the following test categories: ['watchpoint', 'libc++', 'libstdcxx', 'dwo', 'dsym', 'gmodules', 'debugserver', 'objc', 'fork', 'pexpect']
--
Command Output (stderr):
--
PASS: LLDB (C:\build-lldb-native\bin\clang.exe-x86_64) :: test_mulitple_slides (TestMultipleSlides.MultipleSlidesTestCase.test_mulitple_slides)
----------------------------------------------------------------------
Ran 1 test in 2.447s
OK
--
There was a problem hiding this comment.
Also, it may be worth cherry-picking this 58d22f1 to have build commands and outputs in the log without tracing hassle.
There was a problem hiding this comment.
Ty! I'll try that out.
There was a problem hiding this comment.
Looks like after my CMake chages this one is passing now. I'll remove it from the PR
There was a problem hiding this comment.
Ah, sorry I spoke too soon. On windows I am seeing this fail. The test is built without debug information because this is in its makefile:
C_SOURCES := main.c
MAKE_DSYM := NO
include Makefile.rules
main.o: main.c
$(CC) $(CFLAGS_NO_DEBUG) -c $< -o $@
@dzhidzhoev I am curious to know how this is passing for you? Without specifying -g or -gdwarf to Clang, lldb cannot resolve the name first and fails for me locally. But adding either of those flags fixes the problem.
There was a problem hiding this comment.
For comparison, my objdump output shows no symbol table:
PS C:\workspace\llvm-project> C:\workspace\llvm-project\build\bin\llvm-objdump -t C:\workspace\llvm-project\build\lldb-test-build.noindex\functionalities\multiple-slides\TestMultipleSlides.test_mulitple_slides\a.out
C:\workspace\llvm-project\build\lldb-test-build.noindex\functionalities\multiple-slides\TestMultipleSlides.test_mulitple_slides\a.out: file format coff-x86-64
SYMBOL TABLE:
There was a problem hiding this comment.
$ F:/Users/vdzhidzhoev/temp/build-lldb-native/bin/clang.exe main.o -gdwarf -O0 -m64 -IF:\Users\vdzhidzhoev\temp\llvm-project\lldb\packages\Python\lldbsuite\test\make/../../../../..//include -IF:/Users/vdzhidzhoev/temp/build-lldb-native/tools/lldb/include -IF:\Users\vdzhidzhoev\temp\llvm-project\lldb\test\API\functionalities\multiple-slides -IF:\Users\vdzhidzhoev\temp\llvm-project\lldb\packages\Python\lldbsuite\test\make -include F:\Users\vdzhidzhoev\temp\llvm-project\lldb\packages\Python\lldbsuite\test\make/test_common.h -fno-limit-debug-info -fuse-ld=lld --driver-mode=g++ -o "a.out" -v
clang version 20.0.0git (8d8d23fec72ad6ff77e15e7627ab8cc26ac36d4c)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: F:\Users\vdzhidzhoev\temp\build-lldb-native\bin
Build config: +assertions
"F:\\Users\\vdzhidzhoev\\temp\\build-lldb-native\\bin\\lld-link" -out:a.out -defaultlib:libcmt -defaultlib:oldnames -nologo -debug main.o
There was a problem hiding this comment.
For comparison, my objdump output shows no symbol table:
PS C:\workspace\llvm-project> C:\workspace\llvm-project\build\bin\llvm-objdump -t C:\workspace\llvm-project\build\lldb-test-build.noindex\functionalities\multiple-slides\TestMultipleSlides.test_mulitple_slides\a.out C:\workspace\llvm-project\build\lldb-test-build.noindex\functionalities\multiple-slides\TestMultipleSlides.test_mulitple_slides\a.out: file format coff-x86-64 SYMBOL TABLE:
I have the same for a.out. The dump I provided (with "first") was for main.o.
Try launching bin/lldb, "file a.out", "image dump symtab a.out", and there should be "first" symbol. Beware that it will probably show A LOT of entries.
There was a problem hiding this comment.
My output for image dump symtab a.out is empty (just prints the path to a.out). However, if I edit
(lldb) image dump symfile
Dumping debug symbols for 1 modules.
SymbolFile dwarf (C:\workspace\llvm-project\build\lldb-test-build.noindex\functionalities\multiple-slides\TestMultipleSlides.test_mulitple_slides\a.out)
Types:
Compile units:
Symtab, file = C:\workspace\llvm-project\build\lldb-test-build.noindex\functionalities\multiple-slides\TestMultipleSlides.test_mulitple_slides\a.out, num_symbols = 0
Manual DWARF index for (x86_64) 'a.out':
Function basenames:
INFO/0000006f "main"
Function fullnames:
INFO/0000006f "main"
Function methods:
Function selectors:
Objective-C class selectors:
Globals and statics:
INFO/0000005a "second"
INFO/0000002a "first"
Types:
INFO/00000053 "__ARRAY_SIZE_TYPE__"
INFO/0000004c "int"
Namespaces:
Do you have a patched or older version of that file, or have settings in your environment that force -gdwarf to be passed to clang? I ask because although I am now able to see debug info, I still am wondering if there's something in my local environment that is causing different behavior. Additionally, LLDB produces no output when I run image lookup -s first, which means the tests still fail locally since they can't resolve the symbols first and second, but lldb can still resolve first and second if I instead run ta v first or ta v second. Something is preventing it from looking up the symbols for first and second even though they are clearly present as globals
There was a problem hiding this comment.
I was able to figure this out. My CMakeCache had -D LLVM_ENABLE_DIA_SDK="OFF" which I must have set in a previous build. I don't recall why I set this, must have been trying to repro someone else's build and forgot. This indeed passes for me now. Thanks so much for your patience and helping me debug!
This was referenced Jul 26, 2024
|
Closing this as I've been able to bring my CMake configuration in alignment with other contributors building on Windows, and these tests are now passing locally for me. Thanks everyone for the help with debugging! |
|
Thanks for your efforts @kendalharland. If you think any of these details are worth documenting (https://lldb.llvm.org/index.html), please do so. The page sources are in |
kendalharland
deleted the
swiftlang/kendal/xfail-failing-lldb-api-tests
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm currently working on getting the LLDB test suites to pass on Windows x86_64 which is not currently included in LLVM CI. These tests are currently failing in this configuration.
See #100474
See also #75936