lldb's HostTests cause Windows Terminal to completely exit

[DavidSpickett]

Found this on Linaro's Windows on Arm bot machine, but it's not causing failures during bot runs. Build lldb as normal in a Windows Terminal (not conhost) and then:

ninja check-lldb-unit

When this gets to host tests, the current terminal and any other terminal windows you have open will close. Same thing happens if you run HostTests.exe directly.

I narrowed it down to some interaction between these two tests:

FindProcesses
LaunchProcessDuplicatesHandle

Some times I can get what seems like just one to fail, most of the time both have to be enabled.

Either this is specific to being in an interactive session, or being run via buildbot adds some extra layer of process that prevents this happening. Or buildbot is using conhost not windows terminal, this does not happen in conhost.

[llvmbot]

@llvm/issue-subscribers-lldb

Author: David Spickett (DavidSpickett)

Found this on Linaro's Windows on Arm bot. Build lldb as normal in a Windows Terminal (not conhost) and then: ``` ninja check-lldb-unit ``` When this gets to host tests, the current terminal and any other terminal windows will close at some point during the tests.

I narrowed it down to:

TEST(Host, FindProcesses) {
  SubsystemRAII<FileSystem, HostInfo> subsystems;

  if (test_arg != 0) {
    // Give the parent time to retrieve information about self.
    // It will kill self when it is done.
    std::this_thread::sleep_for(std::chrono::seconds(10));
    exit(0);
  }

  bool foundPID = false;
  ProcessLaunchInfo info;
  ProcessInstanceInfoList processes;
  ProcessInstanceInfoMatch match(TestMainArgv0, NameMatch::Equals);
  info.SetExecutableFile(FileSpec(TestMainArgv0),
                         /*add_exe_file_as_first_arg=*/true);
  info.GetArguments().AppendArgument("--gtest_filter=Host.FindProcesses");
  info.GetArguments().AppendArgument("--test-arg=48");
  std::promise<int> exit_status;
  info.SetMonitorProcessCallback([&](lldb::pid_t pid, int signal, int status) {
    exit_status.set_value(status);
  });
  ASSERT_THAT_ERROR(Host::LaunchProcess(info).takeError(), Succeeded());
  ASSERT_TRUE(Host::FindProcesses(match, processes));
  for (const auto &process : processes) {
    if (process.GetProcessID() == info.GetProcessID()) {
      ASSERT_EQ(process.GetExecutableFile().GetFilename(),
                info.GetExecutableFile().GetFilename());
      foundPID = true;
    }
  }
  ASSERT_TRUE(foundPID);
  auto clean_up = llvm::make_scope_exit([&] {
    Host::Kill(info.GetProcessID(), SIGKILL);
    exit_status.get_future().get();
  });
}

Same thing happens if you run HostTests.exe directly.

Working theory is that it is finding the wrong process and managing to SIGKILL the terminal itself.

[DavidSpickett]

Still a mystery, but some things I found:

• Most consistent when FindProcesses and LaunchProcessDuplicatesHandle are both run.
• Does not crash the session if you use conhost.
• Or if you run the terminal in administrator mode.
• Adding std::this_thread::sleep_for(std::chrono::seconds(5)); before FindProcesses looks for the processes dodges the issue.

I thought maybe one test was killing the other's sub-process, but FindProcesses does check that the PID matches the one that it launched.

[DavidSpickett]

More fun details: if you run the unit test exe in conhost, the conhost is fine but any other Windows Terminals you have open will close. I think this is because the new processes are spawned in Windows Terminals not new conhost windows.

[DavidSpickett]

Used procmon to monitor a Windows Terminal that was not running the tests.

Exit Status: -1073741819, User Time: 0.6250000 seconds, Kernel Time: 0.6875000 seconds, Private Bytes: 29,224,960, Peak Private Bytes: 45,359,104, Working Set: 102,559,744, Peak Working Set: 117,100,544

0xC0000005 is an access violation.

So I got Visual Studio to tell me the same thing:

• Run a Windows Terminal
• Attach to it in Visual Studio
• Run test exe in conhost
• Several new Windows Terminal windows are spawned
• They hang around doing nothing for a bit
• Then stop responding
• All Windows Terminal windows close

Exception thrown at 0x00007FFB4451DF0C (Microsoft.Terminal.Control.dll) in WindowsTerminal.exe: 0xC0000005: Access violation reading location 0x000000000000002C.

Callstack:

Microsoft.Terminal.Control.dll!00007ffb4451df0c()
Microsoft.Terminal.Control.dll!00007ffb445464cc()
Microsoft.Terminal.Control.dll!00007ffb44566cd4()
TerminalConnection.dll!00007ffb5a069554()
TerminalConnection.dll!00007ffb5a067ebc()
TerminalConnection.dll!00007ffb5a08afe8()
TerminalConnection.dll!00007ffb5a08a4f8()
kernel32.dll!00007ffb76b922d0()
ntdll.dll!00007ffb790b5b2c()

It's trying to offset off of a null pointer.

Trying to build a debug version of Windows Terminal locally so I can get a source location.

Found microsoft/terminal#18458 which is similar but no one there has a reproducer. I don't either really, it's still "build all of lldb", but hoping to cut it down once I understand more.

When I run the tests exe in conhost, even when the spawned Windows Terminals crash, the exe does finish succesfully and conhost is fine. So this is likely why the buildbot never complained about this, the tests do actually work, it's the tear down having problems and I think problems outside our code.

[DavidSpickett]

This is a Windows Terminal problem, reported as microsoft/terminal#19049.