[asan][win] 'CreateThread' leaks on Windows

[GkvJwa]

When I used asan for detection on Windows, I found that oom occurred every time

Judging from vmmap, there is a large amount of Private Data that has not been released after new.

Due to the call to Virtualalloc, I spent a lot of time finding the cause of the problem

1.Call the Virtualalloc here to create space

llvm-project/compiler-rt/lib/asan/asan_thread.cpp

Lines 96 to 112 in 7ae78a6

	AsanThread *AsanThread::Create(const void *start_data, uptr data_size,
	u32 parent_tid, StackTrace *stack,
	bool detached) {
	uptr PageSize = GetPageSizeCached();
	uptr size = RoundUpTo(sizeof(AsanThread), PageSize);
	AsanThread *thread = (AsanThread *)MmapOrDie(size, __func__);
	if (data_size) {
	uptr availible_size = (uptr)thread + size - (uptr)(thread->start_data_);
	CHECK_LE(data_size, availible_size);
	internal_memcpy(thread->start_data_, start_data, data_size);
	}
	asanThreadRegistry().CreateThread(0, detached, parent_tid,
	stack ? StackDepotPut(*stack) : 0, thread);
	return thread;
	}

2.After start_routine, Destory it

llvm-project/compiler-rt/lib/asan/asan_win.cpp

Lines 138 to 153 in 7ae78a6

	static thread_return_t THREAD_CALLING_CONV asan_thread_start(void *arg) {
	AsanThread *t = (AsanThread *)arg;
	SetCurrentThread(t);
	t->ThreadStart(GetTid());
	ThreadStartParams params;
	t->GetStartData(params);
	auto res = (*params.start_routine)(params.arg);
	t->Destroy(); // POSIX calls this from TSD destructor.
	return res;
	}
	INTERCEPTOR_WINAPI(HANDLE, CreateThread, LPSECURITY_ATTRIBUTES security,
	SIZE_T stack_size, LPTHREAD_START_ROUTINE start_routine,

The problem here is that when a thread ends on Windows, it will not be called next

  t->Destroy();  // POSIX calls this from TSD destructor.

template <typename ThreadProcedure, bool Ex>
static unsigned long WINAPI thread_start(void* const parameter) throw()
{
   ...

    __try
    {
        ThreadProcedure const procedure = reinterpret_cast<ThreadProcedure>(context->_procedure);
        if constexpr (Ex)
        {
            _endthreadex(procedure(context->_context));
        }

static void __cdecl common_end_thread(unsigned int const return_code) throw()
{
    __acrt_ptd* const ptd = __acrt_getptd_noexit();
    if (!ptd)
    {
        ExitThread(return_code);
    }

    __acrt_thread_parameter* const parameter = ptd->_beginthread_context;
    if (!parameter)
    {
        ExitThread(return_code);
    }

    if (parameter->_initialized_apartment)
    {
        __acrt_RoUninitialize();
    }

    if (parameter->_thread_handle != INVALID_HANDLE_VALUE && parameter->_thread_handle != nullptr)
    {
        CloseHandle(parameter->_thread_handle);
    }

    if (parameter->_module_handle != INVALID_HANDLE_VALUE && parameter->_module_handle != nullptr)
    {
        FreeLibraryAndExitThread(parameter->_module_handle, return_code);
    }

Usually the system calls FreeLibraryAndExitThread to end the thread, As a result, memory cannot be released.
And because my test program happened to frequently create threads and end them normally quickly, it eventually led to oom.

[GkvJwa]

@zmodem Hello hans, can you help me look into this problem

I have thought about some ways, such as hook FreeLibraryAndExitThread(NtTerminateThread) to solve this problem. Is there a better way?