Skip to content

kill locking processes before doing actions that write to artifacts directories #49868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 56 additions & 4 deletions eng/common/build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Param(
# Unset 'Platform' environment variable to avoid unwanted collision in InstallDotNetCore.targets file
# some computer has this env var defined (e.g. Some HP)
if($env:Platform) {
$env:Platform=""
$env:Platform=""
}
function Print-Usage() {
Write-Host "Common settings:"
Expand Down Expand Up @@ -108,10 +108,10 @@ function Build {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
# Explicitly set the type as string[] because otherwise PowerShell would make this char[] if $properties is empty.
[string[]] $msbuildArgs = $properties
# Resolve relative project paths into full paths

# Resolve relative project paths into full paths
$projects = ($projects.Split(';').ForEach({Resolve-Path $_}) -join ';')

$msbuildArgs += "/p:Projects=$projects"
$properties = $msbuildArgs
}
Expand Down Expand Up @@ -139,9 +139,59 @@ function Build {
@properties
}

function Stop-ArtifactLockers {
<#
.SYNOPSIS
Terminates dotnet processes that are holding locks on artifacts DLLs

.DESCRIPTION
This function finds dotnet processes spawned from .dotnet/dotnet.exe that have file handles
open to DLL files in the artifacts directory and terminates them to prevent build conflicts.

.PARAMETER RepoRoot
The root directory of the repository. Defaults to current location.
#>
param(
[string]$RepoRoot = (Get-Location).Path
)

$artifactsPath = Join-Path $RepoRoot 'artifacts'

# Exit early if artifacts directory doesn't exist
if (-not (Test-Path $artifactsPath)) {
return
}

# Find dotnet processes spawned from this repository's .dotnet directory
$repoDotnetPath = Join-Path $RepoRoot '.dotnet\dotnet.exe'
$artifactsRoot = Join-Path $RepoRoot 'artifacts'
$dotnetExes = @()
try {
$dotnetExes = Get-Process -Name 'dotnet' -ErrorAction SilentlyContinue | Where-Object { $_.Path -eq $repoDotnetPath -and $_.CommandLine.StartsWith($artifactsRoot) }
Copy link
Preview

Copilot AI Jul 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic differs from the bash version - this checks if CommandLine starts with artifacts path, while bash checks if CommandLine contains artifacts DLL paths. This inconsistency could lead to different behavior between platforms.

Suggested change
$dotnetExes = Get-Process -Name 'dotnet' -ErrorAction SilentlyContinue | Where-Object { $_.Path -eq $repoDotnetPath -and $_.CommandLine.StartsWith($artifactsRoot) }
$dotnetExes = Get-Process -Name 'dotnet' -ErrorAction SilentlyContinue | Where-Object { $_.Path -eq $repoDotnetPath -and $_.CommandLine -match [regex]::Escape($artifactsRoot) }

Copilot uses AI. Check for mistakes.

}
catch {
return
}

if ($dotnetExes.Count -eq 0) {
return
}

# Check each dotnet process for locks on artifacts DLLs
foreach ($process in $dotnetExes) {
try {
$process.Kill()
}
catch {
# Silently continue if we can't check or kill a process
}
}
}

try {
if ($clean) {
if (Test-Path $ArtifactsDir) {
Stop-ArtifactLockers -RepoRoot $RepoRoot
Remove-Item -Recurse -Force $ArtifactsDir
Write-Host 'Artifacts directory deleted.'
}
Expand All @@ -167,6 +217,8 @@ try {
InitializeNativeTools
}

Stop-ArtifactLockers -RepoRoot $RepoRoot

Build
}
catch {
Expand Down
122 changes: 122 additions & 0 deletions eng/common/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -262,8 +262,127 @@ function Build {
ExitWithExitCode 0
}

function Stop-ArtifactLockers {
# Terminates dotnet processes that are holding locks on artifacts shared libraries
#
# This function finds dotnet processes spawned from <repo>/.dotnet/dotnet that have file handles
# open to shared library files in the artifacts directory and terminates them to prevent build conflicts.
#
# Parameters:
# $1 - RepoRoot: The root directory of the repository (required)

local repo_root="${1:-$(pwd)}"
local artifacts_path="$repo_root/artifacts"

# Exit early if artifacts directory doesn't exist
if [[ ! -d "$artifacts_path" ]]; then
return 0
fi

# Find dotnet processes spawned from this repository's .dotnet directory
local repo_dotnet_path="$repo_root/.dotnet/dotnet"
local dotnet_pids=()

# Get all dotnet processes and filter by exact path
if command -v pgrep >/dev/null 2>&1; then
# Use pgrep if available (more efficient)
while IFS= read -r line; do
local pid
local path
pid=$(echo "$line" | cut -d' ' -f1)
path=$(echo "$line" | cut -d' ' -f2-)
if [[ "$path" == "$repo_dotnet_path" ]]; then
dotnet_pids+=("$pid")
fi
done < <(pgrep -f dotnet -l 2>/dev/null | grep -E "^[0-9]+ .*dotnet$" || true)
Copy link
Preview

Copilot AI Jul 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern .*dotnet$ will match any process ending with 'dotnet', not necessarily the exact dotnet executable path. This could match unrelated processes with 'dotnet' in their name.

Suggested change
done < <(pgrep -f dotnet -l 2>/dev/null | grep -E "^[0-9]+ .*dotnet$" || true)
done < <(pgrep -f dotnet -l 2>/dev/null | grep -E "^[0-9]+ dotnet$" || true)

Copilot uses AI. Check for mistakes.

else
# Fallback to ps if pgrep is not available
while IFS= read -r line; do
local pid
local cmd
pid=$(echo "$line" | awk '{print $2}')
cmd=$(echo "$line" | awk '{for(i=11;i<=NF;i++) printf "%s ", $i; print ""}' | sed 's/ $//')
if [[ "$cmd" == "$repo_dotnet_path" ]]; then
dotnet_pids+=("$pid")
fi
done < <(ps aux | grep dotnet | grep -v grep || true)
fi

if [[ ${#dotnet_pids[@]} -eq 0 ]]; then
return 0
fi

# Check each dotnet process for command lines pointing to artifacts DLLs
local pids_to_kill=()
for pid in "${dotnet_pids[@]}"; do
# Skip if process no longer exists
if ! kill -0 "$pid" 2>/dev/null; then
continue
fi

local has_artifact_dll=false

# Get the command line for this process
local cmdline=""
if [[ -r "/proc/$pid/cmdline" ]]; then
# Linux: read from /proc/pid/cmdline
cmdline=$(tr '\0' ' ' < "/proc/$pid/cmdline" 2>/dev/null || true)
elif command -v ps >/dev/null 2>&1; then
# macOS/other: use ps to get command line
cmdline=$(ps -p "$pid" -o command= 2>/dev/null || true)
fi

# Check if command line contains any DLL path under artifacts
if [[ -n "$cmdline" && "$cmdline" == *"$artifacts_path"*.dll* ]]; then
has_artifact_dll=true
fi

if [[ "$has_artifact_dll" == true ]]; then
echo "Terminating dotnet process $pid with artifacts DLL in command line"
pids_to_kill+=("$pid")
fi
done

# Kill all identified processes in parallel
if [[ ${#pids_to_kill[@]} -gt 0 ]]; then
# Send SIGTERM to all processes
for pid in "${pids_to_kill[@]}"; do
kill "$pid" 2>/dev/null || true
done

# Wait up to 5 seconds for all processes to exit
local count=0
local still_running=()
while [[ $count -lt 50 ]]; do
still_running=()
for pid in "${pids_to_kill[@]}"; do
if kill -0 "$pid" 2>/dev/null; then
still_running+=("$pid")
fi
done

if [[ ${#still_running[@]} -eq 0 ]]; then
break
fi

sleep 0.1
((count++))
done

# Force kill any processes still running after 5 seconds
for pid in "${still_running[@]}"; do
if kill -0 "$pid" 2>/dev/null; then
echo "Force killing dotnet process $pid"
kill -9 "$pid" 2>/dev/null || true
fi
done
fi
}

if [[ "$clean" == true ]]; then
if [ -d "$artifacts_dir" ]; then
# Kill any lingering dotnet processes that might be holding onto artifacts
Stop-ArtifactLockers "$repo_root"
rm -rf $artifacts_dir
echo "Artifacts directory deleted."
fi
Expand All @@ -274,4 +393,7 @@ if [[ "$restore" == true ]]; then
InitializeNativeTools
fi

# Kill any lingering dotnet processes that might be holding onto artifacts
Stop-ArtifactLockers "$repo_root"

Build
Loading