Skip to content

Added Code Coverage to build #997

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jeschu1 merged 1 commit into from
Apr 4, 2019
Merged

Added Code Coverage to build #997

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 65 additions & 1 deletion ProjFS.Mac/Scripts/Build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ SCRIPTDIR=$(dirname ${BASH_SOURCE[0]})
SRCDIR=$SCRIPTDIR/../..
ROOTDIR=$SRCDIR/..
PACKAGES=$ROOTDIR/packages
COVERAGEDIR=$ROOTDIR/BuildOutput/ProjFS.Mac/Coverage

PROJFS=$SRCDIR/ProjFS.Mac

Expand All @@ -21,9 +22,72 @@ fi

# Run Tests and put output into a xml file
set -o pipefail
xcodebuild -configuration $CONFIGURATION -project $PROJFS/PrjFS.xcodeproj -scheme 'Build All' test | xcpretty -r junit --output $PROJFS/TestResultJunit.xml || exit 1
xcodebuild -configuration $CONFIGURATION -enableCodeCoverage YES -project $PROJFS/PrjFS.xcodeproj -derivedDataPath $COVERAGEDIR -scheme 'Build All' test | xcpretty -r junit --output $PROJFS/TestResultJunit.xml || exit 1
set +o pipefail

while read -rd $'\0' file; do
COVERAGE_FILE="$file"
done < <(find $COVERAGEDIR -name "*xccovreport" -print0)

if [[ $COVERAGE_FILE == "" ]]; then
echo "Error: No coverage file found"
exit 1
fi

# xcperfect will display pretty output for code coverage. The terminal output isn't supported by our build system yet, but a bug has been filed. Once support is added we'll switch to the prettier format.
#if !(gem list --local | grep xcperfect); then
# echo "Attempting to run 'sudo gem install xcperfect'. This may ask you for your password to gain admin privileges"
# sudo gem install xcperfect
#fi
#xcrun xccov view "$COVERAGE_FILE" --json | TERM=xterm-256color xcperfect

printf "\n\nCode Coverage Report\n"
xcrun xccov view "$COVERAGE_FILE" | tee $PROJFS/CoverageResult.txt

# Fail on any line that doesn't show %100 coverage and isn't on the exclusion list or hpp/cpp
while read line; do
if [[ $line != *"100.00%"* ]] &&
Copy link
Copy Markdown
Member Author

@jeschu1 jeschu1 Apr 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need all these exclusions since we're including too many files in the test report.

@pmj do you know how we could switch the code coverage report to just be on KauthHandler.cpp && VirtualizationRoots.cpp.

Alternatively we could parse the json and remove the files we don't care about in that report. @mjcheetham know a fancy json parser for MacOS as a backup?

Copy link
Copy Markdown
Member

@mjcheetham mjcheetham Apr 3, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use jq which is very small, lightweight and installable by Homebrew.. it's like "sed/awk for JSON"

https://stedolan.github.io/jq/download/

Copy link
Copy Markdown
Member Author

@jeschu1 jeschu1 Apr 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Matt! I had to turn off json since the build output didn't support the output from tool. Once this is supported I think the aim should be to move json full stop and jq looks like a great tool for that.

Copy link
Copy Markdown
Member

@wilbaker wilbaker Apr 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we could parse the json

@jeschu1 for my own understanding, which of the above commands is producing the json?

Copy link
Copy Markdown
Member Author

@jeschu1 jeschu1 Apr 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wilbaker the "--json" option does it

#xcrun xccov view "$COVERAGE_FILE" --json

[[ $line == *"%"* ]] &&
[[ $line != *"KauthHandler_Init"* ]] &&
[[ $line != *"KauthHandler_Cleanup"* ]] &&
[[ $line != *"UseMainForkIfNamedStream"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"HandleVnodeOperation"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"HandleFileOpOperation"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"TryGetVirtualizationRoot"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"CurrentProcessWasSpawnedByRegularUser"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"ShouldHandleFileOpEvent"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"ShouldIgnoreVnodeType"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"WaitForListenerCompletion"* ]] &&
[[ $line != *"KextLog_"* ]] &&
[[ $line != *"Definition"* ]] &&
[[ $line != *"PerfTracer"* ]] &&
[[ $line != *"VirtualizationRoot_GetActiveProvider"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"VirtualizationRoots_Init"* ]] &&
[[ $line != *"VirtualizationRoots_Cleanup"* ]] &&
[[ $line != *"FindOrDetectRootAtVnode"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"FindUnusedIndexOrGrow_Locked"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"FindRootAtVnode_Locked"* ]] && #SHOULD ADD COVERAGE
[[ $line != *"ActiveProvider_"* ]] &&
[[ $line != *"GetRelativePath"* ]] &&
[[ $line != *"VirtualizationRoot_GetRootRelativePath"* ]] &&
[[ $line != *"MockCalls"* ]] &&
[[ $line != *"PerfTracing_"* ]] &&
[[ $line != *"proc_"* ]] &&
[[ $line != *"ParentPathString"* ]] &&
[[ $line != *"SetAndRegisterPath"* ]] &&
[[ $line != *"vn_"* ]] &&
[[ $line != *"vnode_lookup"* ]] &&
[[ $line != *"RetainIOCount"* ]] &&
[[ $line != *"ProviderMessaging_"* ]] &&
[[ $line != *"RWLock_DropExclusiveToShared"* ]] &&
[[ $line != *".xctest"* ]] &&
[[ $line != *".cpp"* ]] &&
[[ $line != *".hpp"* ]]; then
printf "\nError: Not at 100% Code Coverage: $line"
exit 1
fi
done < $PROJFS/CoverageResult.txt

# If we're building the Profiling(Release) configuration, remove Profiling() for building .NET code
if [ "$CONFIGURATION" == "Profiling(Release)" ]; then
CONFIGURATION=Release
Expand Down