Skip to content

Added Code Coverage to build#997

Merged
jeschu1 merged 1 commit into
microsoft:masterfrom
jeschu1:code_coverage
Apr 4, 2019
Merged

Added Code Coverage to build#997
jeschu1 merged 1 commit into
microsoft:masterfrom
jeschu1:code_coverage

Conversation

@jeschu1
Copy link
Copy Markdown
Member

@jeschu1 jeschu1 commented Apr 2, 2019 •
edited
Loading

We include code coverage reports in the build. You can see this in the logs.

    WaitForListenerCompletion()                                                                                                                                                                                                                            0.00% (0/14)       
    GetPid(vfs_context*)                                                                                                                                                                                                                                   100.00% (4/4)      
    GetVNodeAttributes(vnode*, vfs_context*, vnode_attr*)                                                                                                                                                                                                  100.00% (6/6)      
    TryReadVNodeFileFlags(vnode*, vfs_context*, unsigned int*)                                                                                                                                                                                             100.00% (19/19)    
    FileFlagsBitIsSet(unsigned int, unsigned int)                                                                                                                                                                                                          100.00% (4/4)      
    TryGetFileIsFlaggedAsInRoot(vnode*, vfs_context*, bool*)                                                                                                                                                                                               100.00% (11/11)    
    ActionBitIsSet(int, int)                                                                                                                                                                                                                               100.00% (3/3)      
    IsFileSystemCrawler(char const*)                                                                                                                                                                                                                       100.00% (13/13)
    CurrentProcessWasSpawnedByRegularUser()                                                                                                                                                                                                                53.66% (22/41)     
    ShouldHandleFileOpEvent(PerfTracer*, vfs_context*, vnode*, int, short*, FsidInode*, int*)                                                                                                                                                              68.42% (39/57)     
    WaitForListenerCompletion()                                                                                                                                                                                                                            0.00% (0/14)       
    GetPid(vfs_context*)

We will fail on Kext functions not on the exclusion list and is not at 100%.

We hope to move to the json format and a prettier output in the future, once we have full ansci support on the build logs.

@jeschu1 jeschu1 added the WIP label Apr 2, 2019
jeschu1
jeschu1 commented Apr 2, 2019
View reviewed changes
Comment thread ProjFS.Mac/Scripts/Build.sh

# Fail on any line that doesn't show %100 coverage and isn't on the exclusion list or hpp/cpp
while read line; do
if [[ $line != *"100.00%"* ]] &&
Copy link
Copy Markdown
Member Author

@jeschu1 jeschu1 Apr 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need all these exclusions since we're including too many files in the test report.

@pmj do you know how we could switch the code coverage report to just be on KauthHandler.cpp && VirtualizationRoots.cpp.

Alternatively we could parse the json and remove the files we don't care about in that report. @mjcheetham know a fancy json parser for MacOS as a backup?

Copy link
Copy Markdown
Member

@mjcheetham mjcheetham Apr 3, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use jq which is very small, lightweight and installable by Homebrew.. it's like "sed/awk for JSON"

https://stedolan.github.io/jq/download/

Copy link
Copy Markdown
Member Author

@jeschu1 jeschu1 Apr 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Matt! I had to turn off json since the build output didn't support the output from tool. Once this is supported I think the aim should be to move json full stop and jq looks like a great tool for that.

Copy link
Copy Markdown
Member

@wilbaker wilbaker Apr 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we could parse the json

@jeschu1 for my own understanding, which of the above commands is producing the json?

Copy link
Copy Markdown
Member Author

@jeschu1 jeschu1 Apr 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wilbaker the "--json" option does it

#xcrun xccov view "$COVERAGE_FILE" --json

@jeschu1 jeschu1 force-pushed the code_coverage branch 9 times, most recently from cfb3694 to ee795c8 Compare April 2, 2019 20:37
@jrbriggs jrbriggs added affects: engineering Keeping the engineering system healthy affects: tests platform: macOS labels Apr 3, 2019
mjcheetham
mjcheetham reviewed Apr 3, 2019
View reviewed changes
Comment thread ProjFS.Mac/Scripts/Build.sh Outdated
@jeschu1 jeschu1 force-pushed the code_coverage branch 3 times, most recently from b848ac1 to 705acaf Compare April 3, 2019 13:56
@jeschu1 jeschu1 removed the WIP label Apr 3, 2019
nickgra
nickgra approved these changes Apr 3, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@nickgra nickgra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM too!
Offline I proposed an solution to filter away any test bundles that we don't want to measure Code Coverage for, but we can handle that as a separate PR (since we already have to revisit this for the pretty output)

@nickgra
Copy link
Copy Markdown
Member

nickgra commented Apr 4, 2019

/azp run GitHub VFSForGit Mac Functional Tests

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented Apr 4, 2019

No pipelines are associated with this pull request.

@jeschu1 jeschu1 merged commit 2d57b61 into microsoft:master Apr 4, 2019
@jrbriggs jrbriggs added this to the M151 milestone Apr 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mjcheetham mjcheetham mjcheetham left review comments

+2 more reviewers

@wilbaker wilbaker wilbaker approved these changes

@nickgra nickgra nickgra approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

affects: engineering Keeping the engineering system healthy affects: tests platform: macOS

Projects

None yet

Milestone

M151

Development

Successfully merging this pull request may close these issues.

5 participants

@jeschu1 @nickgra @mjcheetham @wilbaker @jrbriggs