-
Notifications
You must be signed in to change notification settings - Fork 6k
Conversation
What's the status of unit testing on Mac? |
bce804f
to
660092e
Compare
It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat. Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you forgot to add the FlutterViewControllerTest.mm file?
Whoops, actually I forgot to remove the line referencing it when I was messing around with trying to figure out the Mac unit tests. Do we actually have functioning unit tests on Mac? If so, how do I run them? |
Never looked into it. @stuartmorgan would be the person to ask. |
Thanks for the pointer! I couldn't figure this out and didn't know who to ask. |
The current state of testing for macOS is not good, certainly. @chinmaygarde added that test so could answer questions about it, or I can look next week when I'm not OOO |
@stuartmorgan Thanks for the pointer, I was able to get the tests running with help from Chinmay. I will attempt to add the Mac unit tests into run_tests.py in this PR. For posterity, the way to directly run the Mac unit tests is to compile for Mac, then run:
|
@gaaclarke Ready for re-review. I was able to add a test and get the Mac unit tests running as a part of run_tests.py. |
|
||
namespace flutter::testing { | ||
|
||
TEST(FlutterViewControllerTest, MacOSTestTest) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this called MacOSTestTest? The name should describe what the test is covering.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, forgot to update that.
[FlutterMethodCall methodCallWithMethodName:@"Clipboard.hasStrings" arguments:nil]; | ||
[viewController handleMethodCall:methodCall result:result]; | ||
ASSERT_TRUE(called); | ||
ASSERT_TRUE(value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect the unit tests on the bots are running in a bootstrap context that doesn't have access to the pasteboard server. If that's the case, perhaps the tests should inject a mock NSPasteboard
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any idea how to include OCMock in the Mac tests? I tried including it in this test file and adding it to Build.gn in various places, but I keep getting compiler errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The same way it's done for iOS? Without knowing what you tried and what exactly didn't work, it's hard to advise further.
Ah you're right, thanks! Running in profile mode fails with the same error locally. If you've got any idea how to get this working let me know. The original FlutterEngineUnittests.mm also fails in profile mode. |
I'm not familiar with At this point I think the best option for this PR would be to revert the addition of the test binary to |
/** | ||
* This just returns the NSPasteboard so that it can be mocked in the tests. | ||
*/ | ||
@property(nonatomic, readonly, nonnull) NSPasteboard* _pasteboard; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still needs to be moved to the private header.
Also, a property name should not start with _
; the property and the ivar are distinct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where exactly should this be moved to? Sorry for my lack of Objective C knowledge. I tried moving this inside the .mm file, but then I couldn't mock it from the test.
👍 About the underscore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FlutterViewController_Internal.h
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah thanks, I missed that.
Though it seems to also give me errors when the test tries to mock it. I just copied this exactly over to the _Internal header. Is there something else I need to do to make it accessible to the test?
FlutterViewControllerTest.mm:32:31: error: no known instance method for selector 'pasteboard'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you include that header in the test file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was it, thanks.
* Returns true iff the clipboard contains nonempty string data. | ||
* | ||
* See also: | ||
* * https://developer.apple.com/documentation/uikit/uipasteboard/1829416-hasstrings, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I didn't explain my earlier question very well: what information is this adding that helps understand the method? The description in the comment seems clear and unambiguous; following that link doesn't tell me anything new AFAICT, so why have it?
It also seems odd to document it in terms of another platform's implementation, given than a) we have a lot of platforms, so referring to one of them is pretty arbitrary, b) that implementation is not relevant here since there's no such method on macOS, and c) that anyone who wants to know how it's done on iOS can easily just go look at the iOS source.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, removed.
@@ -517,7 +528,7 @@ - (void)playSystemSound:(NSString*)soundType { | |||
} | |||
|
|||
- (NSDictionary*)getClipboardData:(NSString*)format { | |||
NSPasteboard* pasteboard = [NSPasteboard generalPasteboard]; | |||
NSPasteboard* pasteboard = [self _pasteboard]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prefer accessing properties with dot syntax: self.pasteboard
.
[pasteboard setString:text forType:NSPasteboardTypeString]; | ||
} | ||
} | ||
|
||
- (BOOL)clipboardHasStrings { | ||
NSDictionary* data = [self getClipboardData:[NSString stringWithFormat:@"%s", kTextPlainFormat]]; | ||
NSString* string = data[@"text"]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at this more closely this time: going through a method whose input and output are explicitly designed for the Flutter method channel contract seems like a very convoluted way to check this. Why not just return [self.pasteboard stringForType:NSPasteboardTypeString]].length > 0;
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, done.
TEST(FlutterViewControllerTest, HasStringsWhenPasteboardEmpty) { | ||
id viewControllerMock = mockViewController(); | ||
|
||
// Call setData to make sure that the pasteboard is empty. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Your test controls the fake clipboard; there is no reason to involve Clipboard.setData
in the test of Clipboard.hasStrings.
It would be good to have a test of setData
, but that should be a separate test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, now I just set up the mock with some given pasteboard string instead of letting the test call setString.
TEST(FlutterViewControllerTest, HasStringsWhenPasteboardFull) { | ||
id viewControllerMock = mockViewController(); | ||
|
||
// Call setClipboardData to make sure there's a string on the pasteboard. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same.
@stuartmorgan That sounds good to me, I've reverted the change to run_tests.py. I'm happy we were able to improve the testing situation a bit even if not quite all the way. |
Waiting on #21111 to fix the Mac iOS Engine test. |
Finally green 🎉 @stuartmorgan Ready for final review when we're back on Monday. |
@stuartmorgan Reminder that this needs a final review when you have a chance. |
Gets ocmock set up so that it can be used by the Mac unit tests (in flutter/engine#20531).
Description
Adding the hasStrings method to Mac, similar to #19859.
Related Issues
Depends on flutter/buildroot#399
#19859
flutter/flutter#60145
Tests
I added the following tests: