Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Upload xcresults to LUCI cloud storage #41647

Merged
merged 10 commits into from
May 15, 2023
Merged

Conversation

cyanglaz
Copy link
Contributor

@cyanglaz cyanglaz commented May 1, 2023

Taking over from #41644

fixes: flutter/flutter#125823

Steps to verify

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the CLA.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@cyanglaz
Copy link
Contributor Author

Scenario app test failure and result uploaded: #41652

@cyanglaz
Copy link
Contributor Author

Unittest failed and result uploaded: #41929

upload scenario app test result

fix script
@cyanglaz cyanglaz marked this pull request as ready for review May 12, 2023 15:36
@cyanglaz cyanglaz requested a review from gaaclarke May 12, 2023 15:36
@@ -752,6 +753,12 @@ def run_objc_tests(ios_variant='ios_debug_sim_unopt', test_filter=None):
ios_unit_test_dir = os.path.join(
BUILDROOT_DIR, 'flutter', 'testing', 'ios', 'IosUnitTests'
)

result_bundle_temp = tempfile.TemporaryDirectory(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way you use TemporaryDirectory is with a with statement to make sure that it gets cleaned up.

@@ -760,13 +767,27 @@ def run_objc_tests(ios_variant='ios_debug_sim_unopt', test_filter=None):
'xcodebuild '
'-sdk iphonesimulator '
'-scheme IosUnitTests '
"-destination name='" + new_simulator_name + "' "
'-resultBundlePath ' + result_bundle_path + " -destination name='" +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the formatting of having a flag at the start of each line.

# Upload the xcresult when the tests fail.
luci_test_outputs_path = os.environ.get('FLUTTER_TEST_OUTPUTS_DIR')
xcresult_bundle = os.path.join(result_bundle_temp, 'ios_embedding.xcresult')
print(xcresult_bundle)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stray print statement.

@@ -41,25 +41,57 @@ fi
# Can also be set via Simulator app Device > Rotate Device Automatically
defaults write com.apple.iphonesimulator RotateWindowWhenSignaledByGuest -int 1

cd $SRC_DIR/out/$FLUTTER_ENGINE/scenario_app/Scenarios
SCENARIO_PATH=$SRC_DIR/out/$FLUTTER_ENGINE/scenario_app/Scenarios
cd $SCENARIO_PATH
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd pushd and pop around here to make sure you don't effect any downstream commands.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I'm just realizing you didn't change this cd, just shifted the code. The push and pop aren't necessary but don't hurt. Thanks for looking into it.

# Zip and upload xcresult to luci.
# First parameter ($1) is the zip output name.
ZIP_AND_UPLOAD_XCRESULT_TO_LUCI () {
# Using absolute directory causes the zip containing all the sub directories.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clear this up please? You are probably running into the difference between having a trailing slash or not fwiw.

ZIP_AND_UPLOAD_XCRESULT_TO_LUCI () {
# Using absolute directory causes the zip containing all the sub directories.
# So use relative directory (./$RESULT_BUNDLE_FOLDER) instead.
echo $1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the echo or make it more useful by adding a label.


# Zip and upload xcresult to luci.
# First parameter ($1) is the zip output name.
ZIP_AND_UPLOAD_XCRESULT_TO_LUCI () {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make the function name lowercase please.

@cyanglaz cyanglaz requested a review from gaaclarke May 15, 2023 18:28
Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

@cyanglaz cyanglaz added the autosubmit Merge PR when tree becomes green via auto submit App label May 15, 2023
@auto-submit auto-submit bot merged commit 0a8308a into flutter:main May 15, 2023
@cyanglaz cyanglaz deleted the upload_xcresult branch May 15, 2023 19:07
test_command[0] = test_command[0] + ' -only-testing:%s' % test_filter
run_cmd(test_command, cwd=ios_unit_test_dir, shell=True)

# except:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaaclarke Just found this is mistakenly commented out, I will create a patch to fix it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR landed before I can remove the auto_submit label :( I was a split second slower :(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, nice job seeing the problem. This wasn't showing up as an error in CI? Do we need another test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wouldn't as I also missed "try"

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request May 15, 2023
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request May 15, 2023
…126876)

flutter/engine@00f20fb...027ca79

2023-05-15 [email protected] Use release_build and os dimension consistently. (flutter/engine#42012)
2023-05-15 [email protected] [Impeller] Add interactive DrawPaint blend test (flutter/engine#42031)
2023-05-15 [email protected] [Impeller] Limit subpass textures and backdrop blurs to the current clip (flutter/engine#42039)
2023-05-15 [email protected] Roll Dart SDK from c302a0252785 to d2b2ac829842 (1 revision) (flutter/engine#42051)
2023-05-15 [email protected] Upload xcresults to LUCI cloud storage (flutter/engine#41647)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
auto-submit bot pushed a commit that referenced this pull request May 15, 2023
CaseyHillers pushed a commit to CaseyHillers/flutter that referenced this pull request May 24, 2023
…lutter#126876)

flutter/engine@00f20fb...027ca79

2023-05-15 [email protected] Use release_build and os dimension consistently. (flutter/engine#42012)
2023-05-15 [email protected] [Impeller] Add interactive DrawPaint blend test (flutter/engine#42031)
2023-05-15 [email protected] [Impeller] Limit subpass textures and backdrop blurs to the current clip (flutter/engine#42039)
2023-05-15 [email protected] Roll Dart SDK from c302a0252785 to d2b2ac829842 (1 revision) (flutter/engine#42051)
2023-05-15 [email protected] Upload xcresults to LUCI cloud storage (flutter/engine#41647)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC [email protected],[email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
autosubmit Merge PR when tree becomes green via auto submit App
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Zip and upload xcresults on failing engine iOS native tests in LUCI
3 participants