Skip to content

Commit c429024

Browse files
TanayParikhwtgodbe
TanayParikh
and
wtgodbe
authored
Playwright 🎭 🛠️ (#45682)
* Version Bumps * Run with `Test: Windows Server x64` * Install Playwright in CI * Add running playwright locally instructions * Test Cleanup * Remove Selenium * Prebuild for Playwright install * Update default-build.yml * Update default-build.yml * Update default-build.yml * Update default-build.yml * Update default-build.yml * Playwright install in test * New template test pipeline * Update components-e2e-tests.yml * Update components-template-tests.yml * Update components-e2e-tests.yml * Update components-e2e-tests.yml * Update components-e2e-tests.yml * Update components-template-tests.yml * Update components-e2e-tests.yml * Update components-e2e-tests.yml * Update components-e2e-tests.yml * Revert new pipeline changes * RestoreAdditionalProjectSources * Update Templates.Blazor.Tests.csproj * PR Feedback * RunBlazorPlaywrightTemplateTests * Socket logic * Fixed socket wait logic Fixes #30761 * Remove redundant process wait for exit We're already in the `Exited` Event: There are two ways of being notified when the associated process exits: synchronously and asynchronously. Synchronous notification means calling the WaitForExit method to block the current thread until the process exits. Asynchronous notification uses the Exited event, which allows the calling thread to continue execution in the meantime. In the latter case, EnableRaisingEvents must be set to true for the calling application to receive the Exited event. https://learn.microsoft.com/dotnet/api/system.diagnostics.process.exited?view=net-7.0 * Update BlazorWasmTemplateTest.cs * Update Templates.Blazor.Tests.csproj * Revert App.Ref, add src in props * Config updates * Update RequiresDelayedBuildProjects.props * Update Templates.Blazor.Tests.csproj * Skip failing tests * Add #46430 * Update Templates.Blazor.Tests.csproj * Update BlazorWasmTemplateTest.cs * Update BlazorWasmTemplateTest.cs --------- Co-authored-by: William Godbe <[email protected]>
1 parent 7d29bfa commit c429024

15 files changed

+53
-495
lines changed

‎.azure/pipelines/ci.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -698,7 +698,7 @@ stages:
698698
# Just uploading artifacts/logs/ files can take 15 minutes. Doubling the cancel timeout for this job.
699699
cancelTimeoutInMinutes: 30
700700
buildArgs: -all -pack -test -binaryLog /p:SkipHelixReadyTests=true /p:SkipIISNewHandlerTests=true /p:SkipIISTests=true
701-
/p:SkipIISExpressTests=true /p:SkipIISNewShimTests=true /p:RunTemplateTests=false
701+
/p:SkipIISExpressTests=true /p:SkipIISNewShimTests=true /p:RunTemplateTests=false /p:RunBlazorPlaywrightTemplateTests=true
702702
$(_InternalRuntimeDownloadArgs)
703703
beforeBuild:
704704
- powershell: "& ./src/Servers/IIS/tools/UpdateIISExpressCertificate.ps1; & ./src/Servers/IIS/tools/update_schema.ps1"

‎eng/RequiresDelayedBuildProjects.props

+1
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,6 @@
1616
<RequiresDelayedBuild Include="$(RepoRoot)src\Grpc\JsonTranscoding\test\Microsoft.AspNetCore.Grpc.Swagger.Tests\Microsoft.AspNetCore.Grpc.Swagger.Tests.csproj" />
1717
<RequiresDelayedBuild Include="$(RepoRoot)src\Grpc\JsonTranscoding\test\testassets\IntegrationTestsWebsite\IntegrationTestsWebsite.csproj" />
1818
<RequiresDelayedBuild Include="$(RepoRoot)src\Grpc\JsonTranscoding\test\testassets\Sandbox\Sandbox.csproj" />
19+
<RequiresDelayedBuild Include="$(RepoRoot)src\ProjectTemplates\test\Templates.Blazor.Tests\Templates.Blazor.Tests.csproj" />
1920
</ItemGroup>
2021
</Project>

‎eng/Versions.props

+1-1
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@
280280
<NewtonsoftJsonVersion>13.0.1</NewtonsoftJsonVersion>
281281
<NSwagApiDescriptionClientVersion>13.0.4</NSwagApiDescriptionClientVersion>
282282
<PhotinoNETVersion>1.1.6</PhotinoNETVersion>
283-
<MicrosoftPlaywrightVersion>1.17.3</MicrosoftPlaywrightVersion>
283+
<MicrosoftPlaywrightVersion>1.28.0</MicrosoftPlaywrightVersion>
284284
<PollyExtensionsHttpVersion>3.0.0</PollyExtensionsHttpVersion>
285285
<PollyVersion>7.2.3</PollyVersion>
286286
<SeleniumSupportVersion>4.8.0</SeleniumSupportVersion>

‎eng/tools/GenerateFiles/Directory.Build.props.in

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
<Project>
22
<PropertyGroup>
33
<DefaultNetCoreTargetFramework>${DefaultNetCoreTargetFramework}</DefaultNetCoreTargetFramework>
4+
<ArtifactsShippingPackagesDir>${ArtifactsShippingPackagesDir}</ArtifactsShippingPackagesDir>
45
<TreatWarningsAsErrors Condition="'$(BuildingInsideVisualStudio)' != 'true'">true</TreatWarningsAsErrors>
56
<LibNetHostAppPackVersion Condition=" '$(LibNetHostAppPackVersion)' == '' ">${LibNetHostAppPackVersion}</LibNetHostAppPackVersion>
67
</PropertyGroup>

‎eng/tools/GenerateFiles/GenerateFiles.csproj

+2-1
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@
2626
MicrosoftNETCoreAppRuntimeVersion=$(MicrosoftNETCoreAppRuntimeVersion);
2727
MicrosoftPlaywrightCLIVersion=$(MicrosoftPlaywrightCLIVersion);
2828
LibNetHostAppPackVersion=$(BundledNETCoreAppPackageVersion);
29-
SupportedRuntimeIdentifiers=$(SupportedRuntimeIdentifiers.Trim())
29+
SupportedRuntimeIdentifiers=$(SupportedRuntimeIdentifiers.Trim());
30+
ArtifactsShippingPackagesDir=$(ArtifactsShippingPackagesDir)
3031
</_TemplateProperties>
3132
</PropertyGroup>
3233

‎src/ProjectTemplates/README.md

+12
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,18 @@ Then, use one of:
7171
previous step, it is NOT advised that you install templates created on your local machine using just
7272
`dotnet new -i [nupkgPath]`.
7373

74+
#### Running Blazor Playwright Template Tests
75+
76+
1. From the root of the repo, build the templates: `.\eng\build.cmd -all -pack`
77+
2. `cd .\src\ProjectTemplates\test\Templates.Blazor.Tests`
78+
3. `dotnet test .\Templates.Blazor.Tests.csproj` with optional `--filter` arg to run a specific test.
79+
80+
The requisite browsers should be automatically installed. If you encounter browser errors, the browsers can be manually installed via the following script, replacing `[TFM]` with the current target TFM (ex. `net8.0`).
81+
82+
```cmd
83+
.\bin\Debug\[TFM]\playwright.ps1 install
84+
```
85+
7486
#### Conditional tests & skipping test platforms
7587

7688
Individual test methods can be decorated with attributes to configure them to not run ("skip running") on certain platforms. The `[ConditionalFact]` and `[ConditionalTheory]` attributes must be used on tests using the skip attributes in order for them to actually be skipped:

‎src/ProjectTemplates/Shared/TemplatePackageInstaller.cs

+5
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,11 @@ private static async Task InstallTemplatePackages(ITestOutputHelper output)
9999
.Where(p => _templatePackages.Any(t => Path.GetFileName(p).StartsWith(t, StringComparison.OrdinalIgnoreCase)))
100100
.ToArray();
101101

102+
if (builtPackages.Length == 0)
103+
{
104+
throw new InvalidOperationException($"Failed to find required templates in {packagesDir}. Please ensure the *Templates*.nupkg have been built.");
105+
}
106+
102107
Assert.Equal(4, builtPackages.Length);
103108

104109
await VerifyCannotFindTemplateAsync(output, "web");

‎src/ProjectTemplates/TestInfrastructure/Directory.Build.props.in

+3
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,7 @@
77

88
<Import Project="${ArtifactsBinDir}GenerateFiles\Directory.Build.props" />
99

10+
<PropertyGroup>
11+
<RestoreAdditionalProjectSources>$(RestoreAdditionalProjectSources);$(ArtifactsShippingPackagesDir)</RestoreAdditionalProjectSources>
12+
</PropertyGroup>
1013
</Project>

‎src/ProjectTemplates/test/Templates.Blazor.Tests/BlazorServerTemplateTest.cs

+13-32
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
using System.Collections.Generic;
66
using System.Linq;
77
using System.Net;
8+
using System.Threading;
89
using System.Threading.Tasks;
910
using Microsoft.AspNetCore.BrowserTesting;
1011
using Microsoft.AspNetCore.Testing;
@@ -25,7 +26,7 @@ public BlazorServerTemplateTest(ProjectFactoryFixture projectFactory)
2526

2627
public override string ProjectType { get; } = "blazorserver";
2728

28-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30761")]
29+
[Theory]
2930
[InlineData(BrowserKind.Chromium)]
3031
public async Task BlazorServerTemplateWorks_NoAuth(BrowserKind browserKind)
3132
{
@@ -77,11 +78,8 @@ await BrowserManager.GetBrowserInstance(browserKind, BrowserContextInfo) :
7778
}
7879
}
7980

80-
public static IEnumerable<object[]> BlazorServerTemplateWorks_IndividualAuthData =>
81-
BrowserManager.WithBrowsers(new[] { BrowserKind.Chromium }, true, false);
82-
83-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
84-
[MemberData(nameof(BlazorServerTemplateWorks_IndividualAuthData))]
81+
[InlineData(BrowserKind.Chromium)]
82+
[Theory]
8583
[SkipOnHelix("https://github.com/dotnet/aspnetcore/issues/30825", Queues = "All.OSX")]
8684
public async Task BlazorServerTemplateWorks_IndividualAuth(BrowserKind browserKind)
8785
{
@@ -134,29 +132,9 @@ public async Task BlazorServerTemplateWorks_IndividualAuth(BrowserKind browserKi
134132

135133
private async Task TestBasicNavigation(IPage page)
136134
{
137-
var socket = await page.WaitForWebSocketAsync();
138-
139-
var framesReceived = 0;
140-
var framesSent = 0;
141-
142-
void FrameReceived(object sender, IWebSocketFrame frame) { framesReceived++; }
143-
void FrameSent(object sender, IWebSocketFrame frame) { framesSent++; }
144-
145-
socket.FrameReceived += FrameReceived;
146-
socket.FrameSent += FrameSent;
135+
// Wait for the page to load, and the connection to idle for >500ms
136+
await page.WaitForLoadStateAsync(LoadState.NetworkIdle, new() { Timeout = 60_000 });
147137

148-
// Receive render batch
149-
await page.WaitForWebSocketAsync(new() { Predicate = (s) => framesReceived == 1 });
150-
await page.WaitForWebSocketAsync(new() { Predicate = (s) => framesSent == 1 });
151-
152-
// JS interop call to intercept navigation
153-
await page.WaitForWebSocketAsync(new() { Predicate = (s) => framesReceived == 2 });
154-
await page.WaitForWebSocketAsync(new() { Predicate = (s) => framesSent == 2 });
155-
156-
socket.FrameReceived -= FrameReceived;
157-
socket.FrameSent -= FrameSent;
158-
159-
await page.WaitForSelectorAsync("nav");
160138
// <title> element gets project ID injected into it during template execution
161139
Assert.Equal("Index", (await page.TitleAsync()).Trim());
162140

@@ -168,10 +146,13 @@ private async Task TestBasicNavigation(IPage page)
168146
await page.WaitForSelectorAsync("h1+p >> text=Current count: 0");
169147

170148
// Clicking the counter button works
171-
await page.ClickAsync("p+button >> text=Click me");
172-
await page.WaitForSelectorAsync("h1+p >> text=Current count: 1");
149+
for (var i = 1; i <= 3; i++)
150+
{
151+
await page.ClickAsync("p+button >> text=Click me");
152+
await page.WaitForSelectorAsync($"h1+p >> text=Current count: {i}");
153+
}
173154

174-
// Can navigate to the 'fetch data' page
155+
// Can navigate to the 'Fetch Data' page
175156
await page.ClickAsync("a[href=fetchdata] >> text=Fetch data");
176157
await page.WaitForSelectorAsync("h1 >> text=Weather forecast");
177158

@@ -180,7 +161,7 @@ private async Task TestBasicNavigation(IPage page)
180161
Assert.Equal(5, await page.Locator("p+table>tbody>tr").CountAsync());
181162
}
182163

183-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
164+
[Theory(Skip="https://github.com/dotnet/aspnetcore/issues/46430")]
184165
[InlineData("IndividualB2C", null)]
185166
[InlineData("IndividualB2C", new [] { "--called-api-url \"https://graph.microsoft.com\"", "--called-api-scopes user.readwrite" })]
186167
[InlineData("SingleOrg", null)]

‎src/ProjectTemplates/test/Templates.Blazor.Tests/BlazorTemplateTest.cs

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ public abstract class BlazorTemplateTest : BrowserTestBase
1818
public BlazorTemplateTest(ProjectFactoryFixture projectFactory)
1919
{
2020
ProjectFactory = projectFactory;
21+
Microsoft.Playwright.Program.Main(new[] { "install" });
2122
}
2223

2324
public ProjectFactoryFixture ProjectFactory { get; set; }

‎src/ProjectTemplates/test/Templates.Blazor.Tests/BlazorWasmTemplateTest.cs

+6-6
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ public BlazorWasmTemplateTest(ProjectFactoryFixture projectFactory)
2323

2424
public override string ProjectType { get; } = "blazorwasm";
2525

26-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
26+
[Theory]
2727
[InlineData(BrowserKind.Chromium)]
2828
public async Task BlazorWasmStandaloneTemplate_Works(BrowserKind browserKind)
2929
{
@@ -59,7 +59,7 @@ private static async Task<IPage> NavigateToPage(IBrowserContext browser, string
5959
return page;
6060
}
6161

62-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
62+
[Theory(Skip="https://github.com/dotnet/aspnetcore/issues/46430")]
6363
[InlineData(BrowserKind.Chromium)]
6464
public async Task BlazorWasmHostedTemplate_Works(BrowserKind browserKind)
6565
{
@@ -107,7 +107,7 @@ private static async Task AssertCompressionFormat(AspNetProcess aspNetProcess, s
107107
Assert.Equal(expectedEncoding, response.Content.Headers.ContentEncoding.Single());
108108
}
109109

110-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
110+
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/45736")]
111111
[InlineData(BrowserKind.Chromium)]
112112
public async Task BlazorWasmStandalonePwaTemplate_Works(BrowserKind browserKind)
113113
{
@@ -142,7 +142,7 @@ public async Task BlazorWasmStandalonePwaTemplate_Works(BrowserKind browserKind)
142142
}
143143
}
144144

145-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
145+
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/45736")]
146146
[InlineData(BrowserKind.Chromium)]
147147
public async Task BlazorWasmHostedPwaTemplate_Works(BrowserKind browserKind)
148148
{
@@ -212,15 +212,15 @@ private static void ValidatePublishedServiceWorker(Project project)
212212
Assert.True(serviceWorkerContents.Contains($"/* Manifest version: {serviceWorkerAssetsManifestVersion} */", StringComparison.Ordinal));
213213
}
214214

215-
[ConditionalTheory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
215+
[ConditionalTheory(Skip="https://github.com/dotnet/aspnetcore/issues/46430")]
216216
[InlineData(BrowserKind.Chromium)]
217217
// LocalDB doesn't work on non Windows platforms
218218
[OSSkipCondition(OperatingSystems.Linux | OperatingSystems.MacOSX)]
219219
public Task BlazorWasmHostedTemplate_IndividualAuth_Works_WithLocalDB(BrowserKind browserKind)
220220
=> BlazorWasmHostedTemplate_IndividualAuth_Works(browserKind, true);
221221

222222
// This test depends on BlazorWasmTemplate_CreateBuildPublish_IndividualAuthNoLocalDb running first
223-
[Theory(Skip = "https://github.com/dotnet/aspnetcore/issues/30882")]
223+
[Theory(Skip="https://github.com/dotnet/aspnetcore/issues/46430")]
224224
[InlineData(BrowserKind.Chromium)]
225225
[SkipOnHelix("https://github.com/dotnet/aspnetcore/issues/30825", Queues = "All.OSX")]
226226
public Task BlazorWasmHostedTemplate_IndividualAuth_Works_WithOutLocalDB(BrowserKind browserKind)

‎src/ProjectTemplates/test/Templates.Blazor.Tests/Templates.Blazor.Tests.csproj

+6-1
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,12 @@
44
<TestGroupName>ProjectTemplates.Blazor.E2ETests</TestGroupName>
55
<DefineConstants>$(DefineConstants);XPLAT</DefineConstants>
66

7+
<RequiresDelayedBuild>true</RequiresDelayedBuild>
8+
79
<RunTemplateTests Condition="'$(RunTemplateTests)' == ''">true</RunTemplateTests>
8-
<SkipTests Condition="'$(RunTemplateTests)' != 'true'">true</SkipTests>
10+
<RunBlazorPlaywrightTemplateTests Condition="'$(RunBlazorPlaywrightTemplateTests)' == ''">$(RunTemplateTests)</RunBlazorPlaywrightTemplateTests>
11+
<SkipTests Condition="'$(RunBlazorPlaywrightTemplateTests)' != 'true'">true</SkipTests>
12+
913
<BaseOutputPath />
1014
<OutputPath />
1115

@@ -47,6 +51,7 @@
4751
<Reference Include="Microsoft.Extensions.Configuration.Json" />
4852
<Reference Include="AngleSharp" />
4953
<Reference Include="System.Net.Http" />
54+
<Reference Include="Microsoft.AspNetCore.Components.WebAssembly" />
5055
<Reference Include="Microsoft.Playwright" Condition="'$(IsPlaywrightAvailable)' == 'true'" />
5156
<Reference Include="Microsoft.Playwright" ExcludeAssets="build" Condition="'$(IsPlaywrightAvailable)' != 'true'" />
5257
<ProjectReference Include="$(RepoRoot)src\Framework\App.Runtime\src\Microsoft.AspNetCore.App.Runtime.csproj"

‎src/ProjectTemplates/test/Templates.Blazor.Tests/package.json

-16
This file was deleted.

0 commit comments

Comments
 (0)