Add MSTest.DesktopTesting thin-layer for Windows desktop E2E testing by Evangelink · Pull Request #7810 · microsoft/testfx

Evangelink · 2026-04-24T10:27:13Z

Summary

Introduces a new MSTest.DesktopTesting NuGet package that provides base classes for end-to-end testing of Windows desktop applications (WinForms, WPF, Win32). Follows the same thin-layer pattern as Microsoft.Playwright.MSTest — a lifecycle adapter over the built-in System.Windows.Automation API with zero external dependencies.

Base class hierarchy

AutomationTest          ← PlaywrightTest equivalent (hierarchy root, extensible)
  └─ ApplicationTest    ← BrowserTest equivalent (process launch/teardown)
       └─ WindowTest    ← PageTest equivalent (MainWindow as AutomationElement)

AutomationTest — root of the hierarchy, extension point for future configuration
ApplicationTest — launches the app via Process.Start, waits for main window handle, manages teardown with CloseMainWindow/Kill. Uses [STATestClass] for COM/STA threading. Configurable via virtual properties (ApplicationPath, ApplicationArguments, ApplicationStartTimeout)
WindowTest — exposes MainWindow as System.Windows.Automation.AutomationElement. Users can layer FlaUI or other libraries on top for richer element interaction

SDK integration

Mirrors the Playwright/Aspire SDK pattern exactly:

<EnableDesktopTesting>true</EnableDesktopTesting> MSBuild property in MSTest.Sdk
DesktopTesting.targets feature file wired into ClassicEngine, VSTest, and NativeAOT targets
Package version templated through Sdk.props.template
Works as both an SDK feature flag and a standalone <PackageReference>

User experience

<Project Sdk="MSTest.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0-windows</TargetFramework>
    <EnableDesktopTesting>true</EnableDesktopTesting>
  </PropertyGroup>
</Project>

[TestClass]
public class MyAppTests : WindowTest
{
    public override string ApplicationPath => @"C:\MyApp\MyApp.exe";

    [TestMethod]
    public void MainWindow_IsVisible()
    {
        Assert.IsNotNull(MainWindow);
    }
}

What's included

New project: src/TestFramework/TestFramework.DesktopTesting/ — the package source
SDK wiring: Feature targets, props template, NativeAOT guard
Sample: samples/public/DemoMSTestSdk/ProjectUsingDesktopTesting/ — Calculator E2E test
Acceptance tests: DesktopTestingSdkTests.cs — tests both MTP runner and VSTest modes using notepad.exe

Design decisions

System.Windows.Automation over FlaUI: Zero external dependencies. Uses the managed UIA wrapper from the Windows Desktop runtime. Users who want a richer API can layer FlaUI on top.
[STATestClass]: Required for COM interop with Windows UI Automation
Separate package + SDK flag: Ships from the same repo, usable both ways

Introduce a new MSTest.DesktopTesting package that provides base classes (AutomationTest, ApplicationTest, WindowTest) for end-to-end testing of Windows desktop applications. Follows the same pattern as Playwright MSTest integration: a thin lifecycle adapter over System.Windows.Automation. - AutomationTest: hierarchy root (extensible) - ApplicationTest: process launch/teardown via Process.Start - WindowTest: exposes MainWindow as AutomationElement SDK integration: - EnableDesktopTesting MSBuild property in MSTest.Sdk - DesktopTesting.targets feature file (ClassicEngine, VSTest, NativeAOT guard) - Version wired through Sdk.props.template Also includes: - Sample project (ProjectUsingDesktopTesting with Calculator tests) - Acceptance tests for both MTP runner and VSTest modes

Copilot

Pull request overview

Adds a new Windows-desktop E2E testing thin layer (MSTest.DesktopTesting) and wires it into MSTest.Sdk behind an <EnableDesktopTesting> feature flag, plus samples and acceptance coverage.

Changes:

Introduces Microsoft.MSTest.DesktopTesting base classes (AutomationTest → ApplicationTest → WindowTest) built on System.Windows.Automation.
Adds MSTest.Sdk feature wiring (DesktopTesting.targets import + version plumbing + NativeAOT guard).
Adds a public sample and an acceptance test that exercises the SDK flag in both MSTest runner and VSTest modes.

Show a summary per file

File	Description
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/DesktopTestingSdkTests.cs	Adds acceptance tests that generate a project enabling the DesktopTesting SDK feature.
src/TestFramework/TestFramework.DesktopTesting/AutomationTest.cs	Adds hierarchy root base class for desktop automation tests.
src/TestFramework/TestFramework.DesktopTesting/ApplicationTest.cs	Adds process lifecycle management for desktop app E2E tests.
src/TestFramework/TestFramework.DesktopTesting/WindowTest.cs	Exposes main window as `AutomationElement` for tests.
src/TestFramework/TestFramework.DesktopTesting/GlobalUsings.cs	Adds MSTest global using for the new package.
src/TestFramework/TestFramework.DesktopTesting/TestFramework.DesktopTesting.csproj	Defines the new packable Windows-desktop testing project/package.
src/TestFramework/TestFramework.DesktopTesting/PublicAPI/PublicAPI.Shipped.txt	Initializes shipped API baseline for the new package.
src/TestFramework/TestFramework.DesktopTesting/PublicAPI/PublicAPI.Unshipped.txt	Declares new public API surface for DesktopTesting.
src/Package/MSTest.Sdk/Sdk/Features/DesktopTesting.targets	Implements the SDK feature target that injects the DesktopTesting package + implicit using.
src/Package/MSTest.Sdk/Sdk/VSTest/VSTest.targets	Imports DesktopTesting feature targets in VSTest mode.
src/Package/MSTest.Sdk/Sdk/Runner/ClassicEngine.targets	Imports DesktopTesting feature targets in ClassicEngine mode.
src/Package/MSTest.Sdk/Sdk/Runner/NativeAOT.targets	Adds a NativeAOT validation error for the feature flag.
src/Package/MSTest.Sdk/Sdk/Sdk.props.template	Adds `EnableDesktopTesting` default + version property.
src/Package/MSTest.Sdk/MSTest.Sdk.csproj	Plumbs DesktopTesting version into SDK props templating.
samples/public/DemoMSTestSdk/ProjectUsingDesktopTesting/ProjectUsingDesktopTesting.csproj	Adds sample project demonstrating the SDK flag.
samples/public/DemoMSTestSdk/ProjectUsingDesktopTesting/CalculatorTests.cs	Adds sample Calculator tests using `WindowTest` + UIA.
Directory.Packages.props	Introduces `MSTestDesktopTestingVersion` and a declared `PackageVersion` entry.

Copilot's findings

Comments suppressed due to low confidence (1)

samples/public/DemoMSTestSdk/ProjectUsingDesktopTesting/CalculatorTests.cs:45

The sample locates buttons by AutomationElement.NameProperty with values "One"/"Two". These names are localized in many Windows locales, so the sample can be flaky or fail outside English. Prefer a locale-independent locator (e.g., AutomationId) or note the limitation explicitly in comments.

        AutomationElement? button1 = MainWindow.FindFirst(
            TreeScope.Descendants,
            new PropertyCondition(AutomationElement.NameProperty, "One"));

        AutomationElement? button2 = MainWindow.FindFirst(
            TreeScope.Descendants,
            new PropertyCondition(AutomationElement.NameProperty, "Two"));

Files reviewed: 17/17 changed files
Comments generated: 5

Copilot · 2026-04-24T10:35:10Z

+        while (AppProcess.MainWindowHandle == IntPtr.Zero && sw.Elapsed < ApplicationStartTimeout)
+        {
+            AppProcess.Refresh();
+            Thread.Yield();


ApplicationSetup uses a tight loop with Thread.Yield() while waiting for MainWindowHandle to become available. This can spin a core for up to ApplicationStartTimeout and cause noisy CPU usage in test runs. Consider replacing the busy-wait with a small delay (or a WaitForInputIdle/poll delay) while still respecting the timeout.

Suggested change

Thread.Yield();

TimeSpan remainingTime = ApplicationStartTimeout - sw.Elapsed;

if (remainingTime > TimeSpan.Zero)

{

Thread.Sleep(remainingTime < TimeSpan.FromMilliseconds(50)

? remainingTime

: TimeSpan.FromMilliseconds(50));

}

Copilot · 2026-04-24T10:35:11Z

+        if (AppProcess is not null && !AppProcess.HasExited)
+        {
+            AppProcess.CloseMainWindow();
+            if (!AppProcess.WaitForExit(5000))
+            {
+                AppProcess.Kill();
+            }
+        }
+
+        AppProcess?.Dispose();
+        AppProcess = null!;


ApplicationTearDown can throw during cleanup due to races (process exits between HasExited check and Kill()/CloseMainWindow()), which would fail the test even if assertions passed. Consider wrapping shutdown/kill in a try/finally and using a safer termination sequence (e.g., handle exceptions, and wait for exit after kill).

Suggested change

if (AppProcess is not null && !AppProcess.HasExited)

{

AppProcess.CloseMainWindow();

if (!AppProcess.WaitForExit(5000))

{

AppProcess.Kill();

}

}

AppProcess?.Dispose();

AppProcess = null!;

Process? appProcess = AppProcess;

try

{

if (appProcess is not null && !appProcess.HasExited)

{

try

{

_ = appProcess.CloseMainWindow();

if (!appProcess.WaitForExit(5000))

{

appProcess.Kill();

_ = appProcess.WaitForExit(5000);

}

}

catch (InvalidOperationException)

{

// The process exited between state checks and shutdown operations.

}

}

}

finally

{

appProcess?.Dispose();

AppProcess = null!;

}

Copilot · 2026-04-24T10:35:11Z

+    <TargetFrameworks Condition=" '$(OS)' == 'Windows_NT' ">net8.0-windows;net9.0-windows</TargetFrameworks>
+    <TargetFrameworks Condition=" '$(OS)' != 'Windows_NT' " />


This project sets TargetFrameworks to empty on non-Windows (<TargetFrameworks ... />). MSBuild requires TargetFramework/TargetFrameworks to be non-empty, so evaluating/restoring this project on non-Windows hosts will fail (even if it’s not intended to build there). Consider always setting the Windows TFMs and using EnableWindowsTargeting (or another approach) to allow restore/build on non-Windows CI, or otherwise ensure the project is never evaluated outside Windows builds.

Suggested change

<TargetFrameworks Condition=" '$(OS)' == 'Windows_NT' ">net8.0-windows;net9.0-windows</TargetFrameworks>

<TargetFrameworks Condition=" '$(OS)' != 'Windows_NT' " />

<TargetFrameworks>net8.0-windows;net9.0-windows</TargetFrameworks>

<EnableWindowsTargeting Condition=" '$(OS)' != 'Windows_NT' ">true</EnableWindowsTargeting>

Copilot · 2026-04-24T10:35:12Z

+  <ItemGroup>
+    <PackageReference Include="MSTest.DesktopTesting" Sdk="MSTest">
+      <Version Condition=" '$(ManagePackageVersionsCentrally)' != 'true' ">$(MSTestDesktopTestingVersion)</Version>
+    </PackageReference>
+    <PackageVersion Include="MSTest.DesktopTesting" Version="$(MSTestDesktopTestingVersion)"
+                    Condition=" '$(ManagePackageVersionsCentrally)' == 'true' " />
+  </ItemGroup>


DesktopTesting.targets unconditionally adds the MSTest.DesktopTesting package when EnableDesktopTesting=true, even for non-Windows OSes/TFMs. Since the feature is Windows-only, this is likely to produce confusing restore/build errors if enabled on unsupported platforms. Consider adding an explicit MSBuild <Error> (or condition the PackageReference) when the OS/TFM isn’t Windows / -windows to fail fast with a clear message.

Copilot · 2026-04-24T10:35:12Z

+        Assert.IsTrue(
+            title.Contains("Calculator", StringComparison.OrdinalIgnoreCase),
+            $"Expected window title to contain 'Calculator', but was '{title}'.");
+    }
+
+    [TestMethod]
+    public void Calculator_NumberButtons_AreVisible()
+    {
+        // Use System.Windows.Automation to locate UI elements
+        AutomationElement? button1 = MainWindow.FindFirst(
+            TreeScope.Descendants,
+            new PropertyCondition(AutomationElement.NameProperty, "One"));
+
+        AutomationElement? button2 = MainWindow.FindFirst(
+            TreeScope.Descendants,
+            new PropertyCondition(AutomationElement.NameProperty, "Two"));
+
+        Assert.IsNotNull(button1, "Could not find the 'One' button.");
+        Assert.IsNotNull(button2, "Could not find the 'Two' button.");


This sample asserts the window title contains the English string "Calculator". On non-English Windows installations the Calculator title is localized, so this sample test can fail even when the feature works. Consider using a non-localized UIA property (e.g., AutomationId) or loosening the assertion to something locale-agnostic (or documenting that the sample assumes an English UI).

This issue also appears on line 39 of the same file.

Suggested change

Assert.IsTrue(

title.Contains("Calculator", StringComparison.OrdinalIgnoreCase),

$"Expected window title to contain 'Calculator', but was '{title}'.");

}

[TestMethod]

public void Calculator_NumberButtons_AreVisible()

{

// Use System.Windows.Automation to locate UI elements

AutomationElement? button1 = MainWindow.FindFirst(

TreeScope.Descendants,

new PropertyCondition(AutomationElement.NameProperty, "One"));

AutomationElement? button2 = MainWindow.FindFirst(

TreeScope.Descendants,

new PropertyCondition(AutomationElement.NameProperty, "Two"));

Assert.IsNotNull(button1, "Could not find the 'One' button.");

Assert.IsNotNull(button2, "Could not find the 'Two' button.");

Assert.IsFalse(

string.IsNullOrWhiteSpace(title),

"Expected the main window to have a non-empty title.");

}

[TestMethod]

public void Calculator_NumberButtons_AreVisible()

{

// Use non-localized AutomationId values to locate UI elements.

AutomationElement? button1 = MainWindow.FindFirst(

TreeScope.Descendants,

new PropertyCondition(AutomationElement.AutomationIdProperty, "num1Button"));

AutomationElement? button2 = MainWindow.FindFirst(

TreeScope.Descendants,

new PropertyCondition(AutomationElement.AutomationIdProperty, "num2Button"));

Assert.IsNotNull(button1, "Could not find the button with AutomationId 'num1Button'.");

Assert.IsNotNull(button2, "Could not find the button with AutomationId 'num2Button'.");

Copilot AI review requested due to automatic review settings April 24, 2026 10:27

Copilot started reviewing on behalf of Evangelink April 24, 2026 10:27 View session

Copilot AI reviewed Apr 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MSTest.DesktopTesting thin-layer for Windows desktop E2E testing#7810

Add MSTest.DesktopTesting thin-layer for Windows desktop E2E testing#7810
Evangelink wants to merge 1 commit intomainfrom
dev/amauryleve/desktop-testing-e2e

Evangelink commented Apr 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Copilot AI Apr 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-            Thread.Yield();
+            TimeSpan remainingTime = ApplicationStartTimeout - sw.Elapsed;
+            if (remainingTime > TimeSpan.Zero)
+            {
+                Thread.Sleep(remainingTime < TimeSpan.FromMilliseconds(50)
+                    ? remainingTime
+                    : TimeSpan.FromMilliseconds(50));
+            }

-        if (AppProcess is not null && !AppProcess.HasExited)
-        {
-            AppProcess.CloseMainWindow();
-            if (!AppProcess.WaitForExit(5000))
-            {
-                AppProcess.Kill();
-            }
-        }
-        AppProcess?.Dispose();
-        AppProcess = null!;
+        Process? appProcess = AppProcess;
+        try
+        {
+            if (appProcess is not null && !appProcess.HasExited)
+            {
+                try
+                {
+                    _ = appProcess.CloseMainWindow();
+                    if (!appProcess.WaitForExit(5000))
+                    {
+                        appProcess.Kill();
+                        _ = appProcess.WaitForExit(5000);
+                    }
+                }
+                catch (InvalidOperationException)
+                {
+                    // The process exited between state checks and shutdown operations.
+                }
+            }
+        }
+        finally
+        {
+            appProcess?.Dispose();
+            AppProcess = null!;
+        }

		<TargetFrameworks Condition=" '$(OS)' == 'Windows_NT' ">net8.0-windows;net9.0-windows</TargetFrameworks>
		<TargetFrameworks Condition=" '$(OS)' != 'Windows_NT' " />

Conversation

Evangelink commented Apr 24, 2026

Summary

Base class hierarchy

SDK integration

User experience

What's included

Design decisions

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants