Skip to content

[1/2] LT-22324: add FieldWorks UI automation support#869

Open
johnml1135 wants to merge 2 commits into
cleanup/non-render-pr-splitfrom
LT-22324-agentic-support
Open

[1/2] LT-22324: add FieldWorks UI automation support#869
johnml1135 wants to merge 2 commits into
cleanup/non-render-pr-splitfrom
LT-22324-agentic-support

Conversation

@johnml1135
Copy link
Copy Markdown
Contributor

@johnml1135 johnml1135 commented May 6, 2026

Summary

This PR keeps only the support-side additions that are independent of the render cleanup branch: FieldWorks live-app automation guidance, screenshot evidence workflow notes, and WinForms MCP setup for UIA2 automation with WinApp/UIA3 fallback guidance.

Stack

Included

  • Add a FieldWorks WinForms/WinApp automation skill under .github/skills/fieldworks-winapp.
  • Add a Windows-focused screenshot capture skill under .github/skills/smart-screenshot-capture.
  • Add winforms-mcp to .vscode/mcp.json with headless net48 settings.
  • Document WinForms MCP/UIA2 and WinApp/UIA3 fallback usage in Docs/mcp.md.

Excluded

  • Render benchmark infrastructure, render baselines, native render/cache changes, and OpenSpec render-speedup artifacts.
  • Skill relocation to .agents/skills, which is split into a separate stacked PR.
  • LT-22324 font-option implementation behavior.

Verification

  • git diff --name-status cleanup/non-render-pr-split...HEAD
  • git diff --check cleanup/non-render-pr-split...HEAD
  • python -m json.tool .vscode/mcp.json
  • git grep -n "LT-22324" -- .github/skills/fieldworks-winapp returned no matches

This change is Reviewable

Copilot AI review requested due to automatic review settings May 6, 2026 20:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@johnml1135 johnml1135 closed this May 6, 2026
@johnml1135 johnml1135 changed the title LT-22324: add agentic support and render tooling LT-22324: add FieldWorks UI automation support May 7, 2026
@johnml1135 johnml1135 reopened this May 7, 2026
@johnml1135 johnml1135 changed the base branch from main to cleanup/non-render-pr-split May 7, 2026 18:05
@johnml1135 johnml1135 changed the title LT-22324: add FieldWorks UI automation support [1/2] LT-22324: add FieldWorks UI automation support May 7, 2026
@jasonleenaylor
Copy link
Copy Markdown
Contributor

Build/Agent/fix-whitespace.ps1 line 13 at r1 (raw file):

}

function Test-HasUtf8Bom {

Was removing this intentional? It seemed this would produce less churn in the repo with it in.

@johnml1135
Copy link
Copy Markdown
Contributor Author

Good catch. That removal wasn't intentional. I restored the BOM-preserving behavior in Build/Agent/fix-whitespace.ps1 so the whitespace fixer keeps existing UTF-8 BOMs and avoids unnecessary churn. The fix is on the branch now in 47111a3.

@johnml1135 johnml1135 force-pushed the LT-22324-agentic-support branch 2 times, most recently from 70d6808 to 29162e5 Compare May 12, 2026 13:35
@github-actions

This comment has been minimized.

@johnml1135 johnml1135 force-pushed the LT-22324-agentic-support branch 2 times, most recently from b721182 to 3aa324a Compare May 12, 2026 16:37
@johnml1135 johnml1135 force-pushed the cleanup/non-render-pr-split branch from fe6f187 to 2488da1 Compare May 12, 2026 16:37
@johnml1135
Copy link
Copy Markdown
Contributor Author

Follow-up after the PR split cleanup: Build/Agent/fix-whitespace.ps1 is no longer part of #869. This PR now contains only the 12 UI automation support files. The BOM-preserving whitespace-fixer change belongs with the base cleanup layer instead, so if you want to review that script change specifically, please look at the base branch/PR rather than this support-only PR.

@johnml1135 johnml1135 requested a review from Copilot May 12, 2026 20:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

.github/skills/fieldworks-winapp/SKILL.md:1

  • The skill name fieldworks-winapp suggests it is WinApp-only, but the skill is explicitly dual-driver (WinForms MCP + WinApp fallback). Consider renaming the skill to reflect its scope (e.g., fieldworks-ui-automation or fieldworks-winforms-winapp) to avoid incorrect discovery/selection by authors and agents.
---

Comment thread .vscode/mcp.json
Comment on lines +24 to +35
"winforms-mcp": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"@fnrhombus/winforms-mcp"
],
"env": {
"HEADLESS": "true",
"TFM": "net48",
"TELEMETRY_OPTOUT": "true"
}
Comment thread Docs/mcp.md
- a route needs UIA3-specific behavior or a control is not exposed correctly through WinForms MCP;
- troubleshooting focus, foreground-window behavior, or non-WinForms surfaces.

Headless WinForms MCP limitations: `send_keys`, drag/drop, and double-click paths use input simulation and require the visible desktop. Prefer `winforms_type_text`, `winforms_set_value`, `winforms_select_item`, `winforms_click_element`, and `winforms_click_menu_item` for headless work.
Comment thread Docs/mcp.md
Comment on lines +127 to +128
- **Headless text input does nothing** – use WinForms MCP `type_text` or `set_value`, not `send_keys`.
- **Headless screenshot is blank or stale** – pass the FieldWorks process ID to `take_screenshot` when the client exposes that parameter.
Comment thread Docs/mcp.md
@@ -13,6 +14,7 @@ servers automatically:
| VS Code 1.101+ | Remote MCP + OAuth support | https://code.visualstudio.com |
@johnml1135 johnml1135 force-pushed the cleanup/non-render-pr-split branch from 2488da1 to daf921a Compare May 12, 2026 20:53
Keep build/test helper cleanup out of the render commit.

Keep agent, installer, and xWorks cleanup separate.

Normalize TonePars generated-file comparisons for Windows paths.
@johnml1135 johnml1135 force-pushed the LT-22324-agentic-support branch from 3aa324a to bc7e8b1 Compare May 12, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants