Skip to content

Fix and simplify ComputerUse sample#5075

Merged
SergeyMenshykh merged 4 commits intomicrosoft:mainfrom
SergeyMenshykh:computer-use-sample
Apr 7, 2026
Merged

Fix and simplify ComputerUse sample#5075
SergeyMenshykh merged 4 commits intomicrosoft:mainfrom
SergeyMenshykh:computer-use-sample

Conversation

@SergeyMenshykh
Copy link
Copy Markdown
Member

@SergeyMenshykh SergeyMenshykh commented Apr 3, 2026

Simplifies the \Agent_Step15_ComputerUse\ sample to improve readability, reduces asset sizes, and expands the README with a detailed explanation of the simulation flow.

Copilot AI review requested due to automatic review settings April 3, 2026 09:42
@markwallace-microsoft markwallace-microsoft added documentation Improvements or additions to documentation .NET labels Apr 3, 2026
@github-actions github-actions bot changed the title Fix ComputerUse sample to send screenshots as inline content instead of requiring file upload .NET: Fix ComputerUse sample to send screenshots as inline content instead of requiring file upload Apr 3, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Agent_Step15_ComputerUse .NET sample to use AIProjectClient.AsAIAgent(...), expands the README to explain the simulated “computer use” state machine, and changes how screenshots are provided to the model (currently via uploaded OpenAI file IDs).

Changes:

  • Refactors the sample program flow to use AIProjectClient.AsAIAgent(...) with a computer tool loop.
  • Reworks screenshot handling utilities (upload/delete + mapping state transitions to screenshots).
  • Expands README documentation with a state-transition table and interaction loop description.

Reviewed changes

Copilot reviewed 4 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step15_ComputerUse/README.md Adds a detailed explanation of the simulated computer-use loop and updates prerequisites/env var guidance.
dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step15_ComputerUse/Program.cs Refactors the sample to AsAIAgent(...) and implements a computer-action loop (currently using uploaded image file IDs).
dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step15_ComputerUse/ComputerUseUtil.cs Implements screenshot upload/delete and state-to-screenshot mapping for the loop.
dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step15_ComputerUse/Agent_Step15_ComputerUse.csproj Updates asset inclusion to .jpg screenshots copied to output.
dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step15_ComputerUse/Assets/*.jpg Adds/updates screenshot assets used by the simulation.

@SergeyMenshykh SergeyMenshykh marked this pull request as draft April 3, 2026 09:52
@SergeyMenshykh SergeyMenshykh changed the title .NET: Fix ComputerUse sample to send screenshots as inline content instead of requiring file upload Fix and simplify ComputerUse sample Apr 3, 2026
@SergeyMenshykh SergeyMenshykh self-assigned this Apr 3, 2026
@SergeyMenshykh SergeyMenshykh moved this to In Review in Agent Framework Apr 3, 2026
@SergeyMenshykh SergeyMenshykh marked this pull request as ready for review April 3, 2026 10:12
@SergeyMenshykh SergeyMenshykh enabled auto-merge April 3, 2026 10:30
@SergeyMenshykh SergeyMenshykh added this pull request to the merge queue Apr 7, 2026
Merged via the queue into microsoft:main with commit 86b49d8 Apr 7, 2026
22 checks passed
@github-project-automation github-project-automation bot moved this from In Review to Done in Agent Framework Apr 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation .NET

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants