Industry insights

Foreground vs background computer use: two ways an agent shares your Mac

An agent can work in the app you are looking at or in apps you are not. Compare foreground and background computer use by input, interruption and task fit.

Back to Blog
Two Mac displays side by side, one showing a person's cursor in an app, the other an agent working out of sight.

Foreground computer use is an agent working in the app you are looking at, on your screen, while you watch. Background computer use is an agent working in apps you are not looking at: on another display, in a virtual machine or in a hosted browser, while you keep using your computer for something else. The difference decides who owns the pointer and keyboard, what happens when you interrupt, and which tasks the agent can reach.

Most discussions of computer-use agents ask how well the model clicks. The first question is simpler: whose screen is it clicking on?

Two ways to share a computer with an agent

A computer-use agent repeats a loop: observe an interface, choose an action, perform it, observe the result. Where that interface lives is a design choice, and it produces two different products.

ForegroundBackground
Where the agent actsThe app in front of you, on your screenAn app on another display, a virtual machine or a hosted browser
What you seeEvery step, as it happensA report, logs or a recording afterwards
Your pointer and keyboardShared with the agent, unless the agent brings its own inputYours
What interruption looks likeYou touch the mouse and the agent's next click lands somewhere elseYou keep working; the agent's environment stays its own
Best forLearning an app, short tasks you want to supervise, questions about what is on screenRepetitive jobs, long tasks, work you do not need to watch

Neither mode is better in general. They fail differently, and they suit different requests.

Where foreground computer use breaks

The simplest foreground agent drives the real cursor and keyboard. That makes it easy to build and easy to trust visually: you see every click. It also means you cannot use the computer while the agent works. Move the mouse to check an email and the next click lands in the wrong place. Type a word and it arrives in the agent's text field. The agent and the person are competing for one set of input devices.

The second problem is focus. Many desktop actions assume a specific window is frontmost. If a notification, a dialog or your own click changes the front window, the agent's next keystroke can reach an app it never meant to touch. A careful foreground agent has to check where it is before every step.

The third problem is scope. A foreground agent sees what you see, which is an advantage for questions about the screen and a limit for anything long. A forty-step job that keeps your screen busy for ten minutes is ten minutes of you waiting.

Where background computer use breaks

Background agents move the work somewhere you are not looking. A hosted browser or a virtual machine gives the agent a clean environment, but that environment has to hold the task's state. A project open in a desktop app on your Mac does not exist in a cloud browser, and copying it there may lose the plugins, licences and editable state the job needs. Our guide to local vs cloud AI agents covers that tradeoff in detail.

Running in the background on your own machine avoids the copy, and brings its own conditions. Some apps behave differently when they are not frontmost or not on a visible display. Some dialogs only appear in front. Interruption still matters: if you stop a background task halfway, an edit may already be applied. Our test of background computer use under interruption separates those recovery questions from tool-call success.

How Braus handles both

Braus started in the background. In Braus open source, the Driver moves app windows to a virtual display within your current Mac session and routes the agent's input there, so you keep working on your Mac without a virtual machine. The windows and files stay on that Mac; the virtual display is not another user account or a separate filesystem. Work that uses this virtual desktop takes turns: while one worker uses it, the others wait. The physical display has to stay awake. The background work notes and the Driver guide describe the details, and the Pro Tools platform notes show what background control takes in an app with its own drawn interface.

The Braus app adds a foreground worker on top of that: the Super Worker. You hold two keys in the app in front of you, say what you need, and Braus looks at that app and answers out loud. It can draw boxes around what you asked for, walk you through the steps while it draws each one, or do the job in that app. It addresses the first foreground problem directly: it never moves your pointer and never types on your keyboard, and you see its own cursor go to each target instead. Only one worker works on your screen at a time.

The two modes also connect. When the rest of a job no longer needs the app in front of you, the Super Worker hands it to the background and the result waits in that worker's chat. You get supervision where it helps and your screen back where it does not. The Super Worker is part of the Braus app, which is distributed by Forte; the guide to talking to Braus from any Mac app on forte-ai.com covers how to use it.

Choosing a mode for a task

Ask four questions before you decide where an agent should work.

Do you need to watch it? If the point is to learn the app, or to confirm each step on something you care about, foreground fits. If you only need the result, background frees your screen.

Where does the task's state live? If the project is open in a desktop app on your machine, an agent on that machine can reach it. A remote environment needs its own copy, with everything that copy needs.

What happens if you interrupt? In the foreground, check whether the agent shares your input devices. In the background, check whether stopping leaves a half-applied change, and inspect the app before retrying.

Does the app cooperate off screen? Some apps need to be in front for certain steps. A background agent that is honest about this brings the app forward for that step only, or tells you it could not confirm the result.

Try both on a small task

Start with something disposable. In the foreground, open a document and ask what is on the screen, then ask where a specific control is and watch the boxes appear. In the background, give a worker a read-only request in an app you are not using, such as summarizing an open TextEdit test document, and keep working while it runs.

Then compare the two on the questions that matter for your work: who owned the pointer, what you could do meanwhile, and how you checked the result. To try the background side from source, follow the Braus quickstart.

All posts