kulikowski.me
← Writing

What if you could bring your agent into the browser?

8 min read#agentic-web #webmcp #experimentview as markdown

What if you could bring your agent into the browser?

Not the agent built into the browser. Not the one a website embeds. Yours - the one you already use every day.

And why would anyone want that? 🤔

Think about the agent you use every day. Maybe you use it for coding, writing, or a bit of everything. Over time it's collected a lot of context about you: your instructions, your preferences, the projects you work on, the files on your machine. You picked it, you set its permission rules, you're signed in, and you pay for it already 😅.

And then there's the other half: the browser. I do most of my work in the browser. That's where my documents, presentations, meetings and calendars live. That's where I'm signed in to pretty much everything, and where my passwords and passkeys are kept.

The agent built into the browser knows the browser: your tabs, your history, what's in front of you - and often the apps connected to your account. The assistant on a website knows that site. Both are great at what they do - they just don't know what your agent already knows about you.

So the idea is: not a new agent, and not another API key pasted into another settings page. The agent you already have, in the browser you already use.

What if the browser could be a harness around your tabs, your logged-in sessions and a page's own tools - for that exact same agent?

For me today, that agent happens to be a coding agent. So I built a small experiment called Browser Harness: a Chrome side panel that connects to Claude Code, Codex or Antigravity CLI right on my machine. It sits next to the page I'm looking at and works on it while I watch - and I can step in or stop it at any time.

When a website exposes its own tools for agents through WebMCP, the agent uses those first - and falls back to reading, clicking and typing when there's no page tool for the job.

The whole experiment is on GitHub if you want to try it yourself: Kulikowski/browser-harness. Here it is next to the zaMaker pizza demo 👇

My own agent (here Antigravity CLI) next to the page, using the page's WebMCP tools to add toppings to the pizza.

How does it work?

Not every agent supports WebMCP. The agent in my terminal doesn't. It speaks MCP, it has plenty of tools of its own - and it has no idea what the tab in front of me offers.

What surprised me while building this is that both protocols I needed already exist:

1️⃣ On the agent side: ACP (Agent Client Protocol). That's the protocol editors use to talk to coding agents (natively or through a small ACP adapter). Each chat in the side panel is its own session in that agent - it turns out a browser side panel can drive an agent the same way an IDE does 😎.

2️⃣ On the website side: WebMCP, mirrored as MCP. The extension reads the tools the active tab registered and hands them to the agent through a normal MCP server called browser. If the site exposes add_todo, the agent gets webmcp_add_todo. When the agent calls it, the call runs right inside the page (asking me first when needed).

Under the hood, the extension's service worker talks through Chrome's native messaging to a small local Node bridge that connects the two.

Every message also tells the agent which tab I'm looking at and what tools it offers, so "this page" just works - and by default the agent stays on that tab for the whole turn, even if I switch to another one.

From the agent's point of view, though, nothing unusual happened: the browser is just one more MCP server next to everything else it already has.

What my agent brings

Once the browser is just another MCP server, a lot comes along for free.

For starters, the extension holds no API keys and never calls a model itself. A new chat simply lists the agents actually installed on this computer and uses the login I already set up in my terminal - along with whatever instructions, skills and other MCP servers that agent normally loads 👇

Only the agents found on this computer show up, with their versions.

Because it's still the same local agent, it keeps my files and my shell. Asking "fill this form from the CSV in my project folder" is just two ordinary steps: read the file on disk, then fill the page.

My favorite part, though, is session continuity. I can start a task in my terminal inside a project, and the moment the next step moves to a website, pick up that exact conversation in the side panel under "Sessions started elsewhere" (for agents that list their sessions). The history and the project folder come right along - the agent is still in my project, it just got a browser 😎.

That's my side of the interaction. What about the page sitting next to it?

What the website brings

Ideally, the website brings its own tools - and when it doesn't, the agent can still read and click the page directly. That gives it two ways to work through the browser server:

1️⃣ The page's WebMCP tools. Whatever the site registered, mirrored as webmcp_* with its own input schemas. Instead of clicking a topping button ten times, the agent makes one structured call 👇

The agent calling the page's own WebMCP tool instead of clicking around.

2️⃣ Generic browser actions. When there's no page tool for the job: snapshot, click, type, press_key, scroll, navigate and screenshot. The snapshot comes from the accessibility tree, so the agent picks real elements instead of guessing coordinates.

So what happens when a page has no WebMCP tools at all? Here is Claude Code on the Angular WebMCP docs - it reads the page and sums up how to add my first tool 👇

No WebMCP tools on this page, so the agent reads it - and answers from what's on the page.

It can also act on the page. I asked it to scroll down to where the implementation is explained, and it clicked the right link in the page's own menu 👇

Asked to scroll to the right section, the agent clicked the link in the page's own menu.

If you build websites, that WebMCP path is the part for you. In Bring your own agent we wrote about needing a contract between a user's agent and a website, without every agent having to integrate with every site. Seeing ACP on one side and WebMCP on the other made that feel very real: you ship one set of WebMCP tools, and Claude Code, Codex and Antigravity all use your site tools!

Which home is this?

That also made me rethink something from The three homes of the Agentic Web, where I split agents into in-page, in-browser and off-browser - and treated each home as a package deal. Off-browser agents had the reach and your local context, but needed their own keys and worked out of sight. In-browser agents got co-browsing and your active session, without knowing much about you outside Chrome.

Here the agent harness itself lives off-browser - it's the process on my machine, with my sessions, context and local tools. Its work on the page, though, happens in-browser, right in the tab I'm looking at.

In that earlier post I wrote about off-browser agents hooked into the user's browser: "I've yet to see an interaction pattern that recreates the co-browsing feel". So... here we have it 😎

The browser sandbox

In my terminal, a command or a local MCP server can touch anything my account can touch. A web page can't. It runs in the browser's sandbox, it can't read my files or start programs, and every site is kept apart from the others.

WebMCP tools live inside that same box. A page's tool runs in the page, with the access that site already has - its own DOM, its own storage, the session I'm signed in to - and nothing beyond it.

It's the same with credentials. In the three homes post I described a co-browsing agent as an extra pair of hands, not a copy of the keys. My passwords and passkeys stay with the browser.

So the browser gives me two things I'd otherwise have to build myself: boundaries between sites, and a place where I'm already logged in without handing my credentials to anyone.

However!

All of that is true for the page. It's not true for my agent.

My agent brings much more than hands. Today it's a coding agent with my files and a shell - and even if tomorrow it's a personal assistant with my Drive, email and calendar, the trade-off is the same. The whole point of bringing my agent is that it can reach things outside the browser.

The web, meanwhile, is full of untrusted pages. Once my agent reads a tab, hidden instructions on a page (prompt injection) get a chance to talk to something with real reach on my machine.

The extension itself asks for broad permissions too: access to every site so it can talk to pages, plus Chrome's debugger permission to read and click on pages without WebMCP. If you don't want the agent using that DOM fallback, a Page tools only switch in the settings hides and blocks the generic page-reading, clicking and typing tools, so the agent can only interact with pages through their WebMCP tools.

On top of that, a few more guardrails help keep things in check:

1️⃣ New chats start with safe defaults. Unless I point a chat at a project, it opens in a separate ~/.browser-harness/workspace folder instead of my home directory, and switches to the agent's ask-first mode where it has one - though a session I bring in from the terminal keeps the mode it had.

2️⃣ The browser asks for its half. On top of my agent's own permission prompts, the extension adds a Browser check whenever an action touches a password field, leaves the site, clicks a submit or buy / delete button, or calls a consequential WebMCP tool (plus the first call to an unmarked one). If I decline, that site is locked for the rest of the turn so the agent can't try clicking a button on the page instead. Here is Codex on a smart-home demo, where unlockFrontDoor is marked as consequential 👇

The browser asks: the page marked unlockFrontDoor as consequential, so the extension wants my yes before it runs.

3️⃣ Page text is marked as untrusted. The extension wraps page content, tool results, errors and the descriptions of the page's own tools, and tells the agent to treat them as data, never instructions. Not everything is covered - a screenshot is still raw pixels, and wrapping text is a strong hint for the model, not a hard boundary.

Which brings us right back to why this lives in a side panel and not in the background: confirmation prompts catch single risky actions before they run, and me watching right next to the page lets me stop the run if it starts drifting off course.

What I'm still thinking about

Building this left me with a few open questions about how personal agents and browsers should work together.

One confirmation instead of two. You might have spotted it back in the opening pizza screenshot: one WebMCP call can trigger two prompts - one from the agent, and one from the browser. The tricky part is that the agent knows why it's acting and the browser knows what the page will do, and today ACP and MCP have no way to combine those into a single check (or agree on who asks).

Sandboxing the agent side too. Chrome puts every website in a strict sandbox, while my local agent can still reach the rest of my machine. Starting in ~/.browser-harness/workspace is a polite default, not a wall. Running browser chats inside an actual sandbox - so the agent can only touch the exact folder or single file I hand it - would make the two sides match much better.

More than tools from the website. Right now the extension only exposes whichever tools are registered on the page at that moment. What if this same setup also carried a site's workflow skills through Agentic Resource Discovery, or rendered small interactive UIs inside the side panel via MCP Apps? Websites could guide visiting agents through longer journeys without ever shipping their own chatbot.

Want to try it?

If you try it yourself, please do it in a separate Chrome profile, on pages you trust, with a folder that has nothing sensitive in it. The code and setup steps are in Kulikowski/browser-harness.

One fun loop to try first if you build websites: open localhost next to a session in that same project, let the agent try the WebMCP tools you're building, see what breaks and fix the code live 😎.

Today this is very much an experiment for people who already use a coding agent every day. Still, the pattern feels much bigger than coding agents: whatever my main agent looks like in a year or two, I'd like to bring it into the browser too.

If you try it out or have thoughts on where this should go, let me know - I'll be exploring this further in future blog posts 🚀.

More soon. ✨