AI agent website readability

We are quickly moving into a world where most interactions on your website will be done by an agent. Instead of a human visiting your website themselves, an agent will be visiting your website on their behalf.

So I just want to share some new information we have about how agents see your website and how you can prepare for this new future. Dan Petrovich recently put out a very interesting blog post talking about how Google's Chromium uses something called a page content agent to understand the content on your website.

They ran a bunch of experiments and extracted exactly how Google sees your website. And this is all happening inside of a Chrome browser.

I'm sure other browsers are starting to do the same thing, but I doubt any of them are as ahead on this as Chrome is. Keep in mind that most browsers do run on chromium.

Using Microsoft Edge, for example, you're still using Chrome. And it's true of most headless browsers as well.

If you're using playwright or puppeteer, you're using chromium...

own. But it's actually a lot more complicated than that.

First issue is that not everything that's rendered on the page is actually included in the HTML. Some elements are loaded by javascript and won't be shown in the HTML.

The page may have retrieved information from your specific account, and that won't show up in the HTML. Or there may be form fields on the page that you've entered information into.

Chrome's page Content Agent can retrieve all of that information. Just a few months ago, the only way for AI to get his information was to actually take screenshots of the page.

Then they might scroll or click a button and take another screenshot. And every screenshot would take a couple seconds to process.

It was incredibly slow and consumed a lot of tokens. What's perhaps most remarkable about Chrome's Page Content Agent is it runs entirely on your browser, so the page extraction happens almost instantly and essentially for free.

And because it's on your hardware, it can also retrieve sensitive information...

can you do about it right now? I recommend visiting your website on Chrome and then opening the Gemini for Chrome in the side panel.

Just ask it to give you the raw output from its page content agent when viewing your website. Then look through the output and make sure it's understanding your website correctly.

In my experience, it does a really good job even if you haven't prepared your website for AI agents. But if you haven't yet set up webmcp, it might have trouble understanding your forms.