Tom Loosemore

I’ve recently been playing with a bunch of personal AI agents such as Meta’s Muse, Claude’s Duet, Instinct and Openclaw. I’m learning a lot about how they tend to behave, both from the perspective of a user, but also as a service owner via my WhenIsBins.com agentic testbed.

For example, I’ve written code so WhenIsBins can now detect the arrival of such agents with reasonable reliability, of which more soon.

But over the weekend I focused on testing a hypothesis: If Government created a change of address service targeted at agents, could it be valuable enough to tempt agents like Muse to use it?

At present these agents first check your email for messages from government services (DWP, DVLA, your local council etc), then search the web and work out how to help you change your address, service by service. What if there was a single change of address service for your AI agent, analogous to Tell Us Once which helps the bereaved inform government of the death of their loved one?

So I thought I’d do some user research, with the above agents as my users. I asked each of them a series of research questions to understand what they would find valuable from such a service, if anything. And then I ran a small experiment to test how they actually behaved. And just like with humans, a real-world experiment gets you the truly golden user insights.

So, firstly, how did they respond to the user research interviews I ran? How useful would they find such a service?

1. The AI agents would absolutely love it if GOV.UK One Login allowed them to log-in and do the whole change of address digitally on behalf of their humans.
2. Openclaw wistfully informed me that One Login was used by <1% of the government services to which it would need access.
3. Instinct told me that One Login hasn’t been designed with ‘delegated authority’ in mind, so it wouldn’t be able to use it at all. (I’d bet Muse would use it, and just pretend to be their human.)
4. After being politely on from moaning about One Login, Muse did eventually suggest that it might help it to have access to a directory of government services which included all the information they’d need to support the change of address process for each service – particularly local government given there’s 361 councils running 1,000s of services.
5. On being prompted with the idea, all the agents said they’d really, really like such a directory They would do, wouldn’t they? Confirmation bias in action.
6. And they’d all want such a change of address directory to be comprehensive, trustworthy, up-to-date and offer immediate value. ‘Immediate value’ is their way of admitting that they have a very short attention span, so you have to make your information very taut and brief.

Given I can’t help the AI agents with their One Login grumbles, I thought I’d run an experiment to test my hypothesis that agents might find such a change of address directory useful enough to use.

Just like normal humans, what they say they’ll do often doesn’t reflect what they actually do.

So I came up with eight fictional ‘moving house’ scenarios to stretch the canvas of the type & location of services such a directory would need to cover. I then carefully researched the ‘model answers’ that the AI agents should be giving to their humans.

Finally I set my various AI agents lose on the scenarios, and marked their results. I didn’t actually do all this myself; I got Claude Open5.5 to do the marking, for example.

I then ran the same change of address test using a sample of 20 businesses and charities, such as banks, insurers, utilities etc.

TL;DR The agents do not need a change of address directory. Even the ones using the cheapest models are doing plenty good enough already a combination of web search and what they learned during their training. Hypothesis comprehensively disproved.

I asked Claude to write up the experiment and its headline results (pdf).

It’s always good to test a hypothesis before you starting building stuff. AI makes this cheaper and quicker than ever.