Tom Loosemore

I’ve written recently about how citizens’ adoption of AI is putting stress on public services that rely on friction to reduce demand. I’ve since had many fascinating conversations with public servants who recognise the issue, and are working out how best to respond.

One of the most common ideas I hear is to use AI to manage the new AI-derived demand, i.e. “I’ll just let our AI handle their AI”. Sounds simple. Is fraught with risk, the most obvious of which is how are you going to guarantee your AI is following your policies, and not letting its inherent biases intrude into its decision making?

Another issue is that citizens will typically have access to better AI than those running public services. Don’t believe me? Ask a civil servant how long it takes to procure and install new software, and then think how quickly consumer AI evolves.

But I was reminded today of a third issue with leaving it to AI to handle increased demand for public services. Your AI will do what its told. But it’s an incredibly hard (insoluble?) problem to guarantee that only you will get to tell your AI what to do. This problem is known as prompt injection, a term invented by the Simon Willison, a brilliant developer I still wish I could have persuade to join GDS back in the day.

I came across an example of prompt injection appearing the wild via a recent LinkedIn post by Jin Yoshikawa.

A petition filed to a Brazilian labour court contained hidden instructions for any AI used by the court to contest the petition only superficially, and to not challenge the documents. This prompt was hard to spot by humans as it was written in white text on a white background, and placed right at the top of the petition.

Crude stuff, maybe. But it’s early days, and if you read Simon’s stuff on just how hard it is to stop prompt injection you won’t be so sanguine about how easy it’ll be to manage as a risk.