When AI Agents Swamp Healthcare

Sam 0:00

Imagine it's 2028 and you have a personal AI agent to help you manage your health. It monitors your blood pressure. It orders your medication. It spots when something's not quite right. It can even liaise with your GP to book an appointment. Now, imagine it's 7.59 a.m. on a Monday morning, and millions of these agents are springing into life, dialing into surgeries across the UK. They might not get through, but they don't get tired. They don't get frustrated, and they simply won't give up. What happens when our health system is flooded with well-intentioned automated demand? And what happens when only people using agents can access health care? It's a very real scenario, and much closer than you think.services. So, today I'm joined by Tedot Vanenin, who heads up Design at NHS England. He led service design for the national COVID testing and vaccination programs, and he oversees the NHS Service Manual and the Design System. Tedo, it's great to be here recording with you today. Thank you so much for taking the time to do this. Hi, Sami. Thank you for inviting me. It's good to be here. How would you summarize the big problem you're trying to solve?

SPEAKER_00 1:49

It comes from the fragmentation of the NHS. What I'm trying to do is simplify the complexity. And NHS is so fragmented, it's something like 30,000 individual organizations and providers within one umbrella, which is the NHS. But when we're trying to design and provide services to the public through that organization, that fragmentation, that creates challenges. I guess when you're talking about big problem, well, that is the big problem that I'm trying to solve for us to us to internally see that complexity so that we can make it simpler and better for the public.

Sam 2:29

That's a mighty problem to take on. Does it ever feel overwhelming?

SPEAKER_00 2:34

It definitely does. And I think everybody who works at NHS would feel the overwhelming. And the overwhelm comes from not just from the complexity of the system. That's what we do. And that's what I've done most of my career is kind of looking at kind of really complex systems. But with the NHS, why we all work here is the responsibility that we feel. And I think that does add to the overwhelm. We have responsibility. We can see how, when services fail, what the impact can be. And we try and help, we try and solve those problems, improve those services. But that's where the complexity then makes it really difficult to do that. But we can see the impact. And we can almost see the kind of if only we could do that, this would make it so much better.

Sam 3:17

You've had a varied career where you've moved between finance, telecoms, and now you're working at NHS England, or you have been for some time now. I'm interested in what the common thread is. What keeps drawing you to complex systems and how has your background affected you?

SPEAKER_00 3:31

I don't know what keeps me drawing into complex systems. I I'm not sure if I specifically looking for those, at least not in the beginning of my career, definitely didn't. I think I've been at NHS now for 10 years. In fact, a few days from this recording, you know, is my tenth anniversary. And I would say this was definitely a choice I wanted to make. And that was kind of choosing a complex system because I knew the NHS and I wanted to be that. But before that, my career, 20 years, has been in the private sector. And I started an engineer. I did my degree in compute engineering. And I think engineering mindset is very much about problem-solving mindset. And I there's so much similarities between engineering and design in that sense. I love the kind of idea of looking at a problem, being able to take it apart in kind of individual parts and see how can we start solving that problem. I've always been a tech geek. I always liked technology. I always liked the kind of the classic thing engineering, kind of taking things apart and trying to put them back together. So I always liked that. I remember we got our first computer as a family. I was nine years old, it was Synchr Spectrum. And he was just, I mean, it's just amazing. It was just a wonderful kind of entry point into computing. And I think that really kicked off the kind of my love for computers.

Sam 4:45

And more recently you've been thinking about game theory and how that can be applied to some of the work you're doing. You've recently written that healthcare is an infinite game played with finite rules. Can you tell me a bit more about what you mean by this?

SPEAKER_00 4:58

So if we look at the kind of what is infinite game and what is a finite game. So if you think about finite game, let's take it out of the healthcare first, it's easier that way. So finite game is something like football or game of tennis. We know when the game starts or we know where the game ends. And we know what the rules are of that game. And we know who's playing that game. And in a finite game, there's always a winner. But if you look at infinite game is we don't know necessarily when it starts and it doesn't really have an endpoint. That is part of the infinite game. We also don't know who is going to play that throughout that game. So we don't know when new players may come and new players may leave. And also the rules are not very clear. So rules may change throughout that. And the point of infinite game is to keep playing. It is literally creating the environment, sustaining the game. So what do we have to create during that game that we can keep playing? And that is the goal of Infinite Game. It's not to win, it's to keep playing. And if we then take that into healthcare, well, you can't win a healthcare. You can't win the NHS. There is no end goal to that. It is it's literally we are here to keep playing the game. And how do we keep the NHS, the healthcare, running and sustain and be viable? And of course, to do that means it needs to provide care to the public. And also we can look at it from the patient's point of view or people's point of view. Your own health is an infinite game. You can't win your health. So these are the infinite game things that we have, and that's the game we are playing within the NHS.

Sam 6:38

I loved your analogy that you used in blog post where you talked about children playing keepy upy with a balloon.

Circular Journeys And Crisis Triggers

SPEAKER_00 6:43

I was thinking, how do I describe it's easy to describe finite game as a game of football or game of tennis. What is the equivalent of infinite game? And I thought, well, that's what we all have done. You give kids a balloon, what they try and do, they try and keep it up in the air. And they jump on sofas and armchairs, and they may create new rules, or you can't touch the floor now, and you have to just be on the here. But the balloon definitely can't touch the floor. And their friends may join or they may leave or whatever else. But very much it's there's no winners and losers in that. What's the balloon in healthcare? It's a good question. I think it depends which lens you're using, and I think that's really important for us. So, like I said, the whole healthcare could be a balloon. How do we keep the healthcare? How do we keep the NHS up in the air? How do we keep that going and not touch the floor? It could be a person's own health. How do they keep their own balloon in the air? Or we could look at it in more detail on a service level. A service, getting screened balcons, getting vaccinated. How do we as a system, how do we as a service provider keep that service in the air as the balloon? You know, how do we make sure that service sustains and is viable? And that means we we have to look at how do we build different components. How does that service evolve over time? When the system around it, when people's needs change, when technology changes around it. That's about keeping that service, the balloon up in the air.

Sam 8:01

You describe healthcare journeys as circular, not linear. Can you walk us through what that looks like for someone living with anxiety or depression?

Services As The Glue Between Gaps

SPEAKER_00 8:10

When we're designing, when we're mapping services, journeys, user journeys, we often do them linearly. We start from the left and something happens and it ends in the right. And I was thinking at the time, I was thinking, well, but people's lives are, whether you are managing a long-term condition or whether you're preventing becoming ill, we are very much following patterns and following kind of set things. There might be a long, kind of longer buttons kind of in our lives, but we do the same things over time. If you're diabetic, you will have your insulins, you have to pick up your medication, you have checks, you have this and that. Same thing if you have asthma, you have certain checks, you do that kind of thing, and they happen routinely. If you are managing it trying to kind of, let's say you're trying to lose weight, therefore you have exercises, you may have diet things, and they may work or may not work for you, and you have to try different things, but there's very much a pattern. And I thought describing that, kind of visualizing that as a circle makes much more sense when we describe how people are going through their lives. But the key thing in those things is that not, you know, the circles may, at some point, people may fall off from that circle. And I call those kind of trigger points. Something happens in person's life that triggers them, that they're gonna fall off on that. And I think that that's the really key thing for us to understand what that is. So they may have a bad episode. So if you look at the kind of example what you described, kind of living with anxiety and depression, there's certain things that they do to try and prevent that, to kind of maintain that kind of situation where they're in. But then something happens. They, for example, if they lose their job, that's a trigger point that may then cause them to have a crisis moment. And therefore that kind of sets them into a different path that we need to understand what that is. But I think what's really important in that is also to understand that is not then just a healthcare or kind of what we consider kind of NHS path. If we look at that, you know, they lose their job, they may ask, they may get universal credit, and that would be then Department of Work and Pension. So GWP, their crisis, they go to, they may go to AE, they may go to their uh GP, they may then get referred to to a local authority to kind of mental health support, and they make it back there for job centre when they're feeling better. So understanding what is the circular paths, but also the trigger points, and where does that get leads them to, and what other circles will follow from that? And what is the new circle, the new, what they need to manage after that. And as the system, it's really important that we understand these overlapping journeys, overlapping circles, if you like, or the common trigger points. Like I said, losing somebody losing a job would be a trigger point possibly for us in case of like I just described, but it would be for DWP as well. And understanding those and kind of looking at, well, how do we as a system make that work? Because when the person's in that situation, when they have a trigger point, we need to make sure they fall into something. We can catch them as a system, we can catch them and we can help them. They are in a moment of crisis when that trigger happens. Something's has happened. They may not be able to navigate that. We need to look after them. We need to make sure that the system works for them. And that's why it's important to understand the kind of circular paths that things may seem to be going on quite a while in a certain way, but then those trigger points happen and something will happen after that.

Sam 11:34

And that leads quite nicely into your other recent blog post where you were talking about services being the glue that stop people falling through. I wondered if you could tell me a bit more what you mean by that and why are services the glue in a complex system like healthcare?

SPEAKER_00 11:49

So let me give you an example. It was the latter part of the pandemic. My wife's uh asthmatic and she needed new medication for to manage her asthma. So she used the NHS app and she ordered a repeat prescription. That went through, that got approved, that went through to our local pharmacy. And she got notifications saying the medication's here, please come and pick it up. Unfortunately, at that point, after she booked that, we had to self-isolate because we both tested positive for COVID. And my wife then uh called the pharmacy and said, Look, we can't I can't pick it up. Can you deliver it? And the pharmacy said, No, we can't deliver it, we have no delivery service. So the only offer that they could give to her was if they cancel that prescription and my wife orders again but sends it to a different pharmacy that would have a delivery, then that would be okay. The added challenge with this one was that my wife knew that being asthmatic and lifelong asthmatic, that a certain amount of prescriptions would trigger a that she needed to go to a review. She needed to go and to the GP practice to do a review of that so they could do the review prescription. And she knew that the next one would trigger that. So she couldn't cancel that and reorder it because she could not attend to review because we are both uh uh self-isolating. So all parts of that NHS app worked brilliantly. She could order the review prescription, she got notification that the prescription and the medication is in the pharmacy. But all in all, she ended up with no medication because we couldn't pick it up until we've finished self-isolation. So every part of that journey worked as it was designed. But overall the outcome, getting the medication to the hand of the patient, didn't happen. And I think that's one example. That's a that's a very personal example to me. But there are many other examples in when we look at different kind of healthcare journeys. And you know, there's a recent research in 2025 that says that 14% of referrals from primary care to secondary care are stuck somewhere between the primary care and secondary care. They are either lost in the somewhere in the system, they are rejected, but nobody seems to know that they're being rejected, or they were never sent. So the result of that is is over 600,000 referrals a year don't go anywhere. So the person who's been referred will not receive the care that they are needing at that time. And the same researchers, 70% of these people had to change that themselves. So they had to become the conduit of understanding what's going on, where's the referral, what time is it, where is it. And that makes the humans, the person, the glue between the capabilities, they are managing that, the gap between the capabilities where people are falling through. But really, that shouldn't be the humans, it would be the service that is the glue. The service, end-to-end service should make sure that people know where they need to go, what time. Or for example, if they are attending a appointment which is operation and they need it fast for eight hours or whatever, that they have done that. Because if they ha don't have that information, then they've fallen through the gap. They are they're failing to kind of attend that. And this is not me blaming anybody. I think this is just something that everybody's trying to do their best, and yet still the people fall through those gaps.

Sam 15:19

I was talking to someone about the comparison between how financial services have got very good at managing that end-to-end customer experience of an application or a process and that side of things. And that's because they have the ability to have someone who owns that end-to-end experience and say, Well, actually, we need to join these things up. We need something that connects this bit with this bit and so on and so forth. It feels like in the NHS that there isn't really anyone that owns an end-to-end experience for a patient. And the that sort of coordination gets outsourced to the patient to try and create their own end-to-end experience.

SPEAKER_00 15:54

And that comes back to the kind of complexity and fragmented nature of DNHS. That's the 30,000 different organizations and help providers within DNHS that a single patient journey covers many of those. How do we all work together as a system to ensure that people don't fall through?

Sam 16:17

And I know inclusion is also really important to you because you've written about exclusion starting at user journey mapping. How does the way we map or plan services lead to this kind of unintentional exclusion?

SPEAKER_00 16:29

Yeah, we used to talk about years ago that we need to design for the etches, the etch cases, and therefore we design for everyone. That's not wrong, but when we look at kind of that, 24% of people in the UK have declared a disability. And that's a disability they had. That's not even including kind of situational disabilities or temporal disabilities that when you've broken your arm or when you're carrying a baby, you only had one arm at uh at use, which is kind of a situational thing. That number is no longer an edge case that we can just say, oh, we're just going to design for the edge case. This is the people we need to design for. You know, we're talking about more than a quarter of the population. We need to make sure we include people that have different needs, that have different ways of thinking, uh, different ways of doing things, different contexts of use and different kind of situations in life. We talk about the diversity of kind of demographic diversity, but also cognitive diversity is really important. And people with different abilities and different disabilities, really important we include those as early on us as we can. So we can really design services that work for people. And it's essential to people who need them, but it's really useful for everyone when we design services that are inclusive and accessible.

Sam 17:52

Which almost goes back to that designing for people on the margins, because it makes the service more accessible for everyone.

SPEAKER_00 17:58

It is exactly that. And but the margin is now a quarter of the population. So it's no longer just uh what we're saying is that we can't ignore that. We can't for us to talk about that we are designing for the margins makes it s easier to ignore that, saying, Oh, they are just the margin, we don't have to worry about them. They are the people. They are, you know, we are all part of that. Both of us are wearing glasses. We are aging population, we start needing different needs, and the needs change all the time. And I think it's that false kind of view of thinking that they are different, we need to design for those people. Because no, we need to design for us, you know, everybody. And that means people with varying needs and different context of use. And we need to take that. That's no longer something that we can take optional.

Sam 18:46

One of the things that strikes me is that it's often the people who are in those kind of those groups who have the highest need for health services.

SPEAKER_00 18:53

It is, exactly. And when we look at kind of digital, digital health services, there's research, there's data that said if designed well and designed properly, they can really have an impact to kind of reduce the health inequality in population.

Building Organisational Memory That Lasts

Sam 19:08

I'd like to move on and talk a bit about organizational memory. This is quite an abstract concept. Maybe maybe before we go into too much detail, can you define a bit what what you mean by organizational memory?

SPEAKER_00 19:21

That's really related to the infinite game theory. Infinite game is about we need to be able to sustain the gameplay. We need to be able to keep playing. And that means we need to evolve. We need to be able to evolve to the changing environment that we are playing that game. But it also means that we need to make sure that the players, us in an organization, that we have the right tools and the right skills to keep playing that game. And it's also about, you know, when we go to kind of talk about sustainability, you know, two kind of economical sustainability, environmental sustainability. And of course, NHS has an ambitious green NHS target as well, which is a big part of that. So this is about organizational memories, about what can we learn from what's happened before? Both in a way that, right, that's already been done. We don't have to do it again, we can just reuse what's already done before. Or what has been done before, and what did we learn from that? Is it time that we try a slightly different way or have things moved? But it's learning from that, what's happened before. And then the sustainability element of that is that what can we reuse? So, what can we create whilst we are playing game, whilst we are going through this, working in in these organizations, organizations, what can we what do we leave behind to the next players, to the next people picking this up? And how can we share that to the wider? So, what we're doing, for example, at NHS England is we have our NHS Service Manual, we have our NHS design system that is part of that. And for us to centrally collate guidance, standards, kind of ways of working, best practice, or the design system is uh is a collection of front-end code and components, design components that anybody can use to create new services, digital services. And that design system is the components are accessible, they are usable, they've gone through user research, we know they work. And very easily other service, other teams in in health and care can use those to create something for themselves. And it doesn't have to look exactly the same, they can change it, things can change. But that means that we've done the work that others can benefit from that. So the system can then the other teams don't have to focus on fixing the same problems over and over again. As the system, as the NHS, we can use the resources and the time to focus on the unique problems that the teams are experiencing, rather than reinventing the wheel of the kind of common things that we have we can solve once. And I think that's really Good example about the kind of organizational memory is that how can we do more of this? How can we all share more about what we are doing that we know works and we can say, look, this may not exactly work for you, but look, there's something here that you can use. You don't have to start again from scratch. And that's how we kind of move forward in this one. That's really the point of Infinite Game, is that we're helping each other. We are in this game together, we're just pushing each other along.

Sam 22:27

So one of the techniques that our team uses when we're working on longer-term projects that design histories, which I'm sure you're very familiar with, may be behind setting them up in in the NHS. And so so if you're not familiar with design histories, obviously you are Terry, but other people listening, design history is just a kind of a backlog of what design decisions have been made on a particular service or product. And I was thinking about those design histories in the context of an organization, and I don't think there's an equivalent for an organizational design history. Like what are the changes we've made as an organization at a kind of high level and why have we made those? Or there certainly isn't in the public domain for big organizations.

SPEAKER_00 23:07

I think the uniqueness of design histories is that they are public, and I think that's a really important part of that. So when we look at the public sector, we can look at what other government organizations have done. You know, what has the Home Office done about that? What has the Department of Education done for certain things? So we can learn from that and we can look at how we can go and talk to, because of they've written something, we can go and talk to the team and talk to the people. And vice versa, of course, for others can read from us. So that to me is the unique thing about design history, which is really important. But I would say if you talk to program managers and project managers, more that kind of thing, you know, we have decision logs, we have risk logs, we have all that, but they are so they are usually internal. We don't publish our decision logs and risk logs into the public. But they exist and they are really useful in that way. Because that is exactly what decisions have we done before. I mean, they could be in spreadsheets or they could be in in somewhere in the internet in a much more fancier tool. But that is really good, that is the similar kind of logic, and that's really important. But I think there is what design histories have over a decision log, which I've seen, is design histories have a lot more of the context and a lot more of the story of that one, not just the decision that was made. Design histories focus on what research was done and what are the user needs and what do we learn from by doing this process. So I think maybe the ask for people holding up decision logs and risk logs is to how can we bring more of that context into those things?

Sam 24:35

Maybe it gets harder when it's around business decisions because a particular decision might not be particularly palatable for the public domain. It might have had to be made for a financial reason or for some other reason that isn't necessarily what people want to hear.

SPEAKER_00 24:49

That's very true. But they even just holding those internally for new people joining the team or new you know within an organization. Can we have those more public so that we can share what other teams have been tackling with? Not everything needs to be public, but I think the organizational memory and the kind of sustaining the infinite game to play means that we are giving our people the tools and the knowledge to continue to play the game and not to chip out over the same things over and over again. What's the benefit of design histories being public? It's the learning of cross-boundaries, across public sector. I think that's the biggest benefit of that. So different government departments can look at different teams across, which going back to the kind of circular and like I said, you know, circular journeys and the trigger points, and those trigger points are not shared just with, you know, they're not just within NHS, they are shared across different departments. So we need to work across public sector. Therefore, understanding what other public sector departments are doing is really important. We need to get into that. So it's that knowledge share across boundaries. But I remember I was working in the private sector at the time when GDS was launched and the service standard was launched, and that was public. And we used that in public in private sector because it was it just made so much sense. And because it was public, we could use that. But we could also point at it and say, look, this is what the government is doing. So making things public, publishing things in that way, other people can use it and use it as inspirational. Like I said, you know, we couldn't say in private sector we're mandated to it, but we could just point at it and look, this makes so much sense. And if the government's doing this, surely we should be doing something similar. So it gives that kind of it gives ammunition, it gives the knowledge to people who need it.

LLM Search For Reusable User Research

Sam 26:34

When I spoke to your colleague Rochelle Gold, probably about a year ago now, she did one of these recordings. She talked about the research, user research repository that she'd been working on and this AI layer that was going over the top of that. Is that something that's been helping with this organizational memory and how's that going?

SPEAKER_00 26:54

It's definitely helping. It's it's really the kind of right kind of tool where we're really we know how important user research is. And user research pro provide the kind of the output the pr they produce a huge amount of information and huge amount of data of that. Our users, and that again, that can be used across different services, different teams. It's really useful. And again, it's kind of like when something new gets kicked off. Most probably we have done some user research on a similar thing, if not exactly the same thing. So giving that background information of where are we even starting from, if or even giving the data we don't have to do it again. User research is expensive, it's time consuming, it's definitely value for money when it's done. But if we can reuse what's already been used before, that's all the benefit that we can do. Uh and so that's really worked. Now, I I've recently done myself uh a couple of times when I had to find information about a kind of wider subject where I needed to know what do we already know about that. Being able to go to use our user research finder, and because it's large language model based, I could ask a question in a way that I wasn't really sure necessarily what I need to ask. So it's lowering the barrier of getting into that information. But you know, I I uh Rochelle would say definitely, and other user researchers, it's not replacing user research. It's not meant to replace user research. Teams still need to do user research where it's absolutely necessary, where we have those gaps. But what it does is that it it gives the more of that knowledge, shares that knowledge. So we are much more educated in when we do start user research, what do we need to do that on? And we can identify those problem areas much more much more in detail when we're starting a new project.

Sam 28:39

How's the LLM layer going? I remember a year ago it was like, wow, that's really innovative and exciting that you're doing this. And now it's become much more normal.

SPEAKER_00 28:49

It's working well. I think large language models used in this kind of way make sense. Whereas before you had to be very specific what you need to find. You know, classic search tools are very you need to know exactly what you're looking for, otherwise you'll it'll it'll may not find it for you. I think large language models are good because you don't have to be so specific. You can ask much more natural language questions. You may still not get the answer from that. You know, the the intelligence on AI is they're not really intelligent, but it does give you a better chance of getting somewhere with it and being able to have the conversation and being able to ask fuller questions is really important and really useful.

Sam 29:28

You've recently moved into a role working in enterprise architecture. And for some people, this might make them think of governance frameworks or technical diagrams. What does enterprise architecture mean to you? And what did you see as the service design opportunity in this space?

SPEAKER_00 29:45

I need to be careful here because I definitely don't claim to understand everything about enterprise architecture yet. I've only been with the team for a couple of months. But I can see similarities in there. And to me, enterprise architecture, yes, governance is part of that and kind of drawing models and pictures with boxes and arrows and kind of frameworks. But what's enterprise architecture to me is about creating a shared map of how the system works and not just focusing on technology, but about the business processes, the data, the decisions, the kind of organizational layer of how and why do we need to do certain things. So service design fits into that relatively well, and that's what I'm really exploring at the moment. If you think about service design, that's designing the kind of experience on a street, let's say that how people move through a street, what do they encounter, how do they feel, how do they interact with the different shops and kind of different elements on a street. The enterprise architect could be about why is that street there, where do you go from that, what are the roads, the plumbing, the power, the kind of grid underneath that and sets it up. So we do need to use both of those to really make the kind of complete picture of what that is. But service design and enterprise architecture come from a different place, but they should end up in the same place.

Sam 30:59

You've described service design and enterprise architecture as practices that create shared language. What are you actually seeing that makes you frame it in that way?

SPEAKER_00 31:08

Both service design and enterprise architecture are about creating repeatable, reusable elements, components of whether they are services, service patterns or other kind of technology patterns or kind of capability patterns. But they need each other to kind of complete that picture. We need that shared language across the system so the different parts of the system talk about the same things at the same time. So when we are connecting different parts of the system together, we know they fit together because we've used the same language and the same components and the same models to kind of bring those together. And combining those, what we are really looking at now is what we call person-centered architecture. So how do we really kind of create an architecture that is led by the person, the user journeys, the user research, the person's needs and their kind of requirements, and then we design the architecture to match that?

Sam 32:01

I guess one reading of that argument is that service design with service design without enterprise architecture can't scale. And enterprise architecture without service design can't actually connect to those real human needs. Does that feel like a fair interpretation?

SPEAKER_00 32:16

We're talking about the same thing, which is about system design services, the capabilities, the way that we do the people, organizational structures, policies. We talk about the same things, but where we're coming from is a different direction. I might be a little bit simplified saying that architecture and enterprise architecture comes from the organizational point of view. What does the organization, the enterprise, need to do to make things happen, to have these services out to serve the people? And service design is coming from the outside in, looking at so what do the people need, what are their journeys, and therefore what services should we provide to them? And the scaling things, I think sometimes in service design, we're designing services, and it's difficult to connect them. And it's not because service designers are not skilled to do that. I think again, this is probably more systematic problem, is that when a service designer works in a team, a part of a team, connecting that service or part of the service that you're responsible at that time to the other parts, you know, we discussed the kind of gaps between capabilities and gaps between services exist. How do we do that connection? How do we make sure that our service connects to the next service seamlessly and glues those capabilities together if you're working in one team? And that's where enterprise architecture can help. Talking about the common language, talking about the same way of modeling things and kind of mapping things in enterprise architecture way, we should be able to do that. So, in that way, enterprise architecture can help with the scaling aspect that service designers often, especially in health and gay, often struggle with because the lens is different and the language is different, but yet we're trying to do the same thing. That's why they work very well together.

Agentic AI And A New Inequality

Sam 34:00

It wouldn't be a podcast in 2026 without some discussion around AI. One thing we've spoken about is the fact that agentic AI could actually widen inequality in the NHS rather than close it. And I wondered if you could talk me through that argument a little bit.

SPEAKER_00 34:19

AI is everywhere at the moment, large language models, and people are using them a lot. Agentic AI, it's a slightly different way. So if you look at kind of what the AI we normally use, we're using it as a chat. So we talk to a to a large language model in response, we ask a question, it response back, and we can have that conversation. It has a memory and we can return to that and we can start new conversations. And we give that agent a task, a goal to achieve, and then it goes and does that. So the model is different. The technologies are similar, but you don't typically interact with the agent. You set it off and it goes and does something. And what's important for us to start to understand, let's take appointment booking as an example. Book an appointment to a GP, the classic 8 a.m. telephone, you have to call up 8 a.m. otherwise you will not get an appointment. NHS app can allow you to make appointments. Now, my own GP practice, I can't use NHS app to book an appointment. They use a different system which I have to use. But I can go in there and at times it says, sorry, there's nothing left. Appointments are full. You have to wait 8 a.m. tomorrow morning to do that. So I'm in the situation again. Let's say I've got a personal health AI agent that I give a task and I tell it to go and why the task could be manage my health, but let's say let's just go and book an appointment for me. It goes and it fills in a form on this third party and it could use NHS app. NHS app has an API, it could use that one, but like I said, my GP practice doesn't use NHS app, so you can't use that one. It could go and fill in this form online, but pretty quickly it would figure out that there's there's this phone number to call at 8 a.m. Currently it's totally reality at the moment that they can use both voice recognition but also voice generation, so they can generate audio. The AI agent could call that 8 a.m. number to my GP practice on my behalf and book an appointment. Now if we're busy, it would try again and it try again and it try again until it gets through. The difference between AI agents and humans is that I would get fed up and bored at some point, I wouldn't call again. The AI agent will not get bored. It will be relentlessly calling that phone number until it gets through. So if we then think about let's scale this up in a couple of years' time, this could be the reality. AI agents are much more prevalent and there's a lot more of them people can use them. And all of a sudden what we've got is AI agents calling these numbers. They may call hospitals to check for test results when they haven't received, they may call these. We discussed the loss referrals, and people have to call where is my referral, where do I need to go? They may call these numbers, they may fill in all any form they can find on the internet for GP practice, hospitals, any kind of healthcare to manage your own health. But when we scale this up to the hundreds or to the thousands of agents doing this all the time relentlessly, they may bring the NHS down. And what's the thing about the equality here is that this is at the moment, they are available, you can do them, but you need to know technology, you need to be able to do this, or you need to be able to pay somebody to build this for you, you can buy them. And in the next few years, that is unlikely to change. I think you know they're gonna be some payment models on these ones, but they're not gonna be for everyone. Just like digital, we can't say everybody can use digital, definitely not everybody can use AI agents. But if those AI agents are then blocking the access to telephone and other kinds of digital forms, they are blocking access to those people who need them to phone their GP practice or to the hospital because those phone lines will be busy, the forms will be will be full, service will be totally down because of the number of requests they get. And what we are potentially creating here is say a really quite wide divide that there are going to be people who can't use technology or don't want to use technology or can't use that, and they're still one part of that. The group of people we're now aiming to be able to use things like NHS app may be pushed out from that because of AI agents take over. And then there's the group of people who either can and program AI agents who can own them or who can afford to pay for them to be built for them, who are then being able to get the healthcare they need. And we need to be really careful that we don't build the world now that does this. And if we are not careful, this is going to happen. And it's not really our choice in the system whether it's going to happen or not. This is out there, these are the patients, these are the people who are going to use these AI agents. We focus on AI, how AI can help the NHS to be more efficient and everything else. But this is a different conversation we're having. This is a conversation that how are we going to build a system and how are we going to architect a system that can respond meaningfully to the scaling up of AI agents so that they don't bring down the health system? So, how are we giving AI agents, how are we building them the APIs that we can control and they can find easy routing so they don't use the phone lines? So, how do we make sure that we build our system to be AI agent-friendly and not block them out because they will find a way and they will then use those routes in that are meant for people who can't use digital?

Sam 39:48

And I wonder how if we get to the stage where AI is making decisions around what should be built, I wonder how AI would tackle that particular challenge.

SPEAKER_00 39:56

That's a good question. Is AI going to be friendly AI and allow other AIs to come through? Don't know. The challenge for us is here is the this is easy for us to ignore that this is not our problem. My worry is that it will become our problem. And this is exactly the kind of where service design and enterprise architecture working together then can really help. And how do we architect a system that allows AIs? But how do we then make it so that those how the services then operate and how do they respond to those AI agents? Because they then become the user of that service, but of course they're completely different users to the human user. And how do we then create services that work for non-AI users, people? And I think that's exactly the kind of the point of combining service design and enterprise architecture together.

Outcomes Over Outputs In Public Services

Sam 40:41

We've talked about a lot of different things today. We've talked about game theory, we've talked about inclusivity, AI. If you could change one rule of the game, so a sort of a structural incentive or a funding mechanism that would improve the way that healthcare is delivered, what would that be?

Closing Thoughts And Healthier

SPEAKER_00 40:58

I think that is it's easy to say, let's get teams together to solve the end-to-end problems. But structural change is really difficult. And I always think about GDS government digital service, this was before the pandemic, but they were really advocating for government departments to form service communities. And I think that's a really good idea because that basically means that you're forming a community around a service and service outcome. You're not transforming anything, you're not moving people around, you're not creating new teams, you're just literally getting people together to look at a problem at hand. And I think that's a really, really good idea. But they need to have the people who can really change the way that we work and make those decisions. So we need those senior leaders to be part of this. It can't be just the service designers and the user researchers and the product people to do this work. We need those senior decision makers in this. And I would say what what you're going to do differently when you come to work next Monday is don't just focus on what's in front of you. You know, you have to look around and see what's in your remit, what's around that. Bruce Lee entered a dracken and his kind of philosophy was that. He said that it's like a finger pointing to the moon. Don't concentrate on the finger or you miss all the heavenly glory. We need to look at the wider picture and are we helping to deliver the outcome, even if we are just responsible, small part of that. And it is so easy in a system that is so under pressure to deliver more with less, is that we're just going to tighten up our focus into the thing that is right in front of us. But really, we need to look at so that we can look at can we take responsibility of either gap, either the one before or the one after? Can we take responsibility of that and make sure that we can carry the user across that gap to the next service who can then take over that? And that then leads to the third point, which is let's not just measure outputs. We need to start measure real outcomes. And I'm sure the wider NHS will absolutely get this. This might be just a challenge for us in digital, but it's not measuring, for example, booking appointment, how many appointments have we booked, but have that person actually successfully been able to attend that appointment is the outcome that we need to look into that. And that is the gap. That's looking over that gap. So measuring the outcome across that gap. So things like can they get to their appointment? Do they know where it is first of all? Is there a car park? Is it on a bus route? Is there a wheelchair ramp for them? Do they have all the instructions? Do they know they need to fast eight hours before an operation? Because if they don't, then they're going to be sent back home. And there's interesting research commissioned by NHS England a couple of years ago, which says that in the adult population in England, uh, 40 42% cannot act on a written health information. So when we send these letters and these invitations and this information to people, 40% of the adult population may not understand what they've been told. That's not because they're stupid or they don't understand that. That is literally the way that we communicate with the people. And that number actually goes up to nearly sixty percent if that information includes numbers. So we're talking forty to sixty percent of adult population in England struggle to understand the information they've been sent. So can we just look at the what how do we make sure that they get the information, they understand the information, and they can get to the appointment ready to be ready. That's benefit when we start measuring, you know, these uh DNAs did not attend. And we we can see these big signs on GP practices and hospitals, you know, so and so many people didn't attend this month. This cost DNH's money. That may not always be their fault. It could be our fault that they didn't understand what they've been instructed to do.

Sam 44:45

Tero, it's been a pleasure talking to you today. Thanks so much for taking the time.

SPEAKER_00 44:50

Thanks, Sam. It's been really good, and it's good to talk about this kind of stuff. It's stuff that, of course, in my head, I write about it, but it's good also just to talk about it, try to bring it to life a little bit.

Sam 45:10

Problems Worth Solving is brought to you by Healthier, the collaborative service design consultancy for transformation in health care and public services. Find out more about how we can help you deliver user centered change at healthier.