Using evidence in scrutiny – new practice guide published
What is evidence? How do we get hold of it, and once we’ve done so, what do we do with it?
Good evidence is the backbone of good scrutiny. Without it, making effective recommendations is impossible. But it is one of the trickiest things to get right. We can suffer from having no information at all – no meaningful data on which to scrutinise the actions of the executive. Or we can suffer from a glut of such information – so much that the act of analysis and organisation seems impossible. How do you assess and weigh the information that’s out there?
With the resource available to support scrutiny at a record low, it hardly seems possible to carry out comprehensive, rigorous evidence-gathering exercises. Even if we wanted to, many of us – officers and members – feel that we probably lack the skills to do so.
Our new publication on evidence and data, Using evidence in scrutiny, published with the support of the Society of Local Authority Chief Executives and Senior Managers (Solace) and the Alliance for Useful Evidence (which is itself supported by Nesta) – aims to give scrutiny practitioners the tools to overcome some of these big challenges. While practitioners need to understand some of the basics of what makes “good” evidence – important issues about reliability and bias, for example – the skills required to intelligently evaluate and use evidence in scrutiny are not as substantial as might first appear. A PhD in a social science is not required – merely an ability to ask the right questions, to make connections, to reveal enough of the truth to provoke action and improvement.
I want to focus in on “making connections” because this is one of the main areas where councillors can “add value”. Evidence does not have a single interpretation, and this interpretation will always change based on its context. So a performance scorecard showing green indicators across the board will look better on its own than it will when it is accompanied by evidence of large number of customer complaints for the same service. What we call “triangulation” – joining and mixing data and evidence from different sources to see where it agrees, and where it disagrees – is a key way for members to make their mark on policy-making by applying a different analysis and perspective to what they have in front of them.
Ultimately, this is about councillors having the confidence to challenge and critique the information placed in front of them. Many of course already do this, but without the evidence on which to base this critique, it can sometimes come across as an attempt to unfairly pillory officers based on gut feeling and political perspective.
Making connections requires getting hold of the information in the first place, and another thing that the paper sets out is the wide range of different sources of evidence that can be used to support scrutiny.
We’ve run a couple of national training days on evidence and data, which have both proven very popular, and may run one again later in the year (follow this link to find out about our upcoming events). In the meantime, we’re always available to help you to understand evidence and data in your own authority. We can run evening, half-day or full-day sessions on the subject for councillors, and for the officers who support them. This isn’t about providing a crash course in research methods, but as I said above, the basic tools to inspire confidence to go out, get the information that’s out there, ask the right questions to find out more, and then do something that might help to make local people’s lives a bit better. If you want us to help you do that, please get in touch.
All news