What good assurance looks like for NHS boards - Part 2

This is the concluding section of a two-part article about the role and importance of the board assurance framework (BAF) in effective NHS governance and leadership.

In the first part of this article, we suggested three questions for boards to consider to assess the quality of the assurances reported to them. Today we share the second set of three questions:

1. How current is the data?

This is an obvious criterion – data should be up-to-date and timely. However, some information – particularly comparative data from national clinical audits or surveys – may only be produced annually, or even less frequently. Users must consider what may have changed since the data was gathered. For example, CQC’s patient survey of community mental health services, published in autumn 2020, was based on fieldwork from late 2019, before the start of the COVID-19 pandemic – another world, as far as the NHS is concerned.

2. Is the data providing assurance against a defined standard?

It is helpful, although not essential, if the assurance includes an assessment against a clearly-defined and accepted standard, for example clinical guidelines from the National Institute for Health and Care Excellence, the NHS Improvement Well Led Framework, or one of the trust’s own corporate policies or standard operating procedures. To use a common phrase, Boards wish to know ‘what good looks like’ and whether their organisations are achieving it.

3. Does the data enable comparisons?

Again, it is helpful, if not always possible, to compare performance over time to show whether the organisation is improving, standing still or deteriorating. Many NHS organisations now incorporate statistical process control charts into their performance and quality dashboards. These can indicate whether monthly variations are statistically significant and therefore merit further investigation, although users of these charts must first understand the principles behind them, and what they show. Data which benchmarks the organisation against its counterparts in the NHS is especially helpful, although it is important to find meaningful comparators so that we do not compare ‘apples with pears’.

Readers may have noted the tensions between some of these criteria. For example, CQC’s national staff survey scores highly on most of them; it benchmarks trusts against their counterparts, the data is analysed by external companies rather than the trust itself, and it undergoes extensive data validation. However, such a large-scale exercise can only be done once a year, and the data collection and analysis process means that although the questionnaire is circulated in autumn, the results are not available until the following March. In a trust that has just experienced a tough winter, staff morale may be lower than the survey results indicate. Thus the staff survey scores poorly against the timeliness criterion. Because few assurances will score highly against all of the criteria, it is essential to triangulate different assurances for each strategic risk, rather than relying on just one or two sources of assurance.

NHS board and committee papers are often crammed with bar graphs, pie charts, tables of figures, but we should not discount qualitative data such as the outputs of focus groups or other staff engagement exercises, which can help to explain why the numbers are as they are. Qualitative data can bring to light less tangible, cultural issues. These are rarely referenced in the board assurance framework, but can prevent a trust from achieving its strategic objectives in just the same way as understaffing, a persistent budget deficit, or unreliable IT systems.

It is good practice for board assurance committees to own two or three risks each from the board assurance framework, and review them regularly. We would encourage them to use the criteria above to form a view about whether they are receiving the right assurance. To inform the discussion, those who compile and update the BAF should do a preliminary assessment of their assurances, and may also wish to consider arranging for the quality of the data within them to be audited.

For board members’ peace of mind, reviewing the board assurance framework in detail will be time well spent.

If you have any questions or comments on this blog, please call us on 07732 681120 or email advice@good-governance.org.uk.