Informing local decisions

The overall aim of the cost impact project − to better inform local decision making − was addressed primarily through the workshops and interviews with commissioners and decision-makers and the review and testing of existing methods and tools. That said, the analysis of NICE’s current methods for public heath also made an important contribution.

5.4.1 Current situation

As noted earlier, the limitations of using CUAs to assess the cost effectiveness of public heath interventions are well documented: these include the QALY,

perspective, time horizon, discounting, health inequalities and equity. Comparators and lack of knowledge about current practice, paucity and quality of data and the complex nature of such interventions were also noted for the serious challenges they pose for economic evaluation. Perhaps not surprisingly the recent reviews of the literature on economic evaluations of public health interventions found that relatively few studies applied a CUA (27% of studies in the review by Drummond et al. 2007 and 14% in the review by McDaid and Needle 2006). These findings are consistent with the workshops and interviews with commissioners and decision-makers and the review of existing ROI methods and tools which found a wide range of criteria and ROI methods in use for supporting local decision-making.

Taken together, the analyses showed that commissioners and decision-makers require a range of different information relating to public health interventions including the direct, indirect and tangible costs and benefits, regardless of sector, presented over the short-, medium- and long-time horizon and which, as far as possible, are quantified and reported in a disaggregated format. The provision of outcomes in natural units and indicators of value for money in the short-term (for example reduction in bed days) was also considered to be very important in the current economic.

As noted by other authors, the information required for decision-making may be

148 better tackled by undertaking a cost consequence analysis (CCA). Not only would this help to address the narrow focus of the current approach (that is, CUA), it would also provide the necessary data for calculating a range of ROI metrics. As we saw from the review of literature and research conducted by Matrix, a range of ROI metrics are in use and decision-makers do not appear to favour one method over all others. On a cautionary note, there was an indication that the metrics may not be that well understood. In addition, care will need to be taken in deciding which metric(s) to use as the research indicated that the different methods for assessing ROI could give rise to different decisions.

It is important to note that CCA is not without its limitations. One of the main weaknesses concerns its lack of transparency (McCabe and Dixon, 2000). In arriving at a decision, McCabe and Dixon note that the decision-maker has to implicitly weigh all the different impacts and relate these to the costs, and then

decide which interventions represent the best value. In addition, the values that drive the decision will be the decision-maker’s values which may or may not reflect the values that society would wish to be used. Interestingly, the ranking exercise with commissioners and decision-makers revealed considerable variations in the

importance placed upon the different criteria both between individuals within a sector as well as across sectors.

The review identified a number of tools that take into account of a variety of

information (criteria). Although the tools varied in their complexity − for example in terms of the number of criteria included, the method of weighting the criteria and the method for combining the criteria − the vast majority were examples of multi-criteria decision analysis (MCDA). As noted earlier, MCDA techniques aim to provide

rational and transparent approaches to decision-making. While MCDA appears to be a promising way forwards, further consideration of its strengths and weaknesses for decision-making and prioritising interventions is required. For example, issues such as which factors should be included in the analysis, the weighting of the factors, how the factors should be combined and how to avoid double counting need to be

explored.

149

5.4.2 Conclusion

Returning to the original aim of better supporting local decisions about investment in public health, it is clear from the research that decision makers prefer to draw on a range of criteria when making investment decisions. It appears that our current methods are insufficient for this purpose and additional methods and metrics will need to be introduced.

The results of the interviews with commissioners and decision makers showed that the majority were reluctant to base their decisions on cost effectiveness alone. The ranking exercise revealed considerable variations in the importance placed upon different criteria by different decision-makers both within and across sectors. There is little evidence on how decision-makers trade off different criteria.

MCDA could be an important tool towards achieving a more rational priority setting process. There are various approaches to MCDA and these are mainly characterised by the criteria included in the MCDA and the methods used for measuring and

weighting the criteria. Further work is required to assess its strengths and

weaknesses. As cautioned by others, we endorse the notion that MCDA should not be seen as a formulaic approach to priority setting, but rather as an aid to decision-making.

150