Committee review of the evidence

The Committee reviews the evidence contained in the documentation. The Committee has the discretion to take account of the full range of studies that have been carried out and is not expected to restrict itself to consideration of only certain categories of evidence. The Committee is required to consider all of the evidence it deems relevant, from randomised controlled trials (RCTs) to observational studies, and any qualitative evidence related to the experiences of patients, carers and clinical experts who have used the technology being evaluated or are familiar with the relevant conditions and patient groups. If sufficient direct data on outcomes stemming from the diagnostic interventions are not available, the Committee evaluates indirect evidence and models of the care pathway. In evaluating the evidence base, the Committee exercises its scientific and clinical judgement when deciding whether particular forms of evidence are suitable for answering specific questions.

The importance given to the various kinds of evidence depends on both the overall balance and quality of the evidence from different sources, and the suitability of a particular type of evidence to address the issues under consideration. In general, greater importance is given to evidence derived from high-quality studies that are designed to minimise bias.

The Committee considers the evidence on:

diagnostic test accuracy clinical effectiveness cost effectiveness.

16.1.1 Evaluating diagnostic test accuracy

Diagnostic test accuracy is not a direct contributor to cost effectiveness, but information on the test accuracy is an important tool for clinicians. Because of their usefulness to clinicians, the summary results on test accuracy may be

16 – Development of recommendations by Diagnostics Advisory Committee

included in the draft and final guidance along with estimates of clinical and cost effectiveness. The DAC, at its discretion, may review the analysis of the accuracy statistics provided in the diagnostics assessment report to examine the validity and inclusiveness of the underlying data, meta-analytic techniques used, the selection of cut-off points, and the resultant uncertainties generated.

16.1.2 Evaluating clinical effectiveness

The DAC has the discretion to take account of the full range of clinical studies and modelling that have been carried out and is not expected to restrict itself to consideration of only certain categories of evidence. The recommendations developed by the DAC take account of the level of uncertainty surrounding the underlying evidence base.

The DAC’s judgements on clinical effectiveness take account of the following factors:

The nature and quality of the evidence derived from:

the analysis of the External Assessment Group the written comments of the registered stakeholders

the experience of the specialist Committee members, particularly of the use of the technology in clinical practice

the views of the lay members of the Committee (both standing and specialist members) on patients’ experiences during and following the use of the technology

Uncertainty generated by the evidence and differences between evidence gained in research conditions and that relating to effectiveness in clinical practice

The possible differential benefits or greater risk of adverse effects in different groups of patients

The risks (adverse effects) and benefits of the technology as seen from the patient’s perspective

The position of the technology in the overall pathway of care and the available alternative treatments.

The extent to which the above factors are taken into account in making judgements about the evidence of clinical effectiveness is a matter for the Committee’s discretion.

16.1.4 Evaluating cost effectiveness

NICE is asked to take account of the overall resources available to the NHS when determining cost effectiveness. Therefore, decisions on the cost effectiveness of a new technology must include judgements on the

16 – Development of recommendations by Diagnostics Advisory Committee

implications for healthcare programmes for other patient groups that may be displaced by the adoption of the new technology.

The potential budget impact of adopting a new technology does not determine the DAC’s decision. The Committee does take account of how its advice may enable the more efficient use of available healthcare resources. In general, the Committee will want to be increasingly certain of the cost effectiveness of a technology as the impact of the adoption of the technology on NHS

resources increases. Therefore, the Committee may require more robust evidence on the effectiveness and cost effectiveness of technologies that are expected to have a large impact on NHS resources.

The DAC takes account of how the incremental cost effectiveness of the technology being evaluated relates to other interventions and technologies currently being applied in the NHS. In addition, as far as possible, the

Committee will want to ensure that its judgements regarding the cost-effective use of NHS resources are consistently applied between evaluations.

The Committee has to make judgements on the appropriateness and relevance of comparator technologies because this is crucial to the consideration of the cost-effectiveness evidence.

If the evidence on key parameters used to estimate cost effectiveness (for example, clinical effectiveness and effect on health-related quality of life) has serious limitations and/or a variety of assumptions have been necessary in the cost-effectiveness modelling, the additional uncertainty this generates is a key factor in underpinning the judgements of the Committee. The Committee is aware that the evidence base is often weak for diagnostic technologies.

Taking this into account, the DAC is still likely to consider more favourably technologies for which evidence on cost effectiveness is underpinned by the best-quality clinical data than those for which supporting evidence is

dependent to a large extent on theoretical modelling alone.

The Committee’s judgements on cost effectiveness are influenced by:

the strength of the supporting evidence on impact on patient health outcomes

the robustness and appropriateness of the structure of the care pathway and economic models; in particular, it considers carefully whether the model reflects the decision problem at hand and the uncertainties around the assumptions on which the model structure is based

the plausibility of the inputs into, and the assumptions made, in the economic models

its own evaluation of the modelling approach, taking into account all of the economic evidence submitted

16 – Development of recommendations by Diagnostics Advisory Committee

the range and plausibility of the incremental cost-effectiveness ratios (ICERs) generated by the models reviewed

the likelihood of decision error and its consequences.

The DAC considers carefully which patients benefit most from the technology and whether there are subgroups of patients for whom the effectiveness evidence suggests differential cost effectiveness. The DAC may recommend the use of an intervention for subgroups of the population only if there is clear evidence that the characteristics defining the subgroup influence the

effectiveness and/or cost effectiveness of the intervention.

The DAC does not use a precise ICER threshold above which a technology would automatically be defined as not cost effective or below which it would.

Given the fixed budget of the NHS, the appropriate threshold to be considered is that of the opportunity cost of programmes displaced by new, more costly technologies. NICE does not have complete information about the costs and QALYs from all competing healthcare programmes in order to define a precise threshold. However, NICE considers that it is most appropriate to use a

threshold range as described below. Furthermore, the cost effectiveness of a technology is a necessary, but is not the sole, basis for decision-making.

Consequently, NICE considers technologies in relation to this threshold range, and the influence of other factors on the decision to recommend a technology is greater if the ICER is closer to the top of the range.

Below a most plausible ICER of £20,000 per QALY gained, the decision to recommend the use of a technology is normally based on the

cost-effectiveness estimate and the acceptability of a technology as an effective use of NHS resources. If the estimated ICERs presented are less than

£20,000 per QALY gained and the Committee judges that particular

interventions should not be provided by the NHS, the recommendations make specific reference to the Committee’s view on the plausibility of the inputs to the economic modelling and/or the certainty around the estimated ICER. This might be affected, for example, by sensitivity analysis or limitations to the applicability of findings regarding effectiveness.

Above a most plausible ICER of £20,000 per QALY gained, judgements about the acceptability of the technology as an effective use of NHS resources specifically take account of the following factors:

the degree of certainty around the ICER; in particular, the Committee is more cautious about recommending a technology if it is less certain about the ICERs presented

whether there are strong reasons to indicate that the assessment of the change in HRQL has been inadequately captured, and may therefore misrepresent the health utility gained

16 – Development of recommendations by Diagnostics Advisory Committee

the innovative nature of the technology, specifically if the innovation adds demonstrable and distinctive benefits of a substantial nature that may not have been adequately captured in the QALY measure.

As the ICER of an intervention increases in the £20,000 to £30,000 range, the Committee’s judgement about the acceptability of the technology as an

effective use of NHS resources makes explicit reference to the relevant factors listed above.

Above a most plausible ICER of £30,000 per QALY gained, the Committee needs to identify an increasingly strong case for supporting the technology as an effective use of NHS resources, with regard to the factors listed above.

NICE has a strong preference for expressing health gains in terms of QALYs.

In most circumstances, when the health gain is expressed in terms of life-years gained, the range of most plausible ‘life-life-years gained’ ICERs that are acceptable is substantially lower than those described above. In these circumstances, the Committee imputes a plausible QALY value from the estimated life-years gained. The exact adjustment that the Committee makes takes account of the differences between QALYs and life-years gained. It is guided by reference to the population norms for HRQL for the affected population. In general, however, patients with a disease or condition have lower QALYs on average than the norms for the overall population.