CR 157: Monitoring the National Road Safety Strategy: Cost-Effectiveness of Road Safety Measures (1995)

Summary

This study reviewed what economic appraisals have been performed on road safety measures within the priority areas identified in the National Road Safety Strategy and Action Plan, and appraised their methodological soundness. The focus has been to establish the cost-effectiveness of measures, and to discuss the role of cost-effectiveness analysis in relation to the current reliance on cost-benefit analysis for resource allocation decisions.

Methodology

A literature search was conducted of Australian and international transport research and related databases, using university and transport sector library networks. The search keywords were framed in terms of road safety, road trauma prevention, effectiveness, evaluation, economic appraisal, cost-effectiveness and cost-benefit analysis. The aim was to identify effectiveness and cost-effectiveness studies that could be utilised in road safety priority settings. Inclusion criteria also extended to studies which presented sufficient effectiveness and cost data to construct a meaningful cost-effectiveness analysis.

Despite the comprehensiveness of the literature search, only a few genuine cost-effectiveness studies of road safety programs were found. As in the transport sector generally, the methodology used by the majority of economic appraisals, and in effectiveness studies that included economic analysis, was cost-benefit analysis. These varied in the strength of effectiveness evidence, and in those most concerned with program planning or funding proposals, the effectiveness was often only a crude estimate.

There was an extensive literature on the effectiveness of road safety programs, but mostly without an economic appraisal component. Again these studies varied widely in the quality of their design, their data, and analysis. Due mainly to the time constraints of this consultancy, it was decided to confine our analysis to only those studies which had an economic component and that were, or could be converted to, cost-effectiveness analysis. A full review of road safety effectiveness studies is probably warranted, but would be a major undertaking, and beyond the scope of this study.

A cost-effectiveness approach was applied to those studies where there was sufficient data for the analysis (ie. studies which included both effectiveness information, even if only an estimate, and cost data). Convertible studies were compiled on a database (MS Accessâ 2) using an economic analysis framework developed for this study. The total number of studies so treated was 48. Specific configurations of the database are presented in table form in the report.

Field consultations were conducted with representative interstate road safety stakeholders, with face to face meetings taking place in Melbourne, Canberra, Sydney, and Adelaide. Advice was sought from the perspectives of NRTAC, state road safety managers, road trauma researchers, insurance offices and road user representatives. Project time-line constraints prevented a wider direct consultation, but there was some indirect feedback on preliminary findings from an Austroads meeting held towards the end of the consultancy.

Overview of Findings

Cost-effectiveness analysis of road injury interventions is a useful supplementary form of analysis to cost-benefit analysis . For some research questions, it may be the preferred form of analysis. It seems clear that it is possible and meaningful to calculate the cost-effectiveness of road safety programs on the basis of cost per injury consequence. It is also possible to rank programs on the basis of their cost per injury consequence. Cost-effectiveness analysis has the strong advantage that it is more transparent in its decision rule, and easier to communicate to interested parties. It allows the decision maker to adopt a benchmark figure for the value of an injury or fatality and say that a particular program has a cost per injury or fatality prevented less or more than they are prepared to pay. If a program cost more than $5 million per life saved, for example, it may be rejected if the benchmark was $3 million.

Cost-effectiveness analysis seems to provide similar absolute values for programs to cost-benefit analysis, where conventional benchmarks are used. That is to say, programs with a benefit-cost ratio (BCR) in excess of an accepted level of say 3:1, tend to have a cost per life year saved of less than $30,000, which is less than many common health care interventions which have been evaluated. While cost-effectiveness analysis may provide similar guidance to cost-benefit analysis on broad notions of economic worth, it does produce different rankings of programs, especially where minor injury is a large component, and depending on how cost savings are treated in the benefit-cost ratio.

The key limitations on the use of published cost-benefit studies of road safety interventions, are that the underlying data on the effectiveness of the programs is most often weak, and that the available techniques of evaluation are often not rigorously applied. Many cost-benefit studies had flaws in design which were inconsistent with the theory and practice of the cost-benefit approach. There is little use of good experimental designs to establish efficacy and too much evaluation is based on intuitive guesswork, often without any supporting sensitivity analysis. Similarly, the lack of appropriate current data on the predicted outcomes of interventions would limit the use of cost-effectiveness analysis for resource allocation across program areas. Consistency in the application and reporting of economic appraisal methods would improve the quality of information available to assist decision making.

Many of the published economic studies were for a group of initiatives at a particular level of expenditure. This makes it very difficult to evaluate the cost-effectiveness of future initiatives at perhaps different levels of expenditure. We might have evaluated a program to improve road conditions at 40 black spot locations, but this might tell us little about the cost-effectiveness of the 41st or another 20 elsewhere. There is a need for studies to evaluate the marginal cost and outcomes for interventions and to consider the transportability of the results. Only one study which we looked at (Cameron, 1993) had sufficient detail to allow a marginal cost-effectiveness analysis to be carried out. In many ways this study was a good example of an evaluation with a valid study design and containing sufficient detail to allow transportability.

In the short term decisions have to be made and we would continue to recommend the use of economic evaluation methods. In the longer term we would recommend an effort to improve the standard and consistency of evaluation methods. In this context it is important to note that, where the safety budget is constrained, there are dangers in the simple application of rankings from cost-benefit or cost-effectiveness ratios. That is to say a simple league table of interventions ranked by these ratios can be a misleading guide to efficiency. There may be an important role for mathematical programming and associated techniques in determining the optimal profile of programs and the level of each program when utilising cost-benefit analysis or cost-effectiveness analysis results in the presence of a budget constraint.

In principle cost-effectiveness analysis could be used to assist in resource allocation decisions at all levels of decision making in road safety. It could be used to determine the optimal mix of programs within a priority area as well as to determine which broad area warrants extra investment. Data requirements are a barrier to each of these tasks, but particularly the latter. In order to determine the best place for additional investment we need to know the cost-effectiveness of the marginal project in each priority area. We have identified only a small number of published studies overall which can provide reliable evidence on both effectiveness and costs, and a very small number of studies in some areas. It would be dangerous, therefore, to rely too heavily on current economic knowledge to allocate resources between priority areas. On the other hand, given a longer time frame, it should be possible to frame research in this way. In the meantime, cost-effectiveness analysis can be used within some broad program areas where there is sufficient reliable data. This seems to be the case, for example, in road environment programs where unpublished data may be available on the expected costs and outcomes of particular investments. In the absence of perfect data a great deal can be done with the use of informed views of the likely effectiveness of interventions and their likely costs. When used with sensitivity or threshold analysis it is possible to show which programs are likely to be cost effective, and which will not be even under the most optimistic assumptions. In this case the choice of economic analysis type is not likely to be important. In these cases the economic worth of a project is likely to be robust no matter the assumed economic value of intangibles such as lives saved or pain and suffering avoided. Where more detailed marginal choices have to be made, however, for example on program expansion, it is not only necessary to have good knowledge of the marginal effectiveness of a program, but also of the impact of the program on the relevant budget. In other words what safety interventions have to be given up in order to expand a particular program.

Conclusions

The major conclusions of this study are as follows:

(1) the value of a project is not influenced by the value of a statistical life;

(2) cost-effectiveness analysis can be adapted to suit a number of policy arenas, e.g. it can be used to compare road safety programs with other types of programs which result in the same benefits, such as health or emergency services, or even air pollution control;

(3) cost-effectiveness analysis enables a transparent, yet systematic approach to program budgeting, that is easier to communicate to decision makers and the general public e.g. cost per crash avoided, cost per fatality prevented.

Key points

If the objective is to minimise injury at least cost then the use of ratio measures (benefit to cost or cost-effectiveness) to rank programs is a good rule of thumb, but a more appropriate technique is the use of simple mathematical programming. This can be done using standard spreadsheet programs on a microcomputer.

Download Complete Document: NRSS_Cost_1 [PDF: 2586 KB]

Type: Research and Analysis Report
Sub Type: Consultant Report
Author(s): A Harris, R Clark, G Verikios, R Carter, D Dunt & S Crowley
Topics: Economic, Methodology
Publication Date: 01/11/95