CR 169: Perceptual Countermeasures: Simulator Validation Study (1997)

Driving simulators have much to offer experimental research programs as they have the potential to provide a safe and controlled environment for testing driving performance without having to expose participants to the hazards of real world driving. However, they can also be disadvantaged if the participant’s behaviour is not normal while using the simulator, that is, if the simulator fails to elicit the same stresses and responses usually elicited while driving.

While validating off-road tests of driving performance would seem to be essential for any simulated driving test, it is rarely undertaken in practice. For the most part, experimental driving research assumes that the laboratory test results are relevant in terms of road behaviour. One might expect that an off-road test that has high face validity is testing on-road driving performance but this is always an assumption without first conducting a rigorous validation test.

A study was undertaken on behalf of the ATSB Road Safety and the New South Wales Roads and Traffic Authority to demonstrate whether the Transport Accident Commission’s Driving Simulator at the Monash University Accident Research Centre was a valid environment for testing perceptual countermeasures. In addition, it aimed to examine the effectiveness of transverse line treatments at reducing travel speed. The study was intended as a precursor to a full experimental program aimed at evaluating a range of low cost road treatments as a counter-measure to excessive speeding.

Study Method

The study set out to compare driving responses obtained on the road with those obtained in the driving simulator. The City of Banyule (formerly the City of Heidelberg) has used transverse line treatments extensively in the approach zones to intersections, roundabouts and curves to reduce accidents on suburban roads and streets. These treatment locations offered an ideal natural road experiment as similar untreated control locations were also available. The transverse lines are made from 1cm thick anti-skid material and thus provide both a visual and a rumble effect during the approach and negotiation of them.

Road Trials. An instrumented vehicle was provided by ARRB Transport Research Limited and 24 participants were recruited to drive this vehicle over a test route containing a selection of treated and untreated road sections. Primary responses collected included speed, deceleration, braking and lateral position, although yaw and lateral acceleration measures were also available.

The test route took approximately 3⁄4 hour to drive after becoming acclimatised with the test vehicle. Primary interest, however, was only with the driver’s responses for up to 100 meters "before" and "during" each treatment and control site. Data were collected on-board during the trial and analysed across 4 sections preceding the treatment and intersection or curve.

Simulator Trials. A similar number of treated and untreated sites were then programmed on the suburban road database of the TAC Driving Simulator, taking care to match both the road and treatment characteristics of each of the road sites. While it was not possible to match precisely the full on-road trial test route, a selection of normal suburban roads and road environments that would have been encountered on the road were used to connect each treatment and control site in the simulator. Primary interest again was in the participant’s driving performance in the 100 metres before and during each treatment and control site.

A different sample of 24 participants then "drove" the simulator route containing these treatment and control sites and their responses were collected for a similar range of performance measures. These data were analysed in the same way so that the road responses were to demonstrate whether the treatment effects found on the road were similarly elicited in the driving simulator.

Validation Findings

Validation can be established at a number of different levels. The least demanding level simply calls for similar patterns of responses in both driving environments. A more demanding test of validity requires statistical significance between the patterns of response on the road and in the driving simulator. A correlation of the differences observed between the treatment and control responses on-road and in the simulator constitutes a severe test of validity of the simulator.

A correlation analysis was undertaken on these data using a canonical correlation co-efficient. Unlike a usual test of correlation, a canonical correlation allows for a test of "no difference" rather than the usual converse and is eminently suitable for tests of validation. The findings are shown in Table 1.

Speed. The speed measure produced the strongest correlation and had the most similar pattern of results between test environments. However, it was less reliable at roundabouts than other test locations. This was probably the result of a lack of reality in the simulated roundabout and the subsequent discomfort it generated among the participants.

Braking. Braking, too, was significant at three of the four locations but was less sensitive in the simulator than on the road generally. This seemed to suggest that the braking motion in the simulator was not well representative of what happens on the road at these sites and probably indicates that the braking mechanism in the simulator would benefit from further development.

Table 1: Results of the validation between on-road and simulator trials

Deceleration. While deceleration is related to foot braking, it is also affected by reductions in engine power and the subsequent deceleration influence. A significant negative correlation was observed between the road and simulator deceleration at the stop sign and a weak positive correlation for the left-hand curve, suggesting that it was a less reliable measure in the simulator at these sites.

Lateral Placement. Lateral placement was only relevant for curve negotiation. While neither the left- or right-hand curves were statistically correlated, their trends were quite similar, albeit less steady on the road. This was a function of the lack of a constant centreline and the variation this produced in the on-road results compared to those collected with a constant centreline in the simulator. Importantly, in both test environments, participants moved further away from the centreline at the treated sites, confirming that this measure was valid in the simulator trials.

Transverse Line Effectiveness

While this study was principally concerned with establishing the validity of testing perceptual countermeasures in a simulated environment, it was also possible to demonstrate the usefulness of transverse line treatments in reducing speed and whether they have purely a perceptual or an alerting influence on driver’s speed choice.

Speed Reduction. The results showed that in either test environment, this low cost road treatment was quite effective at reducing travel speed, both ahead of and in the approach to a potentially hazardous intersection or curve location. Average speed reductions of 2% on the road and 8% in the simulator were observed for the treated sites. Moreover, the speed and braking patterns on the road were slower and more gentle with, than without, the treatment.

Rumble Effects. An additional trial was also conducted in the simulator where the rumble effect of these lines was removed to see what effect this would have on the results. Another group of 24 participants was recruited and tested using the same simulator test route but with no rumble effects apparent on driving over the treated lines. These results were then compared with the previous simulator findings with the rumble effect present.

The only measure which differentiated between the two sets of results was travel speed. While both treatments resulted in slower travel speeds generally, the line and rumble treatment was markedly slower than the line only treatment. This was particularly so for curves and less apparent for the stop and roundabout intersection. While there were signs of a slightly slower speed on the approach to these treatments where the perceptual effect would be expected to be more effective, these differences were not statistically robust. This finding is worthy of further examination in future research efforts.

Operator Discomfort

One disconcerting aspect of the simulator trials was the relatively high number of participants who were unable to complete their trial through sickness or reported a degree of discomfort after completion. Modifying the practice sequence prior to experimentation did reduce the incidence of discomfort substantially. However, most of the difficulty seemed to arise from the roundabout intersection and from other excessive steering movements. While there was no evidence that this discomfort influenced the validation of the simulator, it is important to ensure that future trials be aware of the potential problem and reduce the need for excessive steering wheel movements.

Conclusion

Three major conclusions could be derived from the findings of this study.

• The results of this study confirmed that the TAC Driving Simulator held at the Monash University Accident Research Centre was a suitable test environment for evaluating perceptual countermeasures.
• Transverse lines on suburban roads appear to have a positive effect on speed, often commencing some 2 or 3 seconds before the lines are actually reached. This occurred on the simulator as well as the real road. When an approaching hazard is not visually outstanding such as curves, transverse lines will create a slower approach speed, no matter what material the driver expects the lines to be made with.
• If the transverse lines have an auditory and vibration effects as well as their visual effect, they are likely to have an even larger effect than a visual effect alone.

Type: Research and Analysis Report

Sub Type: Consultant Report

Author(s): Fildes, Godley, Triggs & Jarvis

Topics: Methodology, Road, Speed

Publication Date: 01/04/97