In a fully repeated-measures design, the 31 participants performed four terrain understanding tasks on terrain views shown in each of eight different display formats. The eight formats were created by manipulating three independent variables (IVs): 2 (depth relief format) × 2 (viewing angle) × 2 (terrain fidelity). Two of these variables manipulated the key differences between the natural categories of 2D and static 3D displays (see Fig. 1) by varying depth relief format (shaded vs. topo) and viewing angle (top-down 90° vs. shallow 45°). A third IV contrasted two levels of terrain spatial fidelity (high vs. low). In Fig. 3, the same piece of terrain is shown in each of the eight views, henceforth interchangeably referred to as “display formats.”
Figure 3. The eight display formats formed from the intersection of the depth relief format (shaded vs. topo), viewing angle (45° vs. 90°), and terrain fidelity (sharp vs. smoothed) IVs. The same terrain is shown in the eight display formats, with the starting route shown as a blue dotted line. The small vertical insets show the altitude legends for shaded and topo formats. Average realism scores are beneath each display format, with the highest for the top left format and the lowest for the bottom right. The terrain views shown to participants were equal in size.
Download figure to PowerPoint
Two of the terrain understanding tasks were shape understanding (concealed route laying) and two were relative position (route altitude estimating). Performance on these tasks was subsequently related to intuitions, on a participant by participant basis, to test predictions from the Naïve Realism theory. To determine the realism of the eight display formats, a subset of participants rank ordered the eight display formats in terms of how realistically each depicted terrain, yielding an average realism score for each display format (intra-class correlation, r6 = .73) shown in Fig. 3. By rank ordering the displays in terms of perceived realism, the realism score for the display that actually supported the best performance could be compared with the realism score of the display that participants intuited they would do the task best with. The realism score method allowed us to quantify the display realism needed and the realism intuited, so that the two could be compared. The Naïve Realism theory predicts that participants will intuit needing more display realism than is actually necessary to support a task. Note that the realism scores were used only for analysis, and had no impact on which displays were presented.
Though not directly bearing on the theory, it is illuminating to consider how the different IVs contributed to judgments of realism. Referring to the shaded displays in Fig. 3 (four left-hand displays), participants assessed the sharp displays (top row) as more realistic than their smoothed counterparts (bottom row) for 90° and 45°—sharp 90° shaded (M = 6.5, SD = 1.9) vs. smoothed 90° shaded (M = 3.5, SD = 1.8), t(5) = 4.4, p < .01; sharp 45° shaded (M = 7.0, SD = 2.0) vs. smoothed 45° shaded (M = 4.7, SD = 2.2), t(5) = 2.9, p < .05. Referring to the sharp displays in Fig. 3 (top row), participants assessed the shaded displays as more realistic than their topo counterparts for 45° and 90°—sharp 45° shaded (M = 7.0, SD = 2.0) vs. sharp 45° topo (M = 4.8, SD = 1.0), t(5) = 3.1, p < .05; sharp 90° shaded (M = 6.5, SD = 1.9) vs. sharp 90° topo (M = 3.3, SD = 2.2), t(5) = 2.9, p < .05. Viewing angle did not lead to any significant differences in perceived realism.
A different piece of terrain was shown for each display format to prevent any carry-over of terrain knowledge. The eight pieces of terrain were always presented in the same order, but the order of the display formats, and therefore the display format to terrain pairing, was counterbalanced using a Latin Square design.
Eight swaths of mountainous terrain were cut from a selection of digital elevation models from the U.S. Geological Survey. Identical starting diagonal routes were designated across each piece of terrain, from lower left to upper right. Using real terrain increased external validity but complicated experimental control. Thus, several steps were taken to equate task difficulty as much as possible across the terrain set. First, each piece of terrain was graded against six criteria to ensure the presence of similar features (e.g., mountainous regions) across the set. A terrain swath was rejected if it lacked these features. Second, the digital elevation models were normalized to possess the same altitude range. Third, the starting routes were constructed to be initially visible from the same amount of surrounding terrain (i.e., equally concealed). This was accomplished through minimally adjusting waypoints away from the diagonal straight-line route to attain the same starting exposure score in each terrain piece (see Fig. 3). Finally, pilot testing confirmed that starting routes in each piece of terrain had approximately the same room for improvement in reducing their exposure to the surrounding terrain. The terrain pieces were then presented in different display formats according to the IVs, which are reviewed in turn.
First, terrain spatial fidelity was either high, or lowered by spatially filtering the terrain digital elevation models. To lower fidelity, custom software convolved the digital elevation models with a Gaussian low-pass spatial filter of fixed space constant 3-pixels (a value that pilot testing determined appropriate for the specific terrain pieces used). The spatial filtering had the effect of smoothing the terrain. Henceforth, we refer to the unfiltered, high-fidelity terrain as sharp, and the low-fidelity terrain as smoothed. The filtered or unfiltered digital elevation models were then meshed for visualization.
Second, depth relief format was manipulated by showing the terrain mesh in either shaded or topo depth relief. To accommodate viewing from different viewing angles, the two reliefs were built with a “texture draping” procedure. Shaded relief was created by draping a gray matte texture lit from the conventional northwest (top-left) direction. Topo relief was created by draping a white texture over the terrain mesh and then adding appropriate color-coded contour lines of equal altitude increments. There was no shading in the topo format. A color legend and scale were added to the side of the topo view; see Fig. 3. In both reliefs, colored dots were added at the locations of the highest and lowest altitudes in the scenes in the appropriate color from the topo legend to facilitate scene interpretation.
Third, the scene views were rendered in perspective from either 45° or 90° viewing angles using standard camera geometry. The computer interface interactions for the resulting 90° and 45° views were equated as much as possible while respecting the inherent line of sight ambiguities of the two views (Sedgwick, 1986). In both views, the mouse cursor changed into a thin crosshair when a waypoint was selected and dragged. This allowed participants to clearly pinpoint the new location where the waypoint would be placed.
Finally, for each terrain display the start and finish route locations were indicated by large dark blue dots, with start at lower left, and finish at upper right. The starting route was defined by four equally spaced adjustable waypoints. These were shown as large light blue dots placed diagonally between the start and finish locations. Strings of smaller blue dots defined the segments (route “legs”) between the waypoints; see Fig. 3. The large waypoints could be dragged and dropped with the mouse to define and adjust routes. All waypoints lay on the terrain surface.
After informed consent and Ishihara plate color vision screening, participants completed the Vandenberg Mental Rotation Test (MRT) of spatial ability (Vandenberg & Kuse, 1978). Next, participants were asked to role-play a military surveyor whose primary mission was to lay concealed routes through unfamiliar terrain.
The route-laying mission was divided into four tasks that supported a natural, goal-directed sequence of first laying and gauging a coarse initial route, and then refining and defining it (see Fig. 4). The design of these four tasks allowed us to separately study shape understanding and relative position aspects of terrain appreciation. The objective of the shape understanding tasks was to create a route that was concealed from as much of the surrounding terrain as possible. These tasks required understanding the shape of terrain for gauging lines of sight to and from the route with a richer, more continuous, and global task than the more localized (can A see B?) line of sight judgments used previously (St. John et al., 2001). The objective of the relative position tasks was to estimate the altitude of the waypoints defining the routes as accurately as possible. These tasks required understanding the precise altitudes of route waypoints. The route-laying and altitude-estimating tasks were performed for both an initial route, coarsely defined by four waypoints, and a final route, more precisely defined by 14 waypoints.
Figure 4. The four experimental tasks in order. The route-laying shape-understanding Tasks 1 and 3 (top row), and the altitude-estimating relative position Tasks 2 and 4 (bottom row). The panel below each terrain view shows the estimated altitude profile of the route.
Download figure to PowerPoint
To evaluate route-laying performance, we developed a new shape-understanding metric that assessed the concealment of a route. Overall route concealment was operationalized as the average length of lines of sight extending from the route to all surrounding terrain. Since these lines extended from the route itself to the surrounding terrain, they measured, and showed, the amount of terrain to which the route was exposed; see Fig. 5. We termed the line of sight metric the exposure score, and the corresponding visualization of these lines-of-sight superimposed on the terrain the exposure envelope. Illustrations of the exposure envelope, corresponding exposure score, and the impacts on both resulting from route adjustment were used to explain the goal of minimizing route exposure to the participants; see Fig. 5.
Figure 5. Illustration of the exposure envelope (yellow), exposure score metric, and change score demonstrating a route made less (right) or more (left) exposed. These were used to explain the concept of route exposure and the scoring metric to participants in both experiments, and as actual feedback aids in Experiment 2.
Download figure to PowerPoint
To derive an exposure score, a route was always graded against unfiltered, high-fidelity digital elevation models, in order to use the same standard of performance measurement across trials. Further, this method of grading performance ensured that the task was not simply easier when terrain fidelity was low.
For each display format, participants completed the four tasks in the order illustrated in Fig. 4. These tasks are listed below with the specific terrain appreciation (shape understanding vs. relative position) measured by each task specified:
Speeded initial route laying (shape understanding)
Initial route altitude estimating (relative position)
Self-paced final route laying (shape understanding)
Final route altitude estimating (relative position)
The details of each task are covered in turn.
1. Initial route laying: First, in the speeded initial route-laying task, participants were instructed to reduce the starting route’s exposure to the surrounding terrain. This required participants to quickly search for canyons and valleys through which to lay promising concealed routes. Participants were advised to lay the initial route in a way that it could maximally benefit from the fine adjustments made later in the final route-laying task. To create this initial route, participants used the mouse to drag and drop the four adjustable waypoints (see the coarsely adjusted initial route in Fig. 4, Task 1). The waypoint adjustment range was restricted by the software to prevent the creation of routes with unfeasibly sharp turns or grossly unevenly spaced waypoints. If one of these constraints was violated by a waypoint movement, the software popped the waypoint back to its starting location to prevent that movement. To assess initial route-laying performance, the percent difference between the starting and initial route exposure scores was calculated.
2. Initial altitude estimating: Second, in the initial altitude-estimating task, participants were asked to re-create, as accurately as possible, an altitude profile of the route from the terrain view above. On an altitude panel presented beneath the terrain view, the top and bottom lines represented the highest and lowest points on the terrain, and were marked with the corresponding colored dots from the altitude legend and terrain display. The altitude of the route’s fixed start and finish waypoints was correctly positioned on the panel, whereas the altitude of the four adjustable waypoints was set in the middle of the altitude panel as a default. Using the mouse, participants vertically adjusted each waypoint’s altitude in the panel to reflect their estimates of waypoint altitude (see Fig. 4, Task 2). The selected waypoint was highlighted in both the scene view and altitude panel to clarify which waypoint was being estimated and adjusted. To assess initial altitude estimating performance, altitude estimation error for the initial route was calculated.
3. Final route laying: Third, in the self-paced final route-laying task, participants were instructed to further reduce the exposure of their initial route. To allow for finer adjustments, the initial route was redefined with 14 evenly spaced adjustable waypoints. Unlike the speeded initial route-laying task, participants were instructed to take care and time on this task. To make fine route adjustments, participants adjusted the final route using the mouse to drag and drop waypoints and could also use the keyboard arrow keys for precise single pixel waypoint adjustments (see finely-adjusted route in Fig. 4, Task 3). Since the purpose of this final routing task was to precisely adjust the initial route, rather than to create an entirely new route, the adjustment range of the final route waypoints was further limited by the software to prevent radical departures from the initially laid route. To assess final route laying performance, the percent difference between the initial and final route exposure scores was calculated. Furthermore, initial and final route-laying performance, when summed, yielded a useful, net metric of route laying performance across the entire experiment.
4. Final altitude estimating: Fourth, in the final altitude-estimating task, participants were asked to re-create the altitude profile of the carefully adjusted final route. Participants vertically adjusted the 14 waypoints in the altitude panel to estimate waypoint altitude, just as they had in the initial altitude-estimating task (see Fig. 4, Task 4). To assess final altitude-estimating performance, altitude estimation error for the final route was calculated as in Task 2.
Participants were instructed that they had 7 min to complete all four tasks for each display format. They were told to quickly complete the initial route-laying and altitude-estimating tasks, and to spend the majority of their time and focus on the final route-laying task. To help participants adhere to these instructions, the proctor reset a large timer to 7 min at the beginning of each new terrain view. Participants performed all four tasks on a practice piece of terrain shown in their first display format before proceeding to the eight experimental views.
To test the Naïve Realism predictions about intuitions for each task, participants’ prospective intuitions of which display format would best support their performance were probed after instructions and before the tasks. After the experiment, retrospective intuitions about which display format did support the best performance were also gathered.
The entire procedure, including consent, instructions, data collection, and debrief, took about 2 h.
Several predictions were tested in Experiment 1. First, shaded depth relief was expected to support the route-laying shape-understanding tasks (1 and 3) better than topo, while topo relief was predicted to support the altitude-estimating relative position tasks (2 and 4). Second, lowering terrain fidelity was predicted to support the route-laying shape-understanding task by unmasking scene structure necessary to locate promising regions to conceal the route. Third, the Naïve Realism theory predicted that participants would intuit needing more display realism than necessary across tasks. Finally, we predicted that individuals of lower spatial ability would be more Naïvely Realistic than those of higher spatial ability before and after the experiment.